Creative SEO: Who’s there? Google? Here’s a page, just for you!

It’s been.. uhm.. “rumored” that some sites who require you to pay and login to read their content threat web-crowlers differently and allow them to crawl “restricted” content. Which is nice, since all you have to do to access such sites without paying is to say you’re Google.

After pretending to be Google a few days I’ve noticed something. Many websites seem to give a different page depending on who visits. For example, this is the front page at www.bluecoat.com:

Doesn’t look very fancy, does it? That is because they serve Google (and anyone/thing who pretends to be Google) a completely different page.

Their website actually looks like this - in most browsers:

This is what’s called doing black-hat “search engine optimization”.

Except for one little detail. The problem with all kinds of “dirty trick” black-hat SEO is that it doesn’t work.

And it specially doesn’t work with Google. Se, here’s a little dirty secret about GoogleBot: It sometimes lies about who’s there! It will fetch the / using the normal User-Agent, wait a while, and re-crawl the root page / using a (outdated beta-version of a Linux-only) web browser string.

I don’t actually know what Google (or more correctly, their bot..) thinks of websites who give them a different page. But I do not think their bot likes that kind of SEO. And as mentioned, it’s not like you’re fooling anyone by trying to give search-engines a different page, most of them now check at least 1 page on your site using a “fake” (as in not their own) User-Agent string.

But I actually like getting a simpler “SEO” page. It’s much simpler to find what you’re looking for using a “Web 0.1″ plain text link-list - in most cases…

Just one more little detail regarding SEO: It does not work. Forget about the SE part. Just optimize your sites for human visitors. If they like it then real people who like your site will link to your site and pages on your site, and that’s the only kind of SEO which actuall works. Period.

Explore posts in the same categories: Computer technology, Internet, Marketing, Search Engines

This entry was posted on Wednesday, February 21st, 2007 at 3:36 pm and is filed under Computer technology, Internet, Marketing, Search Engines. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.