Cloaking is the practice of showing a different result to a search engine than is shown to an end user. Classically, this is done by one of three methods:
- Detecting the User Agent Name of the Search Engine Robot
- Detecting the IP Address of the Search Engine Robot (“Bot”)
- Detecting both the User Agent Name and the IP Address of the Bot
Once the Bot is detected, the server is directed to send a different set of content to the Bot than would have been served to an end user. Why would anyone want to do this? Here are a few reasons:
- Your site has Session IDs on it, and this is preventing the Bot from indexing your site properly
- Your site has complex menus, Flash, and Javascript all over every page, and you want to send the Bot your content without all this other stuff in the way.
- Spammers used to use this technique to get ranked for terms they didn’t deserve to rank for.
The first two ideas above sound like perfectly legitimate uses for cloaking. In neither of these two examples are you trying to mislead the search engine. In both cases, you are serving the search engine the right content, just in an easier to process form.
Unfortunately, search engines do not approve of cloaking in any form. In fact, Google has periodically implemented new Bots whose name and IP address are not known to anyone, and sent them out to detect cases of cloaking. The purpose of these Bots is to identify sites that are cloaking, and to raise them to the attention of their human editors for review. Even if your site is squeaky clean, there is no upside to being included in this process.
I have spoken to engineers at Google about this exact issue. While they acknowledge that there are situations where someone might think of implementing cloaking for legitimate reasons, they don’t want to deal with it. They do not approve of cloaking in any form.
However, I know of sites that cloak and have done so successfully for a long time. I have worked with a couple along the way. Based on this practical experience, my interpretation of all this is that they do in fact try to distinguish between “good cloaking” and “bad cloaking”. However, if you have implemented “good cloaking”, it’s at your own risk, and if they accidentally ban you, so be it, it’s your problem.
Let’s look at an example. If you type in “Nirvana CDs” in Google, one of the results returned is:
If you click on the link and view the web page, and then view the source, you get a source file with 1547 lines in it. If you click on the “Cached” link, and view the source, you get a file that is 71 lines long! The reason for this is that CD Universe is providing Google with a version of its content that has all the menu code and Javascript removed from the page. The actual unique content is identical.
Why would they do this? Simple, it greatly reduces Google’s crawl burden, and lets it find the content quickly and efficiently. Since it takes Google almost no time to crawl this page, it can spend more time crawling other pages on the site. These are good things. However, it is still cloaking. The bot is getting a different page than the user.
Note that CD Universe is not trying to “fool” the bot with this technique, so it is what I would call a legitimate reason for cloaking. They have been doing this for years, and Google has not banned them, and may never do so. However, they run the risk that someday Google will tweak its algorithm, and they would get banned. Not for malicious behavior or spamming, but because of behavior that is known to be risky. “Forwarned is Forarmed,” they say.
I would like to briefly mention another situation that is not cloaking. Many sites detect the incoming IP address of the user, and based on the IP address, decide what language the site should be served in. For example, if someone comes in from a German IP address, they get the German version of the site. If their IP address is a US-based address, they get the US English version of the site, and so forth. But there is no reason here to treat the search engine bots any differently. If a bot comes in from a German IP address, give it the German site! Not a problem.
As for cloaking, my advice is don’t do it. It’s like the old commercial about safe driving: “You can be right, but who cares if you are dead right”. The point in this old commercial and the point I would like you to carry away from this article is that it’s not about being right. It’s about using safe and secure techniques to build your business.
If you can find any way, other than cloaking, to solve your problem, then do it. If you can’t, and you must cloak, then get it fixed as soon as possible. Much better safe than sorry when it’s your business on the line.
