I read this morning two recent posts about problems with Analytics on other SEO blogs. Reading these posts has caused me to add my own thoughts to the fray. One of these two posts was done by Michael Martinez at SEOMoz, and it titled How Reliable is Google Analytics.
In it, Michael presents some puzzling data about traffic levels reported by Google Analytics (GA) and what he sees in his log files. Google Analytics, it seems, is significantly under-reporting traffic. In addition, GA is seeing dramatic changes in traffic levels that do not correspond in any way to changes seen in the log files.
It’s fascinating to see that there is such a large disconnect in the data. GA is Javascript based. This means that it should, in principle, be able to provide a very active measurement of the behavior of visitors to the site. Michael mentions one possible source of error as being the robot filtering. Of course, another form of error is that some people have Javascript disabled in their browsers.
However, we don’t think that this should be a substantial source of error.
Lisa Barone over at BruceClay.com offers an excellent post about web analytics titled: Five Problems Facing Web Analytics. In this post, Lisa takes a somewhat broader look at the problem of web analytics. You can summarize some of her posts by saying that it’s really hard work (there is no free lunch).
Lisa based her post off a report by Jim Sterne for DM News’ Special Report on Web Analytics. You can read the whole series of articles in the report and get a better idea as to how Web Analytics fits into your overall strategy.
Making analytics work for you requires a strong analyst, an analytics strategy, and an ongoing investment to measure, tweak, measure again, etc. Lisa also makes the point that any tool you use will make errors. There are many possible sources for these errors. There is the Javascript problem I mentioned above.
Likewise, log file analysis based tools are also prone to errors – for one thing, they don’t count traffic where the page served to the end user is delivered from some third party cache (such as an ISP cache). The most well-known source of errors in analytics software relates to the use of cookies by these programs. Many users have cookies disabled on their machines (last I heard it was 2 or 3%). Also, another small source of error can be the time zone orientation of the tool (when does its day start and end).
One way we have tested analytics tools with our clients is to implement multiple tools so that they can cross check one another. We have done some tests with 3 tools at once at times. When 2 of the tools agree (more or less), and the 3rd is off by a large amount, chances are the other 2 tools are accurate. As Lisa said, this is hard work.
But the tools can provide solid (not perfect) data. Putting GA aside (perhaps you really do get what you pay for), we have had great experiences with this type of data analysis. You can use analytics to improve the conversion of your site. Instead of using the feeling in your gut to design your web pages, analytics allow you to start with your gut and then experiment.
For example, we had one experience, using GA by the way, where we learned that users were not clicking on the links where we expected. It was on a page with an 8 paragraph long informational article. We offered users links that led to our conversion pages near the top of the page, to the left of the menu.
But no one was clicking there. The majority of our visitors read the entire article. They clicked on things near the bottom of the article as a follow-up – our links to conversion pages had scrolled out of sight. The article had generated interest so people were looking for more information from us, but we had hidden it on them. It was a quick fix – we offered a link to our conversion pages at the bottom of the page, and all was well.
Sounds simple, and in actuality, it was. But we had put in the effort to set up the tracking we needed, and we worked hard at it to get it right. We think that this type of investment is well worth the time for most sites.
Follow Us
