Last year in our Web Analytics Shootout report we discussed the effect that Javascript placement has on the collection of analytics data. For example, you can place your analytics Javascript at the top of you page (right after the opening <BODY> tag, or even in the header), or at the bottom of your page (right before the closing </BODY> tag). Recently, we tested this again, and this article will provide the results of that test.
Shootout Results for Javascript Delay
Within the Web Analytics Shootout we conducted a test using an IndexTools tracking pixel at the top of the file and IndexTools tracking Javascript at the bottom of the file. We observed that the tracking pixel was seeing 2% to 4% more traffic than the Javascript, and we measured a delay between the execution of the tracking pixel and the Javascript of 1.4 seconds.
Based on that, we theorized that a small number of people were coming to the site staying long enough for the tracking pixel to execute, but leaving before the site before the analytics Javascript executed. It was a reasonable theory, but the test was not sufficiently rigorous, because the counting methods for the tracking pixel and the Javascript are not identical.
New Test Methodology
With the help of Dennis Mortensen and the IndexTools team, we set up a more disciplined test. In this test, we placed Javascript both at the top and the bottom of the web pages for two sites (Tool Parts Direct and City Town Info).
The scripts were basically identical, except they fed their data into 2 different IndexTools accounts. This allowed us to more accurately measure the impact of placement on counting of visitors to the site. While we implemented the two scripts sitewide, we focused our analysis on the Page Views to the home page.
The reason for this is that one key element of the study was to measure the delay between the execution of the two scripts. This would be hard to do in a rigorous manner across all the pages of the two sites, each of which is quite large and has highly varied types of pages. However, a specific high traffic page is easier to measure with some precision.
It should be noted that there are still some issues as the current traffic load on the site may also be a factor in the JavaScript delay. We have tried to address this by measuring the Javascript delays at different times of day when the sites were under varied loads to provide for a range of times and delays.
Test Results
Our results in this test confirmed what we saw in the initial Shootout report. Here is the raw data for City Town Info:
 |
Here are the raw results for Tool Parts Direct:
 |
Here are the definitions for my cryptic labels in the results:
- Bottom Page Views: The number of page views counted by the JavaScript at the bottom of the page (just before the </BODY> tag).
- Top Page Views: The number of page views counted by the JavaScript at the top of the page (just after the <BODY> tag).
- Delta: Top Page Views – Bottom Page Views
- Min JS Delay – The shortest time we measured between the execution start of the top Javascript and the execution start of the bottom Javascript.
- Max JS Delay – The longest time we measured between the execution start of the top Javascript and the execution start of the bottom Javascript.
- Average JS Delay – The average time we measured between the execution start of the top Javascript and the execution start of the bottom Javascript.
As you can see, there is once again some loss of data in the results. So we have confirmed our theory that we formulated in the original Shootout report.
In addition, with this test, you can see how the longer delay that you see on Tool Parts Direct resulted in an increase in lost page views from 2.42% to 4.53%. Therefore it is reasonable to expect that a slower loading page would see a larger impact.
Significance
As any practitioner of analytics will tell you, the Data Quality Sucks, Let’s Just Get Over it (well maybe they won’t all use Avinash’s words, but they will tell you essentially the same thing). The bottom line in analytics remains that it is its relative measurement capabilities that matter the most. In other words, you need to focus on trends in the data.
For example, if you make a set of changes to your site you want to measure the scope of the impact those changes had. This is simply done by looking at your data before and after the changes were made. If the metrics you chose to look at improved, then it’s a sure bet that the changes you made will help your business. This is what is so wonderful about web analytics – you get much richer data than is available to measure the results of traditional marketing campaigns.
However, removing sources of error is a good and smart thing to do. An error of 4.53% is not insignificant, and slower loading pages will likely see larger errors. In addition, this source of error is easily eliminated, by placing your analytics Javascript at the top of your page.
One downside to this is if the web analytics vendor’s server goes down for some period of time, it will prevent your page from loading. While this is a rare event, it is a definite bummer when it happens. This problem does not exist when you place the Javascript at the bottom of your page, because your page load has completed (other than running the Javascript) when that Javascript finally executes. If the vendor’s server is down, the user can still see the complete page on your site, and click on the next link, and do whatever they came to do.
Suggested Action(s)
Our suggestion would be that if your page load is normally pretty snappy (total page load time of under 3 or 4 seconds), then keep your Javascript at the bottom, and remove any risk related to analytics vendor downtime. The small loss of data you see in this scenario should not be a significant factor in the value of your analytics data.
But, if your page load time is a bit slower (4 seconds or more), you may want to consider placing your analytics Javascript at the top of your web page. Your data loss will be larger, and also the nature of the lost data may start to differ.
When someone comes to your site, and won’t wait 3 or 4 seconds, this is most likely due to a complete lack of interest that the person has in what they saw there. Of course, even pages that normally load quickly can have moments when they won’t load. Your web server could be down, for example. The nature of the lost data in this scenario is largely related to situations you can’t influence.
But, as the total time of delay scales up, you start getting situations where someone sees the link they want on your page while it is still partially loaded. You might not want to lose the ability to track those people. For example, they may be coming to your site from a PPC campaign.
These recommendations also presume that your analytics Javascript is reasonably fast in executing (less than a second). If for some reason you have analytics Javascript that normally takes more than a second to execute, you should be a lot more hesitant to move it towards the top of your page, because it can become a factor in customer perception of your site (because they perceive responsiveness of the site based on when they begin to see the content on screen).
