Digital Change 2007 to Now: Measurement and Testing

When you think about analytics and optimization, you see perhaps the largest changes between now and 2007.

I’ll note what I saw in 2007 and surrounding years.  I’m sure there are plenty of exceptions but this seemed to be what we saw:

• Web analytics was used.  Most sites hooked up a web analytics tool
• With a few exceptions, most companies spent little time gaining insight from the analytics
  • If I only had a nickel for the number of times I came to a client and asked for their usage or web analytics data and was told, “yeah……… we have guy in IT who is over that tool.  He sends out a report I think monthly.  I’m sure he could give you some information.”
• Deeper measurement to understand what a specific user did on a site was almost unheard of
• Almost no one was using A/B testing tools. Yes, Oracle did the first such test in 2000 but Optimizely was founded in 2010 and Maxymiser was founded in 2006.  But we didn’t see it in client sites, especially those not related to ecommerce
• Very few companies had roles related to measurement. Most pushed that to some techie somewhere

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

So it’s safe to say that with a few exceptions, Measurement, testing, and optimization were in their infancy.

Measurement and Testing in 2017

We have seen a huge change in this area.  Frankly, I could turn the time over to a colleague, Ross Monoghan who runs our digital marketing practice, and he would turn this post into a book. But my purpose is to highlight the change rather than the best practices.

To show this change, let me highlight a client who is about to launch a new version of their site with new web content and testing tools. Here’s their reality:

1. They use a team of resources that add up to three full time people
2. The team consists of people focused on SEO, SEM, social analytics, mobile analytics, and web analytics
3. They track far more than just page views and mobile app downloads
4. The product managers and content owners depend on the data for insight on what works on the site
5. New development starts with members of the analytics team providing input and insight on a given area
6. While A/B or multivariate testing is relatively new to the organization, the new governance process incorporates that throughout the lifecycle.  It’s no longer about defining what you want and then implementing. It’s about asking a question, constructing a hypothesis, and testing it. (Note, this is a huge change for the business.   We are asking them not to tell us what they want but to define the outcome and work with us to define content and apps or features that might meet that goal)
7. Like many other organizations, they have multiple apps, sites, subdomains, and third party tools.  To the extend possible, all of these rollup data into a combined report.  They ask any new web or mobile based tools to support their analytics requirements.

So yes, the world has changed.  This particular client example isn’t even at the forefront of possibilities. Other clients are pushing that data back into data lakes and predictive analytics tools to gain further insight and to push personalization to a 1:1 level.  That kind of insight is amazing.  One client found counter-intuitive results about when they are about to lose a client. When they stop calling the help desk, something is about to happen.

Bottom Line

This isn’t your old web master’s web analytics anymore. smart companies harness the data and use it to create better and more personalized experiences for their customers.  Those not spending the time to measure, test, and optimize will find themselves on the trail to being a digital dinosaur.

The really smart ones will continue to optimize their processes to make use of these insights in a more real time fashion.