Google’s announcement of the upcoming retirement of their Optimize A/B testing product took me by surprise. In my mind’s eye I still see experimentation as shiny and exciting technology. I’ll blame this on the finesse of leading software providers, who have adeptly added sophisticated testing and personalization capabilities over the years, right in line with the mounting complexity faced by digital marketers.
The most recent optimization innovations like hyper-personalization, audience patterns and insights, and experimentation automation – all powered by machine learning – have kept geeks like me enamored and sparked the interest of less-nerdy marketers and even IT teams. Setting AI aside, though, these solutions have been around long enough to see two generations of Google testing products enter and exit the stage. They’re downright mature.
Once the demise of Google Optimize began trending everywhere I noticed a peculiarity. Usage seemed to be far lower than I’d have expected given the maturity of the technology. I’ve seen rough guesstimates that a whopping 25,000 Google Optimize accounts have been created during the product’s lifespan, yet only about 5,000 have ever run an experiment. More than 6,000 accounts are estimated to have installed the tool but never used it, with thousands of others conducting just 1 test before shelving it.
Google Optimize’s staying power tells us some organizations found it to be a good fit for their experimentation needs, especially if they wanted to keep things basic and low volume. But others weren’t shy about their frustrations, like an unintuitive user interface, cumbersome workflow, performance issues, and the complexity of trying to leverage GA integration (e.g., for audience targeting).
A Broader View
Taking a broader view, Gartner reported that slightly more than half of large enterprises have implemented some form of A/B testing, whether Google Optimize or one of many other experimentation products. This is strikingly low adoption among a segment that has prioritized investments in improving the customer experience and in martech overall. Contrast this with how marketers were, as a whole, embraced a slew of marketing technologies over the past decade to help improve customer experience and power marketing performance: CRM, CMS, marketing automation, google ads, ABM, video marketing…and so many more. At one point these were each new, but we adopted, adapted, and made them central to how we work.
So…why not experimentation? Why hasn’t every organization made it core to how they shape and refine impactful digital experiences for their customers? And why have many tried but abandoned it, while adopting other martech with relative ease?
The Complexity Factor
The plain fact is many optimization tools “for marketing” were not truly designed for ease of use by marketers. They are of great value and easily usable by data scientists and analytics experts. The problem is (most) marketers are not data scientists.
I’ll share a personal example: I have configured and run experiments using a couple of different legacy platforms. With a master’s degree in tech marketing, stats courses, and even some SPSS under my belt, I still found myself referring back to textbooks and Google to try to decipher a cryptic UI and data labels. I am by no means a data scientist, and I’m fairly certain the slow adoption of testing among marketers overall correlates with the level of frustration we’ve experienced over the years.
I’m also certain of this: We have to figure this out because we need data science to solve the complex marketing problems of 2023 and beyond.
I’m thinking about questions and challenges like:
- Which deals in the pipeline are most likely to close and should be getting most of our sales team’s focus?
- What changes to our commerce site would drive the biggest reduction in cart abandonment?
- Which of these experience design options is more likely to engage our target user on their journey – and should be implemented in our app?
Whether we’re talking about lead scoring, conversion optimization, user testing, or any number of other questions, experimentation powered by data science is the right hammer for the nail.
Next Steps: Harnessing the Power of Experimentation
Thankfully, marketing leaders seem to realize this, with nearly 80 percent reporting plans to increase investment in experimentation over the next 12 months in a recent Econsultancy study. Each of those leaders now faces important questions about how to get set up for success and make the most of their investment to reap measurable KPI improvements.
I’d love to be able to offer all of those marketers – and you – a standard guidebook that will fit all of your organizations but there’s truly no silver bullet. I can at least share a few suggestions that come to mind for any marketing team looking to start or improve an experimentation program:
- Assess experimentation platforms carefully and choose one that will work for you, rather than just making more work for you. Look for those that are more intuitive, better performing, and yield more reliable results than legacy options (like Google Optimize). They exist – Optimizely immediately comes to mind – and they work.
-
Invest in data literacy development for the entire marketing team – not to become data scientists, but to become comfortable in the language of data, to understand the meaning of being a data-driven marketing organization, and to be able to effectively and collaborate engage with the data and analytics teams across your organization to assess problems and challenges.
-
Call on experts to help you build an experimentation program that fits your organization and will be sustainable once the technology vendor and consultants are gone. That includes training and enablement for your team on everything from building hypotheses and prioritizing tests, to configuring and executing experiments, through assessing results and deciding what adjustments to make. Look to your vendor partners for help with defining what a culture of experimentation could look like in your unique organization, honing processes, and roles, and outlining a governance model to prevent friction and costly missteps.
Seizing the Moment
Thanks to Google, experimentation is in the air. I encourage you to seize the moment and strike up a conversation about its potential and your path forward – with your team, your leaders, or just me . ☺
