A Hypothesis is important for understanding what you are trying to prove with your A/B test. A well-formed hypothesis acts as a test guide.
A hypothesis is going to challenge an assumption you have about your website’s performance and/or visitor behavior. What is the assumption you want to validate as right or wrong?
Ask yourself these questions when coming up with your test hypothesis:
- What assumption are you addressing? Is there data to support your assumption?
- What solution are you proposing to address the challenged assumption?
- What is the anticipated outcome of your challenge? What metrics will be impacted if you make the specific change?
Asking those questions will help us ensure the hypothesis is S.O.U.N.D.:
Specific – the hypothesis should clearly define the change that is being tested.
Objective – while the test is proving or disproving an assumption – that assumption should be based upon actual insights – analytics, industry research, or user feedback for example.
User-focused – the hypothesis should address a user pain point. Focusing on user experience will increase test engagement and result in better outcomes.
Needs-based – the hypothesis should address a business need. Spend time on tests that will bring value to the business as well as the user. Keep ROI front of mind.
Data-driven – always make sure the hypothesis has measurable metrics and a clear quantitative goal.
Some examples of a solid hypothesis are:
The current headline on our landing page lacks a clear value proposition, so changing the headline to a more concise and benefit-oriented version will increase conversion rate.
Our promo banners blend in with the page design causing users to scroll by them, so testing a more contrasting color will increase CTA clicks on the banners.
The lead capture form is too long causing users to exit the site, so reducing the number of form fields from 20 to 10 will increase the number of leads.
