A/B testing
What it is
A method for comparing two versions of a webpage or app against one another to determine which works better. This lets you ask focused questions about changes to your product and then collect data about the impact of that change.
Why this method
It takes the guesswork out of your product optimization and enables data-informed decisions to get a more effective product.
How to use
- Identify potential problem content or features that you would like to get data on. Determine what aspect you want to optimise. This could be about anything from driving engagement up or increasing conversion rates to other pages. You can find some of these problem areas by looking at Heatmaps or Google Analytics.
- Create a second version of your product. Keep step one in mind. This second version could include small changes or a complete redesign.
- Conduct the A/B test where half of your users are shown the original version of your product (known as the control or ‘A’) and half of your users are shown the second version (known as ‘B’). Make sure you run the test for a long enough period to gather valid results usually around 1-4 weeks.
- Analyse the results of the test to determine which solution produced the highest conversion or engagement rates, or whatever your goal may be. You can do this by using online tools like Google Analytics to check comparative responses or a statistical significance calculator like this one.
- If your results are more positive or successful for version ‘B’ compared against ‘A’, you know that ‘B’ is the better product and vice versa. Review and run this test with as many ‘B’ versions as you would like.