Improve your digital performance with A/B testing
What do Amazon, Shopify and Airbnb (and many others) have in common? It’s not their business models, their marketing budgets or their launch dates, and it’s definitely not their founders’ star signs.
I’ll end the suspense: they test!
They test different parts of their services, websites and/or mobile apps with real users.
We talked to you about digital optimization a few weeks ago. Now, we’re going to take a look at a simple method to allow you to generate more money with your website*. We will be looking at A/B testing from a conversion rate optimization (CRO) perspective.
Why is it so essential to talk about this now? Because now more than ever, digital is a non-negligible revenue source – the largest, for more and more companies. CRO, much like SEO, is a long-term approach to increase the benefits generated by a website over a sustained period of time.
*For the sake of brevity we are talking about websites, but note that in reality this includes all digital platforms, from email and mobile apps to social media.
“ It turns out that A/B testing is the second most popular CRO method after customer journey analysis and 71% of companies are doing two or more tests per month.”
– Khalid Salef, auteur de Conversion Optimization: Converting Your Website Visitors into Customers
Fig 1. Representation of an A/B test
The division of visitors is usually proportional, so 50% see variation A and 50% see variation B, and relies on specific conditions for segmentation (so we aren’t comparing apples to oranges). However, visitors can also be divided according to other ratios (80/20 for example), depending on the context and what you’re trying to test.
In the end, you’ll look at which version is most effective based on your goals (translated into data), then apply the winning version to the whole page, so that all visitors will see it.
In other words, if your goal is to sell more products, the variation that sells the most products will become your new baseline and will be visible to all visitors.
“The A/B test can be considered the most basic kind of randomized controlled experiment”
– Kaiser Fung, author of multiple books on data science and advanced analytics.
One of the advantages of A/B testing is that the comparison between variations is done over the same time period. This ensures that your results, and the data you collect, are consistent. Had you simply compared data from before and after you made a change to the site, countless external factors could have influenced your results, leading to false conclusions.
Another advantage is that your analysis is based on statistically significant results. Without getting into details, if one of your test variations gives you a significant result with 95% statistical confidence, that means that if you reproduced your experiment 100 times, you would get the same result 95 times. Reassuring, right?
At Adviso, we’ve helped to build hundreds of experiments from the very simple to the very complex, for many clients in many industries. Here are a few examples of simple experiments that have proven their mettle:
Example 1: Add a “popular” label
Gains: Conversion for this page is multiplied by 1.5
Learning: When presenting multiple, complex choices, give clear, visual social proofs to help users make a choice and subscribe to the offering.
Example 2: Make your most important button stand out visually
The famous button colour test. Often the butt of jokes in the CRO world because of its simplicity, in some cases it can still sometimes have a positive effect on conversion.
Gains: +10% of visitors add the product to their shopping carts
Learning: Always make sure the main action you want users to take stands out on the page.
Example 3: Add information on the process to be accomplished
Gains: +9% in forms launched and +19% in forms completed and sent
Learning: Before the user has to start a process that will require some effort, make it easier on them by clarifying what they need to know.
Example 4: Talk about a promotion you’re running
Gains: +38% conversion
Learning: Present your promotions succinctly and in an unmistakable way on the pages of your site that help users make decisions.
To retain: To quickly find ideas for the experiments most likely to have an impact, start by asking yourself the following questions:
- Why are your visitors on your site?
- What obstacles might prevent them from moving forward?
- What might persuade them to move on to the next step?