Contributor
The 5 keys to establishing a culture of experimentation at your company
Contributor
This article was written in collaboration with Camille Radzioch.
Jeff Bezos is known for providing one of the best quotes about the importance of experimentation to the development and success of a company:
« Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day. »
In the fields of software development and information technology (IT)—and particularly amongst MAMAA (Meta, Alphabet, Microsoft, Amazon and Apple)—it is generally understood that continuous experimentation must be at the heart of company culture. Unfortunately, in a good number of other business sectors, this philosophy is not as widespread as it should be. Even amongst marketing teams, which in principle are the best equipped for conducting data reliability testing, A/B testing to optimize ad campaigns or emails is still not a common practice.
And yet the effectiveness of this type of approach is unequivocal. According to a study conducted by Litmus, companies that systematically evaluate the effectiveness of their advertising emails through A/B testing earn marketing returns that are up to 37% higher than companies that don’t. A second study, this time published in the prestigious scientific journal Management Science in 2022, confirmed that experimentation confers a significant competitive advantage to those who employ it. That study showed that companies who adopted A/B testing saw their performance improve from 30% to 100% after only one year of use. The authors of the study (Koning, Hasan and Chatterji) also stated that this substantial effect was not related to a practice of occasional testing of incremental changes, but instead to the use of these tests as part of a larger strategy that is more long-term in nature.
In other words, the key is not employing such tests on an ad hoc basis, but instead establishing an overall strategy of experimentation for the company and applying it systematically and continuously.
A favourable context
According to an article in Harvard Business Review, published in 2021, conditions today are better than ever for launching an “experimental revolution” in marketing. First, access to enormous quantities of first-party data about consumers has been greatly democratized in recent years, particularly through cloud-based technologies like Amazon and Google Cloud. Second, in marketing today it’s easier than ever to centralize all advertising data in the same location, whether they are behavioural (such as Google, Adobe or MixPanel Analytics), CRM or email (in a marketing data warehouse, for example).
Which brings us to an obvious question: If the context is so favourable and the tools needed are within easy reach, why is it that so many companies still don’t have a continuous experimentation program?
If technology is no longer an obstacle to experimentation, then we need to face the fact that it’s organizational culture that is the problem and which must change. So how can such a culture be created? How can a mature A/B testing practice be established? These are the questions we will answer in this article, in addition to offering five ways of encouraging an experimental mindset within your organization.
Experimentation: both a philosophy and a culture
A culture of experimentation is above all a democracy: It’s the active participation of the entire organization that contributes to establishing this mindset, and not just the team specializing in conversion rate optimization (CRO) for your business.
One of the first jobs of the CRO leader will be identifying those with the right profile for contributing to an “experimentation squad.” Seek out people who appear interested in continuous experimentation, who have already put it into practice or who have a particularly logical mindset, which would be consistent with such an approach. Choosing the right team members will be essential for the elected spokesperson of the CRO team, since they will definitely need to be able to count on the support of those colleagues to ensure their ideas are accepted and applied within each department, and even by upper management.
Once the team tasked with establishing the culture of experimentation is created, the CRO team spokesperson will be in charge of orchestrating and hosting a variety of meetings with all specialists. The spokesperson will not only need to be able to promote a fairly complex and not-well-understood field, but also understand how each person can contribute to the company’s culture of experimentation.
Here are five ideas to help you establish this mindset within your organization, as well as a few tips to help you avoid or overcome certain roadblocks you might encounter during the process.
1. Understand the company’s goals
At Adviso, we’re in the habit of holding a workshop-type launch meeting every time we gain a new client in order to arrive at a better conception of their business context.
This is an essential step, because it enables us to understand not only the mission of the company, but also the business goal it would like to achieve in the year to come, with that goal becoming our “North Star Metric” (NSM). Like a pole star, the NSM serves as a compass for the organization by indicating which direction to take, from which a number of sub-goals are derived.
Without this information, it wouldn’t be possible to obtain an overall perspective so as to clearly identify which problems are priorities to be solved, and even less to envision new optimization possibilities for testing.
If you’re not familiar with this process, we invite you to take a look at the “goal tree” model, established by Optimizely. It serves as a decision tree showing targets to be achieved for various key performance indicators (KPI) in order to arrive at your business goals. Do you have a specific objective, for example a certain number of tests to perform per month? Or perhaps a specific percentage increase to achieve a KPI target?
2. Adopt a clear mission and vision
Now that goals have been clearly defined, you need to decide what impact the Experimentation Squad will have on the company and the benefits you would like to obtain from developing a culture of experimentation.
What is the main mandate you will give to the CRO team? How will this team support the achievement of the company’s business goals? How can you encourage the spread of the CRO services offered by the company? How can you position the brand advantageously so that people will spontaneously turn to this team whenever they have ideas, issues or needs that require optimizing their conversion rates?
Every context is different and many questions may arise. Be sure to discuss them not just with your CRO specialists, but also with the selected spokespeople for your other teams.
3. Handle the critics
It is natural that not every collaborator will feel the same sense of investment in this organizational shift. Depending on the degree to which these people feel involved in the process, they will be more or less likely to participate in developing a culture of experimentation within your company.
You might simply face a lack of interest on the part of these collaborators, or, in a more complicated situation, a fear of change.
If teams are fearful of the effect of changing their everyday work methods or workload, reassure them by emphasizing that this new professional challenge is an opportunity to develop skills without having to invest a huge amount of time.
You should also highlight the plus value generated by optimizing conversion rates. If you need ideas on this subject, don’t hesitate to read this article, which presents five good reasons for starting a continuous optimization program. It should help you to construct a really convincing argument!
4. The HiPPO challenge
Have you ever heard of HiPPO? This acronym invented by Avinash Kaushik (a writer specializing in digital marketing and web analytics) stands for “Highest Paid Person’s Opinion”, or in other words, the influence a CEO or director’s opinion can have on decision making before launching an optimization.
You will sometimes have to face ideas or opinions that are entirely based on someone’s intuition, without the least amount of data to support their validity. And yet, at the risk of repeating ourselves, optimizing conversion rates is all about the data! It’s what helps us identify real problems and produce hypotheses to be proven during testing.
Don’t let yourself be intimidated by the title of the person sitting across from you! Instead, explain in a rational way using solid facts why an idea that doesn’t necessarily seem bad may still not be suitable given the company’s goals and business context.
Show goodwill and patience to present your arguments calmly, always supporting your ideas with reliable, proven data
5. Keep your employees engaged
Communicate, communicate, communicate! It’s the key to success when it comes to keeping people interested in your mission.
Keep the teams involved in your culture of experimentation project as informed as possible. Regularly update them on upcoming tests, failures, successes and problems you’ve encountered as the project evolves.
You should especially not forget to document everything in order to construct a complete history of your process. This record will let you remember all the steps you took during the process and all the strategies that contributed to its realization. A historical record is a gold mine of valuable information, both for future collaborations and to ensure the continuation of your work within the company should you leave your position.
Consider also reuniting the Experimentation Squad regularly by holding recurring meetings with team leaders. Whether it’s to discuss new tests, having the group vote on ideas, encourage further thinking during Lunch and Learn sessions or provide training on a particular subject, there’s always a good reason for a meeting!
You should also seize the opportunity to involve new employees by presenting your service to them—it’s a must for creating new allies within the organization!
Finally, be ready to listen to all employees at your company, though without drowning in too many new tests to be validated. Be patient—insist that new ideas have the solid support of data analysis, that these analyses are either quantitative (collected through analytics tools), behavioural (obtained through scroll maps or click maps, which record user sessions, etc.) or qualitative (customer feedback, site survey, themes related to customer service, etc.).
*
Want to launch a continuous optimization program but don’t know where to start? Get in touch with us! Our team of specialists would be happy to help you achieve your business goals.
