Since the emergence of paid search nearly 20 years ago, performance metrics have barely changed. SEM specialists want to maximize clicks and use this as a metric to measure conversion, and simultaneously reduce the cost per click. Taking an example, if your annual AdWords budget was $1,000,000 and you succeeded in reducing your costs by 20% for the same volume of clicks, you would have saved $200,000. It is in this constant search for performance that Adviso develops and deploys innovative tools and solutions. This article will explain how we built one of these internal tools able to maintain a high level of performance for our clients, despite the degree of difficulty that has considerably increased these past few years. This secret weapon, which integrates data science, allows us to maximize the quality and precision of our targeting, all in record time!
Since the year 2000, it has become difficult to obtain these types of results. Competition is fiercer, bidding is saturated, and Google’s algorithm is more fickle. In fact, even achieving 10% savings in CPC is a challenge, especially for highly optimized accounts and on contingent keywords.
Previously, the main way to get a competitive advantage was to use a bid management tool like Marin Software, Kenshoo or Adobe Media Optimizer (now Adobe Advertising Cloud). These tools allowed us to manage highly complex, granular bid formulas on an enormous volume of keywords.
Of course, these platforms are still relevant today, as are the good old instincts specialists. But now, that’s no longer enough in order for the specialist to bring a significant added value. Any advertiser or agency with a big enough budget has access to a Marin or a Google DS3; we therefore have to push innovation further.
To squeeze as much value as possible out of your AdWords data, even with Marin or Kenshoo, requires a huge amount of time for analysis. And time is one thing that’s not readily available in the competitive world of SEM.
That’s why at Adviso, we decided to develop a different approach by looking to science for help… Data science, to be exact.
Through delving into the world of APIs, Big Data, statistical models, and programming in Python and R, we developed a series of tools to help us analyze data in just a few clicks, that would, in the not-so-distant past, have taken hours and hours of work.
The vast majority of accounts we’ve audited have come to the same conclusion. Targeting broad terms doesn’t work anymore. You have to target exact terms if you want to maximize the quality and precision of your ads.
That much is clear. But the real challenge you’re going to face is the time you could end up investing in building and maintaining your structure. Broad match targeting allows you to cover a large volume of search terms without investing a lot of time in setting up the campaign. Time that can then be put into analysis or research into new media opportunities.
Our R&D team decided to attack this issue head on and focused on creating a tool that could take on the laborious task of selecting the best exact keywords to add, in the fastest way possible.
The tool would need to be able to navigate in real time through the hundreds of thousands of searches posted in your account’s search terms report. Then, it needs to identify the most interesting keywords to add as exact terms, while simultaneously flagging broad terms that should be dropped because they are no longer likely to lead to quality clicks.
As a result, the manager can now concentrate on looking for new targeting opportunities, and hand over the arduous work of prioritizing and learning from current targets to a series of algorithms.
We have affectionately christened this process the blindfold. When you look at the Search Term Report screen shot below, you’ll see where the algorithm gets its name.
1- The “Other Search Terms” line – a well-guarded secret
Direct your attention to the final line of the report. It is often overlooked, but so, so important! The line “Other search terms” contains 87% of impressions and 0% of clicks, so it’s essentially a blindfold hiding some of the most useful information for optimizing your campaign targeting: What are the least interesting terms you are currently covering? Up until now, the only solution was to manually add the strongest exact match keywords, and slowly move away from less desirable impressions.
2- The SKAG technique on steroids
In fact, if you kept only the 7 best keywords from the example above as exact matches in separate groups, your new campaign would have a 31% CTR! An amazing success. This technique is known by the acronym SKAG, or Single Keywords Ad Groups, but unfortunately it suffers from a major limitation. It requires a full-time expert to carry out the task of adding target terms one at a time. Especially for large scale operations.
The blindfold takes a similar approach, but with a slightly leaner process. Essentially, you’re up against the fact that if a term is frequently searched but doesn’t get clicked because it’s not relevant, you will never know. You therefore need to maximize your number of “visible” impressions (outside of the blindfold), and constantly move away from broad targeting that generates a high volume of invisible impressions.
You can start by adding the Keyword column to the Search Term Report. Then, combine the keyword report and the search term report into a single one. To do this, you can use Vlookup in Excel or Merge in R Studio. Above all, keep in mind that several search terms can accumulate in a single target. Doing this work manually can amount to a herculean task, and if you have multiple campaigns and accounts, you shouldn’t hesitate to use Google’s API. Once you’ve associated your data, you’ll be left with a decision-making matrix similar to this:
You are now in a position to make informed optimization decisions, as you no longer need to rely on the instincts of your senior specialist. Instead, you have a work tool. All that remains is to make it accessible to every client and for all levels of experience, from the local pizzeria to your major international account.
3- Use of R statistics language : when data science meets SEM
To do this, we opted for R as our statistical language, using the R Shiny library to support the visual interface and the R AdWords library to communicate with the API.
We created the following interface to:
In short, the Blindfold allows to :
If you want to build this type of tool yourself, take a look at these screenshots and video, as well as the libraries shown earlier. You have everything you need to succeed.
For advertisers who would like to see an example of the Blindfold in order to measure the potential impact it could have on their internal or agency-managed AdWords accounts, our team is available to help you produce an analysis with our tool.
Happy SEM revolution!
For a real client, does this automated method have a measurable impact?
We opted for a test using Bayes factors, using the Causal Impact library to verify our three hypotheses.
Hypothesis 1: Reduction of the volume of impressions
The black line tells us that there was an 83% drop in impressions in the test group. The dotted line tells us that the control group stayed stable during the test period. The probability that the results observed aren’t related by chance is 99.9%, or P<0.001.
This confirms the hypothesis that using exact match keywords greatly reduces the terms covered, and allows you to drastically reduce your volume of impressions.
Hypothesis 2: Growth of the CTR
Here, we see a 32% improvement in CTR and a 99.9% probability of impact, confirming that this method reduces the number of impressions and improves the average CTR of your targeting.
Hypothesis 3: Stable level of clicks
The result shows that the number of clicks varies very slightly.
Be careful. To succeed, we still had to improve our assessment services in order to be able to select which keywords to work on first, with some trial and error. We also realized that certain broad keywords still contained interesting search terms that still weren’t visible in the report, which led us to adjust the algorithm so as not to lose those potential visits.
If you do this work over a long period of time, you’ll notice that when the Search Exact Match Impression Share goes up, the average cost per click goes down. Therefore, greater precision increases your quality and reduces your overall cost without losing out on potential visitors. In the example below, the cumulative savings for the 14 weeks following the initial 7 test weeks was $2,500. In other words, with $3,500 you can get results that would have cost $6,000 at your original cost per click.
The level of effort required? Testing an interface and uploading the modification document. The time you save by doing this can be used, among other things, to focus on optimizing your ad copy, something often overlooked in the management of small accounts.
Search Exact Match Impression Share VS CPC
Investment VS Clicks