Use data to make SEO and marketing decisions
Today, the trend is more and more to automate tasks to abandon repetitive work and focus on analysis. It is an evolution of the market and of our daily occupations. However, it is sometimes complicated to navigate in the jungle of data, scripts, languages, formulas and others.
At Adviso, we believe that data is essential in decision-making aimed at prioritizing the sustainability of online activities and how to reach the right targets at the right time. The good news? Everything we need to facilitate these decisions is accessible on the web.
This article follows another recently published text, The Guide to Data Analysis in Times of Crisis . For crisis-specific SEO information, you can skip the prerequisites section by clicking here.
In order to better understand the scripts and methods presented, it is necessary to have some knowledge and some understanding of programming languages and spreadsheet formulas. This knowledge will help to better understand the article.
To redo the scripts and examples presented in the article, you will need the following tools:
- Jupyter Notebook
- Google Colab
Note : The tools presented above are the most popular and easily accessible on the market, but it is quite possible to use other similar tools.
Jupyter Notebook is available on Jupyter and allows you to launch and use Python or R code visually. There are two methods of use here, but we recommend that you go directly through the Anaconda software which integrates the Jupyter Notebook module.
The main advantage of going through Anaconda and Jupyter Notebook is that you can manage working environments for, for example, different versions of Python.
To use Jupyter Notebook, you will need to install software locally, unlike Google Colab. This can be an advantage for mobility without Internet, but will require a good machine for larger calculations.
Google Colab is Google's online data science platform. It integrates into the Google Drive suite and allows you to work on Google's server resources. As for how it works, it is very similar to Jupyter Notebook.
You will only need an internet connection and a Google account to access the interface. All the computing power is done directly on Google's servers.
COLLECT AND ANALYZE DATA FROM GOOGLE TRENDS
By adapting a script available on Google Trends , we are able to " replace " and complement the keyword generation tools.
To do this, we will use the Jupyter Notebook tool and the Python programming language and we will use the pytrends (the equivalent of a Google Trends API) and panda (for data manipulation via Python) libraries.
>> If you want to see the result, click here .
Below, let's review the steps for getting started and using the working environment (installation, script and renders).
1. SETUP AND INSTALLATION
First, the installation of pytrends by the command below:
pip install pytrends
Then installing panda with the same command specifying the panda library :
pip install panda
Then, to load the libraries, copy/paste the code below and run it via Jupyter Notebook or Google Colab:
import pandas as pd
from pytrends.request import TrendReq
pytrend = TrendReq()
We will start using this script via a first function allowing to have keyword suggestions based on Google Trends. It is important to know that this data must be compared to other tools (Google Keyword Planner, Keywordtool.io, Ahrefs or SEMRush for example) if you want to obtain search volumes representing the market you are targeting.
2. KEYWORD SUGGESTION
keywords = pytrend.suggestions(keyword='mot clé')
The keywords variable is created by the pytrend.suggestions function
You just have to insert your keyword in the script and launch everything. You can find examples later in the article.
df_kw = pd.DataFrame(keywords)
We create the framework for the df_kw dataset from the created variable. You can name the dataset whatever you want.
Since there is often “ noise ” in the result, we will remove one of the columns (the 'mid' column) using the drop module .
After analyzing the possibilities of keyword suggestions, it may be interesting to seek a point of view on the trends of the day. This is what pytrends allows, and which we will cover in the next paragraph.
3. DAILY RESEARCH
df_popsearch = pytrend.trending_searches(pn='pays')
The trending_searches module of pytrend allows to create the dataset framework called df_popsearch ( you can name the dataset whatever you want). Searches can be made by country via the pn parameter . The country here is written in letters and not in ISO code .
In order to validate if the result is suitable, we will only display the beginning via the head() module
4. FIND THE "HOT" TOPICS OF THE MOMENT
Daily searches are good, but Google Trends also offers a hot_topic module that allows us to see what is currently the most searched for by Internet users (overall, in a theme, according to a country, etc.)
pytrend.build_payload(kw_list=['mot clé 1’, ‘mot clé 2'])
You can search by keyword or group of keywords for related hot topics .
related_queries = pytrend.related_queries()
We embed the pytrend.related_queries() function in a list named related_queries. You can name this list whatever you want.
Once configured, you must run the line above to get the return of the values. You will have a distinction between top searches ( top ) and rising searches ( rising ).
It is also possible for you to find out, for a chosen keyword, the top or rising information :
Related_queries['mot clé 1']['top']
5. TRENDING TOPICS LIVE FROM GOOGLE TRENDS
Google Trends is known for its Trending Topics “ option ” ; it is even the most used function of the website. In this case, we can explore it using pytrends .
df_todaytt = pytrend.today_searches(pn='CA')
This piece of code allows us to know the Trending Topics of the day. You can manage the geographical part via the variable pn= (here CA for Canada ). We save everything in a dataset named df_todaytt.
6. SCRIPT EXAMPLES AND RETURNS
After all these code steps and possible uses of pytrends , you are surely eager to know the rendering. You are in the right place.
- Keyword suggestions – Here is an example taken with the keyword “ SEO ” :
It is of course necessary to pay attention to the noise of these requests ( Seoul or Securities and Exchange Organization of Iran), because Google Trends remains less qualitative in its proposals (the objective for us being to have ideas of the market).
- Search Trends – Example from Canada:
According to Google Trends, search trends in Canada and at the beginning of April 2020 are as shown above.
Note : If you launch the script on your side, you may have different returns since search trends are linked to seasonality and the targeted country.
- Keywords related to SEO terms – Example taken with queries including the keyword “ SEO ” :
There is still a bit of noise on this request, and it is also the strength of the example which allows to bring a human weighting behind the automation of tasks by scripts.
To continue the analysis, you can export the data to a csv file, for example:
dataframe.to_csv(‘./chemin/d-export/document.csv’, index = False, header = True)
Here we use .to_csv and take the example of a dataset named dataframe.
The parameters index = False and header = True allow respectively to remove the first column with the digits from 0 to N and to keep the headers of the table. Note that these parameters are case-sensitive, so specify False and True with a capital letter at the beginning of the word.
INTEGRATING THESE TOOLS IN TIMES OF COVID-19
This technique can be used at any time to complement and reinforce your digital marketing strategy. We can also see a significant added value in this period of pandemic and better analyze the impact of COVID-19 on the searches of Internet users.
These are keywords, in this case related to the coronavirus, placed in order of importance (100 being the most important and 0 the least important). The terms with the highest search volume are separated from the terms with the highest increase in search volume in a given time.
These are keywords categorized by topic, in this case against “ coronavirus ” .
We can do the same to know the topics similar to e-commerce or delivery.
The global trend of the moment is certainly the search for recipes to make your own bread at home.
When we search for topics related to " baking bread " we can detect expressions to enrich the lexical field of the pages on a cooking site or on an online sales site. This is practical for the development of a page or a group of “pillar page ” pages , or for a Youtube channel when uploading a video.
If we proceed in the same way with the cake recipes, interesting information emerges:
GOOGLE TRENDS: A TOOL WITH ITS LIMITS
Google Trends is a good tool provided by Google for free, but like all tools, it has its weaknesses: limited to keyword comparison, maximum comparison of 5 expressions, linguistic limits with Quebec French for searches in Canada, etc.
Another key point to consider is that Google Trends only offers comparisons of keywords with a monthly search volume greater than 100 . This may be more complex to interpret and that is why it is recommended to use an external tool (Google Search Console, Ahref, SEM Rush, etc.) to access more precise search volumes. For instance :
In addition, do not forget to contextualize these keywords in your overall positioning strategy, your brand or your website. A keyword may experience an increase in search volume, but may not necessarily be of interest to your brand positioning. For example, in times of confinement, don't think covid , think tips, recipes, delivery, help , etc.
PYTRENDS: THE SAME LIMITS AS GOOGLE TRENDS
Overall, we find the same limits in pytrends as in Google Trends (5 keywords, comparison…), with some even more marked limits (geographic targeting).
The advantage of this API is that it automates some or all of your research so you can make marketing decisions more efficiently. By automating, you will be able to connect several tools and several APIs in order to be faster in extracting data and making decisions on verified data.