Dataset covidgilance signals

<p>Research datasets about top signals for covid 19 (coronavirus) for study into Google Trends (GT) and with SEO metrics</p> <p>&nbsp;</p> <p>Website</p> <p>The study is currently published on <a href="https://covidgilance.org">https://covidgilance.org</a> website (in french)</p> <p>&nbsp;</p> <p>Datasets description</p> <p><code>covid signals -&gt; |selection| -&gt; 4 dataset -&gt; |serp.py| -&gt; 4 serp datasets -&gt; |aggregate_serp.pl| -&gt; 4 aggregated dataset of serp -&gt; |prepare datasets| -&gt; 4 ranked top seo dataset</code></p> <p>&nbsp;</p> <p>Original lists of signals (mainly covid symptoms) - dataset</p> <p><strong>Description:</strong> contain the original relevant list of signals for covid19 (here list of queries where you can see, in GT, a relevant signal during the covid 19 period of time)<br> <strong>Name:</strong> covid_signal_list.tsv</p> <p><strong>List of content:</strong></p> <p><strong>- id:</strong> unique id for the topic<br> <strong>- topic-fr:</strong> name of the topic in French<br> <strong>- topic-en:</strong> name of the topic in English<br> <strong>- topic-id:</strong> GT topic id<br> <strong>- keyword fr:</strong> one or several keywords in French for GT<br> <strong>- keyword en:</strong> one or several keywords in English for GT<br> <strong>- fr-topic-url-12M:</strong> link to 12-months French query topic in GT in France<br> <strong>- en-topic-url-12M:</strong> link to 12-months English query topic in GT in US<br> <strong>- fr-url-12M:</strong> link to 12-months French queries in GT in France<br> <strong>- en-url-12M:</strong> link to 12-months English queries topic in GT in US<br> <strong>- fr-topic-url-5M:</strong> link to 5-months French query topic in GT in France<br> <strong>- en-topic-url-5M:</strong> link to 5-months English query topic in GT in US<br> <strong>- fr-url-5M:</strong> link to 5-months French queries in GT in France<br> <strong>- en-url-5M:</strong> link to 5-months English queries topic in GT in US</p> <p>&nbsp;</p> <p>Tool to get SERP of covid signals - tool</p> <p><strong>Description:</strong> query google with a list of covid signals and obtain a list of serps in csv (tsv in fact) file format<br> <strong>Name:</strong> serper.py</p> <p><code>python serper.py</code></p> <p>&nbsp;</p> <p>SERP files - datasets</p> <p><strong>Description</strong> Serp results for 4 datesets of queries <strong>Names:</strong> simple version of covid signals from google.ch in French: serp_signals_20_ch_fr.csv<br> simple version of covid signals from google.com in English: serp_signals_20_en.csv<br> amplified version of covid signals from google.ch in French: serp_signals_covid_20_ch_fr.csv<br> amplified version of covid signals from google.com in English: serp_signals_covid_20_en.csv</p> <p>amplified version means that for each query we create two queries one with the keywords &quot;covid&quot; and one with &quot;coronavirus&quot;</p> <p>&nbsp;</p> <p>Tool to aggregate SERP results - tool</p> <p><strong>Description:</strong> load csv serp data and aggregate the data to create a new csv file where each line is a website and each column is a query. <strong>Name:</strong> aggregate_serp.pl</p> <p>`perl aggregate_serp.pl&gt; aggregated_signals_20_en.csv</p> <p>&nbsp;</p> <p>datasets of top website from the SERP results - dataset</p> <p><strong>Description</strong> a aggregated version of the SERP where each line is a website and each column a query<br> <strong>Names:</strong><br> aggregated_signals_20_ch_fr.csv<br> aggregated_signals_20_en.csv<br> aggregated_signals_covid_20_ch_fr.csv<br> aggregated_signals_covid_20_en.csv</p> <p><strong>List of content:</strong></p> <p><strong>- domain:</strong> domain name of the website<br> <strong>- signal 1:</strong> Position of the query 1 (signal 1) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- signal ...:</strong> Position of the query (signal) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- signal n:</strong> Position of the query n (signal n) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- total:</strong> average position (total of all position /divided by the number of queries)<br> <strong>- missing:</strong> Total number of missing results in the SERP for this website</p> <p>&nbsp;</p> <p>datasets ranked top seo - dataset</p> <p><strong>Description</strong> a ranked (by weighted average position) version of the aggregated version of the SERP where each line is a website and each column a query. TOP 20 have more information about the type and HONcode validity (from the date of collect: September 2020)</p> <p><strong>Names:</strong><br> ranked_signals_20_ch_fr.csv<br> ranked_signals_20_en.csv<br> ranked_signals_covid_20_ch_fr.csv<br> ranked_signals_covid_20_en.csv</p> <p><strong>List of content:</strong></p> <p><strong>- domain:</strong> domain name of the website<br> <strong>- signal 1:</strong> Position of the query 1 (signal 1) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- signal ...:</strong> Position of the query (signal) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- signal n:</strong> Position of the query n (signal n) in the SERP where 30 indicates arbitrary that this website is not present in the SERP<br> <strong>- avg position:</strong> average position (total of all position /divided by the number of queries)<br> <strong>- nb missing:</strong> Total number of missing results in the SERP for this website<br> <strong>- % presence:</strong> % of presence<br> <strong>- weighted avg postion:</strong> combination of avg position and % of presence for final ranking<br> <strong>- honcode:</strong> status of the Honcode certificate for this website (none/valid/expired)<br> <strong>- type:</strong> type of the website (health, gov, edu or media)</p>