Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
35
datasets available to search
ShareScore release 0.9.0
Dataset results
35 results for “news media”
Data set for "Frequency of Prejudice Coverage in News Media Worldwide"
<p>Previous research has identified a post-2010 sharp increase of terms used to denounce prejudice (i.e. racism, sexism, homophobia, Islamophobia, anti-Semitism, etc.) in U.S. and U.K. news media content. Here, we extend previous analysis to an international sample of news media organizations. Thus, we quantify the prevalence of prejudice-denouncing terms and social justice associated terminology (diversity, inclusion, equality, etc.) in over 98 million news and opinion articles across 124 popular news media outlets from 36 countries representing 6 different world regions: English-speaking West, continental Europe, Latin America, sub-Saharan Africa, Persian Gulf region and Asia. We find that the post-2010 increasing prominence in news media of the studied terminology is not circumscribed to the U.S. and the U.K. but rather appears to be a mostly global phenomenon starting in the first half of the 2010s decade in pioneering countries yet largely prevalent around the globe post-2015. However, different world regions’ news media emphasize distinct types of prejudice with varying degrees of intensity. We find no evidence of U.S. news media having been first in the world in increasing the frequency of prejudice coverage in their content. The large degree of temporal synchronicity with which the studied set of terms increased in news media across a vast majority of countries raises important questions about the root causes driving this phenomenon.</p> <p>We provide here a reproducibility data set of counts of target terms and total number of unigrams in each article analyzed plus a Google searchable prefix of the headline of the article for verification of the integrity of the data set and the frequency counts. </p>
Data for manuscript: "Using Word Embeddings to Probe Sentiment Associations of Politically Loaded Terms in News and Opinion Articles from News Media Outlets"
<p>This data set contains material for the purpose of scientific reproducibility of the accompanying manuscript "Using Word Embeddings to Probe Sentiment Associations of Politically Loaded Terms in News and Opinion Articles from News Media Outlets".</p> <p>Note that this data set is distributed with an Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. NonCommercial means you may not use the material for commercial purposes. NoDerivatives means if you remix, transform, or build upon the material, you may not distribute the modified material. Attribution means you must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. See attached license terms for details.</p> <p>The work "Using Word Embeddings to Probe Sentiment Associations of Politically Loaded Terms in News and Opinion Articles from News Media Outlets" describes an analysis of political associations in 27 million diachronic (1975-2019) news and opinion articles from 47 news media outlets popular in the United States. We use embedding models trained on individual outlets content to quantify outlet-specific latent associations between positive/negative sentiment words and terms loaded with political connotations such as those describing political orientation, party affiliation, names of influential politicians and ideologically aligned public figures. </p> <p>News and opinion articles from the outlets listed in Figure 3 are available in the outlet's online domains and/or public cache repositories such as Google cache, The Internet Wayback Machine [31] and Common Crawl [32]. This work has not analyzed video or audio content of news media organizations, except when the outlet explicitly provides a transcript of such content in article form.<br> The temporal coverage of articles from different news outlets is not uniform. For most media organizations, news articles availability in their online domains or Internet cache backups becomes sparse as a function of articles’ age. This is not the case for some news outlets, where availability of news articles goes back to the 1970s. The Supplementary Material (SM) illustrates the time ranges of article data analyzed based on news outlets articles online availability.</p> <p>Textual content included in our analysis is circumscribed to the articles’ headlines and main text and does not include other article elements such as figure captions. Targeted textual content was located in HTML raw data using outlet specific XPath expressions. Tokens were lowercased prior to estimating embedding models. Markup language tags, URLs, nonalphanumeric characters, punctuation, digits, 330 common stop words and multiple spaces were removed prior to estimating word embeddings models.<br> All the analysis scripts and the diachronic word embedding models built from each of the 47 news media outlets analyzed in this work are available in this repository.</p> <p>For the purpose of reproducibility, we also provide in the above repository the articles’ text used to train the news outlets embedding models with the caveat that outlets articles not accessible without a subscription have been excluded. Also, for the included articles, stop words have been removed and the remaining words have been randomly scrambled within a sliding window of size 10 to render the articles incomprehensible to a human reader. These steps have been taken to not infringe articles copyright. These preprocessing steps have only minor impact on Continuous Bag of Words (CBOW) word2vec and the results reported in this work are similar when using the scrambled articles text to train outlet-specific embedding models.</p> <p>We derived outlet-specific word embedding models at every five-year time intervals within the 1975-2019 time range. The gensim [33] implementation of word2vec was used to train the embedding models. The continuous bag of words (CBOW) architecture performed slightly better than the Skip-Gram architecture in commonly used validation metrics so it was used for all subsequent analysis. </p> <p>For training the word embedding models, the following parameters were used: vector dimensions=300, window size=10, negative sampling=10, down sampling frequent words = 0.0001, minimum frequency count of 5 (only terms that appear more than 5 times in the corpus were included into the word embedding model vocabulary), number of training iterations (epochs) through the corpus=5. The exponent used to shape the negative sampling distribution was the default 0.75. </p> <p>Outlet-specific embedding models performance across a range of commonly used semantic, syntactic and analogy tasks was similar to popular pre-trained embedding models trained on corpora such as Twitter or Google books on similarity, association and word analogy tasks, see Supplemeentary Material of the manuscript for detailed validation tests results.</p> <p> </p>
Data for manuscript: "Prevalence of prejudice denoting words in news media discourse: a chronological analysis"
<p>This data set contains frequency counts of target words in 27 million news and opinion articles from 47 popular news media outlets in the United States. The target words are listed in the associated manuscript and are mostly words that denote some type of prejudice. A few additional words not denoting prejudice are also available since they are used in the manuscript for illustration purposes.</p> <p>The textual content of news and opinion articles from the outlets listed in Figure 4 of the main manuscript is available in the outlet's online domains and/or public cache repositories such as Google cache (https://webcache.googleusercontent.com), The Internet Wayback Machine (https://archive.org/web/web.php), and Common Crawl (https://commoncrawl.org). We used derived word frequency counts from these sources. Textual content included in our analysis is circumscribed to articles headlines and main body of text of the articles and does not include other article elements such as figure captions.</p> <p>Targeted textual content was located in HTML raw data using outlet specific xpath expressions. Tokens were lowercased prior to estimating frequency counts. To prevent outlets with sparse text content for a year from distorting aggregate frequency counts, we only include outlet frequency counts from years for which there is at least 1.25 million words of article content from an outlet. This threshold was chosen to maximize inclusion in our analysis of outlets with sparse amounts of articles text per year such as Reason, Alternet or The American Spectator. </p> <p>Yearly frequency usage of a target word in an outlet in any given year was estimated by dividing the total number of occurrences of the target word in all articles of a given year by the number of all words in all articles of that year. This method of estimating frequency accounts for variable volume of total article output over time.</p> <p>The list of compressed files in this data set is listed next:</p> <p>-analysisScripts.rar contains the analysis scripts used in the main manuscript </p> <p>-articlesContainingTargetWords.rar contains counts of target words in outlets articles as well as total counts of words in articles</p> <p>-cableNews.rar contains prevalence of target words in TV cable news. Data is from Stanford Cable TV News Analyzer (https://tvnews.stanford.edu/)</p> <p>-surveyData.rar contains longitudinal survey data used in the manuscript and links to original sources</p> <p>Usage Notes</p> <p>In a small percentage of articles, outlet specific XPath expressions failed to properly capture the content of the article due to the heterogeneity of HTML elements and CSS styling combinations with which articles text content is arranged in outlets online domains. As a result, the total and target word counts metrics for a small subset of articles are not precise. In a random sample of articles and outlets, manual estimation of target words counts overlapped with the automatically derived counts for over 90% of the articles.</p> <p>Most of the incorrect frequency counts were minor deviations from the actual counts such as for instance counting the word "Facebook" in an article footnote encouraging article readers to follow the journalist’s Facebook profile and that the XPath expression mistakenly included as the content of the article main text. Some additional outlet-specific inaccuracies that we could identify occurred in "The Hill" and "Newsmax" news outlets where XPath expressions had some shortfalls at precisely capturing articles’ content. For "The Hill", in years 2007-2009, XPath expressions failed to capture the complete text of the article in about 40% of the articles. This does not necessarily result in incorrect frequency counts for that outlet but in a sample of articles’ words that is about 40% smaller than the total population of articles words for those three years. In the case of "NewsMax", the issue was that for some articles, XPath expressions captured the entire text of the article twice. Notice that this does not result in incorrect frequency counts. If a word appears x times in an article with a total of y words, the same frequency count will still be derived when our scripts count the word 2x times in the version of the article with a total of 2y words. To conclude, in a data analysis of 27 million articles, we cannot manually check the correctness of frequency counts for every single article and hundred percent accuracy at capturing articles’ content is elusive due to the small number of difficult to detect boundary cases such as incorrect HTML markup syntax in online domains. Overall however, we are confident that our frequency metrics are representative of word prevalence in print news media content (see Figure 1 and Figure 2 of main manuscript for supporting evidence).</p>
Data from: Spotted in the news: using media reports to examine leopard distribution, depredation, and management practices outside protected areas in southern India
Open the record for dataset details and reuse information.
Data of A Study on the Presentation of China's COVID-19 News in American Mainstream Media: Party Logic and Mirror Image Effect Hypothesis
Open the record for dataset details and reuse information.
Replication files for Conservative News Media and Criminal Justice: Evidence from Exposure to Fox News Channel by E. Ash and M. Poyker
Open the record for dataset details and reuse information.
Framing Across Borders: News Media Corpus Framing in Mexico and Colombia
<p>The data set contains 140 labeled articles from two relevant news outlets in Colombia and Mexico.</p> <p>The newspapers in the study were chosen for their audience scope, considering they inform on general topics, and have national coverage in print and digital versions. </p> <p>The articles were collected using the search engine on each outlet’s website covering a period of seven semesters: from May 2022 to August 2023. The process produced a unit of analysis of N=140 articles from the two outlets. El Universal 68, El Tiempo 72.</p>
The decreasing/increasing prevalence of the terms "global warming" and "climate change" in news media discourse
<p>I will drop this here as a simple curiosity without much further comment: The decreasing/increasing prevalence of the terms <em>global warming</em> and <em>climate change</em> in news media discourse.</p> <p>https://davidrozado.substack.com/p/gwacc</p>
FactDrill: A Data Repository of Fact-checked Social Media Content to Study Fake News Incidents in India
<p>A dataset containing 22,435 fact-checked social media content to study fake news incidents in India. The dataset comprises news stories from 2013 to the year 2020, covering 13 different languages spoken in the country. There are 14 different attributes present in the dataset.</p>
Understanding Media's Role in Public Perception: Discrepancies in Sentiment During the COVID-19 Pandemic in English and Croatian News Media
Open the record for dataset details and reuse information.
Dataset of user and tweet ids of followers of news media accounts
<p>## Data</p> <p>All files are [xz compressed archives](https://en.wikipedia.org/wiki/XZ_Utils) and<br> can be unpacked with, e.g., `unxz user_ids.dat.xz`. Each file contains one id per<br> line, which are in some cases sorted ascendingly to improve compressibility.</p> <p>The ids of `tweet_ids.dat.xz` and `retweet_ids.dat.xz` correspond to tweet_ids of Twitter,<br> but for convenience we split categorized them into retweets and regular tweets.</p> <p>For each news outlet Twitter account we provide a file with all follower ids which<br> followed them at one point in the end of March 2021. These are suffled and the tweets<br> provided in the files above originate from the first few percent.</p> <p>* `AP.ids.xz` followers of @AP, belonging to the Associated Press news agency<br> * `CNN.ids.xz` followers of @CNN, belonging to the CNN TV news network<br> * `FoxNews.ids.xz` followers of @FoxNews, belonging to the Fox News TV news network<br> * `nytimes.ids.xz` followers of @nytimes, belonging to the New York Times newspaper<br> * `Reuters.ids.xz` followers of @Reuters, belonging to the Reuters news agency<br> * `TIME.ids.xz` followers of @TIME, belonging to the bi-weekly TIME magazine<br> * `washingtonpost.ids.xz` followers of @washingtonpost, belonging to the Washington Post newspaper<br> * `WSJ.ids.xz` followers of @WSJ, belonging to the Wall Street Journal newspaper</p>
Unveiling Global Narratives: A Multilingual Twitter Dataset of News Media on the Russo-Ukrainian Conflict
<p>We present a dataset that collects tweets from news media channels worldwide that pertain to the Russo-Ukrainian war. This dataset spans a period of February 2022-May 2023. The dataset is unique in its global scope, encompassing tweets in various languages and from different parts of the world. Additionally, we extracted information about the stance, sentiment, prominent entities & concepts that occur in tweets to be able to answer questions about the discourse: who says what (prominent entities), who stands (stance) where on what aspect (prominent concepts), how are the aspects portrayed (sentiment). We also downloaded the images attached to the post and classified them to extract image tags for each image. The dataset includes 1,524,826 tweets, out of which 306,295 tweets have images, for 60 languages.<br><br>The source code for the collection and processing of tweets can be found on here: <a href="https://github.com/sherzod-hakimov/ru-ua-news-discourse-twitter"><em>https://github.com/sherzod-hakimov/ru-ua-news-discourse-twitter</em></a></p> <p>Each entry in the dataset is a single JSON line and has the following entries:</p> <pre><code>{ 'tweet_id': 'lang': 'stanza_output': 'stanza_named_entities': 'sentiment': 'stance': 'channel': 'country': 'verified':<br>'image_tags': }</code></pre> <pre> </pre> <p><em><strong>If you need access to the full text of the dataset, please</strong> <strong>contact us via an email: <a href="mailto:sherzodhakimov@gmail.com">sherzodhakimov (at sign) gmail.com</a></strong></em><br><br>If you find the resources useful, please cite us:<br><br>```</p> <p>@inproceedings{hakimov2023unveiling,<br> title={Unveiling Global Narratives: A Multilingual Twitter Dataset of News Media on the Russo-Ukrainian Conflict}, <br> author={Sherzod Hakimov and Gullal S. Cheema},<br> booktitle={Proceedings of the 2024 {ACM} International Conference on Multimedia Retrieval, {ICMR} 2024},<br> year={2024}<br>}<br>```</p>
THE USE OF SOCIO-POLITICAL TERMS IN THE NEWS GENRE OF MEDIA TEXT
Open the record for dataset details and reuse information.
Rewritten Media Bias News Headlines
<p>This dataset was used in the paper "Rewriting Bias: Mitigating Media Bias in News Recommender Systems through Automated Rewriting" by Qin Ruan, Jin Xu, Susan Leavy, Brian Mac Namee, and Ruihai Dong, presented at the 32nd ACM Conference on User Modeling, Adaptation and Personalization (UMAP'24), July 1-4, 2024, in Cagliari, Italy. (ACM, New York, NY, USA, 10.1145/3627043.3659541).</p> <p>The dataset includes seven distinct rewritten versions of news headlines generated using the following methods proposed in the paper: RADJ, RADV, RNOUN, RVERN, RALL, RG3.5, and RG4.0.</p> <p>The rewriting approaches are categorised into two main categories: <strong>Word Replacement </strong>approaches and <strong>Large Language Models </strong>approaches.</p> <p>1.Sentence Rewriting Using Word Replacement :</p> <ul> <li><strong>Replace Adjectives (RADJ)</strong>: This approach replaces adjectives identified as contributing to bias with neutral or opposite ones.</li> <li><strong>Replace Adverbs (RADV)</strong>: This approach replaces adverbs identified as contributing to bias with neutral or opposite ones.</li> <li><strong>Replace Nouns (RNOUN)</strong>: This approach replaces nouns identified as contributing to bias with neutral or more general ones.</li> <li><strong>Replace Verbs (RVERB)</strong>: This approach replaces verbs identified as contributing to bias with neutral or more factual ones.</li> <li><strong>Replace All (RALL)</strong>: This is a combination of all the above approaches.</li> </ul> <p>2.Sentence Rewriting Using Large language Models:</p> <ul> <li><strong>Sentence Rewriting using GPT-3.5 (RG3.5)</strong>: This method rephrases sentences using GPT-3.5 based on a specific prompt.</li> <li><strong>Sentence Rewriting using GPT-4.0 (RG4.0)</strong>: This method rephrases sentences using GPT-4.0 based on a specific prompt.</li> </ul> <p>The dataset consists of original news headlines and their corrsponding rewritten versions prodcued by each of the seven approaches mentioned above. Each rewritten version aims to reduce bias while maintaining the original meaning.</p> <p> </p> <p><strong>Dataset Structure:</strong></p> <p>The dataset consists of a single file with the following columns:</p> <ul> <li>NewsId: A unique identifier for each news article.</li> <li>Original: The original headline of the news article.</li> <li>RG3.5: The headline rewritten using the <a href="https://platform.openai.com/docs/models/gpt-3-5">GPT-3.5</a> model.</li> <li>RG4.0: The headline rewritten using the <a href="https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo">GPT-4.0</a> model.</li> <li>RADJ: The headline with adjectives replaced to reduce bias.</li> <li>RADV: The headline with adverbs replaced to reduce bias.</li> <li>RVERB: The headline with verbs replaced to reduce bias.</li> <li>RNOUN: The headline with nouns replaced to reduce bias.</li> <li>RALL: The headline with adjectives, adverbs, verbs. noun replaced to reduce bias.</li> </ul> <p><strong>Usage:</strong></p> <p>This dataset can be used to study the impact of different sentence rewriting approaches on reudicng bias in enws healines. It is also suitable for researchers interested in natural language processing, media bias reduction, and the application of large language models in generative AI.</p> <p>For more information, please visit <a href="https://github.com/ruanqin0706/MediaBiasinNewsRec">https://github.com/ruanqin0706/MediaBiasinNewsRec</a></p> <p><strong>Contact Information: </strong>qin.ruan@ucdconnect.ie</p>
Hungarian News Media Corpus
<p><strong>Content</strong></p> <p>HTML files scraped from selected online news outlets in Hungary. </p> <ol> <li>24.hu</li> <li>444.hu</li> <li>atlatszo.hu</li> <li>hvg.hu</li> <li>index.hu</li> <li>magyarhirlap.hu</li> <li>magyarnemzet.hu</li> <li>nol.hu</li> <li>nepszava.hu</li> <li>origo.hu</li> </ol> <p><strong>Access</strong></p> <p>The data can only be used for scientific research (see below). To access the data, fill in a data request form. There is no fee for the data access. Once your project is approved, you will receive a temporary download link.</p> <p><strong>License</strong></p> <p>Reproduction is permitted by paragraph 35/A of the <a href="https://net.jogtar.hu/jogszabaly?docid=99900076.tv">Copyright Act of Hungary</a>, which allows storage of copyrighted works for text analysis in the pursuit of scientific research. You are not allowed to make the data public or share with anyone else.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.