Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
219
datasets available to search
ShareScore release 0.9.0
Dataset results
219 results for “wikipedia”
Dataset for "Is Wikipedia Politically Biased?"
<p><span><span>·<span> </span></span></span><span>This work aims to determine whether there is evidence of political bias in English Wikipedia articles.</span></p> <p><span><span>·<span> </span></span></span><span>Wikipedia is one of the most visited domains on the Web, attracting hundreds of millions of unique users per month. Wikipedia content is also routinely used for training Large Language Models (LLMs), which are the core engines driving cutting edge AI systems.</span></p> <p><span><span>·<span> </span></span></span><span>To study political bias in Wikipedia content, we analyze the sentiment (<em>positive</em>, <em>neutral</em> or <em>negative</em>) with which a set of target terms (N=1,628) with political connotations (i.e. names of recent U.S. presidents, U.S. congressmembers, U.S. Supreme Court Justices, or Prime Ministers of Western countries) are used in Wikipedia articles. </span></p> <p><span><span>·<span> </span></span></span><span>We do not cherry pick the set of terms to be included in the analysis but instead use publicly available pre-existing lists of terms from Wikipedia and other sources.</span></p> <p><span><span>·<span> </span></span></span><span>We find a mild to moderate tendency in Wikipedia articles to associate public figures politically aligned right-of-center with more negative sentiment than left-of-center public figures.</span></p> <p><span><span>·<span> </span></span></span><span>These favorable associations for left-leaning public figures are apparent for names of recent U.S. Presidents, U.S. Supreme Court Justices, U.S. Senators, U.S. House of Representatives Congressmembers, U.S. State Governors, Western countries’ Prime Ministers, and prominent U.S. based journalists and media organizations.</span></p> <p><span><span>·<span> </span></span></span><span>Despite being common, these politically asymmetrical sentiment associations are not ubiquitous. We find no evidence of them in the sentiment with which names of U.K. MPs and U.S. based think tanks are used in Wikipedia articles.</span></p> <p><span><span>·<span> </span></span></span><span>We also find larger associations of negative emotions (i.e. <em>anger</em> and <em>disgust</em>) with right-leaning public figures and positive emotion (i.e. <em>joy</em>) with left-leaning public figures.</span></p> <p><span><span>·<span> </span></span></span><span>The trends just described constitute suggestive evidence of political bias embedded in Wikipedia articles.</span></p> <p><span><span>·<span> </span></span></span><span>We also find some of the aforementioned sentiment political associations embedded in Wikipedia articles popping up in OpenAI’s language models. This is suggestive of the potential for biases in Wikipedia content percolating into widely used AI systems.</span></p> <p><span><span>·<span> </span></span></span><span>Wikipedia’s <em>neutral point of view policy</em> (NPOV) aims for articles in Wikipedia to be written in an impartial and unbiased tone. Our results suggest that Wikipedia’s <em>neutral point of view policy</em> is not achieving its stated goal of political viewpoint neutrality.</span></p> <p><span><span>·<span> </span></span></span><span>This report highlights areas where Wikipedia can improve in how it presents political information. Nonetheless, we want to acknowledge Wikipedia’s significant and valuable role as a public resource. We hope this work inspires efforts to uphold and strengthen Wikipedia’s principles of neutrality and impartiality.</span></p> <p>The set of 1,653 target terms used in our analysis, the sample of Wikipedia paragraphs where they occur (as of 2022) and their sentiment and emotion annotations are provided in the files:</p> <p>- WikipediaParagraphsWithTargetNGramsAndSentiment.csv</p> <p>- WikipediaParagraphsWithTargetNGramsAndEmotion.csv</p>
Extracted microbial terms from Wikipedia - Marine Microbiology related pages (unfiltered)
<p>Marine microorganism related terms were extracted from various Wikipedia pages to scan the ODIS graph using the Wikipedia API library. The pages include topics such as "Marine microorganisms", "Marine microbiome", "Marine viruses", "Marine bacteria", "Bacterioplankton", "Bacterial motility", "Marine prokaryotes", "Marine archaea", "Marine protists", "Marine fungi", "Mycoplankton", "Marine microanimals", "Ichthyoplankton", "Marine primary production", "Algae", "Marine microplankton", "Marine microbenthos", "Sea ice microbial communities", "Hydrothermal vent microbial communities", "deep biosphere", and "microbial dark matter". </p> <p> </p> <p>We used the english version of the spaCy library, a language processing (NLP) tool, to process text content extracted from Wikipedia and extract meaningful terms. We passed the combined text content to the spaCy model to generate a Doc object, which contains words and their associated linguistic information. After extracting these words from the Doc object, spaCy keeps track of their frequencies using a dictionary and filters out words that are not purely alphabetic, common stop words, and those shorter than four characters. Then, these words are each converted to lowercase and sorted based on their frequency.</p>
Extracted microbial terms from Wikipedia - Marine Microbiology related pages (refined)
<p>From the dataset titled "Extracted microbial terms from Wikipedia - Marine Microbiology related pages (unfiltered)" available on Zenodo https://doi.org/10.5281/zenodo.12648788, we reviewed the full dataset and narrowed it down to a subset of 100 terms directly related to microbiology. We excluded terms such as "marine", "organisms", "ocean", "life" as they are not strictly related to microbiology. Additionally, terms like "algae" and "plants" were removed since they are not microbial entities. We also avoided including verbs such as "found", "classify", and "form" because they may be related to sentences or methods in microbiology, but not strictly to microbial entities.</p>
A Wikipedia dataset of 5 categories
<p>A subset of articles extracted from the French Wikipedia XML dump. Data published here include 5 different categories : Economy (Economie), History (Histoire), Informatics (Informatique), Health (Medecine) and Law (Droit). The Wikipedia dump was downloaded on November 8, 2016 from https://dumps.wikimedia.org/. Each article is a xml file extracted from the dump and save as UTF8 plain text. The characteristics of dataset is :</p> <ul> <li>Economy : 44'876 articles</li> <li>History : 92'041 articles</li> <li>Informatics : 25'408 articles</li> <li>Health : 22'143 articles</li> <li>Law : 9'964 articles</li> </ul>
Interactions of pharmaceutical companies with world countries, cancers and rare diseases from Wikipedia network analysis
<p>Using the English Wikipedia network of more than 5 million articles we analyze interactions and interlinks between the 34 largest pharmaceutical companies, 195 world countries, 47 rare renal diseases and 37 types of cancer. The recently developed algorithm using a reduced Google matrix (REGOMAX) allows us to take account both of direct Markov transitions between these articles and also of indirect transitions generated by the pathways between them<br> via the global Wikipedia network. This approach therefore provides a compact description of interactions between these articles that allows us to determine the friendship networks between them, as well as the PageRank sensitivity<br> of countries to pharmaceutical companies and rare renal diseases. We also show that the top pharmaceutical companies in terms of their Wikipedia PageRankvare not those with the highest market capitalization.</p>
Wikipedia: wikipedia-eo (Esperanto)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p><p></p>https://en.wikipedia.org/wiki/Esperanto
Wikipedia: wikipedia-vi (Vietnamese)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>
Wikipedia: wikipedia-bg (Bulgarian)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>
OA-Concepts and Wikipedia-Links for "The different AI of Science and Wikipedia"
<p>The files contain the data for the VosViewer analyses in "The Different Artificial Intelligences of Science and Wikipedia" (Korte et al. 2024).</p> <p>OpenAlex:</p> <p>As described in the paper, all works from OpenAlex from 2001 and 2022 with the concept "Artificial Intelligence" and a concept score > 0.3 were downloaded. In August 23 using the OpenAlex API. The files contain for each work all concepts with a concept score > 0.3 in one line each separeted by dots. This allows co-occurrence analyses of concepts in VosViewer. For the analyses in the paper whitespaces and "(" in the concepts were removed.</p> <p>First line for 2001: <br>Random forest. Mathematics. AdaBoost. Statistics. Tree (set theory). Generalization. Support vector machine. Generalization error. Measure (data warehouse). Artificial intelligence. Pattern recognition (psychology). </p> <p>Wikipedia:</p> <p>As described in the paper, all Wikipedia pages of the category "Artificial Intelligence" and of all direct sibling categories were downloaded. In August 23 with the Wikipedia Periodic Revisions tool of El Baff and Hecking (https://github.com/DLR-SC/wikipedia-periodic-revisions): . The files contain for each page all hyperlinks to other Wikipedia pages for the dates of 12.31.2005 and 12.31.2021 in one line each separeted by dots. This allows co-occurrence analyses of links in Wikipedia.</p> <p>First line of 2006: <br>vehicleregistrationplate. corporation. googlesearch. googleplatform. googol. google(disambiguation). menlopark,california. erice.schmidt. sergeybrin. lawrencee.page. georgereyes. internet. [...]</p>
Wikipedia: wikipedia-no (Norwegian)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p><p></p>https://no.wikipedia.org
Wikipedia: Raw English Wikipedia - initial step
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>For connector purposes. Required.
Wikipedia: wikipedia-cs (Czech)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>
Wikipedia: wikipedia-ko (Korean)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>
Wikipedia: wikipedia-ru (Russian)
Wikipedia is a multilingual, web-based, free-content encyclopedia project supported by the Wikimedia Foundation and based on a model of openly editable content. EOL harvests articles from wikipedia that are indexed as species or higher taxa.<p></p>
Wikipedia Compressed SQLite database
<p>Starting point:</p> <ul> <li><a href="http://static.wiki/">static.wiki</a></li> <li><a href="https://news.ycombinator.com/item?id=28012829">HN post</a></li> </ul> <p>We downloaded the 40.3 GiB SQLite database of English Wikipedia and created a compressed version of it with <a href="https://github.com/mlin/sqlite_zstd_vfs/">sqlite_zstd_vfs</a>, our read/write <a href="https://facebook.github.io/zstd/">Zstandard</a> compression layer for SQLite3. The compressed version is 10.4 GiB (26%) and remains randomly accessible over the web in the spirit of the original (although we don't yet have a WebAssembly build of the compression layer).</p> <p>sqlite_zstd_vfs is a building block of our <a href="https://mlin.github.io/GenomicSQLite/">Genomics Extension for SQLite,</a> which bundles it along with other domain-specific features. This demonstrates a general-purpose use case of the compression layer intended for large-scale genomics data storage.</p>
Japanese Wikipedia dataset
<p>This dataset is for the definition modeling task which is a task to generate definitions for phrases.</p> <p>Each entry in .txt contains a phrase to be defined and its definition.</p> <p>人工衛星%wiki.0 pos wikidata 地球を焦点の1つとする楕円軌道を周回する人工の天体 [] []</p> <p>Each entry in .eg contains a context containing the phrase to be defined (marked as <TRG>). <br> 人工衛星%wiki.0 気象衛星とは、気象観測を行う<TRG>である</p>
Wikipedia Edit Event Data 2018 (WikiEvent.2018)
<p>The "Wikipedia Edit Event Data 2018 (WikiEvent.2018)" gives the time, user name, and article title of every edit that any registered and logged-in Wikipedia user performed on any article in the English-language edition of Wikipedia from January 15th, 2001 (the launch of Wikipedia) to January 1st, 2018. </p> <p>The edit event data has been extracted from the file 'enwiki-20180101-stub-meta-history.xml.gz'; which was at that time linked from 'https://dumps.wikimedia.org/enwiki/20180101/'. These files get deleted some months after data collection - however the information is still available in any file 'enwiki-<date>-stub-meta-history.xml.gz' where <date> is 20180101 or later. These data are provided by the Wikimedia Foundation licensed under the GNU Free Documentation License (GFDL) and the Creative Commons Attribution-Share-Alike 3.0 License.</p> <p>The Wikipedia Edit Event Data 2018 comprises one file ('WikiEvent.2018.csv') giving a table with 3 columns and more than 360 million rows in CSV format. Cell delimiter is semicolon (';') and strings are quoted by double-quotes ('"'). The table has a header given in the first row and the three columns are labeled 'time', 'user', and 'article' respectively. The uncompressed size of the file is about 17.1 GB.</p> <p>Time is given by integers representing milliseconds from January 1st, 1970 at 0:00 to the time of the edit. Precision of edit times, however, is given by the second so that the last three digits of every time value are equal to '000'. The edit events are given in ascending order with respect to time (older edits before younger edits). An article is a page in Namespace 0 (the 'main namespace' or 'article namespace') that is not a redirect. An edit (that is, a new revision of an article) is considered to be performed by a registered and logged-in user if the 'contributor' is given in an XML element <username>...</username> in the referenced stub-file. (It is considered to be an anonymous edit if the 'contributor' is given in an XML element <ip>...</ip>. Anonymous edits are not recorded in the table.) Triples (time,user,article) are unique by construction: if the same user edits the same article in the same second more than once, only one of these edits appears in the table.</p> <p>The WikiEvent.2018 data was originally used in: Lerner and Lomi (2020). <strong>Reliability of relational event model estimates under sampling: how to fit a relational event model to 360 million dyadic events.</strong> <em>Network Science</em>, 8(1):97-135. (DOI: <a href="https://doi.org/10.1017/nws.2019.57">https://doi.org/10.1017/nws.2019.57</a>)</p> <p>How to analyze the WikiEvent Data with relational event models is explained in the eventnet tutorial at: <a href="https://github.com/juergenlerner/eventnet/wiki/Large-event-networks-(tutorial)">https://github.com/juergenlerner/eventnet/wiki/Large-event-networks-(tutorial)</a>.</p>
Long document similarity dataset, Wikipedia excerptions for movies collections
<p>Movies-related articles extracted from Wikipedia.</p> <p>For all articles, the figures and tables have been filtered out, as well as the categories and "see also" sections.</p> <p>The article structure, and particularly the sub-titles and paragraphs are kept in these datasets</p> <p> </p> <p><strong>Movies</strong></p> <p>The Wikipedia Movies dataset consists of 100,371 articles describing various movies. Each article may consist of text passages describing the plot, cast, production, reception, soundtrack, and more.</p>
Wikipedia entity dataset
<p>Dataset used in several JsonPath engines.</p>
Poor attention: The wealth and regional gaps in event attention and coverage on Wikipedia
<p>Repository for "Poor attention: The wealth and regional gaps in event attention and coverage on Wikipedia" (currently under review).</p> <p>This repository contains the preprocessed data files as well as code for the main experiments conducted during the creation of the paper. See the GitHub repository for more information: https://github.com/ruptho/wiki-event-bias</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.