Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
55
datasets available to search
ShareScore release 0.7.1
Dataset results
55 results for “Natural language processing”
Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review
<p>Supplementary material outlining complete list of reviewed papers, chronic diseases and their classifications, algorithms used, publication venues, and excluded papers.</p>
Corpus Nummorum - Natural Language Processing Dataset
<p>This Natural Language Processing (NLP) dataset contains a part of the MySQL <a href="https://www.corpus-nummorum.eu/">Corpus Nummorum (CN)</a> database. It covers Greek and Roman coins from ancient Thrace, Moesia Inferior, Troad and Mysia.</p> <p>The dataset contains 7,900 coin descriptions (or designs) created by the members of the CN project. Most of them are actual coin designs which can be linked through our relational database to the matching CN coins, types and their images. However, some of them (about 450) were only created for the training of the NLP model. </p> <p>There are nine different MySQL tables: </p> <ol> <li>data_coins: contains the data of all coins in the CN database</li> <li>data_coins_images: contains data of all images in the CN database</li> <li>data_coins_imagesets: contains the image pairs for the CN coins</li> <li>data_designs: contains every coin description in German, English and Bulgarian</li> <li>data_types: contains the data of alle coin types the Cn database</li> <li>nlp_hierarchy: contains the classes and subclasses of all entity categories </li> <li>nlp_list_entities: contains the data of all nlp entities in the CN database</li> <li>nlp_relation_extraction_en_v2: contains the annotations for the training of our NLP model</li> <li>nlp_training_designs: contains the coin designs used for training our NLP model</li> </ol> <p>Only tables 8 and 9 are important for NLP training, as they contain the descriptions and the corresponding annotations. The other tables (data_...) make it possible to link the coin descriptions with the various coins and types in the CN database. It is therefore also possible to provide the CN image data sets with the appropriate descriptions (<a href="../records/10033993">CN - Coin Image Dataset</a> and <a href="../records/13748799">CN - Object Detection Coin Dataset</a>). The other NLP tables provide information about the entities and relations in the descriptions and are used to create the RDF data for the <a href="https://nomisma.org/datasets">nomisma.org</a> portal. The tables of the relational CN database can be related via the various ID columns using foreign keys.</p> <p>For easier access without MySQL, we have attached two csv files with the descriptions in English and German and the annotations for the English designs. The annotations can be related to the descriptions via the Design_ID column. </p> <p>During the summer semester 2024, we held the "Data Challenge" event at our Department of Computer Science at the Goethe-University. Our students could choose between the Object Detection dataset and a Natural Language Processing dataset as their challenge. We gave the teams that decided to take part in the NLP challenge this dataset with the task of trying out their own ideas. Here are the results:</p> <ul> <li><a href="https://github.com/jasperforth/DataChallenge_LLM_REPipeline">LLM_RE Pipeline</a></li> <li><a href="https://github.com/Axolord/coin-description-embeddings">Coin description embeddings</a></li> <li><a href="https://github.com/calul0/nlp_coin_app/tree/master">NLP coin app</a></li> </ul> <p>Now we would like to invite you to try out your own ideas and models on our coin data.</p> <p>If you have any questions or suggestions, please, feel free to contact us. </p>
A Large Dataset of Tweets on the 2023 Presidential Elections in Nigeria for Natural Language Processing Tasks
<p>The dataset contains tweets related to the 2023 presidential elections in Nigeria. The data was retrieved from the social media network, Twitter (Now X) between February 4<sup>th</sup>, 2023 and April 4<sup>th</sup>, 2023. The hashtags from the official handles and other popular hashtags endorsed and/or representing the candidates of each party were considered for retrieving election related tweets using an API from Twitter social media platform. Three major political parties in Nigeria were considered and they have been labelled as Party A, Party L and Party P in this dataset. The party or group called "General" contains tweets from the Independent National Electoral Commission (INEC) hashtags such as <em>@inecnigeria</em> and <em>#2023election</em> which is not directly for any political party.</p> <p>The dataset has been pre-processed lightly to make it very useful to researcher for a wide range of natural language processing tasks like sentiment analysis, topic modelling, fake news detection, emotion detection, election stance, etc.</p> <p>Details of the dataset collection such as hashtags, retrieved tweets, duplicates removed, and the remaining unique tweets is presented in Table 1. </p> <p> </p> <p>Table 1: Tweets collection and duplicates removal</p> <table> <tbody> <tr> <td> <p>S/N</p> </td> <td> <p>Party</p> </td> <td> <p>Hash tags</p> </td> <td> <p>Retrieved tweets</p> </td> <td> <p>Duplicates tweets</p> </td> <td> <p>Unique tweets</p> </td> </tr> <tr> <td> <p>1</p> </td> <td> <p> X</p> </td> <td> <p><em>@inecnigeria</em></p> <p><em>#2023election</em></p> </td> <td> <p>64,496</p> </td> <td> <p>47,275</p> </td> <td> <p>17,195</p> </td> </tr> <tr> <td> <p>2</p> </td> <td> <p> A</p> <p> </p> </td> <td> <p><em>#TinubuIsComing</em></p> <p><em>#emilokan </em></p> <p><em>#jagabanarmy</em></p> <p><em>#RenewedHope </em></p> <p><em>#BATKSM2023</em></p> </td> <td> <p>263,870</p> </td> <td> <p>231,036</p> </td> <td> <p>32,832</p> </td> </tr> <tr> <td> <p>3</p> </td> <td> <p> </p> <p> L</p> </td> <td> <p><em>#VoteLP</em></p> <p><em>#NigeriaMustBeBright</em></p> <p><em>#PeterObiForPresident2023</em></p> <p><em>#ObiDatti2023 </em></p> <p><em>#PeterObi</em></p> </td> <td> <p>664,083</p> </td> <td> <p>310,857</p> </td> <td> <p>353,226</p> </td> </tr> <tr> <td> <p>4</p> </td> <td> <p> </p> <p>P</p> </td> <td> <p><em>#NigeriaDecides</em></p> <p><em>#VotePDP</em></p> <p><em>#AtikuOkowa2023</em></p> <p><em>#FinalPushToVictory</em></p> <p><em>#RecoverNigeria</em></p> </td> <td> <p>387,450</p> </td> <td> <p>318,425</p> </td> <td> <p>66,227</p> </td> </tr> <tr> <td> <p> </p> </td> <td> <p> </p> </td> <td> <p> </p> </td> <td> <p>1,379,899</p> </td> <td> <p>907,593</p> </td> <td> <p>468,480</p> </td> </tr> </tbody> </table> <p> </p> <p>To encourage NLP tasks, we uploaded in this Version One the following files:</p> <ol> <li>The combined dataset with pre-processed tweets and their meta data but with removed duplicates are in the file labelled “Combined Dataset Pre-processed without duplicates.csv”</li> <li>General statistics on each corpus is in the file labelled “Dataset Statistics.xlsx” </li> <li>The preprocessed corpus from the general group with the tweet contents only is in file labelled “Preprocessed_Tweet only_GENERAL.xlsx”</li> <li>The preprocessed corpus from Party A with the tweet contents only is in file labelled “Preprocessed_Tweet only_Party A.xlsx”</li> <li>The preprocessed corpus from Party L with the tweet contents only is in file labelled “Preprocessed_Tweet only_Party L.xlsx”</li> <li>The preprocessed corpus from Party P with the tweet contents only is in file labelled “Preprocessed_Tweet only_Party P.xlsx”</li> <li>The top 100 frequent tokens are in the file labelled “Top 100 Tokens and weights.xlsx”</li> <li>The top frequent bigrams and their weights are in the file labelled “Top 100 Bigrams and weights.xlsx”</li> <li>The top frequent trigrams and their weights are in the file labelled “Top 100 Trigrams and weights.xlsx”</li> </ol>
Natural Language Processing for Screening Opioid Misuse
ClinicalTrials.gov study NCT05745480. IPD Sharing: UNDECIDED. Countries: 1. Publications: 5.
Natural Language Processing for Headache Medicine
ClinicalTrials.gov study NCT05377437. IPD Sharing: UNDECIDED. Countries: 1. Publications: 1.
Natural Language Processing and Quality Assessment in Primary Care
ClinicalTrials.gov study NCT01023243. IPD Sharing: Not stated. Countries: 1. Publications: 1.
Figure 1d from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 1d A range of sample specimens that demonstrate the wide taxonomic range of specimens encountered in collections. They also demonstrate the diversity of label types, which include handwritten, typed, and printed labels. Note the presence of various barcodes, rulers, and a colour chart in addition to labels describing the origin of the specimen and its identity. - Fossilised animal skin (Natural History Museum 2009)
Figure 1b from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 1b A range of sample specimens that demonstrate the wide taxonomic range of specimens encountered in collections. They also demonstrate the diversity of label types, which include handwritten, typed, and printed labels. Note the presence of various barcodes, rulers, and a colour chart in addition to labels describing the origin of the specimen and its identity. - Pinned insect specimen (Natural History Museum 2018)
Figure 1c from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 1c A range of sample specimens that demonstrate the wide taxonomic range of specimens encountered in collections. They also demonstrate the diversity of label types, which include handwritten, typed, and printed labels. Note the presence of various barcodes, rulers, and a colour chart in addition to labels describing the origin of the specimen and its identity. - Microscope slide (Natural History Museum 2017)
Figure 1a from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 1a A range of sample specimens that demonstrate the wide taxonomic range of specimens encountered in collections. They also demonstrate the diversity of label types, which include handwritten, typed, and printed labels. Note the presence of various barcodes, rulers, and a colour chart in addition to labels describing the origin of the specimen and its identity. - Herbarium specimen (Natural History Museum 2007a)
Figure 11 from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 11 The distribution of languages across the specimen and herbaria. EN=English, FR=French, LA=Latin, ET=Estonian, DE=German, NL=Dutch, PT=Portuguese, ES=Spanish, SV=Swedish, RU=Russian, FI=Finnish, IT=Italian, ZZ=Unknown. The codes for the contributing herbaria are listed in Table 11 (from Dillen et al. 2019).
Supplementary material 1 from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Appendices
Figure 1e from: Owen D, Livermore L, Groom Q, Hardisty A, Leegwater T, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e55789. https://doi.org/10.3897/rio.6.e55789
Figure 1e A range of sample specimens that demonstrate the wide taxonomic range of specimens encountered in collections. They also demonstrate the diversity of label types, which include handwritten, typed, and printed labels. Note the presence of various barcodes, rulers, and a colour chart in addition to labels describing the origin of the specimen and its identity. - Liquid preserved specimen (Natural History Museum 2010)
Figure 2 from: Owen D, Groom Q, Hardisty A, Leegwater T, Livermore L, van Walsum M, Wijkamp N, Spasić I (2020) Towards a scientific workflow featuring Natural Language Processing for the digitisation of natural history collections. Research Ideas and Outcomes 6: e58030. https://doi.org/10.3897/rio.6.e58030
Figure 2 A possible semi-automatic digitisation workflow to extract data from the labels of collection specimens.
Supplemental package of a study on Automated User Story Validation Using Natural Language Processing
Open the record for dataset details and reuse information.
Datasets for "Leveraging Machine Learning and Natural Language Processing Techniques for Agriculture Experiment Station Project Classification"
Open the record for dataset details and reuse information.
Japhug for Natural Language Processing: a single-speaker audio corpus with transcriptions
<p><em>(français ci-dessous)</em></p> <p>This archive contains a dataset (audio files and transcriptions) of a minority language, Japhug (Glottocode: japh1234; closest iso 639-3 code: jya). The archive contains a subset of the Japhug corpus of the Pangloss Collection: it is a single-speaker corpus, consisting of all the audio resources transcribed, for the main speaker of this corpus (Ms. Tshendzin).<br> The corpus is versioned, so that the experiments carried out on these resources (for linguistic research or for Natural Language Processing) are fully reproducible. All relevant information is contained in YAML files (.yml extension; one in French, one in English).<br> The data sub-folder contains the converted and demultiplexed audio files, as well as the annotations associated with each channel of the audio files.<br> The summary files contain, among other things, the list of graphemes used in the language (complex graphemes are particularly important), as well as information on the various resources (audio and annotations), such as their identifiers (DOIs) and links to the original files.<br> From a computational point of view, the list of DOIs of the audios and annotations described in this YAML file is sufficient to generate this corpus at a given time. A corpus like the present one can be viewed as the version, at a given time, of a set of documents in the Pangloss collection: a corpus as it stands at a precise version.</p> <p>Further information is available from <a href="https://gitlab.com/lacito/outilspangloss">https://gitlab.com/lacito/outilspangloss</a></p> <p>---------------</p> <p>Cette archive contient un jeu de données (audios et transcriptions) d’une langue à tradition orale, le japhug (Glottocode: japh1234; code iso 639-3 le plus proche : jya). L’archive contient un sous-ensemble du corpus japhug de la collection Pangloss : c’est un corpus monolocuteur, constitué de l’intégralité des ressources audio transcrites pour la locutrice principale de ce corpus (Mme Tshendzin).<br> Le corpus est versionné, de sorte que les expériences menées sur ces ressources (pour la linguistique ou pour le Traitement automatique des langues) soient reproductibles de façon exacte (en pensant bien à joindre l’algorithme : paramètres, répartitions des fichiers dans les différents ensembles, etc.). Toutes les informations pertinentes se trouvent dans les fichiers YAML (extension .yml ; un en français, un autre en anglais).<br> Le sous-dossier des données contient d’une part les audios convertis et démultiplexés et d’autre part les annotations associées à chaque canal desdits audios.<br> Les fichiers récapitulatifs contiennent notamment la liste des graphèmes utilisés dans cette langue (les graphèmes complexes sont particulièrement importants), ainsi que des informations sur les différentes ressources (audios et annotations), comme les identifiants (DOI), les liens vers les fichiers originaux, etc.<br> Au plan informatique, la liste des identifiants DOI des audios et annotations décrits dans ce fichier YAML suffit pour générer ce corpus à un instant t. Un corpus comme celui-ci peut être vu comme la version à l’instant t d’un ensemble de documents de la collection Pangloss : un corpus arrêté à une version précise.<br> Pour plus de précisions : <a href="https://gitlab.com/lacito/outilspangloss">https://gitlab.com/lacito/outilspangloss</a></p>
Evaluation of a simple score-based Natural Language Processing (NLP) algorithm: Rating Confusion Matrix
<p>Resulting rating confusion matrix for the experiment "Evaluation of a simple score-based Natural Language Processing (NLP) algorithm".</p>
A Free Verbalization Method of Evaluating Sound Design: The Effectiveness of Artificially Intelligent Natural Language Processing Methods and Tools
<p>"Robot" voice sound files. Seventeen sound files were recorded in four formats; raw human voiceover (VO), and three types of robot voice: vocoded voice 1 (“robo”), vocoded voice 2 with music (“kbd”), and a “beep” voice. Each was recorded as 44.1kHz, 24-bit wav files in a professional recording studio. VO was recorded by professional voice actor DB Cooper, who has been the robot voice for several video games, as well as the voice of the DEE BMW internal car AI voice system. Cooper recorded three versions of the emotes on a Sennheiser MKH-416. Professional sound designer pdx Drescher, an expert in robot<br> and interface sound design, created three sets of robot voices from<br> the original voice files. With guidance from one of the authors,<br> pdx was tasked with trying different approaches to turning the VO<br> samples into three different types of robot voice while attempting<br> to maintain the meaning of the original sounds as described in the<br> list above through preserving the prosody/melodic contour of the<br> original. The first set, robo, used some clips from one of pdx’s prior<br> robot voice projects and integrated them to approximate the emo-<br> tional intention of the VO. Clips were re-pitched, manipulated, and<br> modulated using ProTools plugins. For the kbd takes, VO sounds<br> were played into a Shure SM58 microphone. Vocoder patches mod-<br> ified the signal by voice formants, and the pitch was determined<br> by MIDI notes and pitch-bend controllers. Output of the synthe-<br> sizer was then edited with additional synth patches and effects (EQ,<br> modulation, etc.). We made particular use of a plugin called Envy<br> by Cargo Cult, which takes the volume, pitch, and EQ envelopes of<br> one sound (the original VO) and apply them to another sound. This<br> helped make the synth resemble the prosody of the original sound<br> to some degree. The beep sounds underwent a similar development<br> process as the robo takes, but with interface “bleeps and bloops”<br> derived from various sound effects libraries, including the Star Trek<br> LCARS soundset.</p> <p>The following sounds were recorded: 1. Warning calm (“Uh-oh”) 2. Warning alarm (“ah!”) 3. Wrong/<br> error (“rrrrr”) 4. Correct/good (“yay”) 5. Surprise (neutral) (“Oh!”) 6.<br> Surprise (good) (“Oh!”) 7. Surprise (bad) “(ohhh”) 8. Love/adoration<br> (“awww”) 9. Disgust (“ew”) 10. Contempt (“ech”) 11. Guilt (“hmmm”)<br> 12. Confused (“huh?”) 13. Laugh (“ha ha”) 14. Calculating (“hmmm”)<br> 15. Sigh 16. Giggle 17. Pain (“ow ")</p>
Data from: What is in a general plan? using natural language processing to read 461 California city general plans
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.