Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
24
datasets available to search
ShareScore release 0.7.1
Dataset results
24 results for “NER”
PPORTAL_ner: An Annotated Corpus of Portuguese Literary Entities
<h2><a href="https://marianaossilva.github.io/pportal_ner/" target="_blank" rel="noopener">PPORTAL_ner</a></h2> <h3>An Annotated Dataset of Portuguese Literary Entities</h3> <p>The corpus is tailored to Brazilian and Portuguese literary texts, containing annotations for five entity categories, including PER, LOC, GPE, ORG, and DATE. Within a diverse collection of 25 literary works, it offers a total of 125,059 tokens and 5,266 annotated entities. This dataset contributes to the development of potentially more accurate and context-aware NER models, as well as to encourage further exploration within Portuguese literature.<br><br></p> <h2>Corpus Statistics</h2> <p>Our corpus is sourced from <a href="https://doi.org/10.5281/zenodo.5178063">PPORTAL</a>, an extensive repository of metadata containing over 80,000 public domain literary works in the Portuguese language, predominantly derived from Brazil and Portugal. <a href="https://doi.org/10.5281/zenodo.5178063">PPORTAL</a> aggregates data from three digital libraries: <a href="https://www.dominiopublico.gov.br/">Domínio Público</a>, <a href="https://projectoadamastor.org/">Projecto Adamastor</a>, and <a href="https://www.literaturabrasileira.ufsc.br/">Biblioteca Digital de Literatura dos Países Lusófonos (BLPL)</a>.</p> <p>To simplify referencing, this new dataset is called PPORTAL_ner. PPORTAL_ner selection process contains a diverse range of 25 individual literary works, spanning different authors and literary styles. All of these texts were published prior to 1953, adhering to the current criteria for public domain status in Brazil, with the majority falling within the timeframe spanning from 1554 to 1938.</p>
data-augmentation-ner-results
<p>Model evaluation results produced in the context of evaluating data augmentation for Named Entity Recognition over the German legal domain.</p> <p>Detailed information can be found on the <a href="https://github.com/fusion-jena/data-augmentation-ner-legal">Github page</a>.</p>
data-augmentation-ner-datasets
<p>Dataset preparation and augmentation results produced in the context of evaluating data augmentation for Named Entity Recognition over the German legal domain.</p> <p>Detailed information can be found on the <a href="https://github.com/fusion-jena/data-augmentation-ner-legal">Github page</a>.</p> <p>This dataset is based on the <a href="http://github.com/elenanereiss/Legal-Entity-Recognition">German LER dataset</a> that is freely available under the <a href="https://creativecommons.org/licenses/by/4.0/deed.en">CC-BY 4.0 license</a></p>
A Dataset of French Trade Directories from the 19th Century for Nested NER task
<p>This dataset is composed of pages and entries extracted from French directories published between 1798 and 1861.</p> <p>The purpose of this dataset is to evaluate the performance of Nested Named Entity Recognition approaches on 19th century French documents, regarding both clean and noisy texts (due to the OCR engine).</p> <p><strong>Source dataset</strong></p> <p>This dataset has been built from this source dataset :</p> <pre><code class="language-markdown">N. Abadie, S. Baciocchi, E. Carlinet, J. Chazalon, P. Cristofoli, B. Duménieu and J. Perret, A Dataset of French Trade Directories from the 19th Century (FTD), version 1.0.0, May 2022, online at https://doi.org/10.5281/zenodo.6394464.</code></pre> <p><strong>Our experiments // Paper</strong></p> <p>Details about our experiments on nested NER approaches are given in our paper (<a href="https://hal.science/hal-03994759v2">the pre-print version is available here</a>).</p> <pre><code class="language-markdown">Tual, S., Abadie, N., Chazalon, J., Duménieu, B., & Carlinet, E. (2023). A Benchmark of Nested NER Approaches in Historical Structured Documents. Proceedings of the 17th International Conference on Document Analysis and Recognition, San José, California, USA. 2023. Springer. https://hal.science/hal-03994759v2</code></pre> <p>Our code is available on <a href="https://github.com/soduco/paper-nestedner-icdar23-code">Git-Hub</a>.</p> <p><strong>Dataset overview</strong></p> <p>The following list describes the <strong>keys of the .JSON</strong> file which contain the complete materials of our experiments.</p> <p>- id : Entry unique ID in a given page</p> <p>- box : Bounding box of the entry in the scanned directory page</p> <p>- book : Source directory of the entry (*see more information bellow*)</p> <p>- page : Page ID in a given directory</p> <p>- valid_box : Is the bbox of the entry valid ? (*all bbox are valid here*)</p> <p>- text_ocr_ref` : OCR extracted and manually corrected text of the entry</p> <p>- nested_ner_xml_ref : <em> text_ocr_ref</em> with nested ner entities</p> <p>- text_ocr_pero : OCR extracted text of the entry with PERO-OCR engine (best engine according to Abadie et al. experiment)</p> <p>- has_valid_ner_xml_pero : Is entities mapping between nested-ner entities annotated by hand on the ref text and pero ocr text correct ? (in our experiments, we only use entries with True value)</p> <p>- nested_ner_xml_pero : Annotated noisy entries produced with PERO OCR</p> <p>- text_ocr_tess : OCR extracted text of the entry with Tesseract engine (*not used in our expriments*)</p> <p>- nested_ner_xml_tess : Is entities mapping between nested-ner entities annotated by hand on the ref text and tesseract text correct? (not used in our experiments)</p> <p>- has_valid_ner_xml_tess : Annotated noisy entries produced with Tesseract. (not used in our experiments)</p> <p>Nested entities are annotated using XML tags. Our hierachy of entities is a *Part Of* a two-levels hierarchy. It means that bottom entities are contained in a top level entity.</p> <p> </p> <p><strong>Source documents // Copyright and licence</strong></p> <p><em>This section has been copied from the <a href="https://zenodo.org/record/6394464">original dataset description</a>.</em></p> <p>The images were extracted from the original source https://gallica.bnf.fr, owned by the *Bibliothèque nationale de France* (French national library).</p> <p>Original contents from the <em>Bibliothèque nationale de France</em> can be reused non-commercially, provided the mention "Source gallica.bnf.fr / Bibliothèque nationale de France" is kept. </p> <p>=> <strong>Researchers do not have to pay any fee for reusing the original contents in research publications or academic works.</strong></p> <p>Original copyright mentions extracted from <a href="https://gallica.bnf.fr/edit/und/conditions-dutilisation-des-contenus-de-gallica">https://gallica.bnf.fr/edit/und/conditions-dutilisation-des-contenus-de-gallica</a> on March 29, 2022.</p> <p>The original contents were significantly transformed before being included in this dataset.</p> <p>All derived content is licensed under the permissive *Creative Commons Attribution 4.0 International* license.</p> <p>Links to original contents are given in the window bellow :</p>
ETCBC/bhsa: Added NER specs
<p>The TF feature data is identical to the previous release.</p> <p>What is addes is the directory <code>ner</code> at the toplevel. This contains the config and test spec for manual entity markup with the new annotate tool.</p>
COMPRISE_Data12_Y-GV-NER_V1.0
<p>This NER dataset that has four entity types: Personal names, Locations, Organizations, and Dates, on Yoruba data. It contains 1,168 sentences.</p> <p>The purpose of the dataset is to evaluate the performance of different weakly supervised learning techniques for named entity recognition in low-resourced languages.</p>
COMPRISE_Data11_M-NER_V1.0
<p>A named entity recognition (NER) dataset (a subset of the News dataset). It has four entity types: Personal names, Locations, Organizations, and Dates. The dataset supports ten African languages: Amharic, Hausa, Igbo, Kinyarwanda, Luganda, Luo, Nigerian-Pidgin, Swahili, Wolof, and Yoruba.</p>
COMPRISE_Data08_H-VOA-NER_V1.0
<p>A Hausa named entity recognition dataset. This dataset that consists of 1,450 sentences has been annotated for the following four entity types: Personal name, Locations, Organizations, and dates.</p>
BiodivBERT: Pre-processed Datasets for NER and RE Downstream Tasks
<p>Preprocessed datasets for both Named Entity Recognition (NER) and Relation Extraction (RE) used in <a href="https://github.com/fusion-jena/BiodivBERT">BiodivBERT</a> fine-tuning.</p>
name lists for 'Detecting intersectionality in NER models: A data-driven approach'
<p>Name lists used for data augmentation for testing biases (in terms of error disparities) of Name Entity Recognition in Danish NLP pipelines.</p> <p>The following lists are from <a href="https://www.dst.dk/da/Statistik/emner/borgere/navne/navne-i-hele-befolkningen">Statistics Denmark</a>:</p> <ul> <li>majority_first_names_2023_men.csv</li> <li>majority_first_names_2023_women.csv</li> <li>majority_last_names_2023.csv</li> </ul> <p>The following lists are from Eva Villarsen Meldgaard. 2005. <a href="https://nors.ku.dk/publikationer/webpublikationer/muslimske_fornavne/">Muslimske fornavne i danmark</a>. Publisher: Københavns Universitet</p> <ul> <li>minority_first_names_men.csv</li> <li>minority_first_names_men.csv</li> </ul> <p>The list majority_unisex_names.csv is retrieved from <a href="https://familieretshuset.dk/navne/navne/godkendte-fornavne">The Agency of Family Law</a> in Denmark, and the numbers are retrieved from the above lists from Statistics Denmark.</p> <p>The list minority_last_names.csv is retrieved from <a href="https://www.familyeducation.com/baby-names/surname/origin/muslim">FamilyEducation</a>.</p> <p>The list overlapping_names.csv contains first names, which both occur on the list of majority names and the list of minority names.</p>
AttackER: NER Attack Attribution
<p>The folder contains 8 files. The files with .spacy extension are the files that can be used to train models using spaCy and the .JSON files can be used to train NER models using Huggingface transformers. The new_model.zip contains the fine-tuned transformer model using the .spacy files that can be used for NER tasks on Cyber attack attribution. The spacy_run_script.ipynb file can be used to view the contents of the .spacy files as well as run the model inside the .zip file. The script contains the necessary guidelines for the same. Since NER tasks in Huggingface transformers requires a JSON format, this folder contains the necessary train, test and dev files in the .json format.</p>
AskDoc Chemical Entity NER Dataset for Gender Bias Analysis
<p>This repository contains the dataset for the NAACL 2024 paper titled, "A Comprehensive Study of Gender Bias in Chemical Named Entity Recognition Models."</p> <p>The dataset contains two files:</p> <p>askdoc_female.conll - Female AskDoc dataset<br>askdoc_male.conll - Male AskDoc dataset</p> <p>These files contain annotated AskDoc data with chemical mentions from r/AskDocs on Reddit. The female file contains examples from people that self-identify as female and the male file contains data for people that self-identify as male.</p> <p>A comprehensive datasheet is also provided. </p>
Ner Tamid Lamp
Full name: **Ner Tamid Lamp from the Great Synagogue in Oświęcim, Poland** Known as Ner Tamid (Hebrew: "eternal light"), the lamp symbolises the presence of divine light and as such should be kept aflame in a synagogue at all times, regardless whether the temple is open. This tradition is a reminder of the everlasting menorah in the Temple in Jerusalem (also known as The First Temple). The lamp on display was discovered, alongside other objects, during archeological excavations conducted in 2004 on the site once occupied by the [Great Synagogue in Oświęcim](https://muzea.malopolska.pl/en/articles/673), before it was razed to the ground by the Nazis in November 1939. **For more images and further information, visit:** https://muzea.malopolska.pl/en/objects-list/2063 Inventory number: MŻ 9 Localisation of the physical object: Auschwitz Jewish Center, Oświęcim, Poland **Digitalisation: Regional Digitalisation Lab, Małopolska Institute of Culture in Kraków, Poland; "Virtual Museums of Małopolska" project** Source: Objaverse 1.0 / Sketchfab
MultiCardioNER Corpus: Multilingual Adaptation of Clinical NER Systems to the Cardiology Domain
<h1><strong>MultiCardioNER</strong></h1> <p><strong>MultiCardioNER</strong> is a shared task about the adaptation of clinical NER systems to the cardiology domain. It uses a combination of two existing datasets (DisTEMIST for diseases and the newly-released DrugTEMIST for medications), as well as a new, smaller dataset of cardiology clinical cases annotated using the same guidelines.</p> <p>Participants are provided DisTEMIST and DrugTEMIST as training data to use as they see fit (1,000 documents, with the original partitions splitting them into 750 for training and 250 for testing). The cardiology clinical cases (cardioccc) are meant to be used as a development or validation set (258 documents), although participants are encourage to experiment with the documents and annotations as they see fit. The evaluation is done using a different collection of cardiology clinical cases (250).</p> <p>MultiCardioNER proposes two tracks:</p> <p>- Track 1: Spanish adaptation of disease recognition systems to the cardiology domain.<br>- Track 2: Multilingual (Spanish, English and Italian) adaptation of medication recognition systems to the cardiology domain.</p> <p>Please read the README file attached for more information on folder structure and file format.</p> <p><strong>MultiCardioNER</strong> was developed by the Barcelona Supercomputing Center's NLP for Biomedical Information Analysis and used as part of BioASQ 2024. For more information on the corpus, annotation scheme and task in general, please visit: <a href="https://temu.bsc.es/multicardioner" target="_blank" rel="noopener">https://temu.bsc.es/multicardioner</a>. This task is promoted by Spanish and European projects such as DataTools4Heart, AI4HF, BARITONE and AI4ProfHealth.</p> <p><strong>UPDATE MAY 28th 2024: </strong>The test set annotations are now out! We've also included the original background set files, as well as a file with the mappings from the masked filenames used during the evaluation phase to the original filenames. Please check the README for more information.</p> <h2><strong>Resources</strong></h2> <ul> <li><a href="https://temu.bsc.es/multicardioner" target="_blank" rel="noopener">MultiCardioNER website</a></li> <li><a href="http://bioasq.org/" target="_blank" rel="noopener">BioASQ website</a></li> <li><a href="../doi/10.5281/zenodo.6458078" target="_blank" rel="noopener">DisTEMIST Guidelines</a></li> <li><a href="../doi/10.5281/zenodo.11065432" target="_blank" rel="noopener">DrugTEMIST Guidelines</a></li> </ul> <h2><strong>License</strong></h2> <p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p> <h2><strong>Contact</strong></h2> <p>If you have any questions or suggestions, please contact us at:</p> <p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p> <h2><strong>Additional resources and corpora</strong></h2> <p>If you are interested in MultiCardioNER, you might want to check out these corpora and resources:</p> <ul> <li><a href="../records/7614764">DisTEMIST</a> (Corpus of disease mentions and normalization to SNOMED CT)</li> <li><a href="../records/8224056">MedProcNER </a>(Corpus of clinical procedure mentions and normalization to SNOMED CT)</li> <li><a href="../records/10635215">SympTEMIST</a> (Corpus of clinical findings and normalization to SNOMED CT)</li> <li><a href="../records/4270158">PharmaCoNER</a> (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization)</li> <li><a href="../records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization)</li> <li><a href="../records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization)</li> <li><a href="../records/4279323">MEDDOCAN</a> (Corpus of mentions of Personal Health Identifiers (PHI))</li> <li><a href="../records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization)</li> <li><a href="../records/3837305">CodiESP</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version)</li> <li><a href="../records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy)</li> <li><a href="../records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags)</li> <li><a href="../records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries))</li> <li><a href="../records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags)</li> <li><a href="../records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts)</li> </ul>
Ner Tamid lamp
Ner Tamid (Hebrew: eternal light) lamp arose in honor of Pesach Hollander, the owner of a wine store on Kolejowa Street in Oświęcim. He was a long-time member of the town council and the son of a local rabbi. He passed away in 1936. 1936, Oświęcim Auschwitz Jewish Centre https://muzea.malopolska.pl/en/objects-list/2055 Inventory number: MŻ 41 Source: Objaverse 1.0 / Sketchfab
Tutorial: Ein eigenes NER-Modell für die digitale Literaturanalyse trainieren
<p>In der Videoreihe „Named Entity Recognition und digitale Literaturanalyse“ zeigen wir, wie sich der <em>Stanford Named Entity Recognizer</em> installieren und zur Literaturanalyse nutzen lässt. Außerdem stellen wir drei Möglichkeiten zur Verbesserung der Ergebnisse vor und zeigen, unter anderem, wie sich eine generierte Output-Datei in eine XML-Datei umwandeln lässt oder wie du dein eigenes NER-Modell trainieren kannst. <br>In diesem Video zeigen wir Schritt für Schritt, wie ein eigenes Modell trainiert wird, um Eigennamen in einem Text automatisch erkennen zu lassen. Dazu sind keine technischen Vorkenntnisse nötig, der Stanford Named Entity Recognizer sollte aber bereits installiert sein. Abgesehen davon brauchst du den literarischen Text, den du untersuchen möchten, in digitaler Form. Auf der Webseite der Stanford Natural Language Processing Group befinden sich die verwendeten programmiersprachlichen Befehle.</p> <p> </p> <p>Mehr Infos: </p> <ul> <li>Webseite der Stanford NER Group: <a href="https://nlp.stanford.edu">https://nlp.stanford.edu </a></li> <li>Stanford NER Herunterladen: <a href="https://nlp.stanford.edu/software/CRF-NER.shtml">https://nlp.stanford.edu/software/CRF-NER.shtml</a></li> <li>Schriftliche Einführung in die Methodik der NER: <a href="https://fortext.net/routinen/methoden/named-entity-recognition-ner">https://fortext.net/routinen/methoden/named-entity-recognition-ner</a></li> </ul> <p>Übersicht der Videoreihe auf Zenodo:</p> <ol> <li><a href="../records/10372231">Tutorial: Stanford Named Entity Recognizer installieren und deutsche Kategorien laden </a></li> <li><a href="../records/10372239">Tutorial: Stanford Named Entity Recognizer zur digitalen Literaturanalyse nutzen</a> </li> <li><a href="../records/10371086">Tutorial: Ein eigenes NER-Modell für die digitale Literaturanalyse trainieren</a></li> <li><a href="../records/10250582">Fallbeispiel: Figurenkonstellationen in Goethes Werther und Plenzdorfs neuem Werther</a></li> </ol> <p><a href="https://www.youtube.com/watch?v=hYed-ZqEzs8&list=PLu-M0KuYw64pVltr9EazXvscw5DpRv01Q">Hier</a> zur Videoreihe auf Youtube</p>
Thai NER 2.2
<p>This version is fixed wrong tag (DATA -> DATE) from Thai NER 2.1.</p> <p>Dataset</p> <p><strong>Size</strong></p> <ul> <li>Train: 3,938 docs</li> <li>Validation: 1,313 docs</li> <li>Test: 1,313 Docs</li> </ul> <p>Some data come from crowdsourcing between Dec 2018 - Nov 2019. <a href="https://github.com/wannaphong/thai-ner">https://github.com/wannaphong/thai-ner</a></p> <p><strong>Domain</strong></p> <ul> <li>News (It, politics, economy, social)</li> <li>PR (KKU news)</li> <li>general</li> </ul> <p><strong>Source</strong></p> <ul> <li>I use sone data from Nutcha’s theses (<a href="http://pioneer.chula.ac.th/~awirote/Data-Nutcha.zip">http://pioneer.chula.ac.th/~awirote/Data-Nutcha.zip</a>) and improve data by rechecking and adding more tagging.</li> <li>Twitter</li> <li>Blognone.com - It news</li> <li>thaigov.go.th</li> <li>kku.ac.th</li> </ul> <p>And more (the lists are lost.)</p> <p><strong>Tag</strong></p> <ul> <li>DATE - date</li> <li>TIME - time</li> <li>EMAIL - email</li> <li>LEN - length</li> <li>LOCATION - Location</li> <li>ORGANIZATION - Company / Organization</li> <li>PERSON - Person name</li> <li>PHONE - phone number</li> <li>TEMPERATURE - temperature</li> <li>URL - URL</li> <li>ZIP - Zip code</li> <li>MONEY - the amount</li> <li>LAW - legislation</li> <li>PERCENT - PERCENT</li> </ul>
ner for property
<p>n/a</p>
PatchFlow v1.0 age-stratified networks NER
Age-stratified synthetic mobility networks for NER
Deficiency in nucleotide excision repair (NER) family gene activity, especially ERCC3, is associated with non-pigmented hair fiber growth
GEO Series GSE24009. Homo sapiens. 21 samples. Type: Expression profiling by array.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.