Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
28
datasets available to search
ShareScore release 0.9.0
Dataset results
28 results for “Entity Recognition”
An Arabic Dataset for Disease Named Entity Recognition with Multi-Annotation Schemes
<p>This is a novel data descriptor that provides the Arabic natural language processing community with a dataset dedicated to named entity recognition tasks for diseases. The dataset comprises more than 60 thousand words, which were annotated manually by two independent annotators using the inside-outside (IO) annotation scheme. To ensure the reliability of the annotation process, the inter-annotator agreements rate was calculated, and it scored 95.14\%. Due to the lack of research efforts in the literature dedicated for studying Arabic multi-annotation schemes, a distinguishing and a novel aspect of this dataset is the inclusion of six more annotation schemes that will bridge the gap by allowing the researchers to explore and compare the effects of these schemes on the performance of the Arabic named entity recognizers. These annotation schemes are IOE, IOB, BIES, IOBES, IE, and BI. Additionally, five linguistic features, including part-of-speech tags, stopwords, gazetteers, lexical markers, and the presence of the definite article, are provided for each record in the dataset.</p>
Named Entity Recognition Datasets
Open the record for dataset details and reuse information.
Romanian micro-blogging named entity recognition (MicroBloggingNERo)
<p>MicroBloggingNERo is a manually annotated corpus for named entity recognition in Romanian micro-blogging texts. <br> It provides gold annotations for organizations, locations, persons, time expressions, legal references, medical devices,<br> chemicals, anatomical parts and disorders found in micro-blogging texts. The text was anonymized, by replacing all<br> URLs with <url>, user references with <user>, person names, specific locations and organizations with new randomized names. <br> Anonymization was realized in the same way, regardless of the micro-blogging platform specific format.</p> <p>Since names were replaced with new random ones, any resemblance to real individuals is by pure chance of the random names<br> generator. No real person is depicted in the included messages.</p> <p><br> DATA</p> <p>The MicroBloggingNERo corpus is available in different formats: text, span-based, and token-based. </p> <p>Text files are in the folder "text" with .txt extension, in UTF-8 encoding.</p> <p>Span-based annotations are given in BRAT (https://brat.nlplab.org/) ann format. These annotations can be found in folders starting with "ann_".</p> <p>Token-based annotations are given in CONLLUP files, following the CoNLL-U Plus format https://universaldependencies.org/ext-format.html .<br> Part-of-speech tagging was realized using UDPIPE. <br> Named entity annotations are placed in the column "RELATE:NE" (the 11th column) as defined in the "global.columns" metadata field.<br> Automatic processing was performed through the RELATE platform (https://relate.racai.ro).</p> <p>The archive contains: </p> <p>- ann_EVERYTHING <br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations, time, chemicals, medical devices, anatomical parts and disorders. <br> Overlapping annotations of organizations and time entities inside legal references were allowed. </p> <p>- ann_EVERYTHING_LARGEST_SPAN <br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations, time, chemicals, medical devices, anatomical parts and disorders. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. This affects primarily the legal references class.</p> <p>- ann_LEGAL_PER_LOC_ORG_TIME <br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations and time. <br> There are no overlapping annotations. </p> <p>- ann_PER_LOC_ORG_TIME <br> Folder in which all the files are in .ann format and contains annotations of: persons, locations, organizations and time. <br> There are no overlapping annotations. </p> <p>- ann_BIOMEDICAL<br> Folder in which all the files are in .ann format and contains annotations of: medical devices, chemicals, anatomical parts and disorders. <br> There are no overlapping annotations. </p> <p>- conllup_EVERYTHING_LARGEST_SPAN<br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations, time, chemicals, medical devices, anatomical parts and disorders. <br> There are no overlapping annotations. </p> <p>- conllup_LEGAL_PER_LOC_ORG_TIME <br> Folder in which all the files are in .conllup format and contains annotations of: legal references, persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. </p> <p>- conllup_PER_LOC_ORG_TIME <br> Folder in which all the files are in .conllup format and contains annotations of: persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. </p> <p>- conllup_BIOMEDICAL<br> Folder in which all the files are in .conllup format and contains annotations of: medical devices, chemicals, anatomical parts and disorders. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. </p> <p>- text <br> Folder containing the raw texts.</p> <p>- splits.tsv<br> Proposed splits into train,test,valid following a distribution of 70-15-15% for each entity class, based on the ann_EVERYTHING_LARGEST_SPAN folder</p> <p>LICENSING</p> <p>This work is provided under the license CC BY-NC-ND 4.0 (Attribution-NonCommercial-NoDerivatives 4.0 International).<br> The license can be viewed online here: https://creativecommons.org/licenses/by-nc-nd/4.0/ <br> and the full text here: https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode . </p> <p><br> CONTACT</p> <p>Research Institute for Artificial Intelligence "Mihai Draganescu", Romanian Academy<br> Web: http://www.racai.ro <br> Contact emails: vasile@racai.ro , maria@racai.ro , vergi@racai.ro , elena@racai.ro</p>
Romanian Named Entity Recognition in the Legal domain (LegalNERo)
<p>LegalNERo is a manually annotated corpus for named entity recognition in the Romanian legal domain. <br> It provides gold annotations for organizations, locations, persons, time and legal resources mentioned in legal documents (legal references).Starting with version 4, the legal references were annotated using fine-grained legal document types: Law. Ordinance, Publication, Decree, Decision, Treaty, Report, Order, Regulation, Directive, EmergencyOrdinance, Norm, Convention, Code, and Other.<br> Additionally it offers GEONAMES codes for the named entities annotated as location (where a link could be established). </p> <p>The LegalNERo corpus is available in different formats: span-based, token-based and RDF. <br> The Linguistic Linked Open Data (LLOD) version is provided in RDF-Turtle format.</p> <p>CONLLUP files conform to the CoNLL-U Plus format <a href="https://universaldependencies.org/ext-format.html">https://universaldependencies.org/ext-format.html</a> .<br> Part-of-speech tagging was realized using UDPIPE. <br> Named entity annotations are placed in the column "RELATE:NE" (the 11th column) as defined in the "global.columns" metadata field.<br> Similarly GEONAMES references are in the column "RELATE:GEONAMES" (the 12th column, last).<br> Automatic processing was performed through the RELATE platform (<a href="https://relate.racai.ro">https://relate.racai.ro</a>).</p> <p>ANN files conform to BRAT format (<a href="https://brat.nlplab.org/">https://brat.nlplab.org/</a>).<br> <br> The archive contains: </p> <p>- ann_LEGAL_PER_LOC_ORG_TIME_overlap <br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations and time. <br> Overlapping annotations of organizations and time entities inside legal references were allowed. </p> <p>- ann_FGLEGAL_PER_LOC_ORG_TIME_overlap <br> Folder (corresponding to the above entry) in which all the files are in .ann format and contains annotations of: fine-grained legal references, persons, locations, organizations and time. <br> Overlapping annotations of organizations and time entities inside legal references were allowed. </p> <p>- ann_LEGAL_PER_LOC_ORG_TIME <br> Folder in which all the files are in .ann format and contains annotations of: legal references, persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. </p> <p>- ann_FGLEGAL_PER_LOC_ORG_TIME <br> Folder (corresponding to the above entry) in which all the files are in .ann format and contains annotations of: fine-grained legal references, persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. </p> <p>- ann_PER_LOC_ORG_TIME <br> Folder in which all the files are in .ann format and contains annotations of: persons, locations, organizations and time. <br> There are no overlapping annotations. </p> <p>- conllup_LEGAL_PER_LOC_ORG_TIME <br> Folder in which all the files are in .conllup format and contains annotations of: legal references, persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. <br> The annotation of these files was enhanced with GEONAMES codes (where linking was possible). </p> <p>- conllup_FGLEGAL_PER_LOC_ORG_TIME <br> Folder (corresponding to the above entry) in which all the files are in .conllup format and contains annotations of: fine-grained legal references, persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. <br> The annotation of these files was enhanced with GEONAMES codes (where linking was possible). </p> <p>- conllup_PER_LOC_ORG_TIME <br> Folder in which all the files are in .conllup format and contains annotations of: persons, locations, organizations and time. <br> Overlapping annotations were not allowed and only the longest named entities were annotated. <br> The annotation of these files was enhanced with GEONAMES codes (where linking was possible).</p> <p>- rdf <br> Folder containing the corpus in RDF-Turtle format.<br> All the annotations are available here in both span and token format.</p> <p>- text <br> Folder containing the raw texts.</p> <p>- splits_FGLEGAL_PER_LOC_ORG_TIME.tsv<br> This is a proposed split of the documents for training a NER system using the fine-grained entity classes. <br> The split was created randomly, while trying to ensure 15% of each entity type for validation, 15% for testing and 70% for training.<br> </p> <p><strong>NER System</strong></p> <p>A NER model generated using the LegalNERo corpus can be used online in the RELATE platform: https://relate.racai.ro/index.php?path=ner/demo</p> <p>This system was described in: Păiș, Vasile and Mitrofan, Maria and Gasan, Carol Luca and Coneschi, Vlad and Ianov, Alexandru. Named Entity Recognition in the Romanian Legal Domain. In Proceedings of the Natural Legal Language Processing Workshop 2021. Association for Computational Linguistics, Punta Cana, Dominican Republic, pp. 9--18, nov 2021</p> <p><br> <strong>LICENSING</strong></p> <p>This work is provided under the license CC BY-NC-ND 4.0 (Attribution-NonCommercial-NoDerivatives 4.0 International).<br> The license can be viewed online here: https://creativecommons.org/licenses/by-nc-nd/4.0/ <br> and the full text here: https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode . </p> <p><br> <strong>CONTACT</strong></p> <p>Research Institute for Artificial Intelligence "Mihai Draganescu", Romanian Academy<br> Web: <a href="http://www.racai.ro">http://www.racai.ro</a> <br> Contact emails: vasile@racai.ro , maria@racai.ro</p>
Enhancing Name Entity Recognition Through Hybrid Deep Learning in Natural Language Processing
Open the record for dataset details and reuse information.
Named Entity Recognition in the Regesta of Emperor Frederick III. of the Holy Roman Empire
<p>This dataset contains Named Entity Recognition and Entity Linking across 8883 Regesta of Emperor Frederick III. of the Holy Roman Empire. The Regesta are from the Regesta Imperii Edition project: http://www.regesta-imperii.de/en/the-project.html.</p> <p>There are 4554 different names normalized to 2849 distinct entities, identified by the index entry identifier of the yet to be fully published index of the Regesta of Frederick III. All in all, 17258 instances of names are identified.</p> <p><strong>Not all named entities are identified!</strong></p> <p>This dataset is intended as a step stone for recording already identified entities and training NER classifiers for the Regesta Imperii. Some named entities were deliberately omitted.</p> <p>Only those that could be identified with reasonable certainty via the index were included, for example: "Mgf. Albrecht von Brandenburg" is included, while "Albrecht", even when refering to the same entity as the former name, is not. Even of this subset, some entities may have been missed. A newer version may rectify that in the future.</p> <p>The JSONL format is modeled after the <a href="https://github.com/doccano/doccano">Doccano</a> format for entity linking.</p>
Temporally-Informed Analysis of Named Entity Recognition
<p>This repository contains the data set developed for the paper:</p> <p>“Shruti Rijhwani and Daniel Preoțiuc-Pietro. <em>Temporally-Informed Analysis of Named Entity Recognition.</em> In Proceedings of the Association for Computational Linguistics (ACL). 2020.”</p> <p>It includes 12,000 tweets annotated for the named entity recognition task. The tweets are uniformly distributed over the years 2014-2019, with 2,000 tweets from each year. The goal is to have a temporally diverse corpus to account for data drift over time when building NER models.</p> <p>The entity types annotated are locations (LOC), persons (PER) and organizations (ORG). The tweets are preprocessed to replace usernames and URLs with a unique token. Hashtags are left intact and can be annotated as named entities.</p> <p><strong>Format</strong></p> <p>The repository contains the annotations in JSON format.</p> <p>Each year-wise file has the tweet IDs along with token-level annotations. The Public Twitter Search API (<a href="https://developer.twitter.com/en/docs/tweets/search">https://developer.twitter.com/en/docs/tweets/search</a>) can be used extract the text for the tweet corresponding to the tweet IDs.</p> <p><strong>Data Splits</strong></p> <p>Typically, NER models are trained and evaluated on annotations available at the model building time, but are used to make predictions on data from a future time period. This setup makes the model susceptible to temporal data drift, leading to lower performance on future data as compared to the test set.</p> <p>To examine this effect, we use tweets from the years 2014-2018 as the training set and random splits of the 2019 tweets as the development and test sets. These splits simulate the scenario of making predictions on data from a future time period.</p> <p>The development and test splits are provided in the JSON format.</p> <p><strong>Use</strong></p> <p>Please cite the data set and the accompanying paper if you found the resources in this repository useful.</p>
Named Entity Recognition Datasets
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.