Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
13
datasets available to search
ShareScore release 0.9.0
Dataset results
13 results for “Gold Standard Corpus”
CodiEsp corpus: gold standard Spanish clinical cases coded in ICD10 (CIE10) - eHealth CLEF2020
<p><strong>Introduction</strong></p> <p>These are the train, development and test sets of the CodiEsp corpus. Train, development and test have gold standard annotations. In addition, the unannotated background set is also distributed. All documents are released in the context of the CodiEsp track for CLEF ehealth 2020 (<a href="http://temu.bsc.es/codiesp/">http://temu.bsc.es/codiesp/</a>).</p> <p>The CodiEsp corpus contains manually coded clinical cases. All documents are in Spanish language and CIE10 is the coding terminology (it is the Spanish version of ICD10-CM and ICD10-PCS). The CodiEsp corpus has been randomly sampled into three subsets: the train, the development, and the test set. The train set contains 500 clinical cases, and the development and test set 250 clinical cases each. CodiEsp participants must submit predictions for the test and background set, but they will only be evaluated on the test set.</p> <p> </p> <p><strong>Please cite if you use this dataset:</strong></p> <p>Antonio Miranda-Escalada, Aitor Gonzalez-Agirre, Jordi Armengol-Estapé and Martin Krallinger. Overview of automatic clinical coding: annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of CLEF eHealth 2020. In CLEF (Working Notes). 2020</p> <pre><code>@inproceedings{miranda2020overview, title={Overview of automatic clinical coding: annotations, guidelines, and solutions for non-english clinical cases at codiesp track of CLEF eHealth 2020}, author={Miranda-Escalada, Antonio and Gonzalez-Agirre, Aitor and Armengol-Estap{\'e}, Jordi and Krallinger, Martin}, booktitle={Working Notes of Conference and Labs of the Evaluation (CLEF) Forum. CEUR Workshop Proceedings}, year={2020} }</code></pre> <p> </p> <p><strong>Annotation quality</strong></p> <p>Inter-annotator agreement: 88.6% for diagnosis coding, 88.9% for procedure coding and 80.5% for the textual reference annotation. For more information, see the <a href="http://ceur-ws.org/Vol-2696/paper_263.pdf">paper</a>.</p> <p><br> <strong>Zip structure</strong><br> Four folders: train, dev, test and background. Each one of them contains the files for the train, development, test and background corpora, respectively.</p> <ul> <li><strong>train, dev and test</strong> folders have: <ul> <li>3 tab-separated files with the annotation information relevant for each of the 3 sub-tracks of CodiEsp. </li> <li>A subfolder named <em>text_files</em> with the plain text files of the clinical cases.</li> <li>A subfolder named <em>text_files_en</em> with the plain text files machine-translated to English. Due to the translation process, the text files are sentence-splitted.</li> </ul> </li> <li>The <strong>background</strong> folder has only <em>text_files</em> and <em>text_files_en</em> subfolders with the plain text files.</li> </ul> <p><br> <strong>Format</strong><br> The CodiEsp corpus is distributed in plain text in UTF8 encoding, where each clinical case is stored as a single file whose name is the clinical case identifier. Annotations are released in a tab-separated file. Since the CodiEsp track has 3 sub-tracks, every set of documents (train and test) has 3 tab-separated files associated with it. </p> <p>For the sub-tracks CodiEsp-D and CodiEsp-P, the file has the following fields:</p> <pre>articleID ICD10-code </pre> <p>Tab-separated files for the sub-track CodiEsp-X contain extra fields that provide the text-reference and its position:</p> <pre>articleID label ICD10-code text-reference reference-position</pre> <p><br> <strong>Corpus summary statistics</strong><br> The final collection of 1000 clinical cases that make up the corpus had a total of 16504 sentences, with an average of 16.5 sentences per clinical case. It contains a total of 396,988 words, with an average of 396.2 words per clinical case.</p> <p> </p> <p><strong>Resources:</strong></p> <ul> <li><strong><a href="https://temu.bsc.es/codiesp/">Web</a></strong></li> <li><strong><a href="http://ceur-ws.org/Vol-2696/paper_263.pdf">Citation</a>: </strong>Antonio Miranda-Escalada, Aitor Gonzalez-Agirre, Jordi Armengol-Estapé and Martin Krallinger. Overview of automatic clinical coding: annotations, guidelines, and solutions for non-English clinical cases at CodiEsp track of CLEF eHealth 2020. In CLEF (Working Notes). 2020</li> <li><strong><a href="https://doi.org/10.5281/zenodo.3859869">Silver Standard corpus</a></strong></li> <li><strong><a href="https://doi.org/10.5281/zenodo.3730566">Annotation guidelines</a></strong></li> <li><a href="https://www.youtube.com/playlist?list=PL5uSCzf1azhA0crlSVCYMPqMUWd4mXc4x"><strong>YouTube presentations</strong></a></li> <li><a href="https://temu.bsc.es/codiesp/index.php/participants-systems/"><strong>Participant codes</strong></a></li> </ul> <p> </p> <p>For more information, visit the track webpage: http://temu.bsc.es/codiesp/ or email us at encargo-pln-life@bsc.es</p> <p> </p> <p>Copyright (c) 2019 Secretaría de Estado para el Avance Digital</p>
Gold standard corpus, ontologies, and Entity-Quality ontology annotations for evolutionary phenotypes
<p>This data set includes a gold-standard corpus of evolutionary phenotype descriptions (in the form of character state descriptions pulled from a variety of phylogenetic systematics studies), and their corresponding expert-curated annotations with ontology terms in the form of Entity-Quality (EQ) statements. EQ annotatons allow machine-reasoning (through the semantics encoded in the requisite ontologies from which the ontology terms are drawn), and machine-reasoning in turn enables computing metrics for quantifying the semantic similarity between different phenotype descriptions as represented by their EQ annotations.</p> <p>Also included are the ontologies, and the human expert-generated and Semantic Charaparser (i.e., machine) generated EQ annotations used to assess Semantic Charaparser performance relative to inter-curator variation and to the effect of having access to external knowledge. The ontologies include those used as input, the "augmented" ontologies created by human curators in each experiment round, and the merged ontology used to maximize Semantic Charaparser's performance.</p> <p>The production of the gold standard corpus, annotation experiments, and evaluation of the results are described in detail in the following manuscript:</p> <blockquote> <p>Dahdul et al (2018) Annotation of phenotypes using ontologies: a Gold Standard for the training and evaluation of natural language processing systems. BioRxiv https://doi.org/10.1101/322156. Submitted to Database.</p> </blockquote> <p>The analysis code for evaluating the gold standard corpus (and the input data and ontologies for that) are available separately from the following:</p> <blockquote> <p>Manda et al (2018) Code and data for analysis of evolutionary phenotype ontology annotations and gold standard corpus. Zenodo. https://doi.org/10.5281/zenodo.1218010</p> </blockquote> <p>In comparison to the previous version (v1.0.0), this record includes a file of MD5 checksums of the Gold Standard data files. The data files themselves are unchanged.</p>
PretoxTM Corpus: a gold standard corpus of preclinical treatment-related findings annotated from toxicology reports
<p>The PretoxTM Corpus is a gold standard corpus of preclinical treatment-related findings annotated from toxicology reports.</p> <p>Example documents annotated by domain experts, also known as a gold standard corpus, are needed in order to develop, train, and validate text mining tools. To this aim we designed and performed an annotation activity for the development of the corpus of treatment-related findings: the PretoxTM corpus.</p> <p>A treatment-related finding expression enclose several named entities; the most relevant one is the abnormal effect detected; which depending on the study domain of the finding, can be given by a measurement, test, or examination named Study Test and an abnormal Manifestation result obtained for that study test; or by an abnormal Finding in study domains where there is no associated test or measurement (e.g., clinical, macroscopic and microscopic). Other related named entities that could be present to complete the treatment-related finding are; the Specimen of the abnormal observation, the Sex of the subject, the Group of subjects in which the observation was detected and the Dose level administration of the compound. Examples of sentences with treatment-related findings are: “The decrease in food consumption and body weight of the animals from the mid dose onwards is regarded as evidence of general toxicity.” and "At dose level 3, absolute and relative liver weights were increased in male rats.”.</p> <p>Contributions: The PretoxTM corpus was developed by BSC, with the contribution of IMIM and a team of experts from the eTRANSAFE EFPIA partners.</p> <p>License: Creative Commons Attribution-ShareAlike 4.0 International (cc by sa 4.0).</p> <p>The PretoxTM resources have been developed as part of the eTRANSAFE project.</p> <p>For more information about PretoxTM please visit:</p> <p>PretoxTM Corpus Gitlab: <a href="https://gitlab.bsc.es/inb/etransafe/preclinical-toxicological-corpus/">https://gitlab.bsc.es/inb/etransafe/preclinical-toxicological-corpus/</a></p> <p>PretoxTM central documentation: <a href="https://pretoxtm.gitlab.io/documentation/">https://pretoxtm.gitlab.io/documentation/</a></p>
PharmaCoNER corpus: gold standard annotations of Pharmacological Substances, Compounds and proteins in Spanish clinical case reports
<p><strong>Intro:</strong></p><p>The PharmaCoNER corpus (divided into train, dev and test) is a Gold Standard manually annotated dataset used for the the PharmaCoNER shared task posed at BIONLP-ST (at EMNLP). In addition, we include here the PharmaCoNER background set. It contains the train, development and test sets of the two subtasks (subtask-1 and subtask-2) with Gold Standard annotations. In addition, it contains the documents of the background set, without annotations.</p><p>The PharmaCoNER corpus consists of:</p><ul><li>Manually classified clinical case sections derived from Open access Spanish medical publications, named the Spanish Clinical Case Corpus (SPACCC).</li><li>It was manually selected by a practicing oncologist and revised by a clinical documentalist to assure that records were relevant/representative and resembled structure and content relevant to process clinical records.</li><li>The final corpus: 1000 clinical cases 16,504 sentences</li><li>The corpus contains a total of 396,988 words, with an average of 396.2 words per clinical case.</li><li>It covers a range of medical disciplines including oncology, urology, cardiology, pneumology or infections diseases, etc.</li><li>The corpus has been annotated at the mention level by experts in medicinal chemistry and pharmacology following a granular annotation scheme covering four mention types:<ul><li><i>Entity type 1 (NORMALIZABLES)</i>: mentions of chemicals that can be manually normalized to a unique concept identifier (primarily SNOMED-CT).</li><li><i>Entity type 2 (NO_NORMALIZABLES)</i>: mentions of chemicals that could not be normalized manually to a unique concept identifier.</li><li><i>Entity type 3 (PROTEINAS)</i>: mentions of proteins/genes following an adaptation of the BioCreative GPRO track annotation guidelines (includes peptides, peptide hormones & antibodies).</li><li><i>Entity type 4 (UNCLEAR )</i>: cases of general substance class mentions of clinical relevance, including certain pharmaceutical formulations, general treatments, chemotherapy programs, and vaccines.</li><li>Mentions class "<i>UNCLEAR</i>" (not evaluated for the PharmaCoNER track)</li></ul></li></ul><p> </p><p> </p><p><strong>Please, cite: </strong></p><p>A. G. Agirre, M. Marimon, A. Intxaurrondo, O. Rabal, M. Villegas, M. Krallinger, Pharmaconer: Pharmacological substances, compounds and proteins named entity recognition track, in: Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, 2019, pp. 1–10.</p><p> </p><p><strong>Annotation quality</strong></p><p>Inter-annotator agreement: 93% for annotation, 73% for mapping.</p><p>For more information, see the <a href="https://paperswithcode.com/paper/pharmaconer-pharmacological-substances">paper</a>.</p><p> </p><p><strong>Format</strong></p><p>For subtask 1 annotations are distributed in <i>Brat</i> format. (More info at Brat webpage https://brat.nlplab.org/standoff.html)</p><p>For subtask-2, codes are associated with each document are given in a <i>TSV</i> file with the following columns: </p><blockquote><p>filename code</p></blockquote><p> </p><p><strong>Shared task goal:</strong></p><p>In the two subtasks, the goal is to predict the annotations of the test files (either the ANN files or the TSV with the codes) given only the plain text files. </p><p> </p><p><strong>Resources:</strong></p><ul><li><a href="https://temu.bsc.es/pharmaconer/"><strong>Web</strong></a></li><li><a href="https://www.aclweb.org/anthology/D19-5701.pdf"><strong>Citation</strong></a><strong>: </strong>A. G. Agirre, M. Marimon, A. Intxaurrondo, O. Rabal, M. Villegas, M. Krallinger, Pharmaconer: Pharmacological substances, compounds and proteins named entity recognition track, in: Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, 2019, pp. 1–10.</li><li><a href="https://doi.org/10.5281/zenodo.4271908"><strong>Silver Standard corpus</strong></a></li><li><a href="https://doi.org/10.5281/zenodo.3763276"><strong>Annotation guidelines</strong></a></li><li><a href="https://github.com/TeMU-BSC/PharmaCoNER-Tagger"><strong>PharmaCoNER tagger</strong></a></li><li><a href="https://www.youtube.com/watch?v=B3ZzJl5OMkY"><strong>Youtube video(general setting)</strong></a></li><li><a href="https://www.slideshare.net/MartinKrallinger/pharmaconer-pharmacological-substances-compounds-and-proteins-named-entity-recognition-track-at-bionlpost-workshop-november-4-skycity-rm-2-hong-kong-emnlp2019"><strong>Slides PharmacoNER overview talk at BIONLP-ST / EMNLP </strong></a></li></ul><p>For further information, please visit <a href="https://temu.bsc.es/pharmaconer/">https://temu.bsc.es/pharmaconer/</a> or email us at encargo-pln-life@bsc.es</p><p>Copyright (c) 2018 Secretaría de Estado para el Avance Digital (SEAD)</p><p> </p><p><strong>License</strong></p><p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p><p> </p><p><strong>Contact</strong></p><p>If you have any questions or suggestions, please contact us at:</p><p><br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p><p><strong>Additional resources and corpora</strong></p><p>If you are interested in PharmaCoNER, you might want to check out these corpora and resources:</p><ul><li><a href="https://zenodo.org/records/7614764">DisTEMIST</a> (Corpus of disease mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/8224056">MedProcNER </a>(Corpus of clinical procedure mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/8413866">SympTEMIST</a> (Corpus of symptoms, signs and findings mentions and normalization, same document collection)</li><li><a href="https://zenodo.org/records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/4279323">MEDDOCAN</a> (Corpus of mentions of Personal Health Identifiers (PHI), modified synthetic verions of the document collection)</li><li><a href="https://zenodo.org/records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization, different document collection)</li><li><a href="https://zenodo.org/records/3837305">CodiESp</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)</li><li><a href="https://zenodo.org/records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)</li><li><a href="https://zenodo.org/records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)</li></ul>
MEDDOCAN corpus: gold standard annotations for Medical Document Anonymization on Spanish clinical case reports
<p><strong>Intro:</strong></p> <p>Meddocan shared task dataset (divided in train, dev and test). In addition, we include here the Meddocan background set.</p> <p>It contains the training, development and test sets of the Meddocan shared task with Gold Standard annotations.</p> <p>In addition, it contains the documents of the background set, without annotations.</p> <p> </p> <p><strong>Annotation quality</strong></p> <p>Inter-annotator agreement: 98% </p> <p>For more information, see the <a href="http://ceur-ws.org/Vol-2421/MEDDOCAN_overview.pdf">paper</a>. </p> <p> </p> <p><strong>Format:</strong></p> <p>Annotations are distributed in Brat format. See <a href="https://brat.nlplab.org/standoff.html">Brat webpage</a> for more information.</p> <p>In addition, annotations are also distributed in XML format (based on i2b2 XML format).</p> <p>In the <a href="https://temu.bsc.es/meddocan/index.php/resources/">Meddocan webpage</a>, there is a script to convert between MEDDOCAN-Brat, MEDDOCAN-XML, and i2b2 formats.</p> <p> </p> <p><strong>Shared task goal:</strong></p> <p>In the three subtasks, the goal will be to predict the annotations given only the plain text files. </p> <p> </p> <p><strong>Resources:</strong></p> <ul> <li><strong><a href="https://temu.bsc.es/meddocan/">Web</a></strong></li> <li><strong>Citation: </strong>Montserrat Marimon et al. “Automatic De-identification of Medical Texts in Spanish: the MEDDOCAN Track, Corpus, Guidelines, Methods and Evaluation of Results.” In: IberLEF@ SEPLN. 2019, pp. 618–638.</li> <li><strong>Silver Standard corpus</strong></li> <li><a href="https://doi.org/10.5281/zenodo.4279337"><strong>Annotation guidelines</strong></a></li> </ul> <p> </p> <p>For further information, please visit <a href="https://temu.bsc.es/meddocan/">https://temu.bsc.es/meddocan/</a> or email us at encargo-pln-life@bsc.es</p> <p>Copyright (c) 2019 Secretaría de Estado para el Avance Digital (SEAD)</p>
SocialDisNER corpus: gold standard annotations for detection of disease mentions in Spanish tweets
<p><strong>If you use any data from this repository, please cite our scientific paper instead of the Zenodo repo: </strong></p> <p>Luis Gasco Sánchez, Darryl Estrada Zavala, Eulàlia Farré-Maduell, Salvador Lima-López, Antonio Miranda-Escalada, and Martin Krallinger. 2022. <a href="https://aclanthology.org/2022.smm4h-1.48">The SocialDisNER shared task on detection of disease mentions in health-relevant content from social media: methods, evaluation, guidelines and corpora</a>. In <em>Proceedings of The Seventh Workshop on Social Media Mining for Health Applications, Workshop & Shared Task</em>, pages 182–189, Gyeongju, Republic of Korea. Association for Computational Linguistics.</p> <pre><code class="language-json">@inproceedings{gasco2022socialdisner, title = "The {S}ocial{D}is{NER} shared task on detection of disease mentions in health-relevant content from social media: methods, evaluation, guidelines and corpora", author = "Gasco S{\'a}nchez, Luis and Estrada Zavala, Darryl and Farr{\'e}-Maduell, Eul{\`a}lia and Lima-L{\'o}pez, Salvador and Miranda-Escalada, Antonio and Krallinger, Martin", booktitle = "Proceedings of The Seventh Workshop on Social Media Mining for Health Applications, Workshop {\&} Shared Task", month = oct, year = "2022", address = "Gyeongju, Republic of Korea", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.smm4h-1.48", pages = "182--189" }</code></pre> <p> </p> <p><strong>Introduction:</strong><br> The <strong>SocialDisNER corpus</strong> of the SMM4H 2022 – Task 10 task focus on the recognition of disease mentions in tweets written in Spanish after selecting primarily<strong><em> first-hand experience of diseases</em></strong> and other health-relevant content (from patient associations, professional healthcare institutions, and through followers of patient association accounts of a <em>diversity of pathologies</em> including rare diseases, mental health, cancer, etc..).</p> <p><strong>SocialDisNER Gold Standard</strong></p> <p>The Gold Standard corpus was manually annotated by medical experts following the <a href="https://doi.org/10.5281/zenodo.6983041">SMM4H-SocialDisNER guidelines</a>. These guidelines were adapted from previous efforts used to annotate patient clinical records and medical literature. It covers rules for annotating <strong>mentions of diseases</strong> in health-related tweets in Spanish,</p> <p>The training set consists of 5000 tweets written in Spanish and the validation set consists of 2500 tweets written in Spanish. Both sets have been manually annotated by healthcare professionals. The test dataset contains 23430 tweets, although only 2000 will be used to evaluate the systems participating in the task (the rest is background set). We don't plan to publish the test set, but if you want you can test your system from <a href="https://codalab.lisn.upsaclay.fr/competitions/3531">SocialDisNER Codalab</a>.</p> <p><strong>SocialDisNER Large Scale Corpus</strong></p> <p>The large-scale data contains mentions automatically extracted from a set of 85000 tweets. Separate datasets are shown for each entity including diseases, drugs, symptoms, professions, procedures, species, morphology neoplasm, and persons.</p> <p><strong>SocialDisNER co-mention networks</strong></p> <p>We have computed a co-occurrence matrix of the extracted diseases, as well as several co-mention matrices between the disease mentions and the rest of the entities in the large-scale corpora.</p> <p> </p> <p><strong>File structure:</strong></p> <p>The structure of the corpus is: </p> <ul> <li><strong>SocialDisNER_Data:</strong> <ul> <li>training-validation-data folder <ul> <li><strong><em>train-valid-txt-files</em></strong>: folder with training and validation text files. One text file per tweet, the file name corresponds to the tweet id. One sub-directory per corpus split (train and valid). The files named <em>ids_dev_set.txt</em> and<em> ids_train_set.txt </em>contain the list of file identifiers for each of the data splits (validation and train).</li> <li><strong><em>mentions.tsv</em></strong>: This file contains the manually annotated disease mentions. The file has the following fields: <ul> <li><em>tweets_id</em>: This is the id of the tweet, using Twitter API you can query the content of the tweet.</li> <li><em>Begin</em>: This is the position in the tweet where the annotation was found.</li> <li><em>End</em>: This is the position of the last character of the annotation in the tweet.</li> <li><em>Type: </em>This is the type of entity found, in our case "ENFERMEDAD".</li> <li><em>Extraction</em>: This is the literal extraction, in other words, the fragment of text which refers to the annotation. </li> </ul> </li> </ul> </li> <li>test-data folder: <ul> <li><strong>test-data-txt-files</strong>: folder with test text files. One file per tweet, the file name corresponds to the tweet id. The folder contains 23430 tweets to be used as test set of the task. Of them, 2000 will be used to evaluate the participating systems.</li> </ul> </li> </ul> </li> </ul> <p> </p> <ul> <li><strong>SocialDisNER_LargeScale_additionaldata:</strong> <ul> <li>socialdisner_diseases: <ul> <li><strong>tweets_txt:</strong> Folder with large-scale tweet database. One text file per tweet, the file name corresponds to the tweet id.</li> <li><strong>diseases_mentions.tsv</strong>: This file contains the automatically annotated disease mentions from the large-scale SocialDisNER corpus (Silver Standard). The structure is the same than the Golden Standard annotations.</li> </ul> </li> <li>socialdisner_ENTITY: Each folder with this naming convention contains the following data structure. Corpora have been generated with mentions of diseases, drugs, symptoms, professions, procedures, species, morphology neoplasm and persons <ul> <li><strong>tweets_txt:</strong> Folder with large-scale tweet database. One text file per tweet, the file name corresponds to the tweet id.</li> <li><strong>ENTITY_mentions.tsv</strong>: This file contains the automatically annotated mentions of type “ENTITY” from the large-scale SocialDisNER corpus (Silver Standard). The structure is the same than the Golden Standard annotations.</li> </ul> </li> <li>socialdisner_networks: This folder contains tsv files containing the co-mention matrices between the diseases and the rest of the entities of the large-scale socialdisner data. Each file follows the following naming convention: <ul> <li><strong>socialdisner_disease-ENTITY_net.tsv</strong><em>: </em>The tsv file contains a series of columns and rows corresponding to the mentions used for building the matrix. Each column is separated by “;”. The type of each mention is identified by the label in parentheses of each title. The count represents the number of times that mention x and mention y were found in the same tweet of the large-scale dataset.</li> <li><strong>socialdiser_disease_net.tsv</strong>: This tsv file contains the array of socialdisner-disease large-scale corpus co-mentions separated by ";". This file can be loaded into NetworkX to perform disease co-morbidity analysis on the socialdisner-disease large-scale data.</li> </ul> </li> </ul> </li> </ul> <p><em>Note: In previous versions of the dataset the order of the columns in the mentions.tsv file was not in the correct order. From this version onwards the order is correct and adequate to send the predictions of the task.</em></p> <p> </p> <p>For further information, please visit <a href="https://temu.bsc.es/socialdisner/">https://temu.bsc.es/socialdisner/</a></p> <p><strong>Summary statistics:</strong></p> <table> <caption>Manually annotated data</caption> <thead> <tr> <th scope="row"> </th> <th scope="col">Training set</th> <th scope="col">Development set</th> </tr> </thead> <tbody> <tr> <th scope="row"># tweets</th> <td>5000</td> <td>2500</td> </tr> <tr> <th scope="row"># characters</th> <td>1253431</td> <td>516768</td> </tr> <tr> <th scope="row"># tokens</th> <td>211555</td> <td>84478</td> </tr> <tr> <th scope="row">Avg. char / tweet</th> <td>250.69</td> <td>206.71</td> </tr> <tr> <th scope="row">Avg. tok. / tweet</th> <td>42.31</td> <td>33.79</td> </tr> <tr> <th scope="row"># mentions</th> <td>15173</td> <td>4252</td> </tr> <tr> <th scope="row"># unique mentions</th> <td>4407</td> <td>1413</td> </tr> </tbody> </table> <p> </p> <table> <caption>Large-scale annotated data (Silver Standard)</caption> <tbody> <tr> <td> </td> <td><em>Socialdisner-diseases</em></td> <td><em>Socialdisner-pharma</em></td> <td><em>Socialdisner-morphology_neoplasms</em></td> <td><em>Socialdisner-symptoms</em></td> <td><em>Socialdisner-professions</em></td> <td><em>Socialdisner-Procedures</em></td> <td><em>Socialdisnerv-Person</em></td> <td><em>Socialdisner-Species</em></td> </tr> <tr> <td><strong># tweets</strong></td> <td>85077</td> <td>1759</td> <td>8518</td> <td>12624</td> <td>15831</td> <td>11462</td> <td>41033</td> <td>12118</td> </tr> <tr> <td><strong># characters</strong></td> <td>19920670</td> <td>435141</td> <td>2082574</td> <td>3023784</td> <td>4063114</td> <td>2873791</td> <td>10273278</td> <td>2933925</td> </tr> <tr> <td><strong># tokens</strong></td> <td>3236411</td> <td>68269</td> <td>332539</td> <td>521503</td> <td>660071</td> <td>467059</td> <td>1689479</td> <td>486249</td> </tr> <tr> <td><strong>Avg. char / tweet</strong></td> <td>234.15</td> <td>247.38</td> <td>244.49</td> <td>239.53</td> <td>256.66</td> <td>250.72</td> <td>250.37</td> <td>242.11</td> </tr> <tr> <td><strong>Avg. tok. / tweet</strong></td> <td>38.04</td> <td>38.81</td> <td>39.04</td> <td>41.31</td> <td>41.69</td> <td>40.75</td> <td>41.17</td> <td>40.13</td> </tr> <tr> <td><strong># mentions</strong></td> <td>116260</td> <td>1029</td> <td>8943</td> <td>12896</td> <td>18590</td> <td>10080</td> <td>58007</td> <td>14014</td> </tr> <tr> <td><strong># unique mentions</strong></td> <td>16034</td> <td>530</td> <td>541</td> <td>6991</td> <td>3667</td> <td>3841</td> <td>3446</td> <td>1676</td> </tr> </tbody> </table> <p> </p> <p> </p> <p> </p> <p>Do not share the data with other individuals/teams without permission from the task organizer. Tweets IDs are the primary source of information. Tweet texts are provided as support material. By downloading this resource, you agree to the Twitter <a href="https://twitter.com/en/tos">Terms of Service</a>, <a href="https://twitter.com/en/privacy">Privacy Policy</a>, <a href="https://developer.twitter.com/en/developer-terms/agreement">Developer Agreement</a>, and <a href="https://developer.twitter.com/en/developer-terms/policy">Developer Policy</a>.</p> <p> </p> <p> </p>
MEDDOPLACE Corpus: Gold Standard annotations for Medical Documents Place-related Content Extraction
<p><strong>MEDDOPLACE</strong> stands for MEDical DOcument PLAce-related Content Extraction. It is a shared task and set of resources focused on the detection, normalization (entity linking/toponym resolution) and classification of different kinds of places, as well as related types of information such as clinical departments, nationalities or patient movements, in medical documents in Spanish.</p> <p>This repository includes the corpus' <strong>train and test sets</strong> in multiple formats, as well as the <strong>SNOMED gazetteer</strong>, <strong>cross-mapping</strong> between SNOMED and MeSH and the <strong>multilingual silver standard in 8 languages </strong>(Catalan, English, French, Italian, Dutch, Portuguese, Romanian and Swedish). For more information, please check the attached README file.</p> <p>MEDDOPLACE was developed by the Barcelona Supercomputing Center's NLP for Biomedical Information Analysis and used as part of IberLEF 2023. For more information on the corpus, annotation scheme and task in general, please visit: <a href="https://temu.bsc.es/meddoplace">https://temu.bsc.es/meddoplace</a>.</p> <p> </p> <p><strong>Please cite if you use this resource:</strong></p> <p>Salvador Lima-López, Eulàlia Farré-Maduell, Antonio Miranda-Escalada, Vicent Brivá-Iglesias and Martin Krallinger. NLP applied to occupational health: MEDDOPROF shared task at IberLEF 2021 on automatic recognition, classification and normalization of professions and occupations from medical texts. In Procesamiento del Lenguaje Natural, 67. 2021.</p> <pre><code>@article{meddoplace, title={MEDDOPLACE Shared Task overview: recognition, normalization and classification of locations and patient movement in clinical texts}, author={Lima-López, Salvador and Farré-Maduell, Eulàlia and Brivá-Iglesias, Vicent and Gasco-Sanchez, Luis and Krallinger, Martin}, journal = {Procesamiento del Lenguaje Natural}, volume = {71}, year={2023}, issn = {1135-5948},<br>DOI = {10.26342/2023-71-23}, url = {http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6561/3961}, pages = {301--311} }</code></pre> <p><strong>Related Links:</strong></p> <p>- MEDDOPLACE website: <a href="https://temu.bsc.es/meddoplace">https://temu.bsc.es/meddoplace</a></p> <p>- MEDDOPLACE overview paper: <a href="http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6561">http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6561</a></p> <p>- Annotation Guidelines (Spanish): <a href="https://doi.org/10.5281/zenodo.7775234">https://doi.org/10.5281/zenodo.7775234</a></p> <p>- Annotation Guidelines (English): <a href="https://doi.org/10.5281/zenodo.7928145">https://doi.org/10.5281/zenodo.7928145</a></p> <p><strong>License</strong></p> <p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p> <p><strong>Contact</strong></p> <p>If you have any questions or suggestions, please contact us at:</p> <p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p>
MEDDOPROF corpus: complete gold standard annotations for occupation detection in medical documents in Spanish
<p><strong>UPDATE 27/09/2022: </strong>A complete normalization of all mentions in the corpus to SNOMED CT has been added to the 'meddoprof-norm.tsv' file.</p> <p><strong>Description</strong></p> <p>This repository contains the complete MEDDOPROF Gold Standard, a collection of 1,844 clinical cases in Spanish with annotations for occupations, working statuses and activities. MEDDOPROF is a Shared Task celebrated in 2021 that explores the application of natural language processing to occupational health. If you'd like to learn more, please visit: <a href="https://temu.bsc.es/meddoprof">https://temu.bsc.es/meddoprof</a>.</p> <p><strong>Folder and File Structure</strong></p> <p>The corpus' files are presented in the format used by the annotation tool brat. That is, for each clinical case there is a .txt file with the text and a .ann file with its corresponding annotations.</p> <p><em>- meddoprof-ner/</em></p> <p>Clinical cases annotated with these labels: PROFESION (PROFESSION), SITUACION_LABORAL (WORKING_STATUS) or ACTIVIDAD (ACTIVIDAD).</p> <p><em>- meddoprof-class/</em></p> <p>Clinical cases with the same annotations as 'meddoprof-ner' but with these labels instead: PACIENTE (patient), FAMILIAR (family member), SANITARIO (health professional) or OTRO (other).</p> <p><em>- ner_class_joint/</em></p> <p>Clinical cases with both levels of annotation (ner and class) joint (that is, a mention classified as as PROFESOR in meddoprof-ner and as PACIENTE in meddoprof-class would be PROFESION-PACIENTE here).</p> <p><em>- meddoprof-norm.tsv</em></p> <p>Tab-separated file (.tsv) with the mapping of each mention in the corpus to ESCO and SNOMED CT. The file has five columns: filename, mention text, span, ESCO code and SNOMED code.</p> <p>Additionally, two files with the filenames of the train and test partitions are included.</p> <p> </p> <p><strong>Please cite if you use this resource:</strong></p> <p>Salvador Lima-López, Eulàlia Farré-Maduell, Antonio Miranda-Escalada, Vicent Brivá-Iglesias and Martin Krallinger. NLP applied to occupational health: MEDDOPROF shared task at IberLEF 2021 on automatic recognition, classification and normalization of professions and occupations from medical texts. In Procesamiento del Lenguaje Natural, 67. 2021.</p> <pre><code>@article{meddoprof, title={NLP applied to occupational health: MEDDOPROF shared task at IberLEF 2021 on automatic recognition, classification and normalization of professions and occupations from medical texts}, author={Lima-López, Salvador and Farré-Maduell, Eulàlia and Miranda-Escalada, Antonio and Brivá-Iglesias, Vicent and Krallinger, Martin}, journal = {Procesamiento del Lenguaje Natural}, volume = {67}, year={2021}, issn = {1989-7553}, url = {http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6393}, pages = {243--256} }</code></pre> <p><strong>Related Resources:</strong></p> <p>- <a href="http://temu.bsc.es/meddoprof">Web</a></p> <p>- <a href="https://doi.org/10.5281/zenodo.4694768">Training Data</a></p> <p>- <a href="https://doi.org/10.5281/zenodo.4889776">Test set</a></p> <p>- <a href="https://zenodo.org/record/4722741">Codes Reference List</a> (for MEDDOPROF-NORM)</p> <p>- <a href="https://zenodo.org/record/4720833">Annotation Guidelines</a></p> <p>- <a href="https://doi.org/10.5281/zenodo.4524658">Occupations Gazetteer</a></p> <p> </p> <blockquote> <p>MEDDOPROF is part of the IberLEF 2021 workshop, which is co-located with the SEPLN 2021 conference. For further information, please visit <a href="https://temu.bsc.es/meddoprof/">https://temu.bsc.es/meddoprof/</a> or email us at encargo-pln-life@bsc.es</p> <p>MEDDOPROF is promoted by the Plan de Impulso de las Tecnologías del Lenguaje de la Agenda Digital (Plan TL) and the Spanish government's 2020 Proyectos de I+D+i RTI Tipo A (AI4PROFHEALTH - DESCIFRANDO EL PAPEL DE LAS PROFESIONES EN LA SALUD DE LOS PACIENTES A TRAVES DE LA MINERIA DE TEXTOS (PID2020-119266RA-I00)).</p> </blockquote>
Hackathon - TF-TG literature triage Gold Standard corpus
<p>This link contains the gold standard corpus for training the systems aiming at classifying or triaging the documents (PubMed abstracts) in relevant or not relevant for TF-TG relations. Note that this is not purely a curation triage, but we are primarily looking whether the abstracts do describe/characterize TF-TG interactions (positive training set labeled as “1” or they don't (negative training set labeled as “0”). </p> <p>This dataset was manually classified by domain experts into these to classes.</p> <p>Note that in PubMed there is a very considerable class imbalance, implying that there is only a very small fraction of TF-TG relations relevant literature subset. The resulting classifier should be useful to triage the entire PubMed database.</p> <ul> <li> <p>Name: <a href="https://zenodo.org/record/2562939/files/greekc_triage_training_v02.tsv?download=1">greekc_triage_training_v02.tsv</a></p> </li> <li> <p>Example:</p> </li> </ul> <p>11731616 [{"sourceid":"11731616","sourcedb":"PubMed","text":"Homeobox protein Gsh-1-dependent regulation of the rat GHRH gene promoter. Although GHRH is known to play a pivotal role in the regulation of the GHRH-GH-IGF-I axis, the molecular mechanism of GHRH gene expression has not yet been examined. Here we studied the transcriptional regulation of the GHRH gene 5'promoter using an in vitro experimental model system. We especially focused on the role of homeobox transcriptional factor Gsh-1, because a dwarf phenotype and abolished GHRH expression was observed in Gsh-1 knockout mice. First, we cloned human Gsh-1, which showed 87.3% homology with mouse Gsh-1 at the nucleotide level. When the 5'-promoter region of the rat GHRH gene was introduced into the human placental cell line JEG-3, in which we found the endogenous expression of Gsh-1 as well as GHRH mRNA, substantial transcriptional activity of the promoter was recognized. Promoter activity was further enhanced by overexpression of Gsh-1 protein, whereas it was substantially reduced by elimination of Gsh-1 binding sites. EMSA confirmed the actual binding of Gsh-1 on the multiple binding sites of GHRH gene promoter. Finally, coexpression of CREB-binding protein significantly enhanced the Gsh-1-induced GHRH gene expression, suggesting the cooperative role of the coactivator protein. Because Gsh-1 is found to be expressed in the hypothalamus of the adult rat, our data provide evidence that the Gsh-1 homeobox protein plays a key role in the expression of the GHRH gene.","denotations":[{"obj":"Gene:29446","span":{"begin":84,"end":88}},{"obj":"Gene:288457","span":{"begin":17,"end":22}},{"obj":"Gene:29446","span":{"begin":55,"end":59}},{"obj":"Gene:29446","span":{"begin":146,"end":150}},{"obj":"Gene:29446","span":{"begin":193,"end":197}},{"obj":"Gene:29446","span":{"begin":295,"end":299}},{"obj":"Gene:14842","span":{"begin":430,"end":435}},{"obj":"Gene:14601","span":{"begin":477,"end":481}},{"obj":"Gene:14842","span":{"begin":509,"end":514}},{"obj":"Gene:219409","span":{"begin":553,"end":558}},{"obj":"Gene:14842","span":{"begin":599,"end":604}},{"obj":"Gene:29446","span":{"begin":669,"end":673}},{"obj":"Gene:219409","span":{"begin":783,"end":788}},{"obj":"Gene:2691","span":{"begin":800,"end":804}},{"obj":"Gene:288457","span":{"begin":940,"end":945}},{"obj":"Gene:288457","span":{"begin":1010,"end":1015}},{"obj":"Gene:288457","span":{"begin":1068,"end":1073}},{"obj":"Gene:29446","span":{"begin":1107,"end":1111}},{"obj":"Gene:54244","span":{"begin":1152,"end":1172}},{"obj":"Gene:288457","span":{"begin":1200,"end":1205}},{"obj":"Gene:29446","span":{"begin":1214,"end":1218}},{"obj":"Gene:288457","span":{"begin":1304,"end":1309}},{"obj":"Gene:288457","span":{"begin":1408,"end":1413}},{"obj":"Gene:29446","span":{"begin":1473,"end":1477}}]}] 1</p> <ul> <li> <p>Format: tsv-separated columns (PMID, PubAnnotation JSON formated results of Pubtator for this record together with the automatically detected gene mentions using GnormPlus providing the Entrez Gene Identifiers together with the mention offsets, i.e. start and end character positions, corresponding class label of the records (1= relevant, 0=nonrelevant))</p> </li> <li> <p>PubAnnotation format description: <a href="http://www.pubannotation.org/docs/annotation-format/">http://www.pubannotation.org/docs/annotation-format/</a></p> </li> <li> <p>PubTator record retrieval description:</p> </li> </ul> <p>https://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/tmTools/curl.html</p> <p><strong>Remember:</strong> One must notice that this dataset is high imbalanced, with very few cases of the positive class. The additional files could also be explored to build the model or even to build a classifier working at the level of sentences instead of abstracts,</p> <p> </p>
MedProcNER Corpus: Gold Standard annotations for Clinical Procedures Information Extraction
<p><strong>MedProcNER</strong> stands for MEDical PROCedure Named Entity Recognition. It is a shared task and set of resources focused on the <strong>detection, normalization and indexing of clinical procedures in medical documents</strong> in <strong>Spanish</strong>. MedProcNER is complementary to the DisTEMIST corpus (https://temu.bsc.es/distemist) as they both use the same document collection.</p><p><strong>Please cite if you use this dataset:</strong></p><p>Lima-López S, Farré-Maduell E, Gascó L, Nentidis A, Krithara A, Katsimpras G, Paliouras G, Krallinger M. Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. Working Notes of CLEF. 2023.</p><p>@article{lima2023overview, title={Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023}, author={Lima-L{\'o}pez, Salvador and Farr{\'e}-Maduell, Eul{\`a}lia and Gasc{\'o}, Luis and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin}, journal={Working Notes of CLEF}, year={2023} }</p><p>This repository includes the Train Set of the task, which includes a total of 750 documents, plus the annotated Test Set's 250 documents. A gazetteer of possible <strong>SNOMED CT codes</strong> for the normalization and indexing tasks is also part of the bundle as a lexical resource. </p><p>In addition, a cross-mapping file of all SNOMED CT codes to <strong>MeSH</strong> is also included.</p><p>Finally, we release an experimental <strong>multilingual Silver Standard</strong> version derived from the Spanish Gold Standard in <strong>9 languages</strong>: <i><strong>English</strong></i>, <i><strong>Catalan</strong></i>, <i><strong>Italian</strong></i>, <i><strong>French</strong></i>, <i><strong>Portuguese</strong></i>, <i><strong>Romanian</strong></i>, <i><strong>Czech</strong></i>, <i><strong>Dutch</strong></i> and <i><strong>Swedish</strong></i>. </p><p>These documents have been generated using an automatic annotation transfer process that works as follows:</p><ol><li>The text files were translated with a neural machine translation system.</li><li>The annotations were translated with the same neural machine translation system.</li><li>The translated annotations were transferred to the translated text files using a lexical approach and custom dictionaries.</li></ol><p>MedProcNER was developed by the Barcelona Supercomputing Center's NLP for Biomedical Information Analysis and used as part of BioASQ @ CLEF 2023. For more information on the corpus, annotation scheme and task in general, please visit: <a href="https://temu.bsc.es/medprocner">https://temu.bsc.es/medprocner</a>.</p><p> </p><p><strong>Resources:</strong></p><ul><li><a href="https://temu.bsc.es/medprocner"><strong>Web</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/paper-002.pdf"><strong>Citation</strong></a><strong>: </strong>Lima-López S, Farré-Maduell E, Gascó L, Nentidis A, Krithara A, Katsimpras G, Paliouras G, Krallinger M. Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. Working Notes of CLEF. 2023.</li><li><a href="https://doi.org/10.5281/zenodo.7817666"><strong>Annotation guidelines</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/"><strong>Proceedings and participant papers</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/paper-002.pdf"><strong>Overview paper</strong></a></li><li><a href="https://www.slideshare.net/MartinKrallinger/medprocnerproctemist-shared-task-on-clinical-procedure-detection-and-normalization-task-overview-presentation-at-bioasq-workshop-of-clef-2023-conference"><strong>Overview talk slides at BioASQ/CLEF</strong></a></li></ul><p> </p><p><strong>Additional resources and corpora</strong></p><p>If you are interested in MedProcNER, you might want to check out these corpora and resources:</p><ul><li><a href="https://zenodo.org/records/7614764">DisTEMIST</a> (Corpus of disease mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/8413866">SympTEMIST </a>(Corpus of symptoms, signs and findings mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/4270158">PharmaCoNER</a> (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization, same document collection)</li><li><a href="https://zenodo.org/records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/4279323">MEDDOCAN</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, modified synthetic verions of the document collection)</li><li><a href="https://zenodo.org/records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization, different document collection)</li><li><a href="https://zenodo.org/records/3837305">CodiESp</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)</li><li><a href="https://zenodo.org/records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)</li><li><a href="https://zenodo.org/records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)</li></ul><p><strong>License</strong></p><p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p><p><strong>Contact</strong></p><p>If you have any questions or suggestions, please contact us at:</p><p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p>
Cantemist corpus: gold standard of oncology clinical cases annotated with CIE-O 3 terminology
<p><strong>Intro:</strong></p> <p>Cantemist shared task dataset (divided in train, dev1, dev2 and test). In addition, we include here the Cantemist background set.</p> <p>It contains the train, development and test sets of the three subtasks: cantemist-ner, cantemist-norm and cantemist-coding with Gold Standard annotations.</p> <p>In addition, it contains the documents of the background set, without annotations.</p> <p> </p> <p><strong>Please cite if you use this dataset:</strong></p> <p>Miranda-Escalada, A., Farré, E., & Krallinger, M. (2020). Named entity recognition, concept normalization and clinical coding: Overview of the cantemist track for cancer text mining in spanish, corpus, guidelines, methods and results. In <em>Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020), CEUR Workshop Proceedings</em>.</p> <pre><code>@inproceedings{miranda2020named, title={Named entity recognition, concept normalization and clinical coding: Overview of the cantemist track for cancer text mining in spanish, corpus, guidelines, methods and results}, author={Miranda-Escalada, A and Farr{\'e}, E and Krallinger, M}, booktitle={Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020), CEUR Workshop Proceedings}, year={2020} }</code></pre> <p> </p> <p><strong>Format:</strong></p> <p>For subtasks cantemist-norm and cantemist-ner, annotations are distributed in Brat format. See <a href="https://brat.nlplab.org/standoff.html">Brat webpage</a> for more information</p> <p>For subtask cantemist-coding, codes are grouped in a TSV file with the following columns (this follows the format used in <a href="https://temu.bsc.es/codiesp/">CodiEsp shared task</a>): </p> <blockquote> <p>filename code</p> </blockquote> <p> </p> <p><strong>Shared task goal:</strong></p> <p>In the three subtasks, the goal will be to predict the annotations (either the ANN files or the TSV with the codes) given only the plain text files. </p> <p> </p> <p><strong>Resources:</strong></p> <ul> <li><strong><a href="https://temu.bsc.es/cantemist/">Web</a></strong></li> <li><strong><a href="http://ceur-ws.org/Vol-2664/cantemist_overview.pdf">Citation</a>: </strong>Miranda-Escalada, A., Farré, E., & Krallinger, M. (2020). Named entity recognition, concept normalization and clinical coding: Overview of the cantemist track for cancer text mining in spanish, corpus, guidelines, methods and results. In <em>Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2020), CEUR Workshop Proceedings</em>.</li> <li><strong><a href="https://doi.org/10.5281/zenodo.4010899">Silver Standard corpus</a></strong></li> <li><strong><a href="https://doi.org/10.5281/zenodo.3878178">Annotation guidelines</a></strong></li> <li><a href="https://www.youtube.com/playlist?list=PL5uSCzf1azhC24g5dsp5eVMp8BZFWCraX"><strong>YouTube presentations</strong></a></li> <li><a href="https://temu.bsc.es/cantemist/?p=4606"><strong>Participant codes</strong></a></li> </ul> <p> </p> <p>For further information, please visit <a href="https://temu.bsc.es/cantemist/">https://temu.bsc.es/cantemist/</a> or email us at encargo-pln-life@bsc.es</p>
SympTEMIST Corpus: Gold Standard annotations for clinical symptoms, signs and findings information extraction
<p><strong>SympTEMIST</strong> stands for Symptoms TExt MIning Shared Task. It is a shared task and set of resources focused on the <strong>detection of mentions, normalization and indexing of symptoms, signs and findings in medical documents</strong> in <strong>Spanish</strong>. SympTEMIST is complementary to the DisTEMIST (https://temu.bsc.es/distemist) and MedProcNER/ProcTEMIST (https://temu.bsc.es/medprocner) corpora as they all use the same document collection.</p> <p> </p> <p><strong>Please cite if you use this dataset:</strong></p> <p>"Lima-López, S., Farré-Maduell, E., Gasco-Sánchez, L., Rodríguez-Miret, J. and Krallinger, M. (2023). Overview of SympTEMIST at BioCreative VIII: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text. In: <em>Proceedings of the BioCreative VIII Challenge and Workshop: Curation and Evaluation in the era of Generative Models</em>."</p> <p> </p> <p>This repository includes the:</p> <ul> <li><strong>Train </strong>and<strong> Test Set </strong>for the three subtasks</li> <li><strong>SYMPTEMIST gazetteer </strong>of <strong>SNOMED symptoms, signs & findings </strong></li> <li><strong>Multilingual </strong>Silver Standard<strong> </strong>in <strong>9 languages:</strong> <ul> <li><em><strong>English</strong></em></li> <li><em><strong>Portuguese</strong></em></li> <li><em><strong>French</strong></em></li> <li><em><strong>Italian</strong></em></li> <li><em><strong>Romanian</strong></em></li> <li><em><strong>Catalan</strong></em></li> <li><em><strong>Swedish</strong></em></li> <li><em><strong>Dutch</strong></em></li> <li><em><strong>Czech</strong></em></li> </ul> </li> <li><strong>Background set</strong> of over 15,000 clinical cases.</li> <li><strong>Spanish</strong> Silver Standard (predictions by participants over the background set)</li> </ul> <p> </p> <p>Please read the README file attached for more information on folder structure and file format.</p> <p> </p> <p>SympTEMIST was developed by the Barcelona Supercomputing Center's NLP for Biomedical Information Analysis and used as part of BioCREATIVE 2023. For more information on the corpus, annotation scheme and task in general, please visit: https://temu.bsc.es/symptemist.</p> <p> </p> <p><strong>Resources:</strong></p> <ul> <li><a href="https://temu.bsc.es/symptemist/"><strong>Task web</strong></a></li> <li><a href="https://biocreative.bioinformatics.udel.edu/"><strong>BioCreative web</strong></a></li> <li><strong>Citation: </strong>Lima-López, S., Farré-Maduell, E., Gasco-Sánchez, L., Rodríguez-Miret, J. and Krallinger, M. (2023). Overview of SympTEMIST at BioCreative VIII: corpus, guidelines and evaluation of systems for the detection and normalization of symptoms, signs and findings from text. In: <em>Proceedings of the BioCreative VIII Challenge and Workshop: Curation and Evaluation in the era of Generative Models</em>.</li> <li><a href="../records/8246440"><strong>Annotation guidelines</strong></a></li> <li><a href="../records/10103191"><strong>BioCreative/AMIA workshop proceedings</strong></a></li> <li><a href="../doi/10.5281/zenodo.10104546"><strong>Overview paper</strong></a></li> <li><a href="https://www.youtube.com/playlist?list=PLyLDDulunoofv79Pci0-y2rGP5GOBoZhp"><strong>Youtube videos (overview & teams)</strong></a></li> <li><a href="https://www.slideshare.net/MartinKrallinger/symptemist-shared-task-on-symptoms-signs-and-findings-detection-and-normalization-task-overview-talk-at-biocreative-viii-workshop-amia-2023"><strong>SympTEMIST overview talk slides at BioCreative/AMIA workshop</strong></a></li> </ul> <p> </p> <p><strong>License</strong></p> <p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p> <p><strong>Contact</strong></p> <p>If you have any questions or suggestions, please contact us at:</p> <p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p> <p><strong>Additional resources and corpora</strong></p> <p>If you are interested in SympTEMIST, you might want to check out these corpora and resources:</p> <ul> <li><a href="../records/7614764">DisTEMIST</a> (Corpus of disease mentions and normalization to SNOMED CT, same document collection)</li> <li><a href="../records/8224056">MedProcNER </a>(Corpus of clinical procedure mentions and normalization to SNOMED CT, same document collection)</li> <li><a href="../records/4270158">PharmaCoNER</a> (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization, same document collection)</li> <li><a href="../records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)</li> <li><a href="../records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)</li> <li><a href="../records/4279323">MEDDOCAN</a> (Corpus of mentions of Personal Health Identifiers (PHI), modified synthetic verions of the document collection)</li> <li><a href="../records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization, different document collection)</li> <li><a href="../records/3837305">CodiESp</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)</li> <li><a href="../records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)</li> <li><a href="../records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)</li> <li><a href="../records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)</li> <li><a href="../records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)</li> <li><a href="../records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)</li> </ul> <p> </p>
Hackathon - TF-TG relation annotation Gold Standard Corpus
<p>TF-TG relation annotation Gold Standard Corpus.</p> <p>The file contains 130 PMIDs annotated in the following way:</p> <ul> <li>PMID.txt: the original TXT file with the abstract.</li> <li>PMID.ann: the file with the BRAT annotations. Each annotation file contains terms (e.g. T1, T2, ...) and relations (e.g. R1, R2, ...), where each relation is from one term (e.g. T1) to another term (e.g. T2).</li> </ul>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.