Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
48
datasets available to search
ShareScore release 0.9.0
Dataset results
48 results for “Information Extraction”
Invasion Biology WikiProject Scientific Papers: Text Data Mining and LLM-based Information Extraction of Species, Locations, Habitats, and Ecosystems
<p>This dataset contains the abstract and full-text for publication DOIs from the Invasion Biology WikiProject (DOI: <a href="https://www.doi.org/10.5281/zenodo.12518036">10.5281/zenodo.12518036</a>). The data was retrieved using the <a href="https://ask.orkg.org/">ask.orkg.org</a> <a href="https://api.ask.orkg.org/docs#tag/Semantic-Neural-Search/operation/explore_documents_index_explore_get">API</a>. For the <a href="https://github.com/jd-coderepos/invasion-biology-IE/blob/main/scripts/ask-doi-list-fulltext-search.py">script</a> used to obtain the data, refer to the accompanying GitHub repository: <a href="https://github.com/jd-coderepos/invasion-biology-IE/" target="_blank" rel="noopener">https://github.com/jd-coderepos/invasion-biology-IE/</a>.</p> <p>The resulting CSV file includes the following fields: <code>"ASK ID"</code>, <code>"DOI"</code>, <code>"Title"</code>, <code>"Abstract"</code>, and <code>"Full-text"</code>.</p> <p>Of the 49,438 queried DOIs, the ASK database provided:</p> <ul> <li><strong>Total DOIs processed:</strong> 12,636</li> <li><strong>DOIs with neither abstract nor full-text:</strong> 36 (abstract token count was less than 10)</li> <li><strong>DOIs with abstracts but no full-text:</strong> 12,636</li> <li><strong>DOIs with both abstract and full-text:</strong> 2,834</li> </ul> <p>The second part of the dataset contains structured information extracted from the publications using the GPT-4o Large Language Model. This structured data is included in the zipped folder <code>structured-publications.zip</code>.</p> <p>The accompanying GitHub repository provides access to the code and scripts used at various stages of the information extraction (IE) process.</p> <p><strong>Theme of the Study:</strong><br>"Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models."</p>
DrCyZ: Techniques for analyzing and extracting useful information from CyZ.
<p>DrCyZ: Techniques for analyzing and extracting useful information from CyZ.</p> <p>Samples from NASA Perseverance and set of GAN generated synthetic images from Neural Mars.</p> <p>Repository: <a href="https://github.com/decurtoidiaz/drcyz">https://github.com/decurtoidiaz/drcyz</a></p> <p><br> Subset of samples from (includes tools to visualize and analyse the dataset):</p> <p>CyZ: MARS Space Exploration Dataset. [<a href="https://doi.org/10.5281/zenodo.5655473">https://doi.org/10.5281/zenodo.5655473</a>]</p> <p>Images from NASA missions of the celestial body.</p> <p>Repository: <a href="https://github.com/decurtoidiaz/cyz">https://github.com/decurtoidiaz/cyz</a></p> <p>Authors:</p> <p>J. de Curtò c@decurto.be</p> <p>I. de Zarzà z@dezarza.be</p> <p>------------------------------------------<br> File Information from DrCyZ-1.1<br> ------------------------------------------</p> <p> • Subset of samples from Perseverance (drcyz/c).<br> ∙ png (drcyz/c/png).<br> PNG files (5025) selected from NASA Perseverance (CyZ-1.1) after t-SNE and K-means Clustering. <br> ∙ csv (drcyz/c/csv).<br> CSV file.</p> <p> • Resized samples from Perseverance (drcyz/c+).<br> ∙ png 64x64; 128x128; 256x256; 512x512; 1024x1024 (drcyz/c+/drcyz_64-1024).<br> PNG files resized at the corresponding size. <br> ∙ TFRecords 64x64; 128x128; 256x256; 512x512; 1024x1024 (drcyz/c+/tfr_drcyz_64-1024).<br> TFRecord resized at the corresponding size to import on Tensorflow.</p> <p> • Synthetic images from Neural Mars generated using Stylegan2-ada (drcyz/drcyz+).<br> ∙ png 100; 1000; 10000 (drcyz/drcyz+/drcyz_256_100-10000)<br> PNG files subset of 100, 1000 and 10000 at size 256x256.</p> <p> • Network Checkpoint from Stylegan2-ada trained at size 256x256 (drcyz/model_drcyz).<br> ∙ network-snapshot-000798-drcyz.pkl</p> <p> • Notebooks in python to analyse the original dataset and reproduce the experiments; K-means Clustering, t-SNE, PCA, synthetic generation using Stylegan2-ada and instance segmentation using Deeplab (<a href="https://github.com/decurtoidiaz/drcyz/tree/main/dr_cyz+">https://github.com/decurtoidiaz/drcyz/tree/main/dr_cyz+</a>).<br> ∙ clustering_curiosity_de_curto_and_de_zarza.ipynb<br> K-means Clustering and PCA(2) with images from Curiosity.<br> ∙ clustering_perseverance_de_curto_and_de_zarza.ipynb<br> K-means Clustering and PCA(2) with images from Perseverance.<br> ∙ tsne_curiosity_de_curto_and_de_zarza.ipynb<br> t-SNE and PCA (components selected to explain 99% of variance) with images from Curiosity.<br> ∙ tsne_perseverance_de_curto_and_de_zarza.ipynb<br> t-SNE and PCA (components selected to explain 99% of variance) with images from Perseverance.<br> ∙ Stylegan2-ada_de_curto_and_de_zarza.ipynb<br> Stylegan2-ada trained on a subset of images from NASA Perseverance (DrCyZ).<br> ∙ statistics_perseverance_de_curto_and_de_zarza.ipynb<br> Compute statistics from synthetic samples generated by Stylegan2-ada (DrCyZ) and images from NASA Perseverance (CyZ).<br> ∙ DeepLab_TFLite_ADE20k_de_curto_and_de_zarza.ipynb<br> Example of instance segmentation using Deeplab with a sample from NASA Perseverance (DrCyZ).</p>
Single-cell information extracted from IMC example data
<p>If you are working with these files, please cite them as follows:<br><br>Windhager, J., Zanotelli, V.R.T., Schulz, D. et al. An end-to-end workflow for multiplexed image processing and analysis. Nat Protoc (2023). <a href="https://doi.org/10.1038/s41596-023-00881-0">https://doi.org/10.1038/s41596-023-00881-0</a></p><p>This repository contains additional information related to IMC example data available at <a href="https://zenodo.org/record/5949116">zenodo.org/record/5949116</a>. The following files are available and are part of the <a href="https://bodenmillergroup.github.io/IMCDataAnalysis/">IMC Data Analysis workflow</a></p><ul><li><strong>gated_cells.zip:</strong> contains SpatialExperiment objects storing cells that were manually gated based on their expression values to derive ground truth cell phenotype labels.</li><li><strong>spe.rds:</strong> SpatialExperiment object containing the single-cell information (mean intensity per cell and per channel; cellular metadata; channel metadata) of the processed data.</li><li><strong>images.rds:</strong> CytoImageList object containing the spillover-corrected images.</li><li><strong>masks.rds:</strong> CytoImageList object containing the segmentation masks.</li></ul>
TrainTicket microservice testbench extracted information for our work: Evaluating ChatGPT's Proficiency in Understanding and Answering Microservice Architecture Queries Using Source Code Insights
<p>It contains the CSV file output of our tool implemented in the paper: "Evaluating ChatGPT’s Proficiency in Understanding and Answering Microservice Architecture Queries Using Source Code Insights." applied to the TrainTicket microservice testbench. The information in this CSV was used for In-Context-Learning for ChatGPT.</p>
Gold standard annotation of epilepsy clinic letters for the development of information extraction tools
<p>This folder contains 200 synthetic letters, based on hospital outpatient epilepsy clinic consultations, written by neurology consultants, specialist trainees, and epilepsy specialist nurses. The letters were double annotated by trained researchers, according to annotation guidelines, uploaded separately (What and How of annotating with Markup). The 200 .ann annotation files are also included.</p> <p>We used Markup (https://getmarkup.com) for annotation (the configuration file is within the uploaded set of annotations) with an epilepsy concept list based on the Unified Medical Language System (UMLS) ontology. All annotations were compared, reviewed, and corrected to form a gold standard annotation set.</p> <p> </p>
Zero-Shot Information Extraction to Enhance a Knowledge Graph Describing Silk Textiles - English and Spanish neighborhood sub-graphs
<p>Two language-specific sub-graphs (English and Spanish) based on the ConceptNet Knowledge Graph. These two files are required to run the code for reproducing the results reported in the paper <a href="https://aclanthology.org/2021.latechclfl-1.16/">"Zero-Shot Information Extraction to Enhancea Knowledge Graph Describing Silk Textiles"</a> at the <a href="https://sighum.wordpress.com/events/latech-clfl-2021/">LaTeCH-CLfL 2021</a> workshop co-located with <a href="https://2021.emnlp.org/">EMNLP 2021</a>.</p>
An annotated corpus of clinical trial publications supporting schema-based relational information extraction
<p>Repository of an annotated corpus of clinical trial abstracts supporting schema-based relational information extraction and the code for the inter-annotation agreement calculation and the baseline information extraction method.</p>
Extracting abundance information from DNA-based data
<p><span><span><span><span>The accurate extraction of species-abundance information from DNA-based data (metabarcoding, metagenomics) could contribute usefully to the reconstruction of diets and quantitative foodwebs, the inference of species interactions, the modelling of population dynamics and species distributions, the biomonitoring of environmental state and change, and the inference of false positives and negatives. However, capture bias, capture noise, species pipeline biases, and pipeline noise all combine to inject error into DNA-based datasets. This review focuses on methods for correcting the latter two error sources, as the first two are addressed extensively in the ecological survey literature. To extract abundance information from DNA-based data, it is useful to distinguish two concepts. (1) <em>Across</em>-species quantification describes relative species abundances within a single sample. (2) In contrast, <em>within</em>-species quantification describes how the abundance of each individual species varies across samples, where the samples could be a time series, an environmental gradient, or different experimental treatments. In the first part of this paper, we review methods to remove species pipeline biases and pipeline noise. In the second part, we provide a detailed protocol and demonstrate experimentally how to use a 'DNA spike-in' (an internal standard) to remove pipeline noise and recover within-species abundance information.</span></span></span></span></p>
IXI – Information eXtraction from Images | Cortical Volume
<p><strong>This dataset contains cortical volume data, computed from the T1-w images present in the original IXI dataset.</strong></p> <p>Structural pre-processing was conducted using FreeSurfer 6.0, with default parameters, and includes the following steps:</p> <ul> <li>motion correction</li> <li>skull stripping</li> <li>removal of cerebellum and brain stem</li> <li>intensity correction</li> <li>segmentation</li> <li>tassellation</li> <li>smoothing </li> <li>topology correction</li> </ul> <p>Cortical volume was produced for 68 brain regions from the Desikan-Killiany brain atlas (34 per hemisphere, measured in mm3)</p> <p>The original data has been collected as part of the project: <strong>IXI – Information eXtraction from Images (EPSRC GR/S21533/02)<br></strong>Information and data for the original IXI dataset can be found<strong> </strong><a href="https://brain-development.org/ixi-dataset/" target="_blank" rel="noopener">here</a>.</p> <p>This data is made available under the Creative Commons CC BY-SA 3.0 license. If you use the IXI data please acknowledge the source of the IXI data.</p>
The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction - dataset
<p><strong>Brief description</strong></p> <p>The zip file contains two folders. The <strong>"websites"</strong> folder includes crawled web pages from real websites, like a agatameble.pl (an e-shop website), filmweb.pl (a website about films), and ptaki.info (a website about birds). The <strong>"reference-seeds"</strong> folder contains three subfolders, i.e. agatameble.pl, filmweb.pl, and ptaki.info. Each subfolder contains reference-seeds.csv file. The file contains data, i.e. reference instances - carefully labelled ground-truth of corresponding values in each web page of given websites mentioned above.</p> <p><strong>Reference</strong></p> <p>I would appreciate it if you cite the following paper when using the dataset:</p> <p>Marcin Mirończuk The BigGrams: the semi-supervised information extraction system from HTML: an improvement in the wrapper induction, Knowledge and Information Systems, Volume 54, Issue 3, p. 711–776, 2018, (pdf Open Access – http://rdcu.be/u88F lub DOI http://dx.doi.org/10.1007/s10115-017-1097-2)</p>
Relations from Italian Wikipedia using Unsupervised Information Extraction
<p>This dataset contains relations extracted from the Italian Wikipedia by the WikiOIE framework.<br> WikiOIE is based on UDPipe and the Universal Dependencies project for text processing.<br> It easily allows customizing the information extraction (IE) approach to automatically extract triples (subject, predicate, object).<br> This dataset contains relations extracted by two unsupervised IE methods. The former (<strong>simple</strong>) is based only on PoS-tag patterns; the latter (<strong>simpledep</strong>) also uses syntactic dependencies. <br> The extraction process is provided in JSON format.</p> <p>More information and the Java code are available here https://github.com/pippokill/WikiOIE</p> <p>Pierluigi Cassotti, Lucia Siciliani, Pierpaolo Basile,Marco de Gemmis, and Pasquale Lops. 2021. Extracting relations from Italian Wikipedia using unsupervised information extraction. In Proceedings of the 11th Italian Information Retrieval Workshop 2021 (IIR 2021). CEUR-WS.</p>
OIE4PA: Open Information Extraction for the Public Administration
<p>Tenders are powerful means of investment of public funds and represent a strategic development resource.<br> Despite the efforts made so far by governments at national and international levels to digitalise documents related to the Public Administration sector, most of the information is still available in an unstructured format only. <br> With the aim of bridging this gap, we present OIE4PA, our latest study on extracting and classifying relations from tenders of the Public Administration.<br> Our work focuses on the Italian language, where the availability of linguistic resources to perform Natural Language Processing tasks is considerably limited. <br> For evaluation purposes, we built a dataset composed of 2,000 triples extracted from Italian tenders, which have been manually annotated by two human experts.</p> <p>The dataset, compressed in a single zip file, is composed of:</p> <ul> <li>The corpus of 6,262 texts extracted from Italian public tenders (corpus_tenders)</li> <li>The training set of 1,600 annotated triples (training_set)</li> <li>The test set of 400 annotated triples (test_set)</li> <li>The set U of 14,096 triples used for the self-training (u_triples_dd)</li> <li>a compressed archive that contains both the extracted triples and the index for each supervised approach (extraction) <br> </li> </ul>
Extracting abundance information from DNA-based data
Open the record for dataset details and reuse information.
Information Extraction in Handwritten Historical Logbooks
<p>Contains the datasets with tables used in the following paper: Information Extraction in Handwritten Historical Logbooks.</p> <p>The pages starting with "vol003" correspond to the Jeannette corpus, while the ones starting with "Albatross" correspond to the Albatross corpus.</p>
A machine learning framework for extracting information from biological pathway images in the literature
<p><strong>466 target chemicals_selected chemicals:</strong></p> <p>Target chemicals satisfying the criteria for biochemical reactions not covered by MetaNetX and KEGG.</p> <p> </p> <p><strong>466 target chemicals_statistics:</strong><br>Numbers of MetaCyc reactions, papers, and pathway images collected for 466 target chemicals from the bio-based chemicals map (Jang et al., Trends in Biotechnology, 2023).</p> <p> </p> <p><strong>arrow detection_bounding box labels:</strong><br>Bounding box labels for 6,471 images in the training and validation datasets and 100 images in the test dataset. The corresponding images are available in "arrow detection_training and validation datasets.zip".</p> <p> </p> <p><strong>arrow detection_test dataset:</strong></p> <p>Test dataset for arrow detection using Faster R-CNN model. A total of 100 images have been prepared from 89 papers searched through PubMed Central (PMC).</p> <p> </p> <p><strong>arrow detection_training and validation datasets:</strong></p> <p>Training and validation datasets for arrow detection using Faster R-CNN model. A total of 6,471 images have been prepared, including 2,332 images from five different sources and 4,139 augmented images.</p> <p> </p> <p><strong>EBPI outputs:</strong></p> <p>Reaction information extracted using EBPI from 49,846 biological pathway images across 466 target chemicals.</p> <p> </p> <p><strong>text classification_training, validation and test datasets:</strong></p> <p>Dataset for text classification using BioBERT. A total of 59,370 terms have been prepared, including 15,101 “gene” terms, 21,417 “protein” terms, and 22,852 “others” terms by combining the data from MetaCyc and the PaddleOCR results from the papers.</p>
Data and Scripts for the Article "Structural Descriptors and Information Extraction from X-ray Emission Spectra: Aqueous Sulfuric Acid"
<p>Data and scripts for the article titled "Structural Descriptors and Information Extraction from X-ray Emission Spectra: Aqueous Sulfuric Acid".</p> <p>For further details on the contents, see the "readme.md"-file.</p> <p>Article available at <a href="https://doi.org/10.1039/D4CP02454K">10.1039/D4CP02454K</a>.</p>
Geoparsing with Large Language Models: Leveraging the linguistic capabilities of generative AI to improve geographic information extraction
<h2>Geoparsing with Large Language Models</h2> <p>The .zip file included in this repository contains all the code and data required to reproduce the results from our paper. Note, however, that in order to run the OpenAI models, users will required an OpenAI API key and sufficient API credits.</p> <div> <h3>Data</h3> <p>The data used for the paper are in the <code>datasetst</code> and <code>results</code> folders.</p> <ul> <li> <p>**Datasets: **This contains the XML files (LGL and Geovirus) and Json files (News2024) used to benchmark the models. It also contains all the data used to fine-tune the gpt-3.5 model, the prompt templates sent to the LLMs, and other data used for mapping and data creation.</p> </li> <li> <p>**Results: **This contains the results for the models on the three datastes. The folder is separated by dataset, with a single <code>.csv</code> file giving the results for each model on each dataset separately. The <code>.csv</code> file is structured so that each row contains either a predicted toponym and an associated true toponym (along with assigned spatial coordinates), if the model correctly identified a toponym; otherwise the true toponym columns are empty for false positives and the predicted columns are empty for false negatives.</p> </li> </ul> <h3>Code</h3> <p>The code is split into two seperate folders <code>gpt_geoparser</code> and <code>notebooks</code>.</p> <ul> <li>**GPT_Geoparser: **this contains the classes and methods used process the XML and JSON articles (<code>data.py</code>), interact with the Nominatim API for geocoding (<code>gazetteer.py</code>), interact with the OpenAI API (<code>gpt_handler.py</code>), process the outputs from the GPT models (<code>geoparser.py</code>) and analyse the results (<code>analysis.py</code>).</li> <li><strong>Notebooks</strong>: This series of notebooks can be used to reproduce the results given in the paper. The file names a reasonably descriptive of what they do within the context of the paper.</li> </ul> <h3>Code/software</h3> <h3>Requirements</h3> <ul> <li>Numpy</li> <li>Pandas</li> <li>Geopy</li> <li>Scitkit-learn</li> <li>lxml</li> <li>openai</li> <li>matplotlib</li> <li>Contextily</li> <li>Shapely</li> <li>Geopandas</li> <li>tqdm</li> <li>huggingface_hub</li> <li>Gnews</li> </ul> <h3>Access information</h3> <p>Other publicly accessible locations of the data:</p> <ul> <li>The LGL and GeoVirus datasets can also be obtained <a href="https://github.com/milangritta/Pragmatic-Guide-to-Geoparsing-Evaluation" target="_blank" rel="noopener">here<span> (opens in new window)</span></a>.</li> </ul> <h3>Abstract</h3> <div> <p>Geoparsing- the process of associating textual data with geographic locations - is a key challenge in natural language processing. The often ambiguous and complex nature of geospatial language make geoparsing a difficult task, requiring sophisticated language modelling techniques. Recent developments in Large Language Models (LLMs) have demonstrated their impressive capability in natural language modelling, suggesting suitability to a wide range of complex linguistic tasks. In this paper, we evaluate the performance of four LLMs - GPT-3.5, GPT-4o, Llama-3.1-8b and Gemma-2-9b - in geographic information extraction by testing them on three geoparsing benchmark datasets: GeoVirus, LGL, and a novel dataset, News2024, composed of geotagged news articles published outside the models' training window. We demonstrate that, through techniques such as fine-tuning and retrieval-augmented generation, LLMs significantly outperform existing geoparsing models. The best performing models achieve a toponym extraction F1 score of 0.985 and toponym resolution accuracy within 161 km of 0.921. Additionally, we show that the spatial information encoded within the embedding space of these models may explain their strong performance in geographic information extraction. Finally, we discuss the spatial biases inherent in the models' predictions and emphasize the need for caution when applying these techniques in certain contexts.</p> </div> <h3>Methods</h3> <div> <p>This contains the data and codes required to reproduce the results from our paper. The LGL and GeoVirus datasets are pre-existing datasets, with references given in the manuscript. The News2024 dataset was constructed specifically for the paper. </p> <p>To construct the News2024 dataset, we first created a list of 50 cities from around the world which have population greater than 1000000. We then used the GNews python package <a href="https://pypi.org/project/gnews/" target="_blank" rel="noopener">https://pypi.org/project/gnews/<span> (opens in new window)</span></a> to find a news article for each location, published between 2024-05-01 and 2024-06-30 (inclusive). Of these articles, 47 were found to contain toponyms, with the three rejected articles referring to businesses which share a name with a city, and which did not otherwise mention any place names.</p> <p>We used a semi autonmous approach to geotagging the articles. The articles were first processed using a Distil-BERT model, fine tuned for named entity recognicion. This provided a first estimate of the toponyms within the text. A human reviewer then read the articles, and accepted or rejected the machine tags, and added any tags missing from the machine tagging process. We then used OpenStreetMap to obtain geographic coordinates for the location, and to identify the toponym type (e.g. city, town, village, river etc). We also flagged if the toponym was acting as a geo-political entity, as these were reomved from the analysis process. In total, 534 toponyms were identified in the 47 news articles. </p> </div> </div>
SIMARA: a database for key-value information extraction from full-page handwritten documents
<p>We propose a new database for information extraction from historical handwritten documents. The corpus includes 5,393 finding aids from six different series, dating from the 18th-20th centuries. Finding aids are handwritten documents that contain metadata describing older archives. They are stored in the National Archives of France and are used by archivists to identify and find archival documents.</p> <p><br> Each document is annotated at page-level, and contains seven fields to retrieve. The localization of each field is not available in such a way that this dataset encourages research on segmentation-free systems for information extraction.<br> </p>
Preferential information extraction from space-based passive microwave measurements enables accurate characterization of snow depth variability at continental scales
<p>This is a repository contains </p> <p>1)training data (x_data, y_data, snow_max) </p> <p>2) Developed Deep Learning model (snow_model.py)</p> <p>3) Training weights (*.hdf files)</p> <p>for publication " Preferential information extraction from space-based passive microwave measurements enables accurate characterization of snow depth variability at continental scales"</p> <p> </p>
MedProcNER Corpus: Gold Standard annotations for Clinical Procedures Information Extraction
<p><strong>MedProcNER</strong> stands for MEDical PROCedure Named Entity Recognition. It is a shared task and set of resources focused on the <strong>detection, normalization and indexing of clinical procedures in medical documents</strong> in <strong>Spanish</strong>. MedProcNER is complementary to the DisTEMIST corpus (https://temu.bsc.es/distemist) as they both use the same document collection.</p><p><strong>Please cite if you use this dataset:</strong></p><p>Lima-López S, Farré-Maduell E, Gascó L, Nentidis A, Krithara A, Katsimpras G, Paliouras G, Krallinger M. Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. Working Notes of CLEF. 2023.</p><p>@article{lima2023overview, title={Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023}, author={Lima-L{\'o}pez, Salvador and Farr{\'e}-Maduell, Eul{\`a}lia and Gasc{\'o}, Luis and Nentidis, Anastasios and Krithara, Anastasia and Katsimpras, Georgios and Paliouras, Georgios and Krallinger, Martin}, journal={Working Notes of CLEF}, year={2023} }</p><p>This repository includes the Train Set of the task, which includes a total of 750 documents, plus the annotated Test Set's 250 documents. A gazetteer of possible <strong>SNOMED CT codes</strong> for the normalization and indexing tasks is also part of the bundle as a lexical resource. </p><p>In addition, a cross-mapping file of all SNOMED CT codes to <strong>MeSH</strong> is also included.</p><p>Finally, we release an experimental <strong>multilingual Silver Standard</strong> version derived from the Spanish Gold Standard in <strong>9 languages</strong>: <i><strong>English</strong></i>, <i><strong>Catalan</strong></i>, <i><strong>Italian</strong></i>, <i><strong>French</strong></i>, <i><strong>Portuguese</strong></i>, <i><strong>Romanian</strong></i>, <i><strong>Czech</strong></i>, <i><strong>Dutch</strong></i> and <i><strong>Swedish</strong></i>. </p><p>These documents have been generated using an automatic annotation transfer process that works as follows:</p><ol><li>The text files were translated with a neural machine translation system.</li><li>The annotations were translated with the same neural machine translation system.</li><li>The translated annotations were transferred to the translated text files using a lexical approach and custom dictionaries.</li></ol><p>MedProcNER was developed by the Barcelona Supercomputing Center's NLP for Biomedical Information Analysis and used as part of BioASQ @ CLEF 2023. For more information on the corpus, annotation scheme and task in general, please visit: <a href="https://temu.bsc.es/medprocner">https://temu.bsc.es/medprocner</a>.</p><p> </p><p><strong>Resources:</strong></p><ul><li><a href="https://temu.bsc.es/medprocner"><strong>Web</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/paper-002.pdf"><strong>Citation</strong></a><strong>: </strong>Lima-López S, Farré-Maduell E, Gascó L, Nentidis A, Krithara A, Katsimpras G, Paliouras G, Krallinger M. Overview of MedProcNER task on medical procedure detection and entity linking at BioASQ 2023. Working Notes of CLEF. 2023.</li><li><a href="https://doi.org/10.5281/zenodo.7817666"><strong>Annotation guidelines</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/"><strong>Proceedings and participant papers</strong></a></li><li><a href="https://ceur-ws.org/Vol-3497/paper-002.pdf"><strong>Overview paper</strong></a></li><li><a href="https://www.slideshare.net/MartinKrallinger/medprocnerproctemist-shared-task-on-clinical-procedure-detection-and-normalization-task-overview-presentation-at-bioasq-workshop-of-clef-2023-conference"><strong>Overview talk slides at BioASQ/CLEF</strong></a></li></ul><p> </p><p><strong>Additional resources and corpora</strong></p><p>If you are interested in MedProcNER, you might want to check out these corpora and resources:</p><ul><li><a href="https://zenodo.org/records/7614764">DisTEMIST</a> (Corpus of disease mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/8413866">SympTEMIST </a>(Corpus of symptoms, signs and findings mentions and normalization to SNOMED CT, same document collection)</li><li><a href="https://zenodo.org/records/4270158">PharmaCoNER</a> (Corpus of medications, drugs, chemical substances, genes, proteins and vaccine mentions and normalization, same document collection)</li><li><a href="https://zenodo.org/records/7116201">MEDDOPROF</a> (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/8403498">MEDDOPLACE</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/4279323">MEDDOCAN</a> (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, modified synthetic verions of the document collection)</li><li><a href="https://zenodo.org/records/3978041">CANTEMIST</a> (Corpus of cancer tumor morphology mentions and normalization, different document collection)</li><li><a href="https://zenodo.org/records/3837305">CodiESp</a> (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)</li><li><a href="https://zenodo.org/records/7684093">LivingNER</a> (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)</li><li><a href="https://zenodo.org/records/2560344">SPACCC-POS</a> (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-TOKEN</a> (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)</li><li><a href="https://zenodo.org/records/2560338">SPACCC-SPLIT</a> (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)</li><li><a href="https://zenodo.org/records/5602914">MESINESP-2</a> (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)</li></ul><p><strong>License</strong></p><p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p><p><strong>Contact</strong></p><p>If you have any questions or suggestions, please contact us at:</p><p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.