Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
116
datasets available to search
ShareScore release 0.9.0
Dataset results
116 results for “Natural Language”
The Artifact of the ESEC/FSE 2023 Paper Titled "Natural Language to Code: How Far are We?"
<p>In this online repository, we release the source code of each of the selected techniques as well as the experiment results from each technique (which are stored in the <em>Results.zip</em> file). For each technique, we also provide our scripts to fine this approach on the CodeSearchNet-Python dataset. For example, <em>finetune.sh/inference.sh</em> are used to finetune/evaluate CodeBERT and they are under "CodeBERT/CodeBERT".</p> <p> </p> <p>Our evaluation dataset CodeSearchNet is a well-known benchmark and it can be downloaded on its <a href="http://github.com/github/CodeSearchNet">official webpage</a>.</p> <p> </p> <p>The code to calculate the evaluation metrics are reused from <a href="https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-to-code-trans/evaluator/CodeBLEU">CodeBLEU</a>.</p> <p> </p> <p>Below is a piece of code generated by CodeT5. In this case, CodeT5 generates a statement recurrently, which leads to the syntactic error. Despite that, the code itself fulfills certain functionalities, and that is why it can achieve a CodeBLEU of 24.9%.</p> <pre><code class="language-python">def makeMimiLocal(filename): try: with open(filename, 'rb') as f: data = f.read() except IOError: data = b'' data = data.decode('utf-8') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\x00', b'\x00') data = data.replace(b'\ </code></pre> <p> </p> <p>We also release the 100 randomly-selected queries as well as the code generated by ChatGPT in the <em>chatGPT.jsonl</em>.</p>
Natural Language Inference Dataset for Software Engineering
<p>This repository introduces a specialized NLI dataset designed to optimize the performance of the language models targeted for tackling NLP tasks related to the software engineering domain. We consider different texts from the software engineering domain and manually curated the entailment relationships among different sentences. The source of the sentence pairs includes PROMISE dataset, PURE dataset, user guides of different softwares, articles on operating systems (e.g. official documentation of Windows, Mac), databases (e.g. official documentation of MongoDB, Oracle), cyber security (MITRE documentation) and software product descriptions including AWS documentation. Our created text entailment dataset contains 10k sentence pairs manually labeled for balanced classification with the labels Entailment, Contradiction, and Neutral.</p>
Natural Language Annotations for Reasoning about Program Semantics
<p>Natural language annotations about Python statements</p> <p>The dataset is made of pairs of files sharing the prefix of the filename</p> <p>* Annotations are in JSONL format (filenames ending with '_annot'), i.e. JSON objects separated by newline ('\n') characters</p> <p>* Reference source code files are in JSON format. (filenames ending with '_code')</p> <p> </p> <p>Source dataset : Programming Puzzles (Schuster et al. 2021, NeurIPS Dataset and benchmarks track) - MIT License - https://github.com/microsoft/PythonProgrammingPuzzles</p>
Natural selection and language genes in humans
Open the record for dataset details and reuse information.
Data from: Natural language processing systems for pathology parsing in limited data environments with uncertainty estimation
Open the record for dataset details and reuse information.
Dataset for: Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks
<p>Datasets used for Requirement Engineering '24 paper, titled: Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks</p>
Supplementary Material, Towards the LLM-Based Generation of Formal Specifications from Natural-Language Contracts: Early Experiments with Symboleo
<p>This repository contains all the files used in the experiments described in the paper "Towards the LLM-Based Generation of Formal Specifications from Natural-Language Contracts: Early Experiments with Symboleo", which appeared in "RAISE 2025: Requirements engineering for AI-powered SoftwarE", an ICSE 2025 workshop, Ottawa, May 3, 2025.</p>
Supplementary materials of the paper entitled: "Generating Natural Language Requirements via Adversarial Examples in Deep Learning"
<p>Supplementary materials of the paper entitled: "Generating Natural Language Requirements via Adversarial Examples in Deep Learning"</p> <p>File A: Datasets (.txt)</p> <p> A1: Webex</p> <p> A2: Zoom</p> <p> A3: Teams</p> <p> A4: Word</p> <p> A5: PowerPoint</p> <p> A6: Excel</p> <p> </p> <p>File B: Python Code (Both ours and baseline)</p> <p> B1: adversarial_samples.ipynb</p> <p> B2: Baseline.ipynb</p> <p> </p> <p>FIle C: Result Tables (.xlsx)</p> <p> C1: Table of perturbed outputs in Webex </p> <p> C2: Table of perturbed outputs in Zoom</p> <p> C3: Table of perturbed outputs in Teams</p> <p> C4: Table of perturbed outputs in Word</p> <p> C5: Table of perturbed outputs in PowerPoint</p> <p> C6: Table of perturbed outputs in Excel (Excel)</p> <p> </p> <p>File D: Trend of Adversarial Shifts (Graphs)</p> <p> D1: Adversarial shifts of office suit (LSTM)</p> <p> D2: Adversarial shifts of video conferencing suit (LSTM)</p> <p> D3: Non-Adversarial shifts of office suit (LSTM)</p> <p> D4: Non-Adversarial shifts of video conferencing suit(LSTM)</p> <p> D5: Adversarial shifts of office suit (GRU)</p> <p> D6: Adversarial shifts of video conferencing suit (GRU)</p> <p> D7: Non-Adversarial shifts of office suit (GRU)</p> <p> D8: Non-Adversarial shifts of video conferencing suit(GRU)</p> <p> D9: Adversarial shifts of office suit (Bi-LSTM)</p> <p> D10: Adversarial shifts of video conferencing suit (Bi-LSTM)</p> <p> D11: Non-Adversarial shifts of office suit (Bi-LSTM)</p> <p> D12: Non-Adversarial shifts of video conferencing suit(Bi-LSTM)</p> <p> </p> <p>File E: Adversarial Examples (.pdf)</p> <p> E1: Adversarial vs original in Webex</p> <p> E2: Adversarial vs original in Zoom</p> <p> E3: Adversarial vs original in Teams</p> <p> E4: Adversarial vs original in Word</p> <p> E5: Adversarial vs original in PowerPoint</p> <p> E6: Adversarial vs original in Excel</p> <p> </p> <p>File F: Questionnaire</p> <p> </p>
Processed data for the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data"
<p>This is the data used to reproduce the results from "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data".</p>
Scatter plots for the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data"
<p>This file contains the test-score-vs-metric plots generated by the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data".</p>
Generalization metrics for the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data"
<p>This file contains all the generalization metrics that can be used to reproduce the results of "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data".</p>
Rank correlation results for the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data"
<p>This file contains the rank correlation results from the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data".</p>
SemEval 2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials
<p>This is the github repository hosting data and code for Task 2: Safe Biomedical Natural Language Inference for Clinical Trials at <a href="https://semeval.github.io/SemEval2024/" rel="nofollow">Semeval 2024</a>.</p> <p>For additional information about the task, please consult the official <a href="https://sites.google.com/view/nli4ct/home" rel="nofollow">website</a>.</p>
Trends in Natural Language Processing
<p>This dataset contains the supporting parsed corpus as described in the publication: "Analyzing a Decade of Evolution: Trends in<br>Natural Language Processing" </p> <p>This dataset contains a single zip containing data between the years 2010 and 2022, for the conferences:</p> <ul> <li>Meeting of the Association for Computational Linguistics (ACL)</li> <li>Conference on Empirical Methods in Natural Language Processing (EMNLP)</li> <li>American Chapter of the Association for Computational Linguistics (NAACL)</li> <li>Conference on Computational Linguistics (COLING)</li> <li>International Conference on Language Resources and Evaluation (LREC)</li> <li>Conference on Computational Natural Language Learning (CoNLL)</li> <li>European Chapter of the Association for Computational Linguistics (EACL)</li> <li>International Joint Conference on Natural Language Processing (IJCNLP)</li> </ul> <p>The data inlcuded is a PDF and a JSON file for each confernce. The JSON file is constucted from using the python packages SciPDF and PyPDF2. PyPDF2 extract all text from a page and is presented in the 'full_text' field, where as the SciPDF parser utilize machine lerarning to create a smart representation of the data, which is presented in the remaining fields.</p> <p>For further details on how this dataset was generated, please see our <a href="https://github.com/ieeta-pt/nlp-trends" target="_blank" rel="noopener">GitHub</a> repository, and our paper.</p> <p>Citation:</p> <p>AWAITING PUBLICATION</p> <p> </p>
Enhancing Name Entity Recognition Through Hybrid Deep Learning in Natural Language Processing
Open the record for dataset details and reuse information.
Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review
<p>Supplementary material outlining complete list of reviewed papers, chronic diseases and their classifications, algorithms used, publication venues, and excluded papers.</p>
Corpus Nummorum - Natural Language Processing Dataset
<p>This Natural Language Processing (NLP) dataset contains a part of the MySQL <a href="https://www.corpus-nummorum.eu/">Corpus Nummorum (CN)</a> database. It covers Greek and Roman coins from ancient Thrace, Moesia Inferior, Troad and Mysia.</p> <p>The dataset contains 7,900 coin descriptions (or designs) created by the members of the CN project. Most of them are actual coin designs which can be linked through our relational database to the matching CN coins, types and their images. However, some of them (about 450) were only created for the training of the NLP model. </p> <p>There are nine different MySQL tables: </p> <ol> <li>data_coins: contains the data of all coins in the CN database</li> <li>data_coins_images: contains data of all images in the CN database</li> <li>data_coins_imagesets: contains the image pairs for the CN coins</li> <li>data_designs: contains every coin description in German, English and Bulgarian</li> <li>data_types: contains the data of alle coin types the Cn database</li> <li>nlp_hierarchy: contains the classes and subclasses of all entity categories </li> <li>nlp_list_entities: contains the data of all nlp entities in the CN database</li> <li>nlp_relation_extraction_en_v2: contains the annotations for the training of our NLP model</li> <li>nlp_training_designs: contains the coin designs used for training our NLP model</li> </ol> <p>Only tables 8 and 9 are important for NLP training, as they contain the descriptions and the corresponding annotations. The other tables (data_...) make it possible to link the coin descriptions with the various coins and types in the CN database. It is therefore also possible to provide the CN image data sets with the appropriate descriptions (<a href="../records/10033993">CN - Coin Image Dataset</a> and <a href="../records/13748799">CN - Object Detection Coin Dataset</a>). The other NLP tables provide information about the entities and relations in the descriptions and are used to create the RDF data for the <a href="https://nomisma.org/datasets">nomisma.org</a> portal. The tables of the relational CN database can be related via the various ID columns using foreign keys.</p> <p>For easier access without MySQL, we have attached two csv files with the descriptions in English and German and the annotations for the English designs. The annotations can be related to the descriptions via the Design_ID column. </p> <p>During the summer semester 2024, we held the "Data Challenge" event at our Department of Computer Science at the Goethe-University. Our students could choose between the Object Detection dataset and a Natural Language Processing dataset as their challenge. We gave the teams that decided to take part in the NLP challenge this dataset with the task of trying out their own ideas. Here are the results:</p> <ul> <li><a href="https://github.com/jasperforth/DataChallenge_LLM_REPipeline">LLM_RE Pipeline</a></li> <li><a href="https://github.com/Axolord/coin-description-embeddings">Coin description embeddings</a></li> <li><a href="https://github.com/calul0/nlp_coin_app/tree/master">NLP coin app</a></li> </ul> <p>Now we would like to invite you to try out your own ideas and models on our coin data.</p> <p>If you have any questions or suggestions, please, feel free to contact us. </p>
Supplementary material for: Inconsistency Detection in Natural Language Requirements using ChatGPT: a Preliminary Evaluation
<p>Supplementary material including both data and annotations. For each document we present:</p> <ol> <li>annotated results of the chatGPT answers;</li> <li>annotated results of the manual analysis</li> <li>The requirements and the grund truth pairs. Originals requirements are marked with "1" in the third column, mutants are marked with "0". </li> </ol>
Supplementary material for: Inconsistency Detection in Natural Language Requirements using ChatGPT: a Preliminary Evaluation
<p>Supplementary material including both data and annotations. For each document we present:</p> <ol> <li>annotated results of the chatGPT answers;</li> <li>annotated results of the manual analysis</li> <li>The requirements and the grund truth pairs. Originals requirements are marked with "1" in the third column, mutants are marked with "0". </li> </ol>
Datasets for "The Case for Retaining Natural Language Descriptions of Phenotypes in Plant Databases and a Web Application as Proof of Concept", 2023
<p>The collection of a dataset that organized information about plant genes, phenotypes, and annotations from a variety of data sources</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.