Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
112
datasets available to search
ShareScore release 0.9.0
Dataset results
112 results for “language learning”
lilGym: Natural Language Visual Reasoning with Reinforcement Learning, model files
<p>Baselines models for the paper <a href="https://lil.nlp.cornell.edu/lilgym"><em>lil</em>Gym: Natural Language Visual Reasoning with Reinforcement Learning</a>.</p>
Language Learning Aptitude dataset
Open the record for dataset details and reuse information.
Synesthesia in children with difficulties in written language learning
<p>This dataset contains data and results associated with the manuscript “Synesthesia in children with difficulties in written language learning” by Anahita Basirat and Jean-Michel Hupé (<em>Consciousness and Cognition</em>, doi: 10.1016/j.concog.2020.10295).</p> <ul> <li>color.csv: This file contains RGB and hexadecimal codes of the colors used in the consistency tests.</li> <li>summary.csv: This file contains the information about the participants and the results of the consistency test for letters and digits. <ul> <li>A1: session 1, test 1</li> <li>A2: session 1, test 2</li> <li>B1: session 2, test 1</li> <li>B2: session 2, test 1</li> </ul> </li> <li>data: This folder contains raw files of the consistency tests for each participant for the first and the second sessions (respectively, file names ended with a and b). In each raw file, lines 2 to 37 correspond to the first test. Selected letters/digits and colors are shown in columns D and C, respectively. Lines 38 to 73 correspond to the second test. Presented letters/digits and selected colors are shown in columns B and C, respectively.</li> <li>figures: This folder contains the illustration of the results for each participant (as in Figure 3 in the paper).</li> </ul>
Learner Data from a Study on Latin Language Learning
<p>The dataset contains test results from a digital intervention study of the <a href="https://www.projekte.hu-berlin.de/en/callidus-en">CALLIDUS Project</a> in a high school in Berlin. 13 Students were randomly sampled in two groups and completed various linguistic tasks. The focus of the study was to find out whether learning Latin vocabulary in authentic contexts leads to higher lexical competence, compared to memorizing traditional vocabulary lists.</p> <p>The data is available in <a href="https://www.json.org">JSON format</a> as provided by the <a href="https://h5p.org">H5P implementation</a> of <a href="https://www.valamis.com/hub/xapi">XAPI</a>. File names indicate the time of test completion, in the concatenated form of "year-month-day-hour-minute-second-millisecond". This allows us to trace the development of single learners who were fast enough to perform the test twice in a row.</p> <p>Changelog:</p> <p>Version 2.0: Each exercise now has a unique ID that is consistent in the whole dataset, so evaluation/visualization can refer to specific exercises more easily.</p> <p>Version 3.0: A simplified Excel Spreadsheet has been added to enhance the reusability of the dataset. It contains a slightly reduced overview of the data, but the core information (user ID, task statement, correct solution, given answer, score, duration) is still present.</p>
Variational Inference for Learning Representations of Natural Language Edits
<p>Performance Evaluation of Edit Representations (PEER), the dataset we use in the paper <a href="https://arxiv.org/abs/2004.09143">"Variational Inference for Learning Representations of Natural Language Edits"</a>.</p>
Figure 1. The design of the study-The Effect of English Learning Anxiety on Iranian High-School Students' English Language Achievement
<p>The dependent variable in this study was English<br> achievement, and the independent variable was language anxiety. The intervening variable is<br> English proficiency. The control variables are: age of subjects (second year high school students<br> with an average age of 17) and years of experience in English learning (a minimum of 5<br> consecutive years). The schematic design of the study is presented below (Figura 1).</p>
Data and code from: Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome data
<p>We use open source human gut microbiome data to learn a microbial "language" model by adapting techniques from Natural Language Processing (NLP). Our microbial "language" model is trained in a self-supervised fashion (i.e., without additional external labels) to capture the interactions among different microbial species and the common compositional patterns in microbial communities. The learned model produces contextualized taxa representations that allow a single bacteria species to be represented differently according to the specific microbial environment it appears in. The model further provides a sample representation by collectively interpreting different bacteria species in the sample and their interactions as a whole. We show that, compared to baseline representations, our sample representation consistently leads to improved performance for multiple prediction tasks including predicting Irritable Bowel Disease (IBD) and diet patterns. Coupled with a simple ensemble strategy, it produces a highly robust IBD prediction model that generalizes well to microbiome data independently collected from different populations with substantial distribution shift.</p> <p>We visualize the contextualized taxa representations and find that they exhibit meaningful phylum-level structure, despite never exposing the model to such a signal. Finally, we apply an interpretation method to highlight bacterial species that are particularly influential in driving our model's predictions for IBD.</p>
Datasets for Crowd-based Requirements Engineering and aspect-based detection of learning-centered emotion from the text in Serbian language
<h1>Datasets for the paper "Enhancing Software and Learning with Serbian Student Feedback Corpora"</h1> <p>These datasets include student feedback on an Intelligent Tutoring System written in Serbian, annotated with categories for Crowd-based Requirements Engineering (CrowdRE) and aspect-based detection of learning-centered emotions. Four annotators manually annotated each sentence. </p> <p>The CrowdRE dataset includes two JSON files:</p> <ul> <li><strong>crowdre_english.json</strong> - annotated text with columns and classes written in English. Columns are: <ul> <li><em>Comment</em> - the entire student feedback</li> <li><em>Sentence</em> - sentence extracted from the feedback that was annotated</li> <li><em>Intention </em>- class representing the intention of the sentence</li> <li><em>Topic </em>- class representing the topic of the sentence</li> </ul> </li> <li><strong>crowdre_srpski.json</strong> - annotated text with columns and classes written in Serbian. Columns are:<br> <ul> <li><em>Komentar</em> - the entire student feedback</li> <li><em>Recenica </em>- sentence extracted from the entire feedback that was annotated</li> <li><em>Namera </em>- class representing the intention of the sentence</li> <li><em>Tema </em>- class representing the topic of the sentence.</li> </ul> </li> </ul> <p>The dataset for the aspect-based detection of learning-centered emotions includes two JSON files:</p> <ul> <li><strong>emotions_english.json </strong>- annotated text with columns and classes written in English. Columns are: <ul> <li><em>Comment </em>- the entire student feedback</li> <li><em>Sentence </em>- sentence extracted from the feedback that was annotated</li> <li><em>Aspect </em>- class representing the aspect of the sentence</li> <li><em>Emotion </em>- class representing the learning-centered emotion of the sentence</li> </ul> </li> <li><strong>emocije_srpski.json </strong>- annotated text with columns and classes written in Serbian. Columns are: <ul> <li><em>Komentar </em>- the entire student feedback</li> <li><em>Recenica</em> - sentence extracted from the entire feedback that was annotated</li> <li><em>Aspekt</em> - class representing the aspect of the sentence</li> <li><em>Emotion </em>- class representing the learning-centered emotion of the sentence.</li> </ul> </li> </ul> <p>Annotators annotated the dataset based on the annotation procedure and guidelines available <a href="https://github.com/Clean-CaDET/student-feedback-mining">here</a>. </p> <h2>Citation</h2> <p>If you use this in your research, please cite:</p> <blockquote> <p>Vidaković, D., Luburić, N., Kovačević, A., & Slivka, J. Enhancing software and learning with Serbian student feedback corpora. Language Resources & Evaluation (2025). https://doi.org/10.1007/s10579-025-09855-y</p> </blockquote> <p> </p>
DeepThought DPR: Distributed peer review enhanced with natural language processing and machine learning - Dataset I
<p>This is the anonymized dataset obtained from the DPR Experiment run at ESO in Fall 2018. If this dataset is used both this DOI as well as the main paper need to be cited. </p>
Dataset for: "Big data suggest strong constraints of linguistic similarity on adult language learning"
<p>This dataset is adapted from raw data with fully anonymized results on the State Examination of Dutch as a Second Language. This exam is officially administred by the Board of Tests and Examinations (College voor Toetsen en Examens, or CvTE). See cvte.nl/about-cvte. The Board of Tests and Examinations is mandated by the Dutch government.</p> <p>The article accompanying the dataset:</p> <p>Schepens, Job, Roeland van Hout, and T. Florian Jaeger. “Big Data Suggest Strong Constraints of Linguistic Similarity on Adult Language Learning.” <em>Cognition</em> 194 (January 1, 2020): 104056. <a href="https://doi.org/10.1016/j.cognition.2019.104056">https://doi.org/10.1016/j.cognition.2019.104056</a>.</p> <p>Every row in the dataset represents the first official testing score of a unique learner.<br> The columns contain the following information as based on questionnaires filled in at the time of the exam:</p> <p>"L1" - The first language of the learner<br> "C" - The country of birth<br> "L1L2" - The combination of first and best additional language besides Dutch<br> "L2" - The best additional language besides Dutch<br> "AaA" - Age at Arrival in the Netherlands in years (starting date of residence)<br> "LoR" - Length of residence in the Netherlands in years<br> "Edu.day" - Duration of daily education (1 low, 2 middle, 3 high, 4 very high). From 1992 until 2006, learners' education has been measured by means of a side-by-side matrix question in a learner's questionnaire. Learners were asked to mark which type of education they have had (elementary, secondary, or tertiary schooling) by means of filling in for how many years they have been enrolled, in which country, and whether or not they have graduated. Based on this information we were able to estimate how many years learners have had education on a daily basis from six years of age onwards. Since 2006, the question about learners' education has been altered and it is asked directly how many years learners have had formal education on a daily basis from six years of age onwards. Possible answering categories are: 1) 0 thru 5 years; 2) 6 thru 10 years; 3) 11 thru 15 years; 4) 16 years or more. The answers have been merged into the categorical answer.<br> "Sex" - Gender<br> "Family" - Language Family<br> "ISO639.3" - Language ID code according to Ethnologue<br> "Enroll" - Proportion of school-aged youth enrolled in secondary education according to the World Bank. The World Bank reports on education data in a wide number of countries around the world on a regular basis. We took the gross enrollment rate in secondary schooling per country in the year the learner has arrived in the Netherlands as an indicator for a country's educational accessibility at the time learners have left their country of origin.<br> "STEX_speaking_score" - The STEX test score for speaking proficiency.<br> "Dissimilarity_morphological" - Morphological similarity<br> "Dissimilarity_lexical" - Lexical similarity<br> "Dissimilarity_phonological_new_features" - Phonological similarity (in terms of new features)<br> "Dissimilarity_phonological_new_categories" - Phonological similarity (in terms of new sounds)</p> <p><br> A few rows of the data:</p> <p>"L1","C","L1L2","L2","AaA","LoR","Edu.day","Sex","Family","ISO639.3","Enroll","STEX_speaking_score","Dissimilarity_morphological","Dissimilarity_lexical","Dissimilarity_phonological_new_features","Dissimilarity_phonological_new_categories"<br> "English","UnitedStates","EnglishMonolingual","Monolingual",34,0,4,"Female","Indo-European","eng ",94,541,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishGerman","German",25,16,3,"Female","Indo-European","eng ",94,603,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishFrench","French",32,3,4,"Male","Indo-European","eng ",94,562,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishSpanish","Spanish",27,8,4,"Male","Indo-European","eng ",94,537,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishMonolingual","Monolingual",47,5,3,"Male","Indo-European","eng ",94,505,0.0094,0.083191,11,19</p>
Replication Package for "PyTraceBERT: Python Traceback-based Language Model for Detecting Compatibility Issues in Deep Learning Systems"
<p>This package contains the traceback data, pre-trained models, and static word embeddings used in the paper, PyTraceBERT: Python Traceback-based Language Model for Detecting Compatibility Issues in Deep Learning Systems.</p>
Transfer learning and DNA language models enhance transcription factor binding predictions
<p>This is the dataset for replicating the results of the paper called "Transfer learning and DNA language models enhance transcription factor binding predictions" by Ekin Deniz Aksu and Martin Vingron.</p> <p>See https://github.com/ekinda/tfbs_prediction_paper</p>
MTL-QA : A dataset and multi-task learning approach for knowledge graph and natural language question answering
<p>The dataset used for this project is created by enhancing the publicly available MetaQA (Movie Text Audio QA), which is primarily a KGQA dataset pertaining to movies, an extension of WikiMovies. This involves questions requiring 1, 2, and 3 hops which can be answered by using a MetaQA Knowledge Graph. The questions are available in text and audio format. The text has vanilla (original) and its paraphrased version, and is called ntm. </p> <p>In order to develop a dataset to support NLQA, a series of dataset augmentation steps has been performed.</p> <p>The dataset consists of natural language questions and a tagged topic entity as ground truth. This topic entity is used to retrieve textual information related to the question from Wikipedia. The introduction section of the entity's page is used as the context that is required for NLQA. Hence, this dataset has information related to both KGQA and NLQA. Certain preliminary checks and validations are done to only retain those data samples whose context can be used to answer a given question.</p>
Data and code from: Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome data
Open the record for dataset details and reuse information.
Do children learn from their mistakes? A registered report evaluating error-based theories of language acquisition
<p>Error-based theories of language acquisition suggest that children, like adults, continuously make and evaluate predictions in order to reach an adult-like state of language use. However, while these theories have become extremely influential, their central claim - that unpredictable input leads to higher rates of lasting change in linguistic representations – has scarcely been tested. We designed a prime surprisal-based intervention study to assess this claim.</p> <p>As predicted, both 5- to 6-year-old children (n=72) and adults (n=72) showed a pre- to post-test shift towards producing the dative syntactic structure they were exposed to in surprising sentences. The effect was significant in both age groups together, and in the child group separately when participants with ceiling performance in the pre-test were excluded. Secondary predictions were not upheld: there were no verb-based learning effects and there was only reliable evidence for immediate prime surprisal effects in the adult, but not in the child group. To our knowledge this is the first published study demonstrating enhanced learning rates for the same syntactic structure when it appeared in surprising as opposed to predictable contexts, thus providing crucial support for error-based theories of language acquisition.</p>
Data from: Genome-scale annotation of protein binding sites via language model and geometric deep learning
<p>The dataset contains the training and test sets of protein binding sites with DNA, RNA, peptide, protein, ATP, HEM, Zn2+, Ca2+, Mg2+ and Mn2+. Each protein is associated with 3 lines indicating the protein name (PDB accession code and chain), sequence and residue labels (0 for non-binding and 1 for binding), respectively. The ESMFold-predicted structures are also provided.</p>
PRICER: Leveraging Few-Shot Learning with Fine-Tuned Large Language Models for Unstructured Economic Data
<p>Describes the taxonomy used in the paper "PRICER: Leveraging Few-Shot Learning with Fine-Tuned Large Language Models for Unstructured Economic Data", presented at the Second Workshop on Semantic Technologies and Deep Learning Models for Scientific, Technical and Legal Data<em> </em>at the Extended Semantic Web Conference (ESWC) 2024.</p>
SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models
<p>This repository contains the documentation, test datasets and scripts used in the following study:</p> <p>"SonicParanoid2: fast, accurate, and comprehensive orthology inference with machine learning and language models"<br><br>- `sonic-manuscript-master.zip` contains all the scripts to reproduce the study, including those for generating the figures and tables included in the manuscript.</p> <p>- <a href="../api/records/11361985/draft/files/sonicparanoid2.wiki.tar.xz/content" target="_blank" rel="noopener noreferrer">sonicparanoid2.wiki.tar.xz</a> contains a snapshot fo the wiki for SonicParanoid2 as of May 30, 2024</p>
LLMs4OL 2024 Datasets: Toward Ontology Learning with Large Language Models
<p>Ontology learning (OL) from unstructured data has evolved significantly, with recent advancements integrating large language models (LLMs) to enhance various aspects of the process. The LLMs4OL 2024 datasets, were developed to benchmark and advance research in OL using LLMs. This dataset as a key component of the LLMs4OL Challenge, targets three primary OL tasks: Term Typing, Taxonomy Discovery, and Non-Taxonomic Relation Extraction. It encompasses seven domains, i.e. lexosemantics and biological functions, offering a comprehensive resource for evaluating LLM-based OL approaches Each task within the dataset is carefully crafted to facilitate both Few-Shot (FS) and Zero-Shot (ZS) evaluation scenarios, allowing for robust assessment of model performance across different knowledge domains to address a critical gap in the field by offering standardized benchmarks for fair comparison for evaluating LLM applications in OL. </p>
Focused learning by antibody language models using preferential masking of non-templated regions
<p><strong>Motivation.</strong> While existing antibody language models (AbLMs) excel at predicting germline residues, they often struggle with mutated and non-templated residues, which concentrate in the complementarity-determining regions (CDRs) and are crucial for determining antigen-binding specificity. Many of these models are trained using a masked language modeling (MLM) objective with uniform masking probabilities; however, antibody recombination is modular in nature, creating relatively distinct regions of high and low complexity (non-templated and templated, respectively). We sought to determine whether and to what extent AbLMs can improve when trained using an alternative masking strategy based on this observation.</p> <p><strong>Results.</strong> We developed a variation on MLM called <strong><em>Preferential Masking</em></strong>, which alters masking probabilities to amplify training signals from the CDR3. We pre-trained two AbLMs using either uniform or preferential masking and observed that the latter improves pre-training efficiency and residue prediction accuracy in the highly variable CDR3. Preferential masking also improves antibody classification by native chain pairing and binding specificity, suggesting improved CDR3 understanding and indicating that non-random, learnable patterns help govern antibody chain pairing. We further show that specificity classification is largely informed by residues in the CDRs, demonstrating that AbLMs learn meaningful patterns that align with immunological understanding.</p> <p><strong>Files. </strong>The following files are included in this repository:</p> <ul> <li><strong><em>uniform_250k.tar.gz</em></strong>: Model weights for the Uniform-250k model.</li> <li><strong><em>uniform_350k.tar.gz</em></strong>: Model weights for the Uniform-350k model.</li> <li><strong><em>preferential_250k.tar.gz</em></strong>: Model weights for the Preferential-250k model.</li> <li><strong><em>train-eval-test_cdr-mask.tar.gz</em></strong>: Datasets used to train all three models above. Compressed folder containing three files: <em>A_train.csv</em>, <em>A_eval.csv</em>, and <em>B_test.csv</em>. Each row contains a natively paired sequence with its corresponding label-encoded CDR mask, designed to align with the tokenized amino acid sequence. Sequences were obtained from <a href="https://doi.org/10.1038/s41586-022-05371-z">Jaffe et al.</a> and <a href="https://doi.org/10.1016/j.celrep.2024.114307">Hurtado et al</a>. These are referenced in the paper as Dataset A (<em>A_train.csv, A_eval.csv)</em>, and Dataset B (<em>B_test.csv</em>)<em>.</em></li> <li><strong><em>test-set_annotations.tar.gz</em></strong>: Unpaired annotations for all test set (Dataset B) sequences: <em>B_test-set_annotations.csv</em>. Used for Fig. 3 and Fig. 4D. Annotations can be mapped back to the paired sequences using their `sequence_id` and `locus` information.</li> <li><strong><em>pair_classification.tar.gz</em></strong>: Two classification datasets used to train the classifier models in Figure 4: <em>C_native-0_shuffled-1.csv</em> (Dataset C) and <em>D_native-0_shuffled-1.csv</em> (Dataset D). Dataset C sequences were obtained from <a href="https://doi.org/10.1038/s41586-022-05371-z">Jaffe et al.</a> and <a href="https://doi.org/10.1016/j.celrep.2024.114307">Hurtado et al</a> (Dataset B), and Dataset D sequences were obtained from <a href="https://doi.org/10.1038/s41590-022-01230-1">Phad et al</a> and data generated as part of this study.</li> <li><strong><em>CoV_classification.tar.gz</em></strong>: Classification dataset used to train the classifier models in Figure 5: <em>E_hd-0_cov-1.csv </em>(Dataset E). CoV antibody sequences were obtained from <a href="https://doi.org/10.1093/bioinformatics/btaa739">CoV-AbDAb</a>, and healthy donor sequences were obtained from <a href="https://doi.org/10.1038/s41590-022-01230-1">Phad et al</a>.</li> </ul> <p><strong>Code.</strong> All code used for model training, testing, and figure generation is available under the MIT license on <a href="https://github.com/brineylab/preferential-masking-paper">GitHub.</a></p> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.