Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
296
datasets available to search
ShareScore release 0.9.0
Dataset results
296 results for “language models”
Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code
<p>Despite their success, large language models (LLMs) face the critical challenge of hallucinations, generating plausible but incorrect content. While much research has focused on hallucinations in multiple modalities including images and natural language text, less attention has been given to hallucinations in source code, which leads to incorrect and vulnerable code that causes significant financial loss. To pave the way for research in LLMs' hallucinations in code, we introduce Collu-Bench, a benchmark for predicting code hallucinations of LLMs across code generation (CG) and automated program repair (APR) tasks. Collu-Bench includes 13,234 code hallucination instances collected from five datasets and 11 diverse LLMs, ranging from open-source models to commercial ones. <br>To better understand and predict code hallucinations, Collu-Bench provides detailed features such as the per-step log probabilities of LLMs' output, token types, and the execution feedback of LLMs' generated code for in-depth analysis. In addition, we conduct experiments to predict hallucination on Collu-Bench, using both traditional machine learning techniques and neural networks, which achieves 22.03 -- 33.15% accuracy. Our experiments draw insightful findings of code hallucination patterns, reveal the challenge of accurately localizing LLMs' hallucinations, and highlight the need for more sophisticated techniques.</p>
Advancing Large Language Models through Story Energy, Universal Harmony Energy, and SA-UUH-UPP
<p><span>In this groundbreaking exploration of AI, we unveil how Story Energy, Universal Harmony Energy, and the SA-UUH-UPP framework could revolutionize large language models (LLMs). Discover how these advanced concepts push AI beyond current boundaries, enabling deeper contextual understanding, energy-efficient models, and steps toward self-awareness. Whether you’re an AI researcher, developer, or enthusiast, this video provides insights that could redefine the future of AI. Watch now to dive into the next frontier of artificial intelligence!</span></p>
Focused learning by antibody language models using preferential masking of non-templated regions
<p><strong>Motivation.</strong> While existing antibody language models (AbLMs) excel at predicting germline residues, they often struggle with mutated and non-templated residues, which concentrate in the complementarity-determining regions (CDRs) and are crucial for determining antigen-binding specificity. Many of these models are trained using a masked language modeling (MLM) objective with uniform masking probabilities; however, antibody recombination is modular in nature, creating relatively distinct regions of high and low complexity (non-templated and templated, respectively). We sought to determine whether and to what extent AbLMs can improve when trained using an alternative masking strategy based on this observation.</p> <p><strong>Results.</strong> We developed a variation on MLM called <strong><em>Preferential Masking</em></strong>, which alters masking probabilities to amplify training signals from the CDR3. We pre-trained two AbLMs using either uniform or preferential masking and observed that the latter improves pre-training efficiency and residue prediction accuracy in the highly variable CDR3. Preferential masking also improves antibody classification by native chain pairing and binding specificity, suggesting improved CDR3 understanding and indicating that non-random, learnable patterns help govern antibody chain pairing. We further show that specificity classification is largely informed by residues in the CDRs, demonstrating that AbLMs learn meaningful patterns that align with immunological understanding.</p> <p><strong>Files. </strong>The following files are included in this repository:</p> <ul> <li><strong><em>uniform_250k.tar.gz</em></strong>: Model weights for the Uniform-250k model.</li> <li><strong><em>uniform_350k.tar.gz</em></strong>: Model weights for the Uniform-350k model.</li> <li><strong><em>preferential_250k.tar.gz</em></strong>: Model weights for the Preferential-250k model.</li> <li><strong><em>train-eval-test_cdr-mask.tar.gz</em></strong>: Datasets used to train all three models above. Compressed folder containing three files: <em>A_train.csv</em>, <em>A_eval.csv</em>, and <em>B_test.csv</em>. Each row contains a natively paired sequence with its corresponding label-encoded CDR mask, designed to align with the tokenized amino acid sequence. Sequences were obtained from <a href="https://doi.org/10.1038/s41586-022-05371-z">Jaffe et al.</a> and <a href="https://doi.org/10.1016/j.celrep.2024.114307">Hurtado et al</a>. These are referenced in the paper as Dataset A (<em>A_train.csv, A_eval.csv)</em>, and Dataset B (<em>B_test.csv</em>)<em>.</em></li> <li><strong><em>test-set_annotations.tar.gz</em></strong>: Unpaired annotations for all test set (Dataset B) sequences: <em>B_test-set_annotations.csv</em>. Used for Fig. 3 and Fig. 4D. Annotations can be mapped back to the paired sequences using their `sequence_id` and `locus` information.</li> <li><strong><em>pair_classification.tar.gz</em></strong>: Two classification datasets used to train the classifier models in Figure 4: <em>C_native-0_shuffled-1.csv</em> (Dataset C) and <em>D_native-0_shuffled-1.csv</em> (Dataset D). Dataset C sequences were obtained from <a href="https://doi.org/10.1038/s41586-022-05371-z">Jaffe et al.</a> and <a href="https://doi.org/10.1016/j.celrep.2024.114307">Hurtado et al</a> (Dataset B), and Dataset D sequences were obtained from <a href="https://doi.org/10.1038/s41590-022-01230-1">Phad et al</a> and data generated as part of this study.</li> <li><strong><em>CoV_classification.tar.gz</em></strong>: Classification dataset used to train the classifier models in Figure 5: <em>E_hd-0_cov-1.csv </em>(Dataset E). CoV antibody sequences were obtained from <a href="https://doi.org/10.1093/bioinformatics/btaa739">CoV-AbDAb</a>, and healthy donor sequences were obtained from <a href="https://doi.org/10.1038/s41590-022-01230-1">Phad et al</a>.</li> </ul> <p><strong>Code.</strong> All code used for model training, testing, and figure generation is available under the MIT license on <a href="https://github.com/brineylab/preferential-masking-paper">GitHub.</a></p> <p> </p>
Training datasets for "Multi-purpose controllable protein generation via prompted language models"
<div> <p>The datasets used to tune modular prompts of PROPEND fall into three main categories based on their design objectives: tertiary structure, secondary structure, and functional annotation.</p> </div>
Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base
<p>Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base</p>
Impact of Code Language Models on Automated Program Repair (Dataset)
<p>This is the fine-tuning dataset used in the paper Impact of Code Language Models on Automated Program Repair</p>
A Transformer-based Function Symbol Name Inference Model from an Assembly Language for Binary Reversing
<p>This is a dataset and pre-trained model for the official implementation of <a href="https://github.com/agwaBom/AsmDepictor"><strong>AsmDepictor</strong></a>, "A Transformer-based Function Symbol Name Inference Model from an Assembly Language for Binary Reversing", In the 18th ACM Asia Conference on Computer and Communications Security <a href="https://asiaccs2023.org/">AsiaCCS '2023</a></p> <p> </p>
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
<p>QA-Feedback used in the paper: Fine-Grained Human Feedback Gives Better Rewards for Language Model Training</p>
ThoughtSource: A central hub for large language model reasoning data (dataset snapshot)
<p><strong>ThoughtSource is a meta-dataset and software library for chain-of-thought reasoning in large language models (LLMs). </strong></p> <p><strong>This repository contains a snapshot of the openly available ThoughtSource datasets.</strong></p>
Data from: Investigating the human and non-obese diabetic mouse MHC class II immunopeptidome using protein language modelling.
<p><strong>Background</strong>: Identifying peptides associated with the major histocompability complex class II (MHCII) is a central task in the evaluation of the immunoregulatory function of therapeutics and drug prototypes. MHCII-peptide presentation prediction has multiple biopharmaceutical applications, including the safety assessment of biologics and engineered derivatives in silico, or the fast progression of antigen-specific immunomodulatory drug discovery programs in immune disease and cancer. This has resulted in the collection of large–scale data sets on adaptive immune receptor antigenic responses and MHC-associated peptide proteomics. In parallel, recent deep learning algorithmic advances in natural language processing (NLP) and protein language modelling (PLM) have shown potential in leveraging large collections of sequence data and improve MHC presentation prediction. <strong>Methodology</strong>: We trained a compact transformer model (AEGIS) on human and mouse MHCII immunopeptidome data, including a preclinical murine model, and evaluated its performance on the peptide presentation prediction task. <strong>Data</strong>: The data and models used in AEGIS are contained in the uploaded tar files. <strong>Results</strong>: The transformer performs on par with existing deep learning algorithms and that combining datasets from multiple organisms increases model performance (see preprint). We trained variants of the model with and without MHCII information. In both alternatives, the inclusion of peptides presented by the I-Ag7 MHC class II molecule expressed by the non-obese diabetic (NOD) mice enabled the in silico prediction of presented peptides in a preclinical type 1 diabetes model organism, which has promising therapeutic applications.</p>
Results and log of LLM-KG-Bench runs described in article "Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?", Frey et al. 2023
<p>Results and log of LLM-KG-Bench runs described in article ""Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?", Frey et al. 2023, to appear in proceedings for workshop DL4KG@ISWC 2023.</p> <p>For data on task FactExtractStatic please contact authors.</p>
Data for paper on inductive reasoning in large language models
<p>This dataset contains all data that was used for:</p> <p>Han, S. J., Ransom, K. J., Perfors, A. & Kemp, C. (2023). Inductive reasoning in humans and large language models. <em>Cognitive Systems Research</em>.</p> <p>The code for this project can be found <a href="https://github.com/S-J-HAN/InductiveReasoningInLargeLanguageModels">here</a>.</p> <p> </p>
Smart Computing Models, Sensors, and Early Diagnostic Speech and Language Deficiencies Indicators in Child Communication
ClinicalTrials.gov study NCT06633874. IPD Sharing: NO. Countries: 1. Publications: 1.
Automating pharmacovigilance evidence generation: Using large language models to produce context-aware SQL
Open the record for dataset details and reuse information.
Human review for post-training improvement of low-resource language performance in large language models
Open the record for dataset details and reuse information.
Incentivizing news consumption on social media platforms using large language models and realistic bot accounts
Open the record for dataset details and reuse information.
Gene-language models are whole genome representation learners
Open the record for dataset details and reuse information.
Biomedical ELECTRA based deep language representation models for biomedical text mining.
<p>The gzipped tar file contains two biomedical language representation models based on ELECTRA (Clark et al., 2020) deep transformers architecture to be used for down-stream biomedical text mining tasks. </p> <p>Bio-ELECTRA is pre-trained from scratch on PubMed abstracts for 1.8 million steps. Bio-ELECTRA++ is the further pre-trained version of Bio-ELECTRA trained on a corpus of open access full papers from PubMed.</p>
Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment
<p>This repository contains the raw results (by word information-theoretic measures for the experimental stimuli) and the LSTM models analyzed in <a href="https://www.aclweb.org/anthology/2020.acl-main.179/">Recurrent Neural Network Language Models Always Learn English-Like Relative Clause Attachment</a>. The models from the synthetic experiments are given in the synthetic archive, as well as the training data generation script. There is a README included that gives more details for recreating/evaluating results from those experiments.</p> <p>The naming convention for each model in the models directory is:<br> [Language]_hidden[Hidden Units]_batch[Batch Size]_dropout[Dropout Rate]_lr[Learning Rate]_[Model Number].pt</p> <p>Language: en for English and es for Spanish<br> Hidden Units: All models had two layers with 650 hidden units per layer<br> Batch Size: The size of the batch (128 for English, 64 for Spanish)<br> Dropout Rate: All models used a dropout rate of 0.2<br> Learning Rate: All models has a learning rate of 20<br> Model Number: Identifier of the model (English model 0 is the best model from <a href="https://github.com/facebookresearch/colorlessgreenRNNs">Gulordava et al. (2018)</a>) </p> <p> </p> <p> </p>
Model Checking the Multi-Formalism Language FIGARO
<p>for double blind review</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.