Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
296
datasets available to search
ShareScore release 0.9.0
Dataset results
296 results for “language models”
Rank correlation results for the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data"
<p>This file contains the rank correlation results from the paper "Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data".</p>
WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models
<p>#############</p> <h1>WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models</h1> <p>#############</p> <p>Authors: Valentin Gabeff, Marc Russwurm, Devis Tuia & Alexander Mathis</p> <p>Affiliation: EPFL</p> <p>Date: January, 2024</p> <p>Link to the article: <a href="https://link.springer.com/article/10.1007/s11263-024-02026-6">https://link.springer.com/article/10.1007/s11263-024-02026-6</a></p> <p>--------------------------------</p> <p>WildCLIP is a fine-tuned CLIP model that allows to retrieve camera-trap events with natural language from the Snapshot Serengeti dataset. This project intends to demonstrate how vision-language models may assist the annotation process of camera-trap datasets.</p> <p>Here we provide the processed Snapshot Serengeti data used to train and evaluate WildCLIP, along with two versions of WildCLIP (model weights).</p> <p>Details on how to run these models can be found in the project <a href="https://github.com/amathislab/wildclip">github repository</a>.</p> <h2>Provided data (images and attribute annotations): </h2> <p>The data consists of 380 x 380 image crops corresponding to the MegaDetector output of Snapshot Serengeti with a confidence threshold above 0.7. We considered only camera trap images containing single individuals.</p> <p>A description of the original data can be found on LILA <a href="https://lila.science/datasets/snapshot-serengeti">here</a>, released under the <a href="https://cdla.dev/permissive-1-0/" rel="nofollow">Community Data License Agreement (permissive variant)</a>.</p> <p>We warmly thank the authors of LILA for making the MegaDetector outputs publicly available, as well as for structuring the dataset and facilitating its access.</p> <h2>Adapted CLIP model (model weights): </h2> <p>WildCLIP models provided:</p> <ul> <li><strong>[New] WildCLIP_vitb16_t1.pth: </strong>CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Trained on both base and novel vocabulary (see paper for details).</li> <li><strong>[New] WildCLIP_vitb16_t1_lwf.pth: </strong>CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1, and with the additional VR-LwF loss. Trained on both base and novel vocabulary (see paper for details).</li> <li><strong>WildCLIP_vitb16_t1_base.pth:</strong> CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Model used for evaluation and trained on base vocabulary only. (previously named <em>WildCLIP_vitb16_t1.pth</em>)</li> <li><strong>WildCLIP_vitb16_t1t7_lwf_base.pth</strong>: CLIP model with the ViT-B/16 visual backbone trained on data with captions following templates 1 to 7, and with the additional VR-LwF loss. Model used for evaluation and trained on base vocabulary only. (previously named <em>WildCLIP_vitb16_t1t7_lwf.pth</em>)</li> </ul> <p>We also provide the CSV files containing the train / val / test splits. The train / test splits follow camera split from LILA (https://lila.science/datasets/snapshot-serengeti). The validation split is custom, and also at the camera level.</p> <ul> <li><strong>train_dataset_crops_single_animal_template_captions_T1T7_ID.csv</strong>: Train set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1")</li> <li><strong>val_dataset_crops_single_animal_template_captions_T1T7_ID.csv</strong>: Validation set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1")</li> <li><strong>test_dataset_crops_single_animal_template_captions_T1T8T10.csv</strong>: Test set with captions from templates 1, 8, 9 and 10 (columns "all captions")</li> </ul> <p>Details on how the models were trained can be found in the associated <a href="https://link.springer.com/article/10.1007/s11263-024-02026-6" target="_blank" rel="noopener">publication</a>.</p> <h2>References: </h2> <p>If you find our code, or weights, please cite:</p> <pre>@article{gabeff2024wildclip, title={WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models}, author={Gabeff, Valentin and Ru{\ss}wurm, Marc and Tuia, Devis and Mathis, Alexander}, journal={International Journal of Computer Vision}, pages={1--17}, year={2024}, publisher={Springer} }</pre> <p>If you use the adapted Snapshot Serengeti data please also cite their article:</p> <pre>@article{swanson2015snapshot, title={Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna}, author={Swanson, Alexandra and Kosmala, Margaret and Lintott, Chris and Simpson, Robert and Smith, Arfon and Packer, Craig}, journal={Scientific data}, volume={2}, number={1}, pages={1--14}, year={2015}, publisher={Nature Publishing Group} }</pre>
Jargon: A Suite of Language Models and Evaluation Tasks for French Specialized Domains: Data
<p>This repository contains the data for the following paper:</p> <p><span>Vincent Segonne, Aidan Mannion, Laura Cristina Alonzo Canul, Alexandre Daniel Audibert, Xingyu Liu, Cécile Macaire, Adrien Pupier, Yongxin Zhou, Mathilde Aguiar, Felix E. Herron, Magali Norré, Massih R Amini, Pierrette Bouillon, Iris Eshkol-Taravella, Emmanuelle Esperança-Rodier, Thomas François, Lorraine Goeuriot, Jérôme Goulian, Mathieu Lafourcade, et al.. 2024. <a href="https://aclanthology.org/2024.lrec-main.827">Jargon: A Suite of Language Models and Evaluation Tasks for French Specialized Domains</a>. In <em>Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)</em>, pages 9463–9476, Torino, Italia. ELRA and ICCL.</span></p> <p>1) pretraining data for the Jargon specialized language models</p> <p>2) ECTHR_FR dataset for text classification in the French legal domain</p> <p> </p>
Dataset related to: An Empirical Study on Low Code Programming using Traditional vs Large Language Model Support
<p>This repository contains data and prompts related to our research. The files included are:</p> <p><strong>prompt.py</strong>: This script contains the original prompts used in our study.</p> <p><strong>LLM_lowcode.mx20</strong>: This file includes posts and annotation data related to LLM-based low-code platforms.</p> <p><strong>Traditional_lowcode.mx20</strong>: This file includes posts and annotation data related to traditional low-code platforms.</p> <p>The .mx20 files can be opened using the MAXQDA software, which can be downloaded from the official website. MAXQDA offers a 14-day free trial.</p>
Exploring The Potential of GPT-3-based Large Language Model For Melody Generation
<p>Here, we provide the dataset used, all generated melodies and melodies used to conduct subjective listening test.</p>
Risk-of-bias v.2 assessment with large language models
<p>See https://bitbucket.org/aimedtech/fewshot_rob for more information.</p>
Analyzing the Dependability of Large Language Models for Code Clone Generation.
<div> <p>data.zip: <br><br>This dataset includes a collection of ten LeetCode programming problems used in the study "Analyzing the Dependability of Large Language Models for Code Clone Generation". At the top level, you will find a CSV file containing all the initial LeetCode data. Each subdirectory at this level represents a specific LeetCode problem. Within these subdirectories, you will find the original solutions, their behaviors, the input corpus, as well as folders dedicated to various temperatures, models, and code cloning tasks. Additionally, within the "repeated" folder, you will find the original LLM-generated snippets, the preprocessed snippets with the snippet behavior, and the results.</p> <p>characterizing_code_clones_project.zip: </p> <p>This zipped directory encompasses the core scripts and results used in the "Characterizing Code Clones of LLMs" research. The common folder was used to run the whole pipeline, the various parts of the pipeline are each in a folder as well as the various data analysis scripts! </p> <p> </p> </div>
Analyzing the Dependability of Large Language Models for Code Clone Generation
<p>data.zip: <br><br>This dataset includes a collection of ten LeetCode programming problems used in the study "Analyzing the Dependability of Large Language Models for Code Clone Generation". At the top level, you will find a CSV file containing all the initial LeetCode data. Each subdirectory at this level represents a specific LeetCode problem. Within these subdirectories, you will find the original solutions, their behaviors, the input corpus, as well as folders dedicated to various temperatures, models, and code cloning tasks. Additionally, within the "repeated" folder, you will find the original LLM-generated snippets, the preprocessed snippets with the snippet behavior, and the results.</p> <p>characterizing_code_clones_project.zip: </p> <p>This zipped directory encompasses the core scripts and results used in the "Characterizing Code Clones of LLMs" research. The common folder was used to run the whole pipeline, the various parts of the pipeline are each in a folder as well as the various data analysis scripts! </p>
Valentwin: Using Self-Supervised Contrastive Learning on Language Model for Schema Matching Datasets
<div>ValenTwin is a schema matching framework that uses self-supervised contrastive learning to train the model, uses the model to generate embeddings of table columns, then uses different similarity measures to match the column embeddings.</div> <div> </div> <div> <div>We provide two types of zip files for the datasets:<br>1. `data.zip` contains the raw data files, the ground truth files, the sampled data (n=[100, 200, 300, 400, 500] used in the experiments, as well as the contrastive data used to train the model.<br>2. `data-raw.zip` contains only the raw data files and the ground truth files. You can sample the data and generate the contrastive dataset yourself by following step 1 and 2 in the `How to Run` section. <br>Download and unzip one of the zip files to the `data` folder.</div> </div>
Integrated Image-based Deep Learning and Language Models for Primary Diabetes Care
<h2><strong>Example data for the paper "Integrated Image-based Deep Learning and Language Models for Primary Diabetes Care"</strong></h2> <p><strong>Example data for the paper "Integrated Image-based Deep Learning and Language Models for Primary Diabetes Care"</strong></p>
Replication data for the paper "Leveraging Large Language Models for Comprehensive Psychological Analysis: Insights from Four Theoretical Frameworks"
<p>This is a replication data for the paper titled "Leveraging Large Language Models for Comprehensive Psychological Analysis: Insights from Four Theoretical Frameworks" submitted for a blind review.</p> <p>Abstract</p> <p>The rapid advancement of generative Artificial Intelligence (AI) has significantly transformed various research domains. This paper introduces a novel, fully automated methodology for applying Large Language Models (LLMs) to psychological text analysis. The approach includes prompt design for zero-shot and few-shot learning, model internal consistency analysis, autonomous machine evaluation, and additional human validation. Applied to four psychological theories—Self-Determination Theory, the Big Five Personality Traits, Psychological Well-being, and Cognitive Behavioral Therapy—this methodology is tested on a dataset of 25,780 emails written by a senior executive (called Person X) over 16 years. The analysis involves extracting psychological characteristics from the emails and regressing these characteristics against personal, professional, and environmental factors. The results demonstrate that the methodology provides unique insights into the examined psychological theories, offering a detailed understanding of how various factors influence psychological states and traits over time. This research highlights the potential of LLMs in capturing and analyzing complex psychological patterns in large text corpora, contributing a robust framework for future studies and practical applications in psychological assessment and intervention. The findings underscore the transformative impact of generative AI in psychological research, opening new avenues for understanding human behavior through advanced language models.</p> <p>The zipped file contains five csv files:</p> <ol> <li>Email_classification-csv: LLM (GPT-3.5 Turbo) classification of 25,780 emails for four psychological theories: SDT, Big Five, PWB and CBT.</li> <li>SDT_regression_data.csv</li> <li>Big_Five_regression_data.csv</li> <li>PWB_regression_data.csv</li> <li>CBT_regression_data.csv</li> </ol> <p>For 2-5 files the dependent variable is monthy percentage share of emails the were assigned a given value for categories of one of the four psychological theories analyzed. </p> <p>Linear regression model has been applied, where dependent variable is the percentage of emails in a specified category that assigned a specific value in this category. For example in Big Five Traits Model, for the Openness category, for each month we calculated percentage of emails that exhibit <em>High</em> or <em>Low</em> openness, or <em>None</em> if the content of the email does not provide enough information to assess whether the specific need is relevant. Two dependent variables were created: <em>Openness-high</em> and <em>Openness-low</em> and regressed on all independent variables. Regressions were not run for the <em>None</em> values.</p> <p>Descriptions of independent variables:</p> <p>- <em>income_index</em>: Person X salary income and consulting fees in a given month, normalized to [0,1].</p> <p>- <em>card_spending</em>: Person X credit card expenditures in a given month, normalized to [0,1].</p> <p>- <em>abroad_far</em>: dummy variable set to 1 for months when Person X worked in Central Asia</p> <p>- <em>abroad_near</em>: dummy variable set to 1 when Person X worked in other EU country</p> <p>- <em>death_1_war</em>: variable set to 1 in a month when Person X’ farther in law passed away. In the same month Russia invaded Ukraine. The variable was set to .75 in the following month, and to .5 in the month after that.</p> <p>- <em>death_2</em>: variable set to 1 in a month when Person X’ mother passed away. The variable was set to .75 in the following month, and to .5 in the month after that.</p> <p>- <em>court_case</em>: dummy variable set to 1 for months with the emotionally engaging inheritance court case involving other family members.</p> <p>- <em>BIG4_partner</em>: dummy variable set to 1 for months when Person X worked as a partner in BIG4 accounting firm, which resulted in adopting a professional activity sharply different from the usual Person X habits.</p> <p>- <em>AI_company</em>: dummy variable set to 1 for months when Person X worked as C-level executive at a company specializing in artificial intelligence.</p> <p>- <em>elections</em>: dummy variable set to 1 for months when Person X unsuccessfully run in parliamentary elections</p> <p>- <em>covid_lockdown</em>: dummy variable set to 1 for month where Polish government imposed tough measures during two covid lockdowns.</p> <p>- <em>no_receive</em>: number of different email recipients each month, normalized to [0,1].</p> <p>- <em>avg_length</em>: average number of words in emails sent each month, normalized to [0,1].</p> <p> While the email data was collected for January 2008 – March 2014 period, financial data was available from October 2009. There were some months where no emails with more than 10 words were sent, yielding 166 monthly observations used for regressions, before removing outliers.</p> <p> Independent variables were tested for multicollinearity, outlier months were removed, regressions were estimated with robust standard errors, and a range of standard tests were conducted for normality and autocorrelation of residuals, confirming good statistical properties of estimated models.</p> <p>Due to privacy concerns, the email texts cannot be publicly shared. However, the classifications of psychological categories derived from the email texts, along with all other relevant data, are made publicly available in this open access repository, with the consent of email author.</p> <p> </p>
Four psychometrically validated datasets for benchmarking large language models, based on the TIMSS 2008 and 2011 released items.
<p>Four datasets validated according to psychometric principles that can be used to benchmark large language models in terms of achievements in advanced school math, advanced school physics, 8th grade math and 8th grade science.</p> <p>These four datasets are derived from items released by Trends in International Mathematics and Science Study Advanced 2008 and Trends in International Mathematics and Science Study 2011. See <a href="https://nces.ed.gov/timss/released-questions.asp">link</a>.</p> <p>For more information, see our paper <a href="https://arxiv.org/abs/2404.01799">PATCH! Psychometrics-AssisTed benCHmarking of Large Language Models: A Case Study of Mathematics Proficiency</a>.</p>
Replication materials for: Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow
<p>These are the replication materials for the paper:<br>Are Large Language Models a Threat to Digital Public Goods? Evidence from Activity on Stack Overflow<br>By: Maria del Rio-Chanona, Nadzeya Laurentsyeva, and Johannes Wachs. <br>Preprint: https://arxiv.org/abs/2307.07367<br>Under Revision for PNAS Nexus</p> <p> </p>
EditQL: A Textual Query Language for Evolving Models (Reproducibility Package)
Open the record for dataset details and reuse information.
Modeling Languages for Digital Twins - A Survey Among the German Automotive Industry
<p>This repository contains the replication package for the paper _Modeling Languages for Digital Twins: A Survey Among the German Automotive Industry_ by Jérôme Pfeiffer, Dominik Fuchß, Thomas Kühn, Robin Liebhart, Dirk Neumann, Christer Neimöck, Christian Seiler, Anne Koziolek, and Andreas Wortmann. <br>The paper has been submitted to the practice track of [MODELS 2024](https://conf.researchr.org/track/models-2024/models-2024-technical-track#Practice-Track).</p> <h3>Data</h3> <p>This replication package contains all information from the survey:<br>- `results.csv`: A csv version of all data exported from LimeSurvey (German). Personal information from the participants has been removed. This file can be imported to reproduce the extraction results described in our paper.<br>- `survey_german.pdf`: The pdf version of the original survey in German. <br>- `survey_german.md`: A markdown version of the original survey in German. <br>- `survey_english.md`: A markdown version of the survey translated into English. </p> <h3>Selection of participants and distribution</h3> <p>With both versions, the survey can be executed again with a different target audience in English or German. In our case we wanted to reach as much participants from diverse work areas as possible, where we invited the participants by email via an internal mailing list of 189 members of the SofDCar project. To improve the response rate, we implemented two deadline extensions from the initial one-month-long time frame with 2 weeks of additional response time. Together with the deadline extension, we sent a mail to inform and remind the members of the consortium of the survey.</p> <h3>Data extraction</h3> <p>In total, we had 96 participants, of which 43 completed the questionnaire. For incomplete survey responses, we took only the available answers and did not include the missing answers in our data analysis. For data analysis we utilized the commercial Tool IBM SPSS and custom python scripts.</p> <h2>Research Questions </h2> <p>- RQ1: How is the DT understood in the automotive industry?<br> - RQ1.1: For which phases of automotive development are DTs<br>important?<br> - RQ1.2: What are desired properties of DTs?<br> - RQ1.3: What are desired purposes of using DTs?<br> - RQ1.4: How do these purposes change in relation to different phases of automotive development?<br>- RQ2: Which modeling languages and modeling tools are currently employed in the automotive industry?<br> - RQ2.1: Which kinds of models are important during automotive development?<br> - RQ2.2: How important are which models in the phases of automotive development?<br> - RQ2.3: Which tools are used to create and maintain these models?</p>
On Inter-dataset Code Duplication and Data Leakage in Large Language Models
<p>This dataset encompasses the sparse graph referenced in the publication titled "On Inter-dataset Code Duplication and Data Leakage in Large Language Models."</p> <p>This resource is a snapshot of the original <a href="https://github.com/Antolin1/code-inter-dataset-duplication">repository</a>, and the graph is preserved in the <em>interduplication.db</em> database. The schema of this database is easily understandable and is available in the original repository. Each code snippet is identified by a unique identifier (id_within_dataset) that corresponds to its identification within the dataset from which it was extracted. The complete datasets are stored in .jsonl files within their respective folders (e.g., python-150/data.jsonl, codetrans/data.jsonl, etc.).</p> <p> </p> <p> </p> <p> </p>
Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification
<p>Code and dataset for paper "Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification". ICWSM 2025</p> <p>Preprint: https://arxiv.org/abs/2407.17688</p> <p>Citation: </p> <p>@misc{ng2024examininginfluencepoliticalbias,<br> title={Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification}, <br> author={Lynnette Hui Xian Ng and Iain Cruickshank and Roy Ka-Wei Lee},<br> year={2024},<br> eprint={2407.17688},<br> archivePrefix={arXiv},<br> primaryClass={cs.CL},<br> url={https://arxiv.org/abs/2407.17688}, <br>}</p>
Modeling Compliance Specifications in Linear Temporal Logic, Event Processing Language and Property Specification Patterns
<p>Experimental material & data</p>
Data-driven brain network models differentiate variability across language tasks
<p>Data and script associated with the manuscript titled "Data-driven brain network models differentiate variability<br> across language tasks". </p>
Large Language Model
<p>One of the large language models trained in this paper: https://arxiv.org/abs/1810.10045</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.