Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,943
datasets available to search
ShareScore release 0.9.0
Dataset results
1,943 results for “machine learning”
Machine-learning-based seismic detection and location around the Tanlu fault zone in eastern China
<p>REAL, HypoInverse, and HypoDD catalog around the Tanlu fault zone in eastern China.</p>
data and code for Machine Learning-based Denoising of Surface Solar Irradiance simulated with Monte Carlo Ray Tracing
<p>Data and radiative transfer code used for the manuscript "Machine Learning-based Denoising of Surface Solar Irradiance simulated with Monte Carlo Ray Tracing". See Readme for details</p>
Soil dissolved organic carbon (DOC) machine learning model code
Open the record for dataset details and reuse information.
Predictive Design of Ultrastretchable Electrodes with Strain-Insensitive Performance via Robotics- and Machine Learning-Integrated Workflow
Open the record for dataset details and reuse information.
Replication Package for 'How do Machine Learning Models Change?'
<h1>Replication Package: How Do Machine Learning Models Change?</h1> <p> </p> <h2>Overview</h2> <div>This replication package accompanies the paper "<strong>How Do Machine Learning Models Change?</strong>". In this study, we conducted a large-scale analysis of <strong>over 680,000 commits from 100,000 models</strong> and <strong>2,251 releases from 202 of these models</strong> on the Hugging Face (HF) platform. Our goal was to understand how machine learning (ML) models evolve by classifying commit types using a detailed ML change taxonomy and analyzing temporal patterns in their activities using Bayesian networks.</div> <p> </p> <div>Our research addresses three main aspects:</div> <div><strong>1. Categorization of Commit Changes:</strong> We classified over 960,000 commits on HF, providing a detailed breakdown of change types and their distribution.</div> <div><strong>2. Analysis of Commit Sequences</strong>: We examined the sequence and dependencies of commit types using Bayesian networks to identify temporal patterns.</div> <div><strong>3. Release Analysis**</strong>: We investigated the distribution and evolution of release types, analyzing how model attributes and metadata change across successive releases.</div> <p> </p> <div>This package provides all the necessary code, data, and documentation to reproduce the results presented in our paper.</div> <p> </p> <h2>Data Collection and Preprocessing</h2> <h3>Data Collection</h3> <div>We collected data from the Hugging Face platform using the Hugging Face Hub API. The data extraction was performed up to <strong>May 2025,</strong> capturing details from over 1 million models available at that time.</div> <p> </p> <div><strong>- Model & Release Information</strong>: We collected model metadata, commit histories, and release information (Git tags) for our sampled models.</div> <div><strong>- Detailed Commit Changes</strong>: To get a detailed list of files modified in each commit, we implemented a direct Git processing approach. For each model, its repository was temporarily cloned to programmatically extract the list of changed files for every commit SHA.</div> <p> </p> <h3>Data Preprocessing</h3> <div><strong>Commit Diffs</strong></div> <div>We computed the differences for key JSON configuration files (e.g., `config.json`) between commits to identify added, deleted, and updated keys, which served as input for classification.</div> <div> </div> <div><strong>Commit Classification</strong></div> <div>We classified each commit according to Bhatia et al.'s ML change taxonomy using the <strong>Gemini 2.5 Flash LLM</strong>. To ensure the reliability of this process, we implemented a rigorous <strong>two-phase validation</strong>:</div> <div><strong>1. Prompt Refinement (Training): </strong>The prompt was iteratively refined over 6 cycles using a curated training set of 143 commits. The process was guided by comparing LLM classifications against a gold standard created by two human annotators (Human-Human IRR on a subset: 𝜅 = 0.7798). The final refined prompt achieved a Kappa of 0.9068 against the training gold standard.</div> <div><strong>2. Final Validation (Testing): </strong>The validated prompt was tested on an independent, statistically significant sample of 384 commits. The LLM's classifications achieved a Cohen's Kappa of 0.8568 when compared against the test set's gold standard, which itself was validated with a human-human IRR of 𝜅 = 0.8150.</div> <p> </p> <div>We also classified commits into Swanson's categories using a fine-tuned DistilBERT model, as detailed in the paper.</div> <p> </p> <h2>Folder Structure</h2> <div>The replication package is organized as follows. The structure has been designed to separate code, data, and metadata for clarity.</div> <p> </p> <ul> <li>`code/`: Contains all Jupyter notebooks for the study. <ul> <li>`Collection/`: Scripts for data extraction from Hugging Face.</li> <li>`HFExtraction.ipynb`: Collects primary model and commit information.</li> <li>`HFReleasesExtraction.ipynb`: Collects release (tag) specific information.</li> </ul> </li> <li>`Preprocessing/`: Scripts for data cleaning, processing, and classification. <ul> <li>`HFCommitsPreprocessing.ipynb`: Processes commits, computes diffs, and prepares data for classification and analysis.</li> <li>`HFReleasesPreprocessing.ipynb`: Processes and classifies release data.</li> </ul> </li> <li>`Analysis/`: Notebooks for reproducing the analysis for each research question. <ul> <li>`HFFileChanges.ipynb`: Contains the preliminary analysis of file change patterns.</li> <li>`RQ1_Analysis.ipynb`: Analysis for Research Question 1.</li> <li>`RQ2_Analysis.ipynb`: Analysis for Research Question 2.</li> <li>`RQ3_Analysis.ipynb`: Analysis for Research Question 3.</li> </ul> </li> <li>`datasets/`: Contains the key final datasets used in the analysis notebooks. <ul> <li>`commits_datasets/`: Contains the main classified commit dataset. <ul> <li>`HFCommitsClassification_final.csv`: The final dataset with over 960,000 classified commits for RQ1 and RQ2.</li> </ul> </li> <li>`releases_datasets/`: Contains the datasets related to releases. <ul> <li>`HFReleasesClassification.csv`: The final dataset of 2,251 classified releases for RQ3.</li> </ul> </li> <li>`model_metadata.csv`: The extracted internal metadata from model files for RQ3.4.</li> </ul> </li> <li>`metadata/`: Contains configuration files and the data used for the validation process. <ul> <li>`validation_data/`: A sub-folder containing the gold standard data. <ul> <li>`Agreement TOSEM Commit Changes.xlsx`: Excel containing details of the classication and validation processes.</li> <li>`prompt_refinement.txt`: The final, validated prompt used for the LLM classification along its previous iterations.</li> <li>`training_set_ground_truth.json`: Gold standard for the 143-commit training set.</li> <li>`training_set_first_classification.json`: First annotator's labels for the training IRR subset.</li> <li>`training_set_second_classification.json`: Second annotator's labels for the training IRR subset.</li> <li>`test_set_ground_truth.json`: Gold standard for the 384-commit test set.</li> <li>`test_first_classification.json`: First annotator's labels for the training IRR subset.</li> </ul> </li> <li>`test_set_second_classification.json`: Second annotator's labels for the test IRR subset.</li> <li>`tags_metadata.yaml`: Auxiliary metadata file used during preprocessing.</li> </ul> </li> <li>`README.md`: This file.</li> <li>`requirements.txt`: Lists the required Python packages.</li> </ul> <p>*Note: Other intermediate CSV files are provided to facilitate re-running specific parts of the analysis without starting from scratch.*</p> <p> </p> <h2>How to Use This Package</h2> <p> </p> <h3>Setup</h3> <div><strong>1. Create and activate a virtual environment</strong> (recommended).</div> <div>```bash</div> <div>python -m venv venv</div> <div>source venv/bin/activate # On Windows: venv\Scripts\activate</div> <div>```</div> <div><strong>2. Install required packages.</strong></div> <div>```bash</div> <div>pip install -r requirements.txt</div> <div>```</div> <p> </p> <h3>Running the Analysis</h3> <div>The Jupyter notebooks in the `code/` directory are numbered and named to be run in a logical sequence: <strong>Collection -> Preprocessing -> Analysis.</strong> We recommend following this order.</div> <div> </div> <div>- <strong>To reproduce our findings directly,</strong> you can start with the notebooks in `code/Analysis/`. They are configured to load the final, processed datasets provided in the `datasets/` folder.</div> <div>- <strong>To re-run the entire pipeline</strong>, start with the notebooks in `code/Collection/`, followed by `code/Preprocessing/`. Please note that running the full data collection and classification pipeline is time-consuming and may require significant computational resources and appropriate API keys for the LLM.</div> <p> </p> <h3>Key Datasets Provided</h3> <div><strong>- For RQ1 & RQ2:</strong> `datasets/commits_datasets/HFCommitsClassification_final.csv` (100,000 models for RQ1; filtered to 14,343 models for RQ2).</div> <div><strong>- For RQ3.1-3.3:</strong>`datasets/releases_datasets/HFReleasesClassification.csv` (2,251 releases from 202 models).</div> <div><strong>- For RQ3.4: </strong>`datasets/releases_datasets/model_metadata.csv` (from 28 models).</div> <p> </p> <h2>Contact</h2> <div>If you have any questions or encounter issues with this package, please contact the corresponding author. If you find our work useful, please consider citing our paper.</div>
Supplementary Data for 'Machine learning detection of SARS-CoV-2 high-risk variants'
Open the record for dataset details and reuse information.
Application of Machine Learning on MARCUS Aerosols to Remove Ship Stack Contamination
<p>contact qingniu@ou.edu for more info please</p>
Machine Learning Dataset for Rabies Diagnosis and Outbreak Prediction
Open the record for dataset details and reuse information.
ToolBoxSF: Robustly interrogating machine learning-based scoring functions: what are they learning?
<p>This Zenodo repository provides comprehensive resources for the pre-print research paper titled "Robustly interrogating machine learning-based scoring functions: what are they learning?" Our collection includes Singularity containers containing pre-trained models, benchmark datasets, and training/test CSV files, offering valuable insights into the inner workings of machine learning-based scoring functions.</p><p>Key Components:</p><p>Singularity Containers:</p><ul><li>Machine Learning Models: Explore state-of-the-art scoring models used in the study, enabling reproducibility and in-depth analysis.</li><li>Environment Setup: Simplify model deployment and experimentation by utilizing our pre-configured environments.</li></ul><p>Benchmark Datasets:</p><ul><li>Curated benchmark datasets used in the pre-print, facilitating validation and evaluation of scoring functions.</li></ul><p>Training and Test CSV Files:</p><ul><li>Training and test data in CSV format, along with associated metadata.</li><li>Facilitate model testing and comparison using the provided data.</li></ul><p>This Zenodo collection is a valuable resource for researchers, data scientists, and machine learning enthusiasts seeking to replicate the study's findings, explore model behaviors, and conduct further investigations into machine learning-based scoring functions. Detailed documentation and usage instructions are included to support your research efforts at <a href="https://github.com/guydurant/toolboxsf">https://github.com/guydurant/toolboxs</a>f.</p><p>Citation Information: Please cite this Zenodo repository when using our resources in your work, and consider acknowledging the original pre-print when publishing research based on these materials.</p>
MME-only models trained with clean data for JAMES paper "Machine-learned uncertainty quantification is not magic"
<p>This tar file contains all 100 trained models in the MME-only ensemble from Experiment 1 (i.e., those trained with clean data, not with lightly perturbed data). To read one of the models into Python, you can use the method neural_net.read_model in the ml4rt library.</p>
MME-only models trained with lightly perturbed data for JAMES paper "Machine-learned uncertainty quantification is not magic"
<p>This tar file contains all 100 trained models in the MME-only ensemble from Experiment 2 (i.e., those trained with lightly perturbed data). To read one of the models into Python, you can use the method neural_net.read_model in the ml4rt library.</p>
MME/CRPS models trained with clean data for JAMES paper "Machine-learned uncertainty quantification is not magic"
<p>This tar file contains all 100 trained models in the MME/CRPS ensemble from Experiment 1 (i.e., those trained with clean data, not with lightly perturbed data). To pare the ensemble down to 50 models, we randomly select 50. To read one of the models into Python, you can use the method neural_net.read_model in the ml4rt library.</p>
MME/CRPS models trained with lightly perturbed data for JAMES paper "Machine-learned uncertainty quantification is not magic"
<p>This tar file contains all 100 trained models in the MME/CRPS ensemble from Experiment 2 (i.e., those trained with lightly perturbed data). To pare the ensemble down to 50 models, we randomly select 50. To read one of the models into Python, you can use the method neural_net.read_model in the ml4rt library.</p>
Input data and some models (all except multi-model ensembles) for JAMES paper "Machine-learned uncertainty quantification is not magic"
<p>The tar file contains two directories: data and models. Within "data," there are 4 subdirectories: "training" (the clean training data -- without perturbations), "training_all_perturbed_for_uq" (the lightly perturbed training data), "validation_all_perturbed_for_uq" (the moderately perturbed validation data), and "testing_all_perturbed_for_uq" (the heavily perturbed validation data). The data in these directories are unnormalized. The subdirectories "training" and "training_all_perturbed_for_uq" each contain a normalization file. These normalization files contain parameters used to normalize the data (from physical units to z-scores) for Experiment 1 and Experiment 2, respectively. To do the normalization, you can use the script normalize_examples.py in the code library (ml4rt) with the argument input_normalization_file_name set to one of these two file paths. The other arguments should be as follows:</p><p>--uniformize=1</p><p>--predictor_norm_type_string="z_score"</p><p>--vector_target_norm_type_string=""</p><p>--scalar_target_norm_type_string=""</p><p> </p><p>Within the directory "models," there are 6 subdirectories: for the BNN-only models trained with clean and lightly perturbed data, for the CRPS-only models trained with clean and lightly perturbed data, and for the BNN/CRPS models trained with clean and lightly perturbed data. To read the models into Python, you can use the method neural_net.read_model in the ml4rt library.</p>
Demonstration of Portable Performance of Scientific Machine Learning on High Performance Computing Systems
<p>With the largest datasets to date and a diverse set of discoveries to be made, the current generation of scientific analyses are well poised to utilize artificial intelligence (AI) and machine learning (ML) on high performance computing (HPC) resources. Like never before, these workflows can be written in one portable language, python, which thanks to highly-optimized ML libraries achieves excellent cross-platform performance with little to no intervention by the user. In this demonstration, we explore the performance of several scientific AI/ML applications across leading HPC resources and highlight best practices for portable performance.</p>
Machine learning-based high-frequency neuronal spike reconstruction from low-frequency and low-sampling-rate recordings
<p>Supplementary Data for<strong> "Machine learning-based high-frequency neuronal spike reconstruction from low-frequency and low-sampling-rate recordings", Hong et al.</strong></p>
Data for MAPS: Pathologist-level Cell Type Annotation from Tissue Images through Machine Learning
<p>Extracted datasets and processed images used to generate figures in manuscript <i>"MAPS: Pathologist-level Cell Type Annotation from Tissue Images through Machine Learning".</i></p>
Code and Data Repository for "Identifying microbiota community patterns important for plant protection using machine learning in synthetic community experiments"
<p>The repository includes data and R code necessary to reproduce all figures and analyses presented in the study titled "Identifying microbiota community patterns important for plant protection using machine learning in synthetic community experiments".</p>
Cystic Fibrosis Diagnosis in Newborns with Machine Learning]{Newborn Cystic Fibrosis Diagnosis Made Accurate and Efficient with Machine Learning to Reduce False Positives in IRT-Trypsinogen Immunoreactive Screening Program
<p>Datasets to training models in this article and generator code.</p>
Supplement material to the study: Towards understanding the influence of seasons on low groundwater periods based on explainable machine learning
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.