Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
650
datasets available to search
ShareScore release 0.9.0
Dataset results
650 results for “workflows”
A dataset of GitHub Actions workflow histories
<p>This replication package accompagnies the dataset and exploratory empirical analysis reported in the paper "A dataset of GitHub Actions workflow histories" published in the IEEE MSR 2024 conference. (The Jupyter notebook can be found in previous version of this dataset).</p> <p><em><strong>Important notice :</strong> It looks like Zenodo is compressing gzipped files two times without notice, they are "double compressed". So, when you download them they should be named : <code>x.gz.gz</code> instead of <code>x.gz</code>. Notice that the provided MD5 refers to the original file. </em></p> <p><em><strong>2025-10-09 update: update repositories list and observation period</strong>. We now have 3M+ workflows from 49.2K+ repositories. We consider repositories with at least one commit after August 25th, 2024, and they were pulled on August 25th-26th, 2025.</em></p> <p>2025-04-15 update: fix missing metadata and minor notation bug. (unchanged observation period)</p> <p><strong>2024-10-25 update: update repositories list and observation period</strong>. <em>We now have 2.3M+ workflows from 43.3K+ repositories. We consider repositories with at least one commit after January 1st, 2024, and they were pulled on October 7th, 2024.</em></p> <p>2024-07-09 update: fix sometimes invalid <code>valid_yaml</code> flag.</p> <p>2024-04-30: initial version</p> <p>The dataset was created as follow : </p> <ol> <li>First, we used GitHub SEART (on August 25th, 2025) to get a list of every non-fork repositories created at least one year before. having at least 300 commits and at least 100 stars where at least one commit was made in the last year. (The goal of these filter is to exclude experimental and personnal repositories).</li> <li>We checked if a folder <code>.github/workflows</code> existed. We filtered out those that did not contained this folder and pulled the others (on August 25th-26th, 2025).</li> <li>We applied the tool <code>gigawork</code> (version 1.4.2) to extract every files from this folder. The exact command used is <code>python batch.py -d /ourDataFolder/repositories -e /ourDataFolder/errors -o /ourDataFolder/output -r /ourDataFolder/repositories_everything.csv.gz -- -w /ourDataFolder/workflows_auxiliaries</code>. (The script <code>batch.py</code> can be found <a href="https://github.com/cardoeng/gigawork/blob/master/scripts/batch.py" target="_blank" rel="noopener">on GitHub</a>).</li> <li>We concatenated every files in <code>/ourDataFolder/output</code> into a csv (using <code>cat headers.csv output/*.csv > workflows_auxiliaries.csv</code> in <code>/ourDataFolder</code>) and compressed it.</li> <li>We added the column <code>uid</code> via a script available <a href="https://github.com/cardoeng/gigawork/blob/master/scripts/uid.py">on GitHub.</a></li> <li>Finally, we archived the folder with pigz <code>/ourDataFolder/workflows</code> (<code>tar -c --use-compress-program=pigz -f workflows_auxiliaries.tar.gz /ourDataFolder/workflows</code>)</li> </ol> <p>Using the extracted data, the following files were created :</p> <ol> <li><code>workflows.tar.gz</code> contains the dataset of GitHub Actions workflow file histories.</li> <li><code>workflows_auxiliaries.tar.gz</code> is a similar file containing also auxiliary files.</li> <li><code>workflows.csv.gz</code> contains the metadata for the extracted workflow files.</li> <li><code>workflows_auxiliaries.csv.gz</code> is a similar file containing also metadata for auxiliary files.</li> <li><code>repositories.csv.gz</code> contains metadata about the GitHub repositories containing the workflow files. These metadata were extracted using the SEART Search tool. </li> </ol> <p>The metadata is separated in different columns:</p> <ol> <li><code>repository</code>: The repository (author and repository name) from which the workflow was extracted. The separator "/" allows to distinguish between the author and the repository name</li> <li><code>commit_hash</code>: The commit hash returned by git</li> <li><code>author_name</code>: The name of the author that changed this file</li> <li><code>author_email</code>: The email of the author that changed this file</li> <li><code>committer_name</code>: The name of the committer</li> <li><code>committer_email</code>: The email of the committer</li> <li><code>committed_date</code>: The committed date of the commit</li> <li><code>authored_date</code>: The authored date of the commit</li> <li><code>file_path</code>: The path to this file in the repository</li> <li><code>previous_file_path</code>: The path to this file before it has been touched</li> <li><code>file_hash</code>: The name of the related workflow file in the dataset</li> <li><code>previous_file_hash</code>: The name of the related workflow file in the dataset, before it has been touched</li> <li><code>git_change_type</code>: A single letter (A,D, M or R) representing the type of change made to the workflow (Added, Deleted, Modified or Renamed). This letter is given by <code>gitpython</code> and provided as is. </li> <li><code>valid_yaml</code>: A boolean indicating if the file is a valid YAML file.</li> <li><code>probably_workflow</code>: A boolean representing if the file contains the YAML key <code>on</code> and <code>jobs</code>. (Note that it can still be an invalid YAML file).</li> <li><code>valid_workflow</code>: A boolean indicating if the file respect the syntax of GitHub Actions workflow. A freely available JSON Schema (used by gigawork) was used in this goal.</li> <li><code>uid</code>: Unique identifier for a given file surviving modifications and renames. It is generated on the addition of the file and stays the same until the file is deleted. Renamings does not change the identifier.</li> </ol> <p>Both <code>workflows.csv.gz</code> and <code>workflows_auxiliaries.csv.gz</code> are following this format.</p>
ARTE (Article Reproducibility Template & Environment) workflow folder structure
<p>Illustration of a suggested folder and file system for editing articles that are <strong>dynamic and reproducible</strong>.</p> <p>The <a title="TIER Protocol 4.0 Site" href="https://www.projecttier.org/tier-protocol/protocol-4-0/" target="_blank" rel="noopener"><strong>TIER Protocol 4.0</strong></a> served as the foundation for this proposal, which was intended to be modified in <strong>Quarto using Rstudio</strong>, for instance, then published on <a title="Article Template Exemple" href="https://phdpablo.github.io/article-template/" target="_blank" rel="noopener"><strong>GitHub Pages</strong></a>.</p> <p>An example of its deployment is provided here: <a title="Article Template Example" href="https://phdpablo.github.io/article-template/" target="_blank" rel="noopener">https://phdpablo.github.io/article-template/</a></p> <p>According to <a title="Original Article" href="https://periodicos.ufpe.br/revistas/index.php/politicahoje/article/view/245776" target="_blank" rel="noopener">Domingos and Batista's (2021)</a> suggested content (files) recommendations, the figure illustrates the design of the proposal while accounting for the three primary <strong>TIER Protocol 4.0</strong> folders (Data, Scripts, and Output). The proposal for the <strong>TIER Protocol 4.0</strong> was modified to take into account that the research narrative would be edited in a <a title="Site Quarto" href="https://quarto.org/docs/guide/" target="_blank" rel="noopener"><strong>dynamic Quarto-type document</strong></a> (*.qmd), published on <a title="Article Template Example" href="https://phdpablo.github.io/article-template/" target="_blank" rel="noopener">GitHub Pages</a> (docs/) following Rstudio rendering, and potentially even produce a .pdf file of the article; <strong>*.qmd </strong> denotes a collection of <strong>Quarto files </strong>containing the standard sections of a scientific article (Introduction, Theoretical Framework, Methods, etc.); the other folders indicated in the root (adm, docker, renv) are for project management and for controlling dependencies and the environment, if adopted by the researcher.</p> <p>Visit the project repository at <a title="GitHub Repository" href="https://github.com/phdpablo/article-template/" target="_blank" rel="noopener">https://github.com/phdpablo/article-template/</a> for additional details. Visit <a title="OSF" href="https://osf.io/njdq5/" target="_blank" rel="noopener">https://osf.io/njdq5/</a> to view the template on OSF.</p>
Surfalex HF formability study - Workflow 5 - Simulate Marciniak-Kuczynski
<p>This group of MatFlow workflows is the fifth in a set of eight workflow groups developed during our formability study of the Surfalex HF (AA6016A) material. In these workflows, we used Abaqus to perform deformation of a grooved sheet using the finite element method, for different displacement boundary conditions and different groove angles (relative to the major loading direction). Thus, we performed a numerical Marciniak-Kuczynski analysis on the Surfalex material, which allowed us to predict the forming limit curve. We also include various sensitivity studies.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Surfalex HF formability study - Workflow 6 - Generate random volume element
<p>This MatFlow workflow is the sixth in a set of eight workflows developed during our formability study of the Surfalex HF (AA6016A) material. In this workflow, we generate a comparison volume element from a random texture and equiaxed microstructure. This RVE is used in a comparison of the simulated Lankford coefficients between the Surfalex model RVE and this "random" RVE.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Surfalex HF formability study - Workflow 7 - Lankford coefficient
<p>This group of MatFlow workflows is the seventh in a set of eight workflow groups developed during our formability study of the Surfalex HF (AA6016A) material. In these workflows, we perform uniaxial tensile tests on two RVEs: the Surfalex RVE and an RVE with a random texture. We use this data to predict the Lankford coefficient in both cases.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Surfalex HF formability study - Workflow 3 - Fit yield functions
<p>This MatFlow workflow is the third in a set of eight workflows developed during our formability study of the Surfalex HF (AA6016A) material. In this workflow, we fitted three anisotropic yield functions to yield stress data obtained from a large set of multiaxial crystal plasticity simulations. The fitted yield functions were: the Hill1948 yield criterion, the Barlat Yld91 (6-parameter) yield function, and the Barlat Yld2004-18p (18-parameter) yield function. DAMASK was used to perform these simulations.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Surfalex HF formability study - Workflow 1 - Generate volume element
<p>This MatFlow workflow is the first in a set of eight workflows developed during our formability study of the Surfalex HF (AA6016A) material. In this first workflow, we generated a representative volume element (RVE) for the Surfalex HF material. To do this, we sampled 2000 orientations from a CTF file generated from EBSD measurements on the sheet RD-TD plane. The MTEX toolbox was used to sample the texture. The grain morphology was approximated using a Voronoi tessellation that was subsequently stretched by a factor of 1.5 in the RD direction, to mimic the slight grain elongation that was observed. The pre-processing tools in the DAMASK package were used to generated the RVE.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Surfalex HF formability study - Workflow 2 - Fit single crystal parameters
<p>This MatFlow workflow is the second in a set of eight workflows developed during our formability study of the Surfalex HF (AA6016A) material. In this workflow, we calibrated crystal plasticity (CP) parameters using experimental data. The hardening coefficient, the initial and maximum CRSS values, and the hardening exponent, as defined in the CP phenomenological power law adopted by DAMASK were optimised. Starting from trial guesses for these parameters, we optimised the parameters with respect to the experimental stress-strain curve of the Surfalex HF material, using a Levenberg-Marquardt procedure. At each iteration of the process, N+1 CP simulations were performed (for N fitting parameters). Five iterations were used in total.</p> <p>This workflow can be downloaded and explored in a Jupyter notebook, as explained in the <a href="https://github.com/LightForm-group/surfalex_data_explorer">GitHub repository here</a>.</p>
Reconstruction of prokaryotic genomes from ten termite gut metagenomes using two distinct workflows: SnakeMAGs and ATLAS.
<p><strong><em>SnakeMAGs</em></strong> (Nachida Tadrent, Franck Dedeine, Vincent Hervé (Submitted). <em>SnakeMAGs</em>: a simple, efficient, flexible and scalable workflow to reconstruct prokaryotic genomes from metagenomes<em>.</em> <a href="https://doi.org/10.5281/zenodo.7303463">https://doi.org/10.5281/zenodo.7303463</a>; https://github.com/Nachida08/SnakeMAGs) is a workflow for building MAGs (Metagenome Assembled Genomes) from raw Illumina metagenomic reads. During the test phase of the development of this tool, a comparative analysis with another workflow called ATLAS v2.9.1 (<em>Kieser </em>et al, 2020) was performed. To compare these two workflows, we analyzed ten publicly available termite gut metagenomes (accession numbers: SRR10402454; SRR14739927; SRR8296321; SRR8296327; SRR8296329; SRR8296337; SRR8296343; DRR097505; SRR7466794; SRR7466795) from five different studies : Waidele et al, 2019; Tokuda et al, 2018; Romero Victorica et al, 2020; Moreira et al, 2021; and Calusinska et al, 2020.</p> <p>In this repository, we provide the configuration files that were used to launch each of the workflows (SnakeMAGs_config.yaml and ATLAS_config.yaml), as well as the obtained results, <em>i.e. </em>the MAGs reconstructed from each metagenome and their taxonomic classification.</p>
Example input for MAPP workflow
<p>Small test dataset for MAPP workflow:</p> <ul> <li>5mln reads sampled from two paired-end RNA-seq samples of the GEO record: GSE69656</li> <li>100 arbitrarily chosen Position Weight Matrices for RNA binding proteins and their sequence logos</li> <li>configuration template for the workflow (YAML)</li> <li>experiment design table (TSV)</li> <li>Genomic annotation for human: chr21 (GTF)</li> <li>Genomic sequence for human: chr21 (FASTA)</li> <li>PolyAsite Atlas 2.0 annotation: chr21 (BED)</li> </ul>
Dataset: Correlative Light, Electron Microscopy and Raman Spectroscopy Workflow to Detect and Observe Microplastic Interactions with Whole Jellyfish
<p>ABSTRACT</p> <p>Many researchers have turned their attention to understanding microplastic interaction with marine fauna. Efforts are being made to monitor exposure pathways and concentrations, and to assess the impact such interactions may have. To answer these questions, it is important to select appropriate experimental parameters and analytical protocols. This study focuses on medusae of <em>Cassiopea andromeda</em> jellyfish: a unique benthic jellyfish known to favor (sub-)tropical coastal regions which are potentially exposed to plastic waste from land-based sources. Juvenile medusae were exposed to fluorescent poly(ethylene terephthalate) and polypropylene microplastics (< 300 µm), resin embedded, and sectioned before analysis with confocal laser scanning microscopy as well as transmission electron microscopy and Raman Spectroscopy. Results show the fluorescent microplastics were stable enough to be detected with the optimized analytical protocol presented, and that their observed interaction with medusae occurs in a manner which is likely driven by the microplastic properties (<em>e.g.</em> density, hydrophobicity).</p>
Tryps-IN: A streamlined palaeoproteomics workflow enables ZooMS analysis of 10,000-year-old petrous bones from Jordan rift-valley
<p>Poor preservation of collagen in dry and/or arid environments has hindered the application of Zooarchaeology by mass spectrometry (ZooMS) analysis in many regions of the world, and as a result many zooarchaeological investigations have relied exclusively on the morphological assessment of fragmentary remains, due to the inadequate preservation of biomolecules. The climatic conditions of Southwest Asia include extreme temperature fluctuations unconducive to preservation of proteins and DNA. We performed zooarchaeological analysis of remains from the 10,000-year-old site of Shkārat Msaied in Jordan and sub-sampled twenty-eight petrous bones, the hardest bone in the mammalian skeleton, for species identification by ZooMS. Using an unconventional and simplified extraction protocol we call Tryps-IN, in which digestion was performed without removal of the demineralising EDTA, we taxonomically identified several fragments, outperforming the established ZooMS work-flow. A subset of identifications was subsequently confirmed using liquid chromatography coupled to tandem mass spectrometry (LC-MS/MS) protein sequencing. The new methodology presented here opens the possibility of further bioarchaeological investigation of other fragmentary faunal assemblages within this region of archaeological significance. </p>
Demo accompanying the poster "First steps towards a workflow for 3D-models based on IIIF"
<p>This demo illustrates first attempts to develop an automatised workflow to create light-weight 3D-models from the original heavy files which can be viewed and annotated in an IIIF-compatible manner.</p>
Test clustered weather dataset for s2spy workflow
<p>This dataset contains sea surface temperature (SST) over the Pacific and clustered 2 meter temperature (T2M) over North America. It is used in the example workflow of s2spy/lilio packages. The fields used here are processed outputs from the original ERA5 dataset.</p> <p>More details about how this dataset was generated can be found via this link: https://github.com/AI4S2S/cookbook/tree/main/data</p> <p>About the usage of this dataset in the example machine learning workflow of s2spy/lilio, check this link:https://github.com/AI4S2S/cookbook</p> <p>Data used here is generated using Copernicus Climate Change Service information and for more information about licensing, please check the Licence Agreement (https://cds.climate.copernicus.eu/cdsapp/#!/terms/licence-to-use-copernicus-products) for Copernicus Products.</p>
Execution Traces of an MNIST Workflow on a Serverless Edge Testbed
<p>For the evaluation of a Serverless Edge Computing platform, we built an Edge Cloud testbed consisting of several machines:</p> <ul> <li>A Cloud VM</li> <li>An Nvidia Jetson TX</li> <li>Four Raspberry Pi 3b+</li> <li>Two Intel NUCs with i5 processors</li> </ul> <p>We were interested in profiling these devices with a Machine Learning workflow deployed as a serverless application. To that end, we implemented three functions: Preprocessing, Training, and Serving as OpenFaaS functions. The workflow trains an MNIST model.</p> <p> </p>
Test data for running snakePipes : scRNA-seq workflow
<p><strong>Test files for running snakePipes workflows</strong></p> <p><strong>snakePipes</strong> are pipelines built using snakemake and python for the analysis of epigenomic datasets. Please refer to <a href="https://snakepipes.readthedocs.io/en/latest/">this link</a> for further information on snakePipes.</p> <p>This folder contains test files that can be used to run scRNA-seq workflow under snakePipes. To test the workflow, follow the following steps : </p> <ul> <li>Download or prepare genome fasta, indices and annotations for mouse (<strong>mm10</strong>) genome.</li> <li>Download and install snakePipes via `conda create -n snakePipes -c mpi-ie -c bioconda -c conda-forge snakePipes`</li> <li>Update <a href="https://snakepipes.readthedocs.io/en/latest/content/running_snakePipes.html#genome-configuration-file">Genome configuration file</a> with path to indices and annotations.</li> <li>Move to this repository and run the example <strong>command.sh</strong></li> </ul>
Output reports and supplementary data for MTB workflow
<p>The archive contains the following data:</p> <ul> <li>Output of the workflow on all validation samples (tabular summaries and full HTML reports).</li> <li>Database with AMR regions and mutations.</li> </ul> <p> </p>
Porting Workflow for SAGrid applications (November 2014)
<p>This is the application porting workflow for new applications, using the continuous integration service. As of 15.12.2014</p>
RDA Publishing Workflows_ Research Workflows (Responses)
<p>Responses to online questionnaire / call for examples from Research Data Alliance Working Group on Publishing Research Data Workflows.</p> <p>These relate to the article 'Connecting data publication to the research workflow: a preliminary analysis' by the same authors, submitted to the International Digital Curation Conference, 2017</p>
workflow
<p>This dataset uses the CK OO metrics.</p> <p>More information at http://openscience.us/repo/defect/ck/workflow.html</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.