Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
650
datasets available to search
ShareScore release 0.9.0
Dataset results
650 results for “Workflow”
Workshop Material - 3D-e-Chem Structural Cheminformatics Workflows for Computer-Aided Drug Discovery
<p>The workshop at the KNIME user meeting (Berlin 9th of March 2018) is set up to stimulate participants with varying degrees of experience in cheminformatics to learn and apply the different structural cheminformatics tools and workflows developed within the context of the 3D-e-Chem project. You will learn how to construct and apply integrated cheminformatics workflows using the 3D-e-Chem KNIME nodes for the exploitation of G protein-coupled receptor and kinase data (two important pharmaceutical target classes) to obtain useful information for drug discovery.</p> <p>Information on the 3D-e-Chem KNIME nodes and workflows can be found online:</p> <p>3D-e-Chem GitHub website: <a href="http://3d-e-chem.github.io/">http://3d-e-chem.github.io/</a></p>
alawinia/provClustering: Discovering Similar Workflows via Provenance Clustering
<p>Several workflow management systems and scripting languages have adopted provenance tracking, yet many researchers choose to manually capture or instrument their processing scripts to write provenance information to files. The Next Generation Sequencing (NGS) project we are associated with is tracking provenance in such manner. The NGS project is a collaboration between multiple groups at different sites, where each group is collecting and processing samples using an agreed-upon workflow. The workflow contains many stages with varying degrees of complexity. Over time workflow stages are modified, but data samples are only comparable when processed with identical versions of the workflow. However, for various reasons (including the distributed nature of the collaboration) it is not always clear which samples have been processed with which version of the workflow. In this paper, we introduce new techniques for clustering provenance datasets and attempt to discover the ones that are likely to be generated by same workflow. Based on the clustering result, users can identify similar provenance and would be able to categorize them into different clusters for debugging and zoom-in/zoom-out viewing.</p>
Simulated quantification files for "Swimming downstream" workflow (bias 1-6 + uniform 7-12)
<p>Simulated quantification files for "Swimming downstream" workflow <a href="https://doi.org/10.12688/f1000research.15398.3">https://doi.org/10.12688/f1000research.15398.3</a></p> <p>Salmon (0.11.3) with Gibbs samples and kallisto (0.44.0) quantification files for the samples in two condition groups and two balanced batches: with realistic bias (1-6) and samples with uniform coverage (7-12). 24 samples in total. Reference: Gencode v28 human transcripts.</p>
The reference index files used for RNA-seq workflow benchmark in CWL-metrics manuscript
<p>The reference files used in the RNA-Seq workflow benchmark in the manuscript "Accumulating computational resource usage of genomic data analysis workflow to optimize cloud computing instance selection" (https://doi.org/10.1101/456756).</p>
The output and the log files from RNA-Seq workflow benchmark for CWL-metrics manuscript
<p>The output files and log files generated by the workflow executions for RNA-Seq workflow benchmark by CWL-metrics, from the manuscript "Accumulating computational resource usage of genomic data analysis workflow to optimize cloud computing instance selection" (https://doi.org/10.1101/456756).</p>
CWL run of Alignment Workflow (CWLProv 0.6.0 Research Object)
<p>This dataset folder is a <strong>CWLProv Research Object</strong> that captures the <a href="http://commonwl.org/">Common Workflow Language</a> execution provenance, see <a href="https://w3id.org/cwl/prov/0.6.0">CWLProv 0.6.0</a> or use the <a href="https://pypi.org/project/cwlprov/">cwlprov Python tool</a> to explore.</p> <p> </p> <p>The CWL <a href="https://w3id.org/cwl/view/git/923ae325fa3a6d3ee3871539c65afa06ac86ac54/aligner/sbg-alignment-cwl/topmed-alignment.cwl">alignment workflow</a> included in this case study is designed by <a href="https://github.com/DataBiosphere">Data Biosphere</a>. It adapts the alignment pipeline originally developed at Abecasis Lab, The University of Michigan. This <a href="https://github.com/FarahZKhan/topmed-workflows/tree/cwlprov_testing">workflow</a> is part of NIH Data Commons initiative and comprises of four stages.</p> <p>First step, <strong><em>Pre-align</em></strong><em>,</em> accepts a <em>Compressed Alignment Map</em> (CRAM) file (a compressed format for BAM files developed <em>by European Bioinformatics Institute</em> (EBI)) and human genome reference sequence as input and using underlying software utilities of <strong>SAMtools</strong> such as <em>view</em>, <em>sort</em> and <em>fixmate</em> returns a list of fastq files which can be used as input for the next step.</p> <p>The next step <strong><em>Align</em></strong> also accepts the human reference genome as input along with the output files from <em>Pre-align</em> and uses <strong>BWA-mem</strong> to generate aligned reads as BAM files. <strong>SAMBLASTER</strong> is used to mark duplicate reads and <strong>SAMtools</strong> view to convert read files from SAM to BAM format.</p> <p>The BAM files generated after <em>lign</em> are sorted with <strong><em>SAMtool sort</em></strong>'.</p> <p>Finally, these sorted alignment files are merged to produce single sorted BAM file using SAMtools merge in <strong><em>Post-align</em></strong> step.</p> <p> </p> <p><strong>Steps to reproduce</strong></p> <p>This analysis was run using a 16-core Linux cloud instance with 64GB RAM and pre-installed docker.</p> <ol> <li> <p>Install <strong>gsutils</strong><br> </p> <pre><code class="language-bash">export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | \ sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | \ sudo apt-key add - sudo apt-get update && sudo apt-get install google-cloud-sdk</code></pre> <p> </p> </li> <li> <p>Get the data and make the analysis environment ready:<br> </p> <pre><code class="language-bash">git clone https://github.com/FarahZKhan/topmed-workflows.git cd topmed-workflows git checkout cwlprov_testing cd aligner/sbg-alignment-cwl # this is a custom script download google bucket files from json files and create a local json # it needs gsutil to be installed though git clone https://github.com/DailyDreaming/fetch_gs_frm_json.git # Wait... this should download ~18Gb. python2.7 fetch_gs_frm_json/dl_gsfiles_frm_json.py topmed-alignment.sample.json </code></pre> <p> </p> </li> <li> <p>Run the following commands to create the CWLProv Research Object:</p> <pre><code class="language-bash">time cwltool --no-match-user --provenance alignmnentwf0.6.0 --tmp-outdir-prefix=/CWLProv_workflow_testing/intermediate_temp/temp --tmpdir-prefix=/CWLProv_workflow_testing/intermediate_temp/temp topmed-alignment.cwl topmed-alignment.sample.json.new zip -r alignment_0.6.0_linux.zip alignment_0.6.0_linux sha256sum alignment_0.6.0_linux.zip > alignment_0.6.0_linux.zip.sha25</code></pre> <p> </p> </li> </ol>
NGSAP-VC : Genomic Variant Calling as an Installable GALAXY Workflow Using NGS data.
<p>Implementation of genomic variants calling as an installable GALAXY workflows using NGS data. Repository contains two separate sets of simulated ebola test data. One for SNPs and INDELs calling and another for Structural Variants calling.</p>
FIGURE 2 in Digitization workflows for paleontology collections
FIGURE 2. Example implementation of the workflow modules described herein.
Snakemake workflow for Nanopore-only bacterial assembly and QC
<p>This repository contains a Snakemake workflow tailored for assembling bacterial genomes from long-read data generated with R10.4.1 simplex reads from Oxford Nanopore Technologies and includes several QC steps on the resulting assemblies. The latest version of the scripts can be found at https://gitlab.ilvo.be/genomics/wgs/nanopore-only-bacterial-assembly-snakemake.</p>
Data Science Tasks used in "AI Support for Data Scientists: An Empirical Study on Workflow and Alternative Code Recommendation"
<p>This entry contains the supplementary files for a scientific article. </p> <p>The dataset contains the necessary files for the two data science tasks used in the experiment study from scientific article "AI Support for Data Scientists: An Empirical Study on Workflow and Alternative Code Recommendation"</p> <p> </p>
Demo-Dataset for publication "FAIR workflows in Earth system modelling: a use case with semantic data management"
<p>This demodataset is intended to be used to test the workflow described in the publication by Lennartz & Schlemmer "FAIR workflows in Earth System modelling: a use case with semantic data management". It contains example model output for an arbitrary biogeochemical model tracer (here: dissolved organic carbon, DOC) from an ocean model as a 4-dimensional dataset (latitude, longitude, depth, time), the corresponding grid point locations as well as a textfile specifying parameter inputs for the model. The file structure is adapted for seamless integration into the workflow described in Lennartz & Schlemmer, which builds on the open source semantic research data management system LinkAhead. The dataset contains the following structure: The folder DataAnalysis stores data required for data analysis, such as the grid point locations in the file TMM_grid_v2018a.mat. The folder SimulationData stores model output in the folder 2022_TMM, containing the parameter input file nl_in.txt and the model output TR_monthly.mat. Related instructions can be accessed here: https://gitlab.com/salexan/fairworkflows-demodataset .</p>
Workflow Automation of SEM Acquisitions and Feature Tracking
<p>Data related to the paper <em>Workflow Automation of SEM Acquisitions and Feature Tracking. </em>Includes images used for nanoparticle analysis and for tracking.</p>
Agreement of multiple night- and daytime filtering approaches of eddy covariance-derived net ecosystem CO2 exchange over a mountain forest. Reproducible workflow.
<p>Datasets and python scripts to reproduce results from the publication <em>Agreement of multiple night- and daytime filtering approaches of eddy covariance-derived net ecosystem CO2 exchange over a mountain forest.</em></p> <p>See README.txt for a description of the single files.</p>
Data from: Detection of the endangered European weather loach (Misgurnus fossilis) via water and sediment samples: testing multiple eDNA workflows.
<p>The European weather loach (<i>Misgurnus fossilis</i>) is classified as highly endangered in several countries of Central Europe. Populations of <i>M. fossilis</i> are predominantly found in ditches with low water levels and thick sludge layers and are thus hard to detect using conventional fishing methods. Therefore, environmental DNA (eDNA) monitoring appears particularly relevant for this species. In previous studies, <i>M. fossilis</i> was surveyed following eDNA water sampling protocols, which were not optimized for this species. Therefore, we created two full factorial study designs to test six different eDNA workflows for sediment samples and twelve different workflows for water samples. We used qPCR to compare the Threshold cycle (Ct) values of the different workflows, which indicate the target DNA amount in the sample, and spectrophotometry to quantify and compare the total DNA amount inside the samples. We analyzed 96 water samples and 48 sediment samples from a pond with a known population of <i>M. fossilis</i>. We tested several method combinations for long-term sample preservation, DNA capture and DNA extraction. Additionally, we analyzed the DNA yield of samples from a ditch with a natural <i>M. fossilis</i> population monthly over one year to determine the optimal sampling period. Our results showed that the long-term water preservation method commonly used for eDNA surveys of <i>M. fossilis </i>did not lead to optimal DNA yields, and we present a valid long-term sample preservation alternative. A cost-efficient high salt DNA extraction led to the highest target DNA yields and can be used for sediment and water samples. Furthermore, we were able to show that in a natural habitat of <i>M. fossilis</i>, total and target eDNA were higher between June and September, which implies that this period is favorable for eDNA sampling. Our results will help to improve the reliability of future eDNA surveys of <i>M. fossilis</i>.</p>
Tutorial output for Tourmaline amplicon sequence processing workflow
<p>Tutorial output for the <a href="https://github.com/aomlomics/tourmaline">Tourmaline</a> amplicon sequence processing workflow.</p> <p>Tourmaline was run on the test data provided in the directory <a href="https://github.com/aomlomics/tourmaline/tree/master/00-data">00-data</a>, which were downloaded along with the rest of the repository using this command:</p> <pre><code>git clone https://github.com/aomlomics/tourmaline</code></pre> <p>Reference data were downloaded and symlinked using these commands:</p> <pre><code>cd tourmaline/01-imported wget https://data.qiime2.org/2021.2/common/silva-138-99-seqs-515-806.qza wget https://data.qiime2.org/2021.2/common/silva-138-99-tax-515-806.qza ln -s silva-138-99-seqs-515-806.qza refseqs.qza ln -s silva-138-99-tax-515-806.qza reftax.qza</code></pre> <p>Paths in 00-data/manifest_pe.csv and 00-data/manifest_se.csv were edited to match the local paths.</p> <p>Output for all modes of the workflow were then generated in series:</p> <pre><code>conda activate qiime2-2021.2 snakemake dada2_pe_report_unfiltered snakemake dada2_pe_report_filtered snakemake dada2_se_report_unfiltered snakemake dada2_se_report_filtered snakemake deblur_se_report_unfiltered snakemake deblur_se_report_filtered</code></pre> <p> </p>
ACORBA: Automated workflow to measure Arabidopsis thaliana root tip angle dynamic
<p>Raw data linked to the research article "ACORBA: Automated workflow to measure Arabidopsis thaliana root tip angle dynamic"</p>
IWC : Test data for VGP workflows v2.0
<p>Test data for VGP workflows in <a href="https://github.com/galaxyproject/iwc">iwc</a>. Pipeline VGP assembly v2.0</p>
How are software repositories mined? A systematic literature review of workflows, methodologies, reproducibility, and tools
<p>This is the excel spreadsheet dataset containing our analysis of papers performing mining software repositories research from the conferences ICSE, ESEC/FSE, and MSR from the years 2018 - 2020. The data is broken into columns and can be explained at a high-level as follows:</p> <p>Column Content</p> <p>1 The paper being analyzed</p> <p>2 Does the paper state the data they analyzed is available</p> <p>3 Does the paper perform some sort of data analysis or sampling using data others have compiled in the past</p> <p>4 Does the paper state a timestamp for when they begin their work</p> <p>5 Does the paper state the use of systems pre-built to help with MSR work</p> <p>6 - 18 Forms of sampling researchers may have employed to select their data</p> <p>19 What datasets (if any) were used in the analysis</p> <p>20 What tools (if any) were used in the analysis</p> <p>21 How they performed their data sampling workflow</p> <p>22 How they performed their data filtering workflow</p> <p>23 How they performed their data retrieval workflow</p> <p>24 Did they create any scripts in each of these workflows</p> <p>25 - 33 Did they publish a replication package and what is contained within</p> <p>34 Is the paper describing a tool for research or not</p> <p>35 Short description of the paper read</p> <p>36 A high-level category of the work performed in each paper</p>
A computational workflow for cell line profiling by Imaging Mass Cytometry.
<p>Imaging Mass Cytometry Data as 32-bit single TIFF with computational analysis from the manuscript: <strong>A computational workflow for cell line profiling by Imaging Mass Cytometry.</strong></p> <p><strong><span lang="EN-US">Breast cancer cell lines SKBR3 MCF7 HCC1143 IMC data and CellProfiler pipelines.zip</span></strong></p> <p><strong><span lang="EN-US">Elongated cell lines HeLa SKOV3 BJ IMC data and CellProfiler pipelines.zip:</span></strong></p> <p><strong><span lang="EN-US">Small cell lines A431 HT29 BxPC3 IMC data and CellProfiler pipelines.zip</span></strong></p> <p><strong><span lang="EN-US">U937 PMA-differentiated cells IMC data and CellProfiler pipeline.zip</span></strong></p> <p><strong><span lang="EN-US">A431 Cisplatin Study IMC data and CellProfiler pipeline.zip</span></strong></p> <p><span lang="EN-US">Contains 1 folder per cell line or drug treatment of single TIFF 32-bit markers exported from MCD/txt files (including Xe131 channel) and their respective cpproj. pipeline file for IMC Cell Line Profiler workstream reproducible analysis</span></p> <p><strong><span lang="EN-US">IMC Cell Line Profiler high dimensional and correlation analysis R scripts.zip</span></strong></p> <p><span lang="EN-US">Contains three adaptable R scripts for high dimensional analysis, correlation analysis and combination of both scripts for Machine Learning classified datasets.</span></p> <p><strong><span lang="EN-US">Breast cancer cell lines nuclear state classification by CellProfiler Analyst MLs.zip</span></strong></p> <p><span lang="EN-US">Contains SQLite databases, properties files, training datasets, nuclear classes visual rendering, and classifier model files with outputs for two machine learning classifiers (Random Forest and Fast Gentle Boosting) per breast cancer cell line for CellProfiler Analyst workflow reproducibility.</span></p> <p><strong><span lang="EN-US">A431 Cisplatin Study IMC data nuclear state classification by CellProfiler Analyst MLs.zip</span></strong></p> <p><span lang="EN-US">Contains SQLite databases, properties files, training datasets, classifier model with outputs for Fast Gentle Boosting and Random Forest per treatment for CellProfiler Analyst workflow reproducibility.</span></p> <p><strong><span lang="EN-US">IMC Cell Line Profiler pseudo-color images with Ki-67 marker Cytoplasm marker and Cell-ID nuclei (Fig2 Fig3), visual nuclei and whole-cell segmentation contours rendered images (Fig4).</span></strong></p> <p><strong><span lang="EN-US">Non-compensated and compensated multiTIFF 32-bit cells lines with Cellprofiler masks SCE objects and FCS files and Datatables.zip</span></strong></p> <p>Contains publicly available compensation matrix (<a href="https://zenodo.org/records/7575859">https://zenodo.org/records/7575859</a>) , R compensation script (<strong>Compensation IMC data with CATALYST.R)</strong>, compensated and non-compensated multiTIFF stacks 32-bit per cell line experiment, exported CellProfiler 16-bit masks per cell line dataset, R single cell experiment script (<strong>Conversion IMC data to Single Cell Experiments Objects and FCS.R)</strong> with inputs and outputs (fcs files, sce files, panel files, metadata files),R<strong> </strong>conversion single cell experiment to datatable script<strong> (Conversion SCE to Datatable and analysis.R)</strong>.</p> <p><strong><span lang="EN-US">Step-by-step guide to assist users with the IMC Cell Line Profiler computational workflow.</span></strong></p>
Research Workflows and Open Science - Data Set
<p>Data set accompanying the report "Research Workflows and Open Science", a systematic study of open science research workflows.</p> <p>The data set summarises the open science characteristics exhibited by the analysed workflows. The first two columns ‘<strong>workflow ID</strong>’ and ‘<strong>URL</strong>’ are dedicated to the ID we used to identify each workflow and to the publications related to the workflows respectively.</p> <p>The remaining columns are dedicated to the characteristics exhibited by the analysed workflows and are named The remaining columns are dedicated to the characteristics exhibited by the analysed workflows and are named following the different categories identified:</p> <ul> <li> <p>'<strong>used/open science infrastructure/virtual</strong>'</p> <ul> <li> <p>If a workflow relies on a virtual open infrastructure (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>used/open science infrastructure/physical</strong>'</p> <ul> <li> <p>If a workflow relies on a physical open infrastructure (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>used/open scientific knowledge/open source software</strong>'</p> <ul> <li> <p>If a workflow relies on open source software (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>used/open scientific knowledge/open hardware</strong>'</p> <ul> <li> <p>If a workflow relies on open hardware (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>used/open scientific knowledge/open research data</strong>'</p> <ul> <li> <p>If a workflow (re)uses open research data (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>used/open scientific knowledge/open educational resources</strong>'</p> <ul> <li> <p>If a workflow (re)uses open educational resources (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>produced/open scientific knowledge/(open access) scientific publication</strong>'</p> <ul> <li> <p>If a workflow envisages the release of a scientific publication (e.g. papers, reports, data management plans, preprints, study designs) under an open access licence (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>produced/open scientific knowledge/open source software</strong>'</p> <ul> <li> <p>If a workflow envisages the release of software (e.g. code, analysis scripts) under an open access licence (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>produced/open scientific knowledge/open research data</strong>'</p> <ul> <li> <p>If a workflow envisages the release of open research data (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>produced/open scientific knowledge/open educational resources</strong>'</p> <ul> <li> <p>If a workflow envisages the release of open educational resources (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>transparency/transparency type</strong>'</p> <ul> <li> <p>degree of transparency of a workflow, defined in terms of which research products are openly shared and when in order to document the research processes (‘built-in’ if transparent, ‘enabled’ if capable of being transparent, ‘opaque’ otherwise)</p> </li> </ul> </li> <li> <p>'<strong>transparency/sharing type</strong>'</p> <ul> <li> <p>workflow categories based on when the research products are shared (‘end’ for sharing at the end of the workflow, mixed for sharing part of the research products during the workflow and the rest at the end of it, ‘iterative’ for sharing iteratively during or at the end of the related workflow phase, and ‘user-dependent’, where it is ultimately up to the researcher to decide when to share the research products since the workflow offers different paths to follow while imposing no sharing constraint.)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/collaboration implementation</strong>'</p> <ul> <li> <p>If a workflow implements collaborative practices (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open engagement of societal actors/crowdfunding</strong>'</p> <ul> <li> <p>If a workflow envisages crowdfunding (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open engagement of societal actors/crowdsourcing</strong>'</p> <ul> <li> <p>If a workflow envisages crowdsourcing (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open engagement of societal actors/scientific volunteering</strong>'</p> <ul> <li> <p>If a workflow envisages scientific volunteering (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open engagement of societal actors/citizen and participatory science</strong>'</p> <ul> <li> <p>If a workflow envisages citizen and participatory science (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open dialogue with other knowledge systems/indigenous peoples</strong>'</p> <ul> <li> <p>If a workflow envisages the establishment of a dialogue with indigenous peoples (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open dialogue with other knowledge systems/marginalised scholars</strong>'</p> <ul> <li> <p>If a workflow envisages the establishment of a dialogue with marginalised scholars (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>collaboration/open dialogue with other knowledge systems/local communities</strong>'</p> <ul> <li> <p>If a workflow envisages the establishment of a dialogue with local communities (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>assessment</strong>'</p> <ul> <li> <p>If a workflow implements assessment processes for the evaluation of the research products created (yes/no)</p> </li> </ul> </li> <li> <p>'<strong>automation</strong>'</p> <ul> <li> <p>If a workflow includes automated processes (yes/no)</p> </li> </ul> </li> </ul>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.