Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
12
datasets available to search
ShareScore release 0.9.0
Dataset results
12 results for “RO-Crate”
RO-Crate created using Autosubmit version 4.0.100 workflow running kinow/auto-mhm-test-domains
<p>This dataset is an <a href="https://www.researchobject.org/ro-crate/">RO-Crate</a> representation of an execution of an example Autosubmit workflow that executes <a href="https://mhm-ufz.org/">mHM</a>, the UFZ mesoscale Hydrologic Model. The example was created with Autosubmit version 4.0.100. The source code of the workflow can be found at <a href="https://github.com/kinow/auto-mhm-test-domains">https://github.com/kinow/auto-mhm-test-domains</a>, commit <a href="https://github.com/kinow/auto-mhm-test-domains/commit/c2672568fb14706c6e66481bc7ceac467db8261e">c267256</a>. This dataset was <a href="https://github.com/ResearchObject/workflow-run-crate/pull/61">validated by the RO-Crate community</a>.</p>
Packing provenance using CPM RO-Crate profile
<p>This dataset is an <a href="https://www.researchobject.org/ro-crate/">RO-Crate</a> that bundles artifacts of an AI-based computational pipeline execution. It is an example of application of the <a href="https://by-covid.github.io/cpm-ro-crate/0.1/">CPM RO-Crate profile</a>, which integrates the <a href="https://doi.org/10.1038/s41597-022-01537-6">Common Provenance Model</a> (CPM), and the <a href="https://www.researchobject.org/workflow-run-crate/profiles/process_run_crate">Process Run Crate profile</a>.</p> <p>As the CPM is a groundwork for the<em> ISO 23494 Biotechnology — Provenance information model for biological material and data</em> provenance standards series development, the resulting profile and the example is intended to be presented at one of the ISO TC275 WG5 regular meetings, and will become an input for the<em> ISO 23494-5 Biotechnology — Provenance information model for biological material and data — Part 5: Provenance of Data Processing</em> standard development.</p> <p><strong>Description of the AI pipeline</strong></p> <p>The goal of the AI pipeline whose execution is described in the dataset is to train an AI model to detect the presence of carcinoma cells in high resolution human prostate images. The pipeline is implemented as a set of python scripts that work over a filesystem, where the datasets, intermediate results, configurations, logs, and other artifacts are stored. In particular, the AI pipeline consists of the following three general parts:</p> <ul> <li> <p><strong>Image data preprocessing</strong>. Goal of this step is to prepare the input dataset – whole slide images (WSIs) and their annotations – for the AI model. As the model is not able to process the entire high resolution images, the preprocessing step of the pipeline splits the WSIs into groups (training and testing). Furthermore, each WSI is broken down into smaller overlapping parts called patches. The background patches are filtered out and the remaining tissue patches are labeled according to the provided pathologists’ annotations.</p> </li> <li> <p><strong>AI model training</strong>. Goal of this step is to train the AI model using the training dataset generated in the previous step of the pipeline. Result of this step is a trained AI model.</p> </li> <li> <p><strong>AI model evaluation</strong>. Goal of this step is to evaluate the trained model performance on a dataset which was not provided to the model during the training. Results of this step are statistics describing the AI model performance.</p> </li> </ul> <p>In addition to the above, execution of the steps results in generation of log files. The log files contain detailed traces of the AI pipeline execution, such as file paths, model weight parameters, timestamps, etc. As suggested by the CPM, the logfiles and additional metadata present on the filesystem are then used by a provenance generation step that transforms available information into the CPM compliant data structures, and serializes them into files. </p> <p>Finally, all these artifacts are packed together in an RO-Crate.</p> <p>For the purpose of the example, we have included only a small fragment of the input image dataset in the resulting crate, as this has no effect on how the Process Run Crate and CPM RO-Crate profiles are applied to the use case. In real world execution, the input dataset would consist of terabytes of data. In this example, we have selected a representative image for each of the input dataset parts. As a result, the only difference between the real world application and this example would be that the resulting real world crate would contain more input files. </p> <p><strong>Description of the RO-Crate</strong></p> <p><strong>Process Run Crate related aspects</strong></p> <p>The Process Run Crate profile can be used to pack artifacts of a computational workflow of which individual steps are not controlled centrally. Since the pipeline presented in this example consists of steps that are executed individually, and that the pipeline execution is not managed centrally by a workflow engine, the process run crate can be applied. </p> <p>Each of the computational steps is expressed within the crate’s ro-crate-metadata.json file as a pair of elements: 1) SW used to create files; 2) specific execution of that SW. In particular, we use the SoftwareSourceCode type to indicate the executed python scripts and the CreateAction type to indicate actual executions. </p> <p>As a result, the crate consists the seven following “executables”:</p> <ul> <li> <p>Three python scripts, each corresponding to a part of the pipeline: preprocessing, training, and evaluation.</p> </li> <li> <p>Four provenance generation scripts, three of which implement the transformation of the proprietary log files generated by the AI pipeline scripts into CPM compliant provenance files. The fourth one is a meta provenance generation script.</p> </li> </ul> <p>For each of the executables, their execution is expressed in the resulting ro-crate-metadata.json using the CreateAction type. As a result, seven create-actions are present in the resulting crate.</p> <p>Input dataset, intermediate results, configuration files and resulting provenance files are expressed according to the underlying RO Crate specification.</p> <p><strong>CPM RO-Crate related aspects</strong></p> <p>The main purpose of the CPM RO-Crate profile is to enable identification of the CPM compliant provenance files within a crate. To achieve this, the CPM RO-Crate profile specification prescribes specific file types for such files: CPMProvenanceFile, and CPMMetaProvenanceFile.</p> <p>In this case, the RO Crate contains three CPM Compliant files, each documenting a step of the pipeline, and a single meta-provenance file. These files are generated as a result of the three provenance generation scripts that use available log files and additional information to generate the CPM compliant files. In terms of the CPM, the provenance generation scripts are implementing the concept of provenance finalization event. The three provenance generation scripts are assigned SoftwareSourceCode type, and have corresponding executions expressed in the crate using the CreateAction type.</p> <p><strong>Remarks</strong></p> <p>The resulting RO Crate packs artifacts of an execution of the AI pipeline. The scripts that implement individual steps of the pipeline and provenance generation are not included in the crate directly. The implementation scripts are hosted on github and just referenced from the crate’s ro-crate-metadata.json file to their remote location.</p> <p>The input image files included in this RO-Crate are coming from the <a href="http://gigadb.org/dataset/100439">Camelyon16 dataset</a>.</p>
Linking provenance and its metadata for an AI-based computation using CPM and RO-Crate
<p>This dataset is a prototype implementation of a mechanism for linking provenance information and its metadata, also called provenance of provenance or meta-provenance. This dataset is an <a href="https://www.researchobject.org/ro-crate/">RO-Crate</a> that bundles artifacts of an AI-based computational pipeline. The resulting RO-Crate contains (directly or by a reference) artifacts of the pipeline execution, such as input dataset, intermediate and final results, configuration files, pipeline implementation, log files, or provenance files. The RO-Crate is based on the <a href="https://w3id.org/cpm/ro-crate/0.2">CPM RO-Crate profile</a>, which integrates the <a href="https://doi.org/10.1038/s41597-022-01537-6">Common Provenance Model</a> (CPM) and <a href="https://w3id.org/ro/wfrun/process/0.2">Process Run Crate profile</a>. The description of the AI pipeline and an explanation of how the CPM RO-Crate profile is applied to bundle the pipeline execution artifacts is provided in our <a href="https://doi.org/10.5281/zenodo.7676924">previous work</a>.</p> <p>As this dataset aims to demonstrate the mechanism for linking provenance and meta-provenance, the input dataset used for the AI model training and testing is reduced only to a few images, as the size of the input dataset does not affect the mechanism. The images used in the input are from the <a href="http://gigadb.org/dataset/100439">Camelyon16 dataset</a>.</p> <p> </p>
Demonstration of "RO-Crate for Testbeds: Automated Packaging of Experimental Results"
<p>Demonstrative experiment data of the paper "RO-Crate for Testbeds: Automated Packaging of Experimental Results".</p> <p>Demonstrator shows experimental artifacts as a RO-Crate package.</p>
Example of Workflow Run RO-Crate Output in Sapporo
<p>This is an archive of the contents under <a href="https://github.com/sapporo-wes/sapporo-service/tree/main/tests/ro-crate/ro-crate_dir">https://github.com/sapporo-wes/sapporo-service/tree/main/tests/ro-crate/ro-crate_dir</a> as of <a href="https://github.com/sapporo-wes/sapporo-service/releases/tag/1.5.1">version 1.5.1 of GitHub - sapporo-wes/sapporo-service</a>. For more details, please refer to <a href="https://github.com/sapporo-wes/sapporo-service/blob/main/tests/ro-crate/README.md">https://github.com/sapporo-wes/sapporo-service/blob/main/tests/ro-crate/README.md</a>.</p>
Photocatalysis Ontology - Dataset and RO-Crates packages
<p>In this package are the datasets extracted from the Artleafs database, as well as the RO-Crate packages and the RDF dataset generated from them for the project to create RO-Crates using the PHCAT ontology. You can also find python scripts used to transform the extracted CSV data into a new RDF dataset allowing you to create more RO-Crate packages if desired.</p> <p><strong>-./data:</strong> contains the set of data extracted from the database in CSV format.</p> <p><strong>- ./resources:</strong> contains the generated RO-Crate packages as well as the mapping files used and the RDF subsets of each article.</p> <p><strong>- ./OutputPhotocatalysisMapping.ttl:</strong> is the file in turtle format in charge of storing the global RDF data set after the translation of the database data.</p> <p>- ./<strong>documentation: </strong>in this folder is the documentation in HTML format of W3C.</p> <p>The rest of the folders and files contain mapping rules and scripts used in the data transformation process. For more information check the following GitHub repository: https://github.com/oeg-upm/photocatalysis-ontology.</p> <p> </p>
Workflow Run RO-Crate capturing provenance from WSI conversion
<p>Example of <a href="https://www.researchobject.org/workflow-run-crate/profiles/">Workflow Run RO-Crate</a> capturing provenance data from an execution of the <a href="https://github.com/crs4/fair-crcc-img-convert/tree/main">fair-crcc-img-convert</a> workflow on a whole-slide image from the <a href="https://doi.org/10.7937/25T7-6Y12">Cancer Moonshot Biobank - Prostate Cancer Collection (CMB-PCA)</a>.</p><ul><li>Slide ID: MSB-02917-01-02, generated by Natasha Honomichl</li><li>Image License: <a href="https://creativecommons.org/licenses/by/4.0/">CC BY 4.0</a></li></ul><p>Note that the license for the RO-Crate is CC BY 4.0, except for the workflow, which is licensed under the <a href="https://www.gnu.org/licenses/gpl-3.0.en.html">GPL-3.0</a>.</p>
Recording provenance of workflow runs with RO-Crate (RO-Crate and mapping)
<p>RO-Crate for the manuscript that describes Workflow Run Crate, includes mapping to PROV using SKOS/SSSOM.</p>
RO-Crate from staged WfExS working directory 047b6dfc-3547-4e09-92f8-df7143038ff4 (overbridging templon)
<p>This RO-Crate, which follows Workflow Run RO-Crate Process and Workflow profiles, was generated from the execution of the workflow execution <a href="https://github.com/wombat-p/WOMBAT-Pipelines">WOMBAT-Pipelines</a> by <a href="https://github.com/inab/WfExS-backend">WfExS-backend</a> workflow orchestrator. It contains both the provenance of the workflow execution, and snapshots of the used workflow, inputs, containers and generated outputs, included as payloads of the RO-Crate.</p>
BY-Covid synthetic RO-Crates
<p>RO-CRATE description profiles of the BY-COVID project. </p>
RO-Crate from staged WfExS working directory a37fee9e-4288-4a9e-b493-993a867207d0 (meer oxometalate)
<p>This RO-Crate, which follows Workflow Run RO-Crate Process and Workflow profiles, was generated from the execution of the workflow execution <a href="https://github.com/inab/Wetlab2Variations/tree/31348ed533961f84cf348bf1af660ad9de6f870c/cwl-workflows" target="_blank" rel="noopener">Wetlab2Variations</a> by <a href="https://doi.org/10.5281/zenodo.12589121" target="_blank" rel="noopener">WfExS-backend 1.0.0a0</a> workflow orchestrator. It contains both the provenance of the workflow execution, and snapshots of the used workflow, inputs, containers and generated outputs, included as payloads of the RO-Crate.</p>
Test of upload RO-crate zip
<p>Test</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.