Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
11
datasets available to search
ShareScore release 0.9.0
Dataset results
11 results for “reproducibility challenge”
"The sound comes from a meadow in the Sierra Nevada Mountains in California. The meadow is at an elevation of 2400 meters near a mountain named Olancha Peak, which is 3700 meters in altitude. Ihave a group of friends with which Ibackpack (trek) into the mountains. Our goal was to spend some time in the mountains and hike to the top of Olancha Peak (…) By the time we reached the meadow, we were in a forest and there was still snow on the ground in some places. We took the trip in June of 2006. The Sierra Nevada Mountains are a large mountain range. Much of the range is protected by national parks or preserved areas we call 'wilderness areas' (…) Ihave been backpacking for nearly 40 years and Iwill hopefully continue with this challenging activity for 40 years more! Many of my friends are much younger than Iam and it gives me much satisfaction to be able to have as much or more stamina for this activity than they have! When we are on these trips, we hike up peaks, catch fish, drink some whiskey around campfires and enjoy our time in the beautiful solitude. My memories of this trip were of the steep, hot hike from the desert to the cool meadow; the overall beauty of the nature, the absolute solitude of our campsite near the meadow; the strenuous hike to the top of Olancha Peak; the camaraderie of my friends; and, of course the sound of the frogs in the meadow. The frog sounds were astounding to me and Iwould listen in awe of the creature's instinctual desire to reproduce and continue the existence of their kind. Surely there were different species in the meadow for some of the frog sounds were different than others. The sounds only occurred after the Sun went down for the evening. Istood next to the creek in the meadow and recorded the sounds using my digital camera." [Peter/plentz1960]16 in Collecting Sounds. Online Sharing of Field Recordings as Cultural Practice
"The sound comes from a meadow in the Sierra Nevada Mountains in California. The meadow is at an elevation of 2400 meters near a mountain named Olancha Peak, which is 3700 meters in altitude. Ihave a group of friends with which Ibackpack (trek) into the mountains. Our goal was to spend some time in the mountains and hike to the top of Olancha Peak (…) By the time we reached the meadow, we were in a forest and there was still snow on the ground in some places. We took the trip in June of 2006. The Sierra Nevada Mountains are a large mountain range. Much of the range is protected by national parks or preserved areas we call 'wilderness areas' (…) Ihave been backpacking for nearly 40 years and Iwill hopefully continue with this challenging activity for 40 years more! Many of my friends are much younger than Iam and it gives me much satisfaction to be able to have as much or more stamina for this activity than they have! When we are on these trips, we hike up peaks, catch fish, drink some whiskey around campfires and enjoy our time in the beautiful solitude. My memories of this trip were of the steep, hot hike from the desert to the cool meadow; the overall beauty of the nature, the absolute solitude of our campsite near the meadow; the strenuous hike to the top of Olancha Peak; the camaraderie of my friends; and, of course the sound of the frogs in the meadow. The frog sounds were astounding to me and Iwould listen in awe of the creature's instinctual desire to reproduce and continue the existence of their kind. Surely there were different species in the meadow for some of the frog sounds were different than others. The sounds only occurred after the Sun went down for the evening. Istood next to the creek in the meadow and recorded the sounds using my digital camera." [Peter/plentz1960]16
Paper is not enough: Crowdsourcing the T<sub>1</sub> mapping common ground via the ISMRM reproducibility challenge
Dataset provided for NeuroLibre preprint. Author repo: https://www.github.com/rrsg2020/note NeuroLibre fork:https://github.com/roboneurolibre/note <p>For details, please visit the corresponding <a href="https://github.com/neurolibre/neurolibre-reviews/issues/23">NeuroLibre technical screening.</a></p> <p><strong><a href="https://neurolibre.org" target="NeuroLibre">https://neurolibre.org</a></strong></p>
SC20 Reproducibility Challenge Artifacts
<p>As a part of the reproducibility initiative at the SC20 conference, students participating in the Student Cluster Competition where given the reproducibility challenge, which is a directed reproducibility exercise that reproduces a paper from the previous year’s conference. At SC20, the paper, <em>MemXCT: memory-centric X-ray CT reconstruction with massive parallelization (</em><a href="https://doi.org/10.1145/3295500.3356220">https://doi.org/10.1145/3295500.3356220)</a>, from SC19 was used. In the directed exercise, students were asked to reproduce performance characteristics of the research on a variety of architectures. This file contains the experimental setup and outputs of their work from the reproducibility challenge. It also includes the direction that students were given to the students for the exercise.</p> <p>The students then published their reports as polished critiques of the reproducibility of the original paper along with a new revision of the original paper, titled <em>MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography</em> <em>Imaging</em> (<a href="https://doi.org/10.1109/TPDS.2021.3128032">https://doi.org/10.1109/TPDS.2021.3128032</a>), in a Special Section of the IEEE Transactions on Parallel and Distributed Systems (TPDS). The paper along with the revised papers are available at <a href="https://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=9696253&punumber=71"> https://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=9696253&punumber=71</a>.</p>
ISMRM 2022/23 Reproducibility Team Challenge: Replication results for MReplIcators team
<p>This repository contains the <em><strong>MR</strong></em>epl<em><strong>I</strong></em>cators team’s replication attempt results from the 2022-23 International Society of Magnetic Resonance in Medicine (ISMRM) Challenge ‘Repeat It With Me: Reproducibility Team Challenge’ (<a href="https://challenge.ismrm.org/">ISMRM Challenge – ISMRM's Challenge Forums</a>).</p> <p>The challenge aims to “raise awareness of the importance of reproducible science through fostering collaboration between labs by reproducing ISMRM abstracts”.</p> <p>The Reproducer sub-team was formed by Ebony R. Gunwhy (ERG) and Jemima Pilgrim-Morris (JPM) from POLARIS (<a href="https://www.sheffield.ac.uk/polaris">POLARIS (Pulmonary, Lung and Respiratory Imaging Sheffield) | POLARIS | The University of Sheffield</a>) at The University of Sheffield and the team worked on reproducing abstract #1700 "Sensitivity Analysis of the Bloch Equations" (ISMRM 2022) by Author sub-team Nick Scholand and Martin Uecker from The Institute of Medical Engineering at Graz University of Technology, Austria and Institute for Diagnostic and Interventional Radiology University Medical Center, Göttingen (1).</p> <p>In this archive, a zipped results folder contains the independent results and notes from each member of the replicator sub-team (in ‘results/JPM’ and ‘results/ERG’, respectively). A more detailed overview of the abstract and replication (motivation, approach, challenges, and outcomes) can also be found in ‘Abstract replication details.pdf’. A shorter summary is provided here:</p> <p><em><strong>A successful replication ‘attempt’ was defined as…</strong></em></p> <ul> <li> <p>Installation of provided software</p> </li> <li> <p>Creation of simulations with the BART (2) simulation interface</p> </li> </ul> <p><em><strong>A fully successful replication was defined as…</strong></em></p> <ul> <li> <p>Successful recreation of the figures provided in the abstract</p> </li> <li> <p>Examining goodness-of-fit by matching the provided ground truth dataset to some tolerances (normalised root mean square error; NRMSE)</p> </li> <li> <p>Using quantitative measures of reproducibility (e.g., coefficient of variation) between original data computed by author sub-team and data computed by replicator sub-team</p> </li> <li> <p>Application of technique to further sequences and tissue/sequence parameters</p> </li> </ul> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p><em><strong>Contact between Replicator and Author sub-teams</strong></em></p> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p>An initial meeting was conducted to discuss how we should organise our participation as a team in this challenge. Some starting information was exchanged, in addition to the original abstract:</p> <p><strong>(A)</strong> Publicly published information about the studies:</p> <p>(i) arXiv preprint of more detailed manuscript (2)</p> <p>(ii) GitHub repository associated with the arXiv preprint, containing steps for reproducing the work relating to Figure 2 of the abstract (<a href="https://github.com/mrirecon/bloch-moba/tree/main/02_sens_analysis">https://github.com/mrirecon/bloch-moba/tree/main/02_sens_analysis</a>)</p> <p>(iii) GitHub repositories with tutorials on using the BART reconstruction toolbox (<a href="https://github.com/mrirecon/bart-workshop">https://github.com/mrirecon/bart-workshop</a>)</p> <p><strong>(B)</strong> The author sub-team also created an additional GitHub repository made publicly available, containing interactive Google Colab and Binder notebooks introducing BART and its simulation framework, and providing a more step-by-step guide to allow for an exact replication of the work in the abstract (<a href="https://github.com/mrirecon/ismrm-2022-sensitivity-analysis-bloch-eq">https://github.com/mrirecon/ismrm-2022-sensitivity-analysis-bloch-eq</a>). The author sub-team also uploaded the ground truth dataset (their own simulation results, located within ‘02_irbssfp/ref/’ and ‘03_unprep_irbssfp/ref/) from the abstract to this repository.</p> <p>In the first instance, the Replicator sub-team attempted a replication using only the publicly available information outlined in (A). Each member of the Replicator sub-team created their own GitHub forks of the repository listed in (A)(ii) to provide a starting point from which to document their replication from. These can be found at <a href="https://github.com/JemimaPM/bloch-moba-jpm">https://github.com/JemimaPM/bloch-moba-jp</a>m (JPM) and <a href="https://github.com/EbonyGunwhy/bloch-moba-erg/tree/MReplIcators">https://github.com/EbonyGunwhy/bloch-moba-erg/tree/MReplIcators</a> (ERG) and contain a commit history of all replication steps taken.</p> <p>The additional repository described in (B) was only used to download the ground truth dataset for quantitatively assessing the replication results. This limitation was imposed to allow the Replicator sub-team autonomy in reproducing the work and to aid in gaining a more thorough understanding of the methodology involved. It was agreed that if required, the replicator sub-team would take help from the extra repository described in (B) and provide feedback to the authors afterwards summarising what degree of information was useful for replicating the work. Further interaction was limited to creating issues on GitHub and to quick queries via email, as needed.</p> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p><em><strong>Expected outcomes</strong></em></p> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p>The original study was based on numerical experiments. Therefore, the results were expected to be directly reproducible, however it was expected that possible differences may occur due to numerical noise. The major reason for this is the single floating-point arithmetic used in the provided BART software. Its presence can be tested by reproducing the study results on various machines multiple times. Numerical noise is expected to change between different hardware. Therefore, both members of the Replicator sub-team attempted to independently replicate the results on their own hardware. The used hardware in this challenge is specified below.</p> <p><em>Replicators:</em></p> <p>ERG: Intel(R) Core(TM) i5-1145G7 CPU @ 2.60GHz (4 cores, 8 logical processors)</p> <p>JPM: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz (4 cores, 8 logical processors)</p> <p><em>Authors:</em></p> <p>Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz (4 cores, 8 logical processors)</p> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p><em><strong>Replication results</strong></em></p> <p><em><strong>----------------------------------------------------------------------------</strong></em></p> <p>Prior to starting the replication attempt, we agreed on the NRMSE tolerances below which we would define the replication as successful. These were NRMSE = 0.005 for the DQ derivative and NRMSE = 0.0001 for the SA derivative.</p> <p>Due to time constraints, only a direct replication was performed. This was deemed successful as both replicators were able to reproduce the results within these tolerances. As expected, differences occurred between hardware, with 0% NMRSE achieved in the replication attempt of JPM. In addition, the reproduced figures were qualitatively compared to the abstract figures and these were found to agree.</p> <p>Further notes and results from the Replicator sub-team attempts can be found within the ‘results’ folder for each respective replictor.</p> <p> </p> <p><strong><em>===========================================</em></strong></p> <p><em><strong>Carbon footprint</strong></em></p> <p>This algorithm runs in 0.08 mins on 4 CPUs and on:</p> <ul> <li> <p>Intel i5-10400F: draws 572.04 Wh. Based in the United Kingdom, this has a carbon footprint of 132.21 g CO2e, which is equivalent to 0.14 tree-months (<a href="http://calculator.green-algorithms.org//?runTime_hour=12&runTime_min=0&appVersion=v2.2&locationContinent=Europe&locationCountry=United%20Kingdom&locationRegion=GB&coreType=CPU&numberCPUs=4&CPUmodel=Core%20i5-10400F&memory=12&platformType=personalComputer&PSFradio=Yes&PSF=1">Green Algorithms i5-10400F (green-algorithms.org)</a>)</p> </li> <li> <p>Intel i7-4930K: draws 1.10 kWh. Based in the United Kingdom, this has a carbon footprint of 253.13 g CO2e, which is equivalent to 0.28 tree-months (<a href="http://calculator.green-algorithms.org//?runTime_hour=12&runTime_min=0&appVersion=v2.2&locationContinent=Europe&locationCountry=United%20Kingdom&locationRegion=GB&coreType=CPU&numberCPUs=4&CPUmodel=Core%20i7-4930K&memory=12&platformType=personalComputer&PSFradio=Yes&PSF=1">Green Algorithms i7-4930K (green-algorithms.org)</a>)</p> </li> </ul> <p>(calculated using green-algorithms.org v2.2 [1]. Note: at time of writing, the Green Algorithms calculator did not include our specific CPUs, therefore here we have calculated with similar processors. A request has been made on the Green Algorithms GitHub repository for our processors to be added in future).</p> <p><em><strong>===========================================</strong></em></p> <p><em><strong>References</strong></em></p> <p><em>1. Scholand N, & Uecker M. Sensitivity Analysis of the Bloch Equations [abstract]. In: Proceedings of the 31st Annual Meeting of ISMRM, London, 2022. Abstract nr 1700.</em></p> <p><em>2. Uecker M, Virtue P , Ong F, Murphy MJ, Alley MT, Vasanawala SS, Lustig M. Software Toolbox and Programming Library for Compressed Sensing and Parallel Imaging [abstract]. ISMRM Workshop on Data Sampling and Image Reconstruction, Sedona, 2013.</em></p> <p><em>3. Scholand N, Wang X, Roeloffs V, Rosenzweig S, Uecker M. Quantitative Magnetic Resonance Imaging by Nonlinear Inversion of the Bloch Equations. arXiv:2209.08027 [Preprint]. 2022. doi: <a href="https://doi.org/10.48550/arXiv.2209.08027">10.48550/arXiv.2209.08027</a></em></p> <p><em>4. Lannelongue L, Grealey J, Inouye M. Green Algorithms: Quantifying the Carbon Footprint of Computation. Adv. Sci. 2021;8(12):2100707. doi: <a href="https://doi.org/10.1002/advs.202100707">10.1002/advs.202100707</a></em></p> <p> </p> <p> </p>
Results of the ISMRM 2020 joint Reproducible Research & Quantitative MR study groups reproducibility challenge on phantom and human brain T<sub>1</sub> mapping
<p>This dataset includes both raw data and processed T1 maps obtained from the 2020 challenge on inversion recovery T1 mapping organized by the International Society in Magnetic Resonance in Medicine (ISMRM) Reproducible Research Study Group (RRSG). For a comprehensive overview of the data distribution submitted for this challenge, please visit <a href="https://rrsg2020.db.neurolibre.org/">https://rrsg2020.db.neurolibre.org</a>. It's important to note that this dataset exclusively comprises ISMRM-NIST system phantom data.</p> <p>Dataset provided for NeuroLibre preprint. Author repo: https://github.com/rrsg2020/paper NeuroLibre fork:https://github.com/roboneurolibre/paper</p> <p>For details, please visit the corresponding <a href="https://github.com/neurolibre/neurolibre-reviews/issues/14">NeuroLibre technical screening.</a></p> <p><strong><a href="https://neurolibre.org">https://neurolibre.org</a></strong></p>
Evaluation of Reproducibility of a Sporozoite Challenge Model for Plasmodium Vivax in Human Volunteers
ClinicalTrials.gov study NCT00367380. IPD Sharing: Not stated. Countries: 1. Publications: 69.
ISMRM Reproducible Research Study Group: Data for the paper "CG-SENSE revisited: Results from the first ISMRM reproducibility challenge"
<p>Challange data (brain/heart) and supplementary data (cardiac/rawdata_sprial) for the paper "CG-SENSE revisited: Results from the first ISMRM reproducibility challenge".</p>
Challenges of ELA-based Function Evolution using Genetic Programming - Reproducability files
<p>This repository contains the data and code for the paper "Challenges of ELA-based Function Evolution using Genetic<br> Programming"</p> <p> </p> <p>This repository consists of separated folders, which contain the following data:</p> <p>## Code:</p> <p>This is the main code used to run the GP functions. The main executable is 'main_gp.py', which executes a single run of the GP system (based on the passed-in argument, which is an index from 0-71 in our experiments).</p> <p>The data for the BBOB functions are generated using the 'preliminary' folder and the 'get_ela_preliminary.py' file.</p> <p> </p> <p>## Data_GP:</p> <p>This contains the full logs from each GP run, separated by target function and dimension.</p> <p> </p> <p>## data_random_func:</p> <p>This contains the same kind of data but for the Random Function Generator.</p> <p> </p> <p>## Reproducibility:</p> <p>This contains all code used to analyse and visualize the resulting data. The notebook is structured in the same way as the paper, separated by figure.</p>
Nasal Allergen Challenge - Reproducibility of Biomarkers and Effect of Topical Steroid Treatment
ClinicalTrials.gov study NCT03431961. IPD Sharing: NO. Countries: 1. Publications: 1.
Reproducibility of Malaria Challenge in Healthy Volunteers
ClinicalTrials.gov study NCT01058226. IPD Sharing: Not stated. Countries: 1. Publications: 1.
Clinical Trial for the Development of a Safe Malaria Challenge Model That Can be Reproduced in Humans
ClinicalTrials.gov study NCT00935623. IPD Sharing: Not stated. Countries: 1. Publications: 0.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.