Curated dataset of 18 computational experiments-(E1-E18)

<h2><strong>Overview</strong></h2> <p>This dataset, <strong>"Multi-Domain Experiment Dataset for Evaluating Reproducibility Tools (E1&ndash;E18),"</strong> is curated to facilitate the evaluation and benchmarking of reproducibility frameworks. It provides a structured and diverse collection of scientific experiments, enabling researchers and developers to test and compare different tools designed for computational reproducibility.</p> <h2><strong>Dataset Composition</strong></h2> <p>The dataset consists of 18<strong> experiments (E1&ndash;E18)</strong> covering multiple scientific domains, including <strong>computer science, human-computer interaction (HCI), medicine, artificial intelligence, climate change, and economics</strong>. These experiments range from simple computational scripts to complex setups requiring integrated databases, multiple programming languages, and domain-specific computational environments.</p> <h2><strong>Experiment Sources</strong></h2> <p>To ensure a well-balanced dataset, experiments were sourced from <strong>peer-reviewed scientific conferences</strong> and <strong>open-access repositories</strong>:</p> <ul> <li> <p><strong>Computer Science:</strong></p> <ul> <li><strong>Software Engineering:</strong> Experiments from the <strong>IEEE/ACM International Conference on Software Engineering (ICSE 2022)</strong>, a premier venue for software engineering research.</li> <li><strong>Databases:</strong> Selected experiments from the <strong>International Conference on Very Large Databases (VLDB 2021)</strong>, a leading database conference.</li> <li><strong>Human-Computer Interaction (HCI) and User Studies:</strong> <ul> <li>Experiments sourced from the <strong>European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023)</strong>.</li> <li>These experiments focus on user studies related to software engineering and usability research.</li> </ul> </li> </ul> </li> <li> <p><strong>Interdisciplinary Fields (Collected from Zenodo):</strong></p> <ul> <li><strong>Artificial Intelligence (AI):</strong> Experiments covering machine learning and AI-driven methodologies.</li> <li><strong>Climate Change:</strong> Computational experiments and simulations addressing environmental research.</li> <li><strong>Medicine:</strong> Experiments focusing on medical and healthcare applications.</li> <li><strong>Economics:</strong> Computational economic models and data analysis experiments.</li> </ul> </li> <li> <p><strong>Related Work on Reproducibility Tools:</strong></p> <ul> <li>We included <strong>experiments previously used to evaluate reproducibility tools</strong> and methodologies.</li> <li>This ensures alignment with prior research and enhances comparability across tools.</li> <li>Among these, two key studies&nbsp;<a href="https://ieeexplore.ieee.org/document/9041769"><strong>SciInc</strong></a> and <strong><a href="https://doi.ieeecomputersociety.org/10.1109/eScience.2017.51">SciUnit</a> </strong>provided three fully documented experiments: <ul> <li><strong>Chicago Food Inspections Evaluation;</strong></li> <li><strong>Variable Infiltration Capacity;</strong></li> <li><strong>Incremental Query Execution.</strong></li> </ul> </li> </ul> </li> </ul> <h2><strong>Methodology for Dataset Curation</strong></h2> <p>To construct this dataset, we employed a structured selection process:</p> <ol> <li> <p><strong>Scientific Conference Selection:</strong></p> <ul> <li>We identified key research areas within <strong>computer science</strong> and selected experiments from <strong>ICSE 2022</strong>, <strong>VLDB 2021</strong>, and <strong>ESEC/FSE 2023</strong> (focusing on user studies in HCI).</li> </ul> </li> <li> <p><strong>Zenodo Repository Search:</strong></p> <ul> <li>Targeted searches were conducted using the keywords <strong>"Medical," "Artificial Intelligence," "Climate Change,"</strong> and <strong>"Economics."</strong></li> <li>We filtered results to include only <strong>software repositories</strong>.</li> <li>From the <strong>top 100</strong> search results in each category, <strong>five experiments were randomly selected per domain</strong>.</li> </ul> </li> <li> <p><strong>Reproducibility Tools &amp; Related Work:</strong></p> <ul> <li>We incorporated experiments <strong>previously used to evaluate existing reproducibility tools</strong>.</li> <li>This selection ensures <strong>comparability and continuity</strong> with past reproducibility studies.</li> </ul> </li> </ol> <p>&nbsp;</p>