Find research datasets worth reusing

Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.

datasets available to search

ShareScore release 0.7.1

Dataset results

3 results for “Black-box testing”

Learn how ShareScore rates datasets ↗

zenodo36/100

Flakify: A Black-Box, Language Model-based Predictor for Flaky Tests – Replication Package

This is the replication package associated with the paper: Flakify: A Black-Box, Language Model-based Predictor for Flaky Tests. We explain how to use it to reproduce the results reported in the paper. A maintainable version of this replication package is available on GitHub (<a href="https://github.com/uOttawa-Nanda-Lab/Flakify">https://github.com/uOttawa-Nanda-Lab/Flakify</a>). Flakify Test Smell Detector This is a step-by-step guideline to detect test smells in the source code of test cases and retain statements that match them. Requirements: <ul> <li>Eclipse IDE (the version we used was 2021-12)</li> <li>The libraries (the .jar files in the <code>lib\</code> directory)</li> </ul> Input Files: This is a list of input files that are required to accomplish this step: <ul> <li> dataset/FlakeFlagger/FlakeFlagger_filtered_dataset.csv </li> <li> dataset/FlakeFlagger/FlakeFlagger_class_files/ </li> <li> dataset/IDoFT/IDoFT_filtered_dataset.csv </li> <li> dataset/IDoFT/IDoFT_class_files/ </li> </ul> The <code>dataset/FlakeFlagger/FlakeFlagger_filtered_dataset.csv</code> and <code>dataset/IDoFT/IDoFT_filtered_dataset.csv</code> are used to obtain the label (flaky=1 or non-flaky=0) and project name for each test case parsed from <code>dataset/FlakeFlagger/FlakeFlagger_class_files/</code> and <code>dataset/IDoFT/IDoFT_class_files/</code>, respectively. Output Files: <ul> <li> dataset/FlakeFlagger/FlakeFlagger_dataset.csv </li> <li> dataset/FlakeFlagger/FlakeFlagger_test_cases_full_code/ </li> <li> dataset/FlakeFlagger/FlakeFlagger_test_cases_preprocessed_code/ </li> <li> dataset/IDoFT/IDoFT_dataset.csv </li> <li> dataset/IDoFT/IDoFT_test_cases_full_code/ </li> <li> dataset/IDoFT/IDoFT_test_cases_preprocessed_code/ </li> </ul>   Replicating the experiment To detect test smells and retain only code statements related to them, the <code>src/FlakifySmellsDetector.java</code> file should be compiled and run using the Eclipse IDE by having all the .jar files in the classpath. The pre-generated executable Jar file <code>src/FlakifySmellsDetector.jar</code> can be executed using the shell script <code>src/FlakifySmellsDetector.sh</code> after changing paths for each dataset as needed, using the following commands: <pre><code class="language-bash">bash FlakifySmellsDetector.sh FlakeFlagger bash FlakifySmellsDetector.sh IDoFT</code></pre> It will generate the dataset required to run Flakify's flaky test prediction model for the datasets given as input. The class file containing each of the test cases is then parsed to produce the corresponding full code and pre-processed code of the test case. The full and pre-processed source code of all test cases are also combined and saved in a CSV file, along with test smells found, project names, and labels.   Flakify Replication This is the guideline for replicating the experiments we used to evaluate Flakify for classifying test cases as flaky and non-flaky using both cross-validation and per-project validation. Requirements: This is a list of all required python packages: <ul> <li>python =3.8.5</li> <li>imbalanced_learn= 0.8.1</li> <li>numpy= 1.19.5</li> <li>pandas= 1.3.3</li> <li>transformer= 4.10.2</li> <li>torch=1.5.0</li> <li>scikit_learn= 0.22.1</li> </ul> Input Files: This is a list of input files that are required to accomplish this step: <ul> <li>dataset/FlakeFlagger/Flakify_FlakeFlagger_dataset.csv</li> <li>dataset/IDoFT/Flakify_IDoFT_dataset.csv</li> </ul> This file contains the full code and pre-processed code of the test cases in both FlakeFlagger and IDOFT datasets, along with their ground truth labels (flaky and non-flaky). Output File: <ul> <li> results/Flakify_cross_validation_results_on_FlakeFlagger_dataset.csv </li> <li> results/Flakify_per_project_results_on_FlakeFlagger_dataset.csv </li> <li> results/Flakify_model_trained_on_FlakeFlagger_dataset.pt </li> <li> results/Flakify_cross_validation_results_on_IDoFT_dataset.csv </li> <li> results/Flakify_per_project_results_on_IDoFT_dataset.csv </li> <li> results/Flakify_model_trained_on_IDoFT_dataset.pt </li> </ul>   Replicating Flakify experiments Cross-Validation To run the Flakify experiment using cross-validation on the two datasets, navigate to <code>src\</code> folder and run the following commands: <pre><code class="language-bash">bash Flakify_predictor_cross_validation.sh FlakeFlagger bash Flakify_predictor_cross_validation.sh IDoFT</code></pre> This will generate the classification results into <code>results/Flakify_cross_validation_results_on_FlakeFlagger_dataset.csv</code> and <code>results/Flakify_cross_validation_results_on_IDoFT_dataset.csv</code> for the cross-validation experiments on both datasets. It will also save the weights of the two models trained on the FlakeFlagger and IDoFT datasets into <code>results/Flakify_model_trained_on_FlakeFlagger_dataset.pt</code> and <code>results/Flakify_model_trained_on_IDoFT_dataset.pt</code>, respectively.   Per-project Validation To run the Flakify experiment using per-project validation on the two datasets, navigate to <code>src\</code> folder and run the following commands: <pre><code class="language-bash">bash Flakify_predictor_per_project.sh FlakeFlagger bash Flakify_predictor_per_project.sh IDoFT</code></pre> This will generate the classification results into <code>results/Flakify_per_project_results_on_FlakeFlagger_dataset.csv</code> and <code>results/Flakify_per_project_results_on_IDoFT_dataset.csv</code> for the whole per-project validation experiments on both datasets.   FlakeFlagger Replication This is the guideline for replicating the experiments we used to evaluate the two versions of FlakeFlagger, white-box and black-box, for classifying test cases as flaky and non-flaky using cross-validation on the FlakeFlagger dataset. Requirements: This is a list of all required python packages: <ul> <li>python =3.8.5</li> <li>imbalanced_learn= 0.8.1</li> <li>pandas= 1.3.3</li> <li>scikit_learn= 0.22.1</li> </ul> Input File: This is a list of input files that are required to accomplish this step: <ul> <li>dataset/FlakeFlagger/FlakeFlagger_filtered_dataset.csv</li> <li>dataset/FlakeFlagger/FlakeFlaggerFeaturesTypes.csv</li> <li>dataset/FlakeFlagger/Information_gain_per_feature.csv</li> </ul> Output File: <ul> <li>results/FlakeFlagger_black-box_results.csv</li> <li>results/FlakeFlagger_white-box_results.csv</li> </ul>   Replicating FlakeFlagger experiments To run the FlakeFlagger experiments, navigate to <code>src\</code> folder and run the following command: <pre><code class="language-bash">bash FlakeFlagger_predictor.sh white-box bash FlakeFlagger_predictor.sh black-box</code></pre> This will generate the classification results into <code>results/FlakeFlagger_white-box_results.csv</code> and <code>results/FlakeFlagger_black-box_results.csv</code> for both white-box and black-box experiments, respectively.

opencc-by-4.0Aug 2022View details →

zenodo36/100

LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models - Replication Package

LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models This is the replication package associated with the paper "LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models". Replication Package Contents: This replication package contains all the necessary data and code required to reproduce the results reported in the paper. We provide the results of the Fault Detection Rate (FDR), Total Minimization Time (MT), Time Saving Rate (TSR) , statistical tests for all the minimization budgets (i.e., 25%, 50%, and 75%), results for the preliminary study, results for UniXcoder/Cosine with preprocessed code on 16 projects. Data: We provide in the Data directory the data used in our experiments, which is the source code of test cases (Java test methods) of 17 projects collected from Defects4J. Code: We provide in the Code directory the code (Python) and bash files required to run the experiments and reproduce the results. Results: We provide in the Results directory the detailed results for our approach (called LTM). We also provide the summarized results of LTM and a baseline (ATM) for comparison purposes. Additional technical details about ATM can be found at https://zenodo.org/record/7455766. _________________________________ LTM's Similarity Measurement: The source code of this step is in the Code/LTM/Similarity directory. Requirements: To run this step, Python 3 is required (we used Python 3.10). Also, the required libraries in the Code/LTM/Similarity/requirements.txt file should be installed, as follows: cd Code/LTM/Similarity pip install -r requirements.txt Input: <ul> <li>Data/LTM/TestMethods</li> </ul> Output: <ul> <li>Data/LTM/similarity_measurements</li> </ul> Running the experiment: To measure the similarity between all pairs of test cases, the following bash script should be executed: bash measure_similarity.sh The source code of test methods of each project in the Data/LTM/TestMethods is parsed to generate pairs of test cases. This steps includes test methods tokenization, test methods embeddings extraction and similarity calculation. Then, all similarity scores are stored in Data/LTM/similarity_measurements folder. Due to the large size of the calculated similarity scores (60 GB), they were not uploaded on Zenodo, but they can be available upon request. LTM's Test Suite Minimization: The source code of this step is in the Code/LTM/Search directory. Requirements: To run this step, Python 3 is required (we used Python 3.10). Also, the required libraries in the Code/LTM/Search/requirements.txt file should be installed, as follows: cd Code/LTM/Search pip install -r requirements.txt Input: Data/LTM/similarity_measurements Output: Results/LTM/minimization_results Running the experiments: To minimize the test suite for each project version, the following bash script should be executed: bash minimize.sh The similarity scores of all test case pairs per project version are parsed by the search algorithm (Genetic Algorithm). Each experiment runs ten times using three minimization budgets (25%, 50%, and 75%). The results are stored in the Results/LTM/minimization_results directory. LTM's Evaluation: To evaluate the minimization results for each version and each project, the following bash script should be executed: cd Code/LTM/Evaluation bash evaluate_per_version.sh cd Code/LTM/Evaluation bash evaluate_per_project.sh This will evaluate the FDR, MT and TSR results for each version and each project for each minimization budget. These results are stored in the Results/LTM directory. Note that for each version, the FDR is either 1 or 0. For each project, the FDR ranges from 0 to 1.

opencc-by-4.0Jul 2024View details →

zenodo36/100

ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search – Replication Package

This is the replication package associated with the paper "ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search" accepted at the 45th IEEE/ACM International Conference on Software Engineering (ICSE 2023) – Technical Track. Cite this paper using the following: @inproceedings{pan2023atm,   title={ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search},   author={Pan, Rongqi and Ghaleb, Taher A. and Briand, Lionel},   booktitle={Proceedings of the 45th IEEE/ACM International Conference on Software Engineering},   year={2023},   pages={1--12} } Replication Package Contents: The replication package contains all the necessary data and code required to reproduce the results reported in the paper. We also provide the results for other minimization budgets, and detailed FDR, execution time, and statistical test results. In addition, we provide the data and code required to reproduce the results of baselines techniques: FAST-R and random minimization. Data: We provide in the Data directory the data used in our experiments, which is based on 16 projects from <a href="https://github.com/rjust/defects4j">Defects4J</a>, whose characteristics can be found in Data/subject_projects.csv. Code: We provide in the Code directory the code and scripts (Java, Python, and Bash) required to run the experiments and reproduce the results. Results: We provide in the Results directory the results for each technique independently, and also a summary of all results together for comparison purposes. The source code for this step is in the Code/ATM/CodeToAST directory. The source code for this step is in the Code/ATM/Similarity directory. _________________________________ ATM - Code to AST transformation: Requirements: * Eclipse IDE (we used 2021-12) * The libraries (the .jar files in the Code/ATM/CodeToAST/lib directory) Input: All zipped data files should be unzipped before running each step. * Data/test_suites/all_test_cases.zip → Data/test_suites/all_test_cases * Data/test_suites/changed_test_cases.zip → Data/test_suites/changed_test_cases * Data/test_suites/relevant_test_cases.zip → Data/test_suites/relevant_test_cases Output: * Data/ATM/ASTs/all_test_cases * Data/ATM/ASTs/changed_test_cases Running the experiment: To generate ASTS for all test cases in the project test suites, the Code/ATM/CodeToAST/src/CodeToAST.java file should be compiled and run using the Eclipse IDE by including all the required .jar files in the Code/ATM/CodeToAST/lib directory as part of the classpath. A bash script is provided along with a pre-generated .jar file in the Code/ATM/CodeToAST/bin directory to run this step, as follows: <pre><code class="language-bash">cd Code/ATM/CodeToAST bash transform_code_to_ast.sh</code></pre> Each test file in the Data/test_suites/all_test_cases and Data/test_suites/changed_test_cases directories is parsed to generate a corresponding AST for each test case method (saved in an XML format in Data/ATM/ASTs/all_test_cases and Data/ATM/ASTs/changed_test_cases for each project version) _________________________________ ATM - Similarity Measurement: Requirements: * Eclipse IDE (we used 2021-12) * The libraries (the .jar files in the Code/ATM/Similarity/lib directory) Input: * Data/test_suites/all_test_cases * Data/test_suites/changed_test_cases Output: * Data/ATM/similarity_measurements Running the experiment: To measure the similarity between each pair of test cases, the Code/ATM/Similarity/src/SimilarityMeasurement.java file should be compiled and run using the Eclipse IDE by including all the required .jar files in the Code/ATM/Similarity/lib directory as part of the classpath. A bash script is provided along with a pre-generated .jar file in the Code/ATM/Similarity/bin directory to run this step, as follows: <pre><code class="language-bash">cd Code/ATM/Similarity bash measure_similarity.sh</code></pre> ASTs of each project in the Data/ATM/ASTs/all_test_cases and Data/ATM/ASTs/changed_test_cases directories are parsed to create pairs of ASTs containing one test case from the Data/ATM/ASTs/all_test_cases directory with another test case from the Data/ATM/ASTs/changed_test_cases directory (redundant pairs are discarded). Then, all similarity measurements are saved in the Data/ATM/similarity_measurements.zip file. __________________________________________ Search-based Minimization Algorithms: The source code for this step is in the Code/ATM/Search directory. Requirements: To run this step, Python 3 is required (we used Python 3.10). Also, the libraries in the Code/AMT/Search/requirements.txt file should be installed, as follows: <pre><code class="language-bash">cd Code/ATM/Search pip install -r requirements.txt</code></pre> Input: * Data/ATM/similarity_measurements Output: * Results/ATM/minimization_results Running the experiment: To minimize the test suites in our dataset, the following bash script should be executed: <pre><code class="language-bash">bash minimize.sh</code></pre> All similarity measurements are parsed for each version of the projects, independently. Each version is run 10 times using three minimization budgets (25%, 50%, and 75%). Genetic Algorithm (GA) is run using four similarity measures, namely top-down, bottom-up, combined, and tree edit distance. NSGA-II is run using two combinations of similarity measures: top-down & bottom-up and combined & tree edit distance. The minimization results are generated in the Results/ATM/minimization_results directory. __________________ Evaluate results: To evaluate and summarize the minimization results, run the following: <pre><code class="language-bash">cd Code/ATM/Evaluation bash evaluate.sh</code></pre> This will generate summarized FDR and execution time results (per-project and per-version) for each minimization budget, which can all be found in Results/ATM. In this replication package, we provide the final, merged FDR with execution time results. _________________________________ Running FAST-R experiments ATM was compared to <a href="https://github.com/ICSE19-FAST-R/FAST-R">FAST-R</a>, a state-of-the-art baseline, which is a set of test case minimization techniques called: FAST++, FAST-CS, FAST-pw, and FAST-all, which we adapted to our data and experimental setup. Requirements: To run this step, Python 3.7 is required. Also, the libraries in the Code/FAST-R/requirements.txt file should be installed, as follows: <pre><code class="language-bash">cd Code/FAST-R pip install -r requirements.txt</code></pre> Input: * Data/FAST-R/test_methods * Data/FAST-R/test_classes Output: * Results/FAST-R/test_methods/FDR_and_Exec_Time_Results_[budget]%_budget.csv * Results/FAST-R/test_classes/FDR_and_Exec_Time_Results_[budget]%_budget.csv To run FAST-R experiments, the following bash script should be executed: <pre><code class="language-bash">bash fast_r.sh test_methods #method level bash fast_r.sh test_classes #class level</code></pre> Results are generated in .csv files for each budget. For example, for the 50% budget, results are saved in FDR_and_Exec_Time_Results_50%_budget.csv in the Results/FAST-R/test_methods and Results/FAST-R/test_classes directories. _________________________________ Running the random minimization experiments ATM was also compared to random minimization as a standard baseline. Requirements: To run this step, Python 3 is required (we used Python 3.10). Also, the libraries in the Code/RandomMinimization/requirements.txt file should be installed, as follows: <pre><code class="language-bash">cd Code/RandomMinimization pip install -r requirements.txt</code></pre> Input: N/A Output: * Results/RandomMinimization/FDR_and_Exec_Time_Results_[budget]%_budget.csv To run the random selection experiments, the following bash script should be executed: <pre><code class="language-bash">bash random_minimization.sh</code></pre> Results are generated in .csv files for each budget. For example, for the 50% budget, results are saved in FDR_and_Exec_Time_Results_50%_budget.csv in the Results/RandomMinimization directory.

opencc-by-4.0Dec 2022View details →

ScienceDex guides

Understand access before you commit

These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.

Compare curated datasets

Allen Brain Atlas

Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.

allen-brain-atlas

neuroscienceopenDocumentation, web resources, and API references are available online.

Last verified 2026-04-30Open record

Annotated Behaviour and Observability Dataset (ABODe)

ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.

abode-home-cage

behavioral-neuroscienceopenThe DataShare record exposes download links for annotations, documentation, license text, and the zipped per-snippet data directory.

Last verified 2026-04-30Open record

DANDI Archive for NWB datasets

DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.

dandi-nwb

electrophysiologyopenPublished Dandiset metadata and archive endpoints are available through the production DANDI API.

Last verified 2026-04-30Open record

International Brain Laboratory public data

The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.

ibl

behavioral-neuroscienceopenPublic sessions can be searched and loaded from the IBL public data server through ONE.

Last verified 2026-04-29Open record

OpenNeuro

OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.

openneuro

neuroscienceopenPublished datasets are available on demand over the internet.

Last verified 2026-04-29Open record