Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
73
datasets available to search
ShareScore release 0.9.0
Dataset results
73 results for “Unit Test”
Does Unit-Tested Code Crash? A Case Study of Eclipse
<ul> <li>The "jacoco.xml" file contains the code coverage of Eclipse Project generated by JaCoCo</li> <li>The "incidents (stack traces).zip" file contains the stacktraces files (in JSON format) with the specific popular release and product that we used for our research.</li> <li>All The Eclipse stacktraces can be downloaded <a href="http://software-data.org/datasets/aeri-stacktraces/downloads/incidents_full.tar.bz2">here.</a></li> <li>The "output.csv" contains the data that we produced after combining JaCoCo Report and stacktraces.</li> </ul>
Diversity-Driven Unit Test Generation (Data Set)
<p>The goal of automated unit test generation tools is to create a set of test cases for the software under test that achieve the highest possible coverage for the selected test quality criteria. The most effective approaches for achieving this goal at the present time use meta-heuristic optimization algorithms to search for new test cases using fitness functions defined on existing sets of test<br> cases and the system under test. Regardless of how their search algorithms are controlled, however, all existing approaches focus on the analysis of exactly one implementation, the software under test, to drive their search processes, which is a limitation on the information they have available. In this paper we investigate whether the practical effectiveness of white box unit test generation tools can be increased by giving them access to multiple, diverse implementations of the functionality under test harvested from widely available Open Source software repositories. After presenting a basic implementation of such an approach, DivGen (Diversity-driven Generation), on top of the leading test generation tool for Java (EvoSuite), we assess the performance of DivGen compared to EvoSuite when applied in its traditional, mono-implementation oriented mode (MonoGen). The results show that while DivGen outperforms MonoGen in 33% of the sampled classes for mutation coverage (+16% higher on average), MonoGen outperforms<br> DivGen in 12.4% of the classes for branch coverage (+10% higher average).</p>
ETHZ-SED EEW unit-test dataset
<p>This is a preliminary version of a unit-test dataset for EEW algorithms developed at SED-ETHZ based on the data from the MLh 4.3 earthquake on October 25, 2020 in Elm (Switzerland). Further information about this earthquake dataset can be found in Massin, Clinton & Böse (2021).</p>
Figure 1. Experimental units for feeding rates tests with terrestrial isopods and fecal pellets from different food sources. A in Coprophagy in detritivores: methodological design for feeding studies in terrestrial isopods (Crustacea, Isopoda, Oniscidea)
Figure 1. Experimental units for feeding rates tests with terrestrial isopods and fecal pellets from different food sources. A) Treatment access; coprophagy is allowed. B) Treatment removal; coprophagy and bacterial activity on feces are avoided. C) Treatment net; coprophagy is avoided and bacterial activity on feces allowed. D) Fecal pellet from carrot (left) and decomposing leaf (right) consumption.
Unit Test Trace Analysis Data
<p>In order to find performance changes at code level, Peass (https://github.com/DaGeRe/peass) uses unit tests as proxy for the performance of a software. The performance of a unit test may not change if the called source stays the same. Therefore, PeASS' regression test selection identifies which tests to execute. For further analysis of the performance changes, the changed source is analysed and methods, which are called and which contain source code changes, are extracted.</p> <p>This dataset provides the results of the regression test selection and source code analysis of Apache Commons Compress, CSV, DBCP, fileupload, imaging, IO, JCS, numbers, pool and text and httpcomponents-core.</p> <p>To execute the analysis, execute the following steps:</p> <pre><code class="language-bash">tar -xvf peass_metadata_commons.tar # extract folder chmod +x getCalls.sh # Make script executable, since Zenodo provides scripts without x flag ./getCalls.sh # Execute analysis</code></pre> <p>This consumes a significant amount of hard disc space and contains long filenames; therefore execution on eCryptfs might yield problems.</p>
Unit test generation for common and uncommon behaviors: dataset
<p>The data used for the 'Unit test generation for common and uncommon behaviors' Master's thesis.</p> <p>The folder structure used in the archives is the same as the one used by the linked evaluation tool.</p> <p>The archives contain the following:</p> <ul> <li>cubtg-eval-1.4.tar.gz: tests generated by EvoSuite</li> <li>stadard.log.tar.gz: EvoSuite log files created during test generation</li> <li>cubtg-eval-1.4-pit-1.6.tar.gz: result of running PIT on the generated tests</li> </ul>
MIDAS2 unit testing
<p>MIDAS2 unit test and custom genome collection template.</p> <p>Github: https://github.com/czbiohub/MIDAS2</p>
Artefact to our paper "An Empirical Study of Automated Unit Test Generation for Python"
<p><strong>Artefact for “An Empirical Study of Automated Unit Test Generation for Python”</strong></p> <p>Together with our paper “An Empirical Study of Automated Unit Test Generation for Python”, we provide this artefact for future use.</p> <p><strong>Pynguin Version</strong></p> <p>We used Pynguin 0.25.2 for our experiments. The releases of Pynguin are archieved by Zenodo, too. Pynguin 0.25.2 is available under DOI <a href="https://doi.org/10.5281/zenodo.6836225">10.5281/zenodo.6836225</a>.</p> <p><strong>Preparation of the Environment</strong></p> <p>We use the <a href="https://python-poetry.org"><code>poetry</code></a> dependency-management tool to manage all dependencies for this artefact. Install this tool if you have not done yet. Furthermore, let <code>poetry</code> create a virtual environment for the experiment by execution <code>poetry install</code>.</p> <p><strong>Execution of the Experiment</strong></p> <p>The execution scripts make several assumptions that are based on our infrastructure. We maintain a SLURM cluster infrastructure that defines different <code>constraints</code> for different machines.</p> <p>Furthermore, we assume some paths to be present: we assume every computing machine to have writable mount points at <code>/local/${USER}</code> and <code>/local/hdd/${USER}</code>. On our machines, both are mount points on the local hard disk/SSD of the computing machines. Additionally, we have a shared mount <code>/scratch/${USER}</code>, which is mounted via NFS from a central file server. This mount point is also mounted on all computing machines.</p> <p>We assume the created and packaged Docker image to be located at <code>/scratch/lukasczy/pynguin.tar</code>. You can change this path by editing the XML files. These XML files contain the basic definitions of the jobs: they specify the SLURM constraint, the version of the Pynguin Docker container, the used Pynguin configurations as well as the modules used for the experiments. These modules have to reside under <code>projects</code>, as they come with this artefact.</p> <p>The Python script <code>execution.py</code> generates the actual run scripts from the XML file. It generates all scripts necessary to run a SLURM array job consisting of all runs for the experiment. Further general settings for the SLURM array job are present in this file.</p> <p>The Bash script <code>run_experiment.sh</code> executes the full execution pipeline; one has to specify the variable <code>EXPERIMENT_NAME</code> to match the name of the respective XML file who's defined experiment shall be executed.</p> <p><em>Important:</em> Executing the full experiment can take several days, depending on your computing infrastructure! We do therefore provide the raw result CSVs for further inspection.</p> <p><strong>Data Analysis</strong></p> <p>All raw data resides in the <code>data</code> folder:</p> <ul> <li><code>loc_data.csv</code> contains all information about the lines of code in each module. This file was created using the <code>extract_locs_and_types.py</code> script in the root folder. Please note that executing this script requires that the <code>cloc</code> utility tool is installed on your system's path.</li> <li><code>types.csv</code> and <code>types_per_module.csv</code> contain type information extract from the modules at different granularity level. They are also generated using the aforementioned script.</li> <li><code>results-assertion.csv.xz</code> contains the raw results from the experiment for RQ3 that evaluates the effectiveness of the assertions.</li> <li><code>results.csv.xz</code> contains the raw results from the experiment for RQ1 and RQ2.</li> </ul> <p>We provide the Jupyter Notebook that generated the plots, tables, and various LaTeX macros in the <code>notebooks</code> folder. Please note that if you want to reexecute this notebook, you might have to change the <code>PAPER_EXPORT_PATH</code> constant in cell <code>[2]</code> to a suitable location on your machine. Executing this notebook requires a installation of a TeX system to be available on your system because the plots are generated using <code>pdflatex</code> and <code>matplotlib</code>s pgf backend.</p> <p><strong>Further Data</strong></p> <p>The folder <code>projects</code> contains all the projects in the versions stated in our paper. The folder <code>run-logs</code> contains all the run logs from our experiment executions.</p>
Excavated test units at the Piqua site
Test units excavated by students at the Wright State University Field School in Archaeology at the Piqua site, May 2021. Source: Objaverse 1.0 / Sketchfab
Replication Kit for Paper: "Are There Any Unit Tests? An Empirical Study on Unit Testing in Open Source Python Projects"
<p>Replication Kit for the Paper "Are there any Unit Tests? An Empirical Study on Open Source Python Projects" by Fabian Trautsch, Jens Grabowski.</p> <p>You can cite the paper via:</p> <p>@inproceedings{trautsch2017there,<br> title={Are There Any Unit Tests? An Empirical Study on Unit Testing in Open Source Python Projects},<br> author={Trautsch, Fabian and Grabowski, Jens},<br> booktitle={Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST)},<br> pages={207--218},<br> year={2017},<br> organization={IEEE}<br> }</p> <p> </p> <p>Contents:<br> 1) Used version of the vcsSHARK<br> - located in “vcsSHARK”<br> 2) Used version of the testImpSHARK<br> - located in “testImpSHARK”<br> 3) Analysis implementations<br> - located in “testImpSHARK/testimpshark/analysis”<br> 4) Raw Data CSV Files<br> - located in “testImpSHARK/testimpshark/analysis/data<br> 5) Raw MongoDB<br> - located in “mongo_backup”</p> <p><br> Usage:<br> 1) Usage instructions for the vcsSHARK is given on its github homepage (http://ftrautsch.github.io/vcsSHARK/index.html) or directly in the “vcsSHARK/pyvcsshark/main.py” file</p> <p>2) Usage instructions for the testImpSHARK:<br> - if only one revision should be analyzed use “testImpSHARK/main.py”<br> - if all revisions should be analyzed use “testImpSHARK/execution.py”<br> - in both files concrete instructions can be found</p> <p>3) Each analysis file is commented. For some of them (rq1_boxplot.py and rq4.py the connection to the MongoDB must be changed). For the R files, the path to the data must be adapted. Otherwise, the files can be directly executed.</p> <p>4) The MongoDB can be restored via:<br> mongorestore --gzip --archive=smartshark040816.gz --db smartshark --host <HOST> --port <PORT> --username <USERNAME> --password <PASSWORD> --authenticationDatabase <AUTHENTICATION_DATABASE></p> <p><br> Tests:<br> 1) The tests can be run directly via the unittest framework of python: e.g., python -m unittest tests/test_common.py</p>
Replication package of "Higher Fault Detection Through Novel Density Estimators in Unit Test Generation"
<p>Replication package for the paper "Higher Fault Detection Through Novel Density Estimators in Unit Test Generation" accepted at the Symposium on Search-based Software Engineering (SSBSE) 2024.</p> <p> </p> <p> </p>
Adaptive Parameter Control for Search-Based Unit Test Generation — Replication Package
<h1>Running the experiments</h1> <p>Prerequisites for running the experiments:</p> <ul> <li><a href="https://www.docker.com/" target="_blank" rel="noopener">Docker</a></li> <li><a href="https://python-poetry.org/" target="_blank" rel="noopener">Poetry</a></li> </ul> <p>Steps to run the experiments:</p> <ol> <li>Download the experiment zip file you wish to run (<code>single_parameter_experiment.zip</code> or <code>multi_parameter_experiment.zip</code>).</li> <li>Un-zip the file.</li> <li>Open a terminal and navigate to the unzipped folder (e.g. <code>cd single_parameter_experiment</code>).</li> <li>Run <code>poetry install --only main</code> to install all dependencies.</li> <li>To run the experiment, run <code>poetry run python run_experiment.py</code>.</li> <li>All results can be found in the folder <code>data/</code>.</li> </ol> <p>The modules used for the experiment are defined in the file <code>experiment_modules.py</code> and to see the experiment configuration, look in <code>run_experiment.py</code>.</p> <p><strong>Warning</strong>: the experiments take several weeks to run on a single machine, therefore it is advisable to split the experiments based on modules and run them in parallel.</p> <h1>Running the analysis</h1> <p>Prerequisites for running the analysis:</p> <ul> <li><a href="https://conda.io/projects/conda/en/latest/user-guide/getting-started.html" target="_blank" rel="noopener">Conda</a></li> </ul> <p>Steps to run the analysis:</p> <ol> <li>Download the analysis zip file (<code>analysis-adaptive-parameter-control.zip</code>).</li> <li>Un-zip the file.</li> <li>Open a terminal and navigate to the unzipped folder (e.g. <code>cd analysis-adaptive-parameter-control</code>).</li> <li>Run the following command to install the conda environment and all dependencies: <code>conda env create -f environment.yml</code></li> <li>If you want to re-run the Bayesian models locally on your machine, follow the <strong>optional</strong> step below, otherwise download and unzip the trace data from the replication package, i.e., <code>Trace data single.zip</code> and <code>Trace data multi.zip</code>. </li> <li>Place the <code>.nc</code> files in the corresponding folder: <code>analysis-adaptive-parameter-control/single_parameter/</code> or <code>analysis-adaptive-parameter-control/multi_parameter/</code>. <ol> <li>E.g. the <code>coverage_rate_model_single_parameter.nc</code> goes in the <code>single_parameter</code> folder, while the <code>coverage_rate_model_multi_parameter.nc</code> goes in the <code>multi_parameter</code> folder.</li> </ol> </li> <li>Navigate to the notebooks folder (<code>Notebooks/</code>).</li> <li>Open a notebook of choice (<code>coverage_rate_multi_parameter.ipynb</code>, <code>coverage_rate_single_parameter.ipynb</code>, <code>final_coverage_multi_parameter.ipynb</code>, <code>final_coverage_single_parameter.ipynb</code>, <code>overhead_model_multi_parameter.ipynb</code>, or <code>overhead_model_single_parameter.ipynb</code>).</li> <li>Navigate to the section called "Data analysis" and run all cells in order.</li> </ol> <h3>(Optional) Running the Bayesian models locally before the analysis.</h3> <ol> <li>Navigate to the notebooks folder (<code>Notebooks/</code>).</li> <li>Open a notebook of choice (<code>coverage_rate_multi_parameter.ipynb</code>, <code>coverage_rate_single_parameter.ipynb</code>, <code>final_coverage_multi_parameter.ipynb</code>, <code>final_coverage_single_parameter.ipynb</code>, <code>overhead_model_multi_parameter.ipynb</code>, or <code>overhead_model_single_parameter.ipynb</code>).</li> <li>Navigate to the section called "Model specification" and run the three notebook cells.</li> </ol> <p><strong>Warning</strong>: this will take a long time, if you don't have the time, use the following alternative instead</p> <h1>Data</h1> <p>The data from when we ran the experiments is available in the <code>Single data.zip</code> and <code>Multi data.zip</code> files.</p> <p>The structure of these are the following:</p> <ul> <li>There are folders for each module the experiment was run on, further divided into each unique run. All these folders include: <ul> <li>Coverage reports.</li> <li>Complete logs for the unique run.</li> <li>A timeline over controlled parameter values during the test generation process.</li> <li>The complete Pynguin configuration for the run.</li> <li>The generated test suite.</li> </ul> </li> <li>There is one <code>statistics.csv</code> file containing some information about each run and their branch coverage timelines.</li> </ul> <h1>Running the parameter assignment analysis</h1> <p>Prerequisites for running the parameter assignment analysis:</p> <ul> <li><a href="https://conda.io/projects/conda/en/latest/user-guide/getting-started.html" target="_blank" rel="noopener">Conda</a></li> </ul> <p>Steps to run the parameter assignment analysis:</p> <ol> <li>Download the analysis zip file (<code>parameter-assignment.zip</code>).</li> <li>Un-zip the file.</li> <li>Open a terminal and navigate to the unzipped folder (e.g. <code>cd parameter-assignment</code>).</li> <li>Run the following command to install the conda environment and all dependencies: <code>conda env create -f environment.yml</code></li> <li>Navigate to the notebooks folder (<code>Notebooks/</code>).</li> <li>Open the notebook <code>parameter_assignment_analysis.ipynb</code>.</li> <li>Run all cells in order.</li> </ol>
Binary data file used for analysis_common_envelope unit testing in the Phantom smoothed particle hydrodynamics and magnetohydrodynamics code
<p>** this file is automatically downloaded as part of the Phantom github actions tests **</p> <p>This is an example snapshot from a Phantom simulation of a common envelope interaction, taken from the paper by <a href="https://ui.adsabs.harvard.edu/abs/2022MNRAS.517.3181G">González-Bolívar et al. (2022)</a>. It is posted here primarily in order to perform unit and regression testing on the <a href="https://github.com/danieljprice/phantom/blob/master/src/utils/analysis_common_envelope.f90">analysis_common_envelope</a> module in the Phantom smoothed particle hydrodynamics and magnetohydrodynamics code (<a href="http://adsabs.harvard.edu/abs/2018PASA...35...31P">Price et al. 2018</a>).</p>
Dataset for "Provide Developer-Approved Descriptive Names for Unit Tests"
<p>Dataset for "Provide Developer-Approved Descriptive Names for Unit Tests"</p> <p>Author: Jianwei Wu, James Clause</p> <p>Please contact at wjwcis@udel.edu for any questions.</p>
Genetic Testing to Understand and Address Renal Disease Disparities Across the United States
ClinicalTrials.gov study NCT04191824. IPD Sharing: YES. Countries: 1. Publications: 1.
Impact of a Procalcitonin Testing and Treatment Algorithm on Antibiotic Use and Outcomes in the Pediatric Intensive Care Unit
ClinicalTrials.gov study NCT03440918. IPD Sharing: NO. Countries: 1. Publications: 1.
HPV DNA Testing Through Mobile Mammography Unit
ClinicalTrials.gov study NCT03551028. IPD Sharing: NO. Countries: 1. Publications: 2.
Genetic Testing to Understand and Address Renal Disease Disparities Across the United States Pharmacogenetic Substudy
ClinicalTrials.gov study NCT06748040. IPD Sharing: YES. Countries: 1. Publications: 1.
Does Unit-Tested Code Crash? A Case Study of Eclipse: Replication Package
<p><strong>Does Unit-Tested Code Crash? A Case Study of Eclipse: Replication Package</strong></p> <p>This is a replication package associated with the paper titled “Does Unit-Tested Code Crash? A Case Study of Eclipse”. Below is a description of the package’s contents.</p> <p><strong>Data</strong></p> <p>Data files associated with the paper are provided in the <code>data</code> directory.</p> <p><strong>Text file <code>tested-crashed.txt</code></strong></p> <p>Data specifying whether methods were tested and whether they crashed (according to the criteria adopted in the study). Extracted from <code>matches.xlsx</code>. The data are used as input for Fisher’s test (RQ1).</p> <p><strong>Spreadsheet <code>matches.xlsx</code></strong></p> <p>Test coverage data and calculations associated with failed methods, class coverage, and matching method coverage results are provided in an Excel spreadsheet. Below is the description of the spreadsheet’s contents.</p> <p>Worksheet <em>Test Coverage</em></p> <p>Contains the data regarding the JaCoCo test code coverage analysis.</p> <ul> <li>Class: The name of the class in which a method appears in JVM internal form notation</li> <li>Method: The method’s name</li> <li>Parameters: The method’s arguments in JVM parameter descriptor format; required to handle Java’s {} polymporphism</li> <li>Class Has Unit Test: Whether the corresponding class has associated unit test code</li> <li>Class Unit-Test Line Density: The ratio of lines in class’s test code over those in the class’s implementation code</li> <li>Covered Instructions / Branches / Lines: As reported by JaCoCo</li> <li>Total Instructions / Branches / Lines: As reported by JaCoCo</li> <li>Covered Instructions / Branches / Lines ratio: The ratio between the two preceding values; 1 for methods without any branches</li> <li>Top-1 / Top-6 / Top-10 : In how many stack traces the method appears within; the top-10 / top-6 / the very first stack frame(s)</li> <li>Tested: TRUE if the method is considered tested by having a test code coverage above the median (0.966) and an associated test class</li> <li>Crashed: TRUE if the method has crashed as evidenced by its appearance on the topmost stack frame</li> <li>Stack trace file names: in which the method appeared</li> </ul> <p>Worksheet <em>Test Existence</em></p> <p>Contains the data of the analysis regarding the existence of test code.</p> <ul> <li>Class: Class containing implementation code</li> <li>TestClassNames: Classes that contain tests for the above</li> <li>Number of relevant tests</li> <li>Lines in class test code</li> <li>Lines of class</li> <li>Class Unit-Test Line Density: The ratio between the two above</li> </ul> <p>Worksheet <em>Metrics</em></p> <p>Contains the derivation of metrics reported in the paper. In the cases of tables these are formatted in LaTeX for direct incorporation into the text.</p> <p><strong>Spreadsheet <code>jacoco.xlsx</code></strong></p> <p>Complete test coverage data obtained from JaCoCo are provided in an Excel spreadsheet. Below is the description of the spreadsheet’s contents.</p> <p>Worksheet <em>Data</em></p> <p>Contains the following method code coverage fields as reported by JaCoCo, as well as the calculated percentages.</p> <ul> <li>Class</li> <li>Method</li> <li>Parameters</li> <li>Covered Instructions</li> <li>Total Instructions</li> <li>% Covered Instructions</li> <li>Covered Branches</li> <li>Total Branches</li> <li>% Covered Branches</li> <li>Covered Lines</li> <li>Total Lines</li> <li>% Covered Lines</li> </ul> <p>Worksheet <em>Metrics</em></p> <p>Contains the derivation of numbers reported in the preliminary quantitative analysis and Figure 2.</p> <p>Compressed tar archive <code>eclipse-src.tar.gz</code></p> <p>Contains the Eclipse source code used for running the Eclipse tests with JaCoCo code coverage analysis. It was obtained from the Eclipse source code repositories as follows.</p> <ul> <li>Clone the Eclipse aggreagator repository into a directory named z by running: <code>git clone -b master --recursive git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git z</code></li> <li>In the <code>z</code> directory, checking out the used release by running <code>cd z && git submodule foreach git checkout M20160212-1500</code></li> <li>Checking out the release for the main repository by running: <code>git checkout M20160212-1500</code></li> <li>Applying the patch <code>eclipse-src.diff</code></li> </ul> <p><strong>Patch file <code>eclipse-src.diff</code></strong></p> <p>See above.</p> <p><strong>Zip file <code>incidents.zip</code></strong></p> <p>Contains the 126,026 incidents (crash report stack traces and meta-data) associated with <em>EclipseProduct</em> <code>org.eclipse.epp.package.java.product</code> and <em>BuildID</em> <code>4.5.2.M20160212-1500</code>. This is a subset from the two million incidents available as the <a href="http://software-data.org/datasets/aeri-stacktraces/downloads/incidents_full.tar.bz2">AERI stack traces data set</a>.</p> <p>The subset of incidents was extracted from the full AERI data set with the following command.</p> <pre><code class="language-bash">for f in *; do grep -q org.eclipse.epp.package.java.product $f && grep -q 4.5.2.M20160212-1500 $f && mv $f selected-files/ done</code></pre> <p><br> <strong>Compressed file <code>jacoco.xml.gz</code></strong></p> <p>Contains the results of the JaCoCo code coverage analysis over the Eclipse testing.</p> <p><strong>Code</strong></p> <p>The following scripts are provided in the <code>src</code> directory</p> <ul> <li><code>extract.py</code>: script for extracting crash (incidents) and coverage (JaCoCo) data</li> <li><code>unit-tested-classes.py</code>: script for finding the classes with associated unit test code</li> <li><code>merge.py</code>: script for matching crash (incidents) with coverage (JaCoCo) data</li> <li><code>fisher.r</code>: R script for running Fisher’s test</li> </ul>
Understanding the Differences in the Unit Tests Produced by Humans and Coverage-Directed Automated Generation
<p>Automated test generation - the use of tools to create all or part of test cases - has a critical role in controlling the cost of testing. A particular area of focus in automated test generation research is unit testing. Unit tests are intended to test the functionality of a small isolated unit of code - typically a class. </p> <p>In automated test generation research, it is not abnormal to compare the effectiveness of the test cases generated by automation to those written by humans. Indeed, a common premise of automation research - implicitly or explicitly - is that effective automation can replace human effort. The hypothesis postulated is that, if we make enough advances, a tool could replace the tremendous effort expended by a human tester to create those unit tests. </p> <p>This observation leads to two natural questions. Do the tests produced by humans and automation differ in the types of faults they detect? If so, in what ways are the tests produced and the faults detected different? Understanding when and how to deploy automation requires a clearer understanding of how the tests produced by humans and automation are different, and how those differences in turn affect the ability of those test cases to detect faults. Insight into the differences between human and automation-produced test cases could lead not only to improvements in the ability of automation to replace human effort, but improvements in our ability to use automation to augment human effort. The goal of this study is to explore and attempt to quantify those differences. </p> <p>In this study, we make use of the EvoSuite test generation framework for Java. We generate test suites targeting two configurations - a traditional single-criterion configuration targeting Branch Coverage over the source code and a more sophisticated multi-objective configuration targeting eight criteria. Controlling for coverage level, we compare the suites generated by EvoSuite to those written by humans for five mature, popular open-source systems in terms of both their syntactic structure and their ability to detect 45 different types of faults. Our goal is not to declare a "winner'", but to identify the areas where humans and automation differ in their capabilities, and - in turn - to make recommendations on how human and automation effort can be combined to overcome gaps in the coverage of the other. We aim to identify lessons that will improve human practices, lead to the creation of more effective automation, and present natural opportunities to both augment and replace human effort. </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.