Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
8
datasets available to search
ShareScore release 0.9.0
Dataset results
8 results for “automated unit testing”
Artefact to our paper "An Empirical Study of Automated Unit Test Generation for Python"
<p><strong>Artefact for “An Empirical Study of Automated Unit Test Generation for Python”</strong></p> <p>Together with our paper “An Empirical Study of Automated Unit Test Generation for Python”, we provide this artefact for future use.</p> <p><strong>Pynguin Version</strong></p> <p>We used Pynguin 0.25.2 for our experiments. The releases of Pynguin are archieved by Zenodo, too. Pynguin 0.25.2 is available under DOI <a href="https://doi.org/10.5281/zenodo.6836225">10.5281/zenodo.6836225</a>.</p> <p><strong>Preparation of the Environment</strong></p> <p>We use the <a href="https://python-poetry.org"><code>poetry</code></a> dependency-management tool to manage all dependencies for this artefact. Install this tool if you have not done yet. Furthermore, let <code>poetry</code> create a virtual environment for the experiment by execution <code>poetry install</code>.</p> <p><strong>Execution of the Experiment</strong></p> <p>The execution scripts make several assumptions that are based on our infrastructure. We maintain a SLURM cluster infrastructure that defines different <code>constraints</code> for different machines.</p> <p>Furthermore, we assume some paths to be present: we assume every computing machine to have writable mount points at <code>/local/${USER}</code> and <code>/local/hdd/${USER}</code>. On our machines, both are mount points on the local hard disk/SSD of the computing machines. Additionally, we have a shared mount <code>/scratch/${USER}</code>, which is mounted via NFS from a central file server. This mount point is also mounted on all computing machines.</p> <p>We assume the created and packaged Docker image to be located at <code>/scratch/lukasczy/pynguin.tar</code>. You can change this path by editing the XML files. These XML files contain the basic definitions of the jobs: they specify the SLURM constraint, the version of the Pynguin Docker container, the used Pynguin configurations as well as the modules used for the experiments. These modules have to reside under <code>projects</code>, as they come with this artefact.</p> <p>The Python script <code>execution.py</code> generates the actual run scripts from the XML file. It generates all scripts necessary to run a SLURM array job consisting of all runs for the experiment. Further general settings for the SLURM array job are present in this file.</p> <p>The Bash script <code>run_experiment.sh</code> executes the full execution pipeline; one has to specify the variable <code>EXPERIMENT_NAME</code> to match the name of the respective XML file who's defined experiment shall be executed.</p> <p><em>Important:</em> Executing the full experiment can take several days, depending on your computing infrastructure! We do therefore provide the raw result CSVs for further inspection.</p> <p><strong>Data Analysis</strong></p> <p>All raw data resides in the <code>data</code> folder:</p> <ul> <li><code>loc_data.csv</code> contains all information about the lines of code in each module. This file was created using the <code>extract_locs_and_types.py</code> script in the root folder. Please note that executing this script requires that the <code>cloc</code> utility tool is installed on your system's path.</li> <li><code>types.csv</code> and <code>types_per_module.csv</code> contain type information extract from the modules at different granularity level. They are also generated using the aforementioned script.</li> <li><code>results-assertion.csv.xz</code> contains the raw results from the experiment for RQ3 that evaluates the effectiveness of the assertions.</li> <li><code>results.csv.xz</code> contains the raw results from the experiment for RQ1 and RQ2.</li> </ul> <p>We provide the Jupyter Notebook that generated the plots, tables, and various LaTeX macros in the <code>notebooks</code> folder. Please note that if you want to reexecute this notebook, you might have to change the <code>PAPER_EXPORT_PATH</code> constant in cell <code>[2]</code> to a suitable location on your machine. Executing this notebook requires a installation of a TeX system to be available on your system because the plots are generated using <code>pdflatex</code> and <code>matplotlib</code>s pgf backend.</p> <p><strong>Further Data</strong></p> <p>The folder <code>projects</code> contains all the projects in the versions stated in our paper. The folder <code>run-logs</code> contains all the run logs from our experiment executions.</p>
Understanding the Differences in the Unit Tests Produced by Humans and Coverage-Directed Automated Generation
<p>Automated test generation - the use of tools to create all or part of test cases - has a critical role in controlling the cost of testing. A particular area of focus in automated test generation research is unit testing. Unit tests are intended to test the functionality of a small isolated unit of code - typically a class. </p> <p>In automated test generation research, it is not abnormal to compare the effectiveness of the test cases generated by automation to those written by humans. Indeed, a common premise of automation research - implicitly or explicitly - is that effective automation can replace human effort. The hypothesis postulated is that, if we make enough advances, a tool could replace the tremendous effort expended by a human tester to create those unit tests. </p> <p>This observation leads to two natural questions. Do the tests produced by humans and automation differ in the types of faults they detect? If so, in what ways are the tests produced and the faults detected different? Understanding when and how to deploy automation requires a clearer understanding of how the tests produced by humans and automation are different, and how those differences in turn affect the ability of those test cases to detect faults. Insight into the differences between human and automation-produced test cases could lead not only to improvements in the ability of automation to replace human effort, but improvements in our ability to use automation to augment human effort. The goal of this study is to explore and attempt to quantify those differences. </p> <p>In this study, we make use of the EvoSuite test generation framework for Java. We generate test suites targeting two configurations - a traditional single-criterion configuration targeting Branch Coverage over the source code and a more sophisticated multi-objective configuration targeting eight criteria. Controlling for coverage level, we compare the suites generated by EvoSuite to those written by humans for five mature, popular open-source systems in terms of both their syntactic structure and their ability to detect 45 different types of faults. Our goal is not to declare a "winner'", but to identify the areas where humans and automation differ in their capabilities, and - in turn - to make recommendations on how human and automation effort can be combined to overcome gaps in the coverage of the other. We aim to identify lessons that will improve human practices, lead to the creation of more effective automation, and present natural opportunities to both augment and replace human effort. </p>
Dataset for Automated Unit Test Generation via Chain of Thought Prompt and Reinforcement Learning
<p>This is the replication package including three types datasets: training dataset with CoT prompts, reward dataset for training reward model, rl dataset for optimizing policy model. The training dataset includes filter_test_cot_rule_50k.csv, filter_train_cot_rule_50k.csv, and filter_valid_cot_rule_50k.csv. These three datasets includes multiple fields (i.e., src_fm, intention, plan, elaboration, gpt_test, src_fm_cot_gpt, target, src_fm_fc_ms_ff,src_fm_intention,src_fm_plan,src_fm_elaboration,idx,rule_cot,rule_cot_nlp,combine_cot,src_fm_rule_cot_nlp,src_fm_cot_nlp_gpt,gpt_cot_filter,src_fm_plan_intention). The reward dataset includes test_athena.json, train_athena.json, and valid_athena.json three files. The rl dataset includes three files: filter_test_cot_gpt_rl.csv, filter_train_cot_gpt_rl.csv, filter_valid_cot_gpt_rl.csv. These files include mulitple fields: src_fm,intention,plan,elaboration,gpt_test,src_fm_cot_gpt,target,src_fm_fc_ms_ff,src_fm_intention,src_fm_plan,src_fm_elaboration,gpt_cot_filter.</p>
Artefact for "Combining Type Inference and Automated Unit Test Generation for Python"
<p>Contains the artefact for our ASE 2023 submission “Combining Type Inference and Automated Unit Test Generation for Python”.</p>
Artefact to the paper "Combining Type Inference and Automated Unit Test Generation for Python" submitted to ISSTA 2024
Open the record for dataset details and reuse information.
Overlap between Automated Unit and Acceptance Testing -- a Systematic Literature Review (Dataset)
<p>This is the dataset used to conduct a mapping study followed by a systematic literature review on the overlap between automated unit and acceptance testing. It is composed of a full export from Scopus and its cleaned up version with the classification for both the MS and SLR.</p> <p>A full reproduction package is available on <a href="http://gitlab.com/mde4asd/hybrid-software-testing-reproduction-pkg">gitlab</a>.</p> <p>This work has been published at EASE 2022.</p>
Automated Unit Test Generation for Python—Experiments
<p>This archive contains the used projects, the raw result data, and the Jupyter Notebook used for evaluation of our work</p> <p>“Automated Unit Test Generation for Python”</p> <p>accepted at the 12th Symposium for Search-Based Software Engineering (SSBSE 2020), October 7–8, 2020, Bari, Italy.</p>
Dataset for Automated Unit Test Generation via Chain of Thought Prompt and Reinforcement Learning
<p>This is the replication package including three types datasets: training dataset with CoT prompts, reward dataset for training reward model, rl dataset for optimizing policy model. The training dataset includes filter_test_cot_rule_50k.csv, filter_train_cot_rule_50k.csv, and filter_valid_cot_rule_50k.csv. These three datasets includes multiple fields (i.e., src_fm, intention, plan, elaboration, gpt_test, src_fm_cot_gpt, target, src_fm_fc_ms_ff,src_fm_intention,src_fm_plan,src_fm_elaboration,idx,rule_cot,rule_cot_nlp,combine_cot,src_fm_rule_cot_nlp,src_fm_cot_nlp_gpt,gpt_cot_filter,src_fm_plan_intention). The reward dataset includes test_athena.json, train_athena.json, and valid_athena.json three files. The rl dataset includes three files: filter_test_cot_gpt_rl.csv, filter_train_cot_gpt_rl.csv, filter_valid_cot_gpt_rl.csv. These files include mulitple fields: src_fm,intention,plan,elaboration,gpt_test,src_fm_cot_gpt,target,src_fm_fc_ms_ff,src_fm_intention,src_fm_plan,src_fm_elaboration,gpt_cot_filter.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.