Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
18
datasets available to search
ShareScore release 0.9.0
Dataset results
18 results for “Regression Testing”
swmm-nrtestsuite: Regression Test Suite for OWA SWMM
<p>Open Water Analytics (OWA) Stormwater Management Model (SWMM)</p> <p>Project Link: <br> https://github.com/OpenWaterAnalytics/swmm-nrtestsuite</p> <p> </p>
RTPTorrent: An Open-source Dataset for Evaluating Regression Test Prioritization
<p>This dataset is designed to be used in evaluation studies of regression test prioritization techniques. It includes 20 open-source Java projects from GitHub and over 100,000 logs of real-world build logs from TravisCI. The projects span a wide range with regard to size, number of contributors, and maturity of open-source Java projects available on GitHub.</p> <p>Futher, the dataset includes the results of baseline approaches to ease the comparison of new techniques applied to the dataset.</p> <p>A readme file with a more detailed description of the structure of the dataset is included. For even more information see the corresponding MSR 2020 publication.</p> <p>Versions:</p> <ul> <li> 2020-09-23 (version 1.1) <ul> <li>Updated archived `deeplearning4j` repository with a fork that contains all of the original commits. Repository at the original GitHub location had been replaced. Defect identified by Daniel Elsner (Technische Universität München).</li> <li>Renamed root folder from MSR2 to rtp-torrent</li> </ul> </li> <li>2020-05-25 (version 1.0) <ul> <li>Initial release</li> </ul> </li> </ul> <p> </p>
Regression-Test History Data for Flaky Test-Research, Dataset
<p>The dataset comprises developer test results of Maven projects with flaky tests across a range of consecutive commits from the projects' git commit histories. The Maven projects are a subset of those investigated in an <a href="https://doi.org/10.1145/3428270">OOPSLA 2020 paper</a>. The commit range for this dataset has been chosen as the flakiness-introducing commit (FIC) and iDFlakies-commit (see the OOPSLA paper for details). The commit hashes have been obtained from the <a href="https://github.com/TestingResearchIllinois/idoft/blob/main/tic-fic-data.csv">IDoFT dataset</a>.</p> <p>The dataset will be presented at the <a href="https://conf.researchr.org/home/icse-2024/ftw-2024">1st International Flaky Tests Workshop 2024 (FTW 2024)</a>. Please refer to <a href="https://doi.org/10.1145/3643656.3643901">our extended abstract</a> for more details about the motivation for and context of this dataset.</p> <p>The following table provides a summary of the data.</p> <table> <tbody> <tr> <td><strong>Slug (Module)</strong></td> <td><strong>FIC Hash</strong></td> <td><strong>Tests</strong></td> <td><strong>Commits</strong></td> <td><strong>Av. Commits/Test</strong></td> <td><strong>Flaky Tests</strong></td> <td><strong>Tests w/ Consistent Failures</strong></td> <td><strong>Total Distinct Histories</strong></td> </tr> <tr> <td>TooTallNate/Java-WebSocket</td> <td> 822d40</td> <td>146</td> <td> 75</td> <td> 75</td> <td>24</td> <td> 1</td> <td>2.6x10^9</td> </tr> <tr> <td>apereo/java-cas-client (cas-client-core)</td> <td> 5e3655</td> <td>157</td> <td> 65</td> <td>61.7</td> <td> 3</td> <td> 2</td> <td>1.0x10^7</td> </tr> <tr> <td>eclipse-ee4j/tyrus (tests/e2e/standard-config)</td> <td> ce3b8c</td> <td>185</td> <td> 16</td> <td> 16</td> <td>12</td> <td> 0</td> <td> 261</td> </tr> <tr> <td>feroult/yawp (yawp-testing/yawp-testing-appengine)</td> <td> abae17</td> <td> 1</td> <td>191</td> <td>191</td> <td> 1</td> <td> 1</td> <td> 8</td> </tr> <tr> <td>fluent/fluent-logger-java</td> <td> 5fd463</td> <td> 19</td> <td>131</td> <td>105.6</td> <td>11</td> <td> 2</td> <td>8.0x10^32</td> </tr> <tr> <td>fluent/fluent-logger-java</td> <td> 87e957</td> <td> 19</td> <td>160</td> <td>122.4</td> <td>11</td> <td> 3</td> <td>2.1x10^31</td> </tr> <tr> <td>javadelight/delight-nashorn-sandbox</td> <td> d0d651</td> <td> 81</td> <td>113</td> <td>100.6</td> <td> 2</td> <td> 5</td> <td>4.2x10^10</td> </tr> <tr> <td>javadelight/delight-nashorn-sandbox</td> <td> d19eee</td> <td> 81</td> <td> 93</td> <td>83.5</td> <td> 1</td> <td> 5</td> <td>2.6x10^9</td> </tr> <tr> <td>sonatype-nexus-community/nexus-repository-helm</td> <td> 5517c8</td> <td> 18</td> <td> 32</td> <td> 32</td> <td> 0</td> <td> 0</td> <td> 18</td> </tr> <tr> <td>spotify/helios (helios-services)</td> <td> 23260</td> <td>190</td> <td>448</td> <td>448</td> <td> 0</td> <td> 37</td> <td> 190</td> </tr> <tr> <td>spotify/helios (helios-testing)</td> <td> 78a864</td> <td> 43</td> <td>474</td> <td>474</td> <td> 0</td> <td> 7</td> <td> 43</td> </tr> </tbody> </table> <p> </p> <p>The columns are composed of the following variables:</p> <ul> <li><strong>Slug (Module):</strong> The project's GitHub slug (i.e., the project's URL is https://github.com/{Slug}) and, if specified, the module for which tests have been executed.</li> <li><strong>FIC Hash:</strong> The flakiness-introducing commit hash for a known flaky test as described in this OOPSLA 2020 paper. As different flaky tests have different FIC hashes, there may be multiple rows for the same slug/module with different FIC hashes. </li> <li><strong>Tests:</strong> The number of distinct test class and method combinations over the entire considered commit range.</li> <li><strong>Commits:</strong> The number of commits in the considered commit range</li> <li><strong>Av. Commits/Test:</strong> The average number of commits per test class and method combination in the considered commit range. The number of commits may vary for each test class, as some tests may be added or removed within the considered commit range.</li> <li><strong>Flaky Tests:</strong> The number of distinct test class and method combinations that have more than one test result (passed/skipped/error/failure + exception type, if any + assertion message, if any) across 30 repeated test suite executions on at least one commit in the considered commit range.</li> <li><strong>Tests w/ Consistent Failures:</strong> The number of distinct test class and method combinations that have the same error or failure result (error/failure + exception type, if any + assertion message, if any) across all 30 repeated test suite executions on at least one commit in the considered commit range.</li> <li><strong>Total Distinct Histories:</strong> The number of distinct test results (passed/skipped/error/failure + exception type, if any + assertion message, if any) for all test class and method combinations along all commits for that test in the considered commit range.</li> </ul>
Dataset for Understanding and Improving Regression Test Selection in Continuous Integration
<p>This is the dataset for the paper "Understanding and Improving Regression Test Selection in Continuous Integration". The dataset consists of the logs collected from running RTS tools on Travis CI along with a CSV summary of those results.</p>
PyGBe regression test meshes
<p>This zip-file includes the meshes needed to run the regression tests of PyGBe, a Barba group open source Python GPU code for Boundary elements that does molecular electrostatics calculations with a continuum approach. https://github.com/barbagroup/pygbe</p> <p>Note:</p> <ul> <li> Lysozyme folder: contains the meshes for all the Lysozyme test.</li> <li>sphere folder: contains the meshes related to the one-sphere cases and two-spheres ones. </li> </ul>
TRAVEL: A Dataset with Toolchains for Test Generation and Regression Testing of Self-driving Cars Software
<p><strong>Introduction</strong></p> <p>This repository hosts the Testing Roads for Autonomous VEhicLes (TRAVEL) dataset. TRAVEL is an extensive collection of virtual roads that have been used for testing lane assist/keeping systems (i.e., driving agents) and data from their execution in state of the art, physically accurate driving simulator, called <a href="https://www.beamng.tech/">BeamNG.tech</a>. Virtual roads consist of sequences of <strong>road points</strong> interpolated using Cubic splines.</p> <p>Along with the data, this repository contains instructions on how to install the tooling necessary to generate new data (i.e., test cases) and analyze them in the context of test regression. We focus on test selection and test prioritization, given their importance for developing high-quality software following the DevOps paradigms.</p> <p>This dataset builds on top of our previous work in this area, including work on</p> <ul> <li>test generation (e.g., <a href="https://doi.org/10.1145/3293882.3330566">AsFault</a>, <a href="https://doi.org/10.1145/3368089.3409730">DeepJanus</a>, and <a href="https://doi.org/10.1145/3460319.3464811">DeepHyperion</a>) and the SBST CPS tool competition (<a href="https://doi.org/10.1109/SBST52555.2021.00011">SBST2021</a>),</li> <li>test selection: <a href="https://arxiv.org/abs/2111.04666">SDC-Scissor</a> and related <a href="https://github.com/ChristianBirchler/sdc-scissor">tool</a></li> <li>test prioritization: <a href="https://arxiv.org/abs/2107.09614">automated test cases prioritization work for SDCs</a>.</li> </ul> <p><strong>Dataset Overview</strong></p> <p>The TRAVEL dataset is available under the <code>data</code> folder and is organized as a set of <em>experiments</em> folders. Each of these folders is generated by running the <code>test-generator</code> (see below) and contains the configuration used for generating the data (<code>experiment_description.csv</code>), various statistics on generated tests (<code>generation_stats.csv</code>) and found faults (<code>oob_stats.csv</code>). Additionally, the folders contain the raw test cases generated and executed during each experiment (<code>test.<TEST_ID>.json</code>).</p> <p>The following sections describe what each of those files contains.</p> <p>Experiment Description</p> <p>The <code>experiment_description.csv</code> contains the settings used to generate the data, including:</p> <ul> <li><strong>Time budget</strong>. The overall generation budget in hours. This budget includes both the time to generate and execute the tests as driving simulations.</li> <li><strong>The size of the map</strong>. The size of the squared map defines the boundaries inside which the virtual roads develop in meters.</li> <li><strong>The test subject</strong>. The driving agent that implements the lane-keeping system under test. The TRAVEL dataset contains data generated testing the BeamNG.AI and the end-to-end Dave2 systems.</li> <li><strong>The test generator</strong>. The algorithm that generated the test cases. The TRAVEL dataset contains data obtained using various algorithms, ranging from naive and advanced random generators to complex evolutionary algorithms, for generating tests.</li> <li><strong>The speed limit</strong>. The maximum speed at which the driving agent under test can travel.</li> <li><strong>Out of Bound (OOB) tolerance</strong>. The test cases' oracle that defines the tolerable amount of the ego-car that can lie outside the lane boundaries. This parameter ranges between 0.0 and 1.0. In the former case, a test failure triggers as soon as any part of the ego-vehicle goes out of the lane boundary; in the latter case, a test failure triggers only if the entire body of the ego-car falls outside the lane.</li> </ul> <p>Experiment Statistics</p> <p>The <code>generation_stats.csv</code> contains statistics about the test generation, including:</p> <ul> <li><strong>Total number of generated tests</strong>. The number of tests generated during an experiment. This number is broken down into the number of <strong>valid tests</strong> and <strong>invalid tests</strong>. Valid tests contain virtual roads that do not self-intersect and contain turns that are not too sharp.</li> <li><strong>Test outcome</strong>. The test outcome contains the number of <strong>passed tests</strong>, <strong>failed tests</strong>, and <strong>test in error</strong>. Passed and failed tests are defined by the OOB Tolerance and an additional (implicit) oracle that checks whether the ego-car is moving or standing. Tests that did not pass because of other errors (e.g., the simulator crashed) are reported in a separated category.</li> </ul> <p>The TRAVEL dataset also contains statistics about the failed tests, including the overall number of failed tests (<strong>total oob</strong>) and its breakdown into OOB that happened while driving <strong>left</strong> or <strong>right</strong>. Further statistics about the diversity (i.e., <strong>sparseness</strong>) of the failures are also reported.</p> <p>Test Cases and Executions</p> <p>Each <code>test.<TEST_ID>.json</code> contains information about a test case and, if the test case is valid, the data observed during its execution as driving simulation.</p> <p>The data about the test case definition include:</p> <ul> <li><strong>The road points</strong>. The list of points in a 2D space that identifies the center of the virtual road, and their interpolation using cubic splines (<strong>interpolated_points</strong>)</li> <li><strong>The test ID</strong>. The unique identifier of the test in the experiment.</li> <li><strong>Validity flag and explanation</strong>. A flag that indicates whether the test is valid or not, and a brief message describing why the test is not considered valid (e.g., <code>the road contains sharp turns</code> or <code>the road self intersects</code>)</li> </ul> <p>The test data are organized according to the following JSON Schema and can be interpreted as <code>RoadTest</code> objects provided by the <a href="https://github.com/se2p/tool-competition-av/blob/main/code_pipeline/tests_generation.py">tests_generation.py</a> module.</p> <pre><code>{ "type": "object", "properties": { "id": { "type": "integer" }, "is_valid": { "type": "boolean" }, "validation_message": { "type": "string" }, "road_points": { §\label{line:road-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "interpolated_points": { §\label{line:interpolated-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "test_outcome": { "type": "string" }, §\label{line:test-outcome}§ "description": { "type": "string" }, "execution_data": { "type": "array", "items": { "$ref" : "schemas/simulationdata" } } }, "required": [ "id", "is_valid", "validation_message", "road_points", "interpolated_points" ] } </code></pre> <p>Finally, the <strong>execution data</strong> contain a list of timestamped state information recorded by the driving simulation. State information is collected at constant frequency and includes absolute position, rotation, and velocity of the ego-car, its speed in Km/h, and control inputs from the driving agent (steering, throttle, and braking). Additionally, execution data contain OOB-related data, such as the lateral distance between the car and the lane center and the OOB percentage (i.e., how much the car is outside the lane).</p> <p>The simulation data adhere to the following (simplified) JSON Schema and can be interpreted as Python objects using the <a href="https://github.com/se2p/tool-competition-av/blob/main/self_driving/simulation_data.py">simulation_data.py</a> module.</p> <pre><code>{ "$id": "schemas/simulationdata", "type": "object", "properties": { "timer" : { "type": "number" }, "pos" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel_kmh" : { "type": "number" }, "steering" : { "type": "number" }, "brake" : { "type": "number" }, "throttle" : { "type": "number" }, "is_oob" : { "type": "number" }, "oob_percentage" : { "type": "number" } §\label{line:oob-percentage}§ }, "required": [ "timer", "pos", "vel", "vel_kmh", "steering", "brake", "throttle", "is_oob", "oob_percentage" ] } </code></pre> <p><strong>Dataset Content</strong></p> <p>The TRAVEL dataset is a lively initiative so the content of the dataset is subject to change. Currently, the dataset contains the data collected during the SBST CPS tool competition, and data collected in the context of our recent work on test selection (<a href="https://arxiv.org/abs/2111.04666">SDC-Scissor work</a> and <a href="https://github.com/ChristianBirchler/sdc-scissor">tool</a>) and test prioritization (<a href="https://arxiv.org/abs/2107.09614">automated test cases prioritization work for SDCs</a>).</p> <p>SBST CPS Tool Competition Data</p> <p>The data collected during the SBST CPS tool competition are stored inside <code>data/competition.tar.gz</code>. The file contains the test cases generated by <a href="https://github.com/mahshidhelali/tool-competition-av">Deeper</a>, <a href="https://github.com/ERATOMMSD/frenetic-sbst21">Frenetic</a>, <a href="https://github.com/TayYim/adafrenetic-sbst22">AdaFrenetic</a>, and <a href="https://github.com/alessiogambi/swat-sbst2021">Swat</a>, the <strong>open-source</strong> test generators submitted to the competition and executed against BeamNG.AI with an aggression factor of 0.7 (i.e., conservative driver).</p> <table> <thead> <tr> <th>Name</th> <th>Map Size (m x m)</th> <th>Max Speed (Km/h)</th> <th>Budget (h)</th> <th>OOB Tolerance (%)</th> <th>Test Subject</th> </tr> </thead> <tbody> <tr> <td>DEFAULT</td> <td>200 × 200</td> <td>120</td> <td>5 (real time)</td> <td>0.95</td> <td>BeamNG.AI - 0.7</td> </tr> <tr> <td>SBST</td> <td>200 × 200</td> <td>70</td> <td>2 (real time)</td> <td>0.5</td> <td>BeamNG.AI - 0.7</td> </tr> </tbody> </table> <p>Specifically, the TRAVEL dataset contains 8 repetitions for each of the above configurations for each test generator totaling 64 experiments.</p> <p>SDC Scissor</p> <p>With SDC-Scissor we collected data based on the <a href="https://github.com/ERATOMMSD/frenetic-sbst21">Frenetic</a> test generator. The data is stored inside <code>data/sdc-scissor.tar.gz</code>. The following table summarizes the used parameters.</p> <table> <thead> <tr> <th>Name</th> <th>Map Size (m x m)</th> <th>Max Speed (Km/h)</th> <th>Budget (h)</th> <th>OOB Tolerance (%)</th> <th>Test Subject</th> </tr> </thead> <tbody> <tr> <td>SDC-SCISSOR</td> <td>200 × 200</td> <td>120</td> <td>16 (real time)</td> <td>0.5</td> <td>BeamNG.AI - 1.5</td> </tr> </tbody> </table> <p>The dataset contains 9 experiments with the above configuration. For generating your own data with SDC-Scissor follow the instructions in its <a href="https://github.com/ChristianBirchler/sdc-scissor">repository</a>.</p> <p><strong>Dataset Statistics</strong></p> <p>Here is an overview of the TRAVEL dataset: generated tests, executed tests, and faults found by all the test generators grouped by experiment configuration. Some 25,845 test cases are generated by running 4 test generators 8 times in 2 configurations using the SBST CPS Tool Competition code pipeline (SBST in the table). We ran the test generators for 5 hours, allowing the ego-car a generous speed limit (120 Km/h) and defining a high OOB tolerance (i.e., 0.95), and we also ran the test generators using a smaller generation budget (i.e., 2 hours) and speed limit (i.e., 70 Km/h) while setting the OOB tolerance to a lower value (i.e., 0.85). We also collected some 5, 971 additional tests with SDC-Scissor (SDC-Scissor in the table) by running it 9 times for 16 hours using Frenetic as a test generator and defining a more realistic OOB tolerance (i.e., 0.50).</p> <p><strong>Generating new Data</strong></p> <p>Generating new data, i.e., test cases, can be done using the <a href="https://github.com/se2p/tool-competition-av">SBST CPS Tool Competition pipeline</a> and the driving simulator <a href="https://www.beamng.tech/">BeamNG.tech</a>.</p> <p>Extensive instructions on how to install both software are reported inside the SBST CPS Tool Competition pipeline <a href="https://github.com/se2p/tool-competition-av/blob/main/documentation/README.md">Documentation</a>; therefore, below we only summarize the overall installation process.</p> <p>Installation</p> <ol> <li> <p>Install Python 3.7. (We tested the code using 3.7.9, so we suggest to install that version of Python)</p> </li> <li> <p>Clone the <a href="https://github.com/se2p/tool-competition-av">SBST CPS Tool Competition pipeline</a></p> </li> <li> <p>Request a (free) copy of the driving simulator at <a href="https://register.beamng.tech/">https://register.beamng.tech</a> and, after receiving the registration key file (<code>tech.key</code>) and the download link, download and install the latest distribution of the software (<code>0.24</code>).</p> </li> <li> <p>Create a python virtual environment (<code>python -m venv .venv</code>) inside the root of the SBST CPS Tool Competition pipeline and activate it. Please, be sure to call the virtual environment exactly <code>.venv</code> and place it in the correct position as the scripts to collect new data assume this setup.</p> </li> <li> <p>Install inside the virtual environment all the SBST CPS Tool Competition pipeline requirements (<code>pip install -r requirements.txt</code>).</p> </li> </ol> <p>If you have successfully setup the python environment, activate the virtual environment, and run</p> <pre><code>python.exe competition.py --help </code></pre> <p>and check that the command outputs the <a href="https://github.com/se2p/tool-competition-av/blob/main/documentation/GUIDELINES.md#technical-considerations">usage</a> of the SBST CPS Tool Competition pipeline.</p> <p>Data Collection</p> <p>Collecting new data is as simple as starting the SBST CPS Tool Competition pipeline with suitable parameters for test generation (<code>--time-budget</code>, <code>--module-name</code>, <code>--module-path</code> and <code>--class-name</code>), test executor (<code>--executor</code>), and test definition (<code>--map-size</code>, <code>--speed-limit</code> and <code>--oob-tolerance</code>).</p> <p>To ease the collection of new data, we include the (PowerShell) scripts we used to create the current TRAVEL dataset. Those scripts can be found inside the <code>data-collection</code> folder and are named after the test generator and configuration they implement. For instance, <code>Frenetic-DEFAULT.ps1</code> will execute the Frenetic test generator in the DEFAULT configuration, whereas <code>Deeper-SBST.ps1</code> will execute the Deeper test generation in the SBST configuration. The description of these configurations is reported in the <a href="https://github.com/se2p/msr-travel-dataset#sbst-cps-tool-competition-data">table above</a>).</p> <p>To run those scripts, open a PowerShell, <code>cd</code> to the folder containing the script (<code>data-collection</code>), and invoke the script you want. Assuming you have configured the SBST CPS Tool Competition pipeline as described above, the scripts will start the code pipeline and the BeamNG.tech driving simulator.</p> <p>Once the run is finished, you will find the results under the <code>results</code> folder inside the SBST CPS Tool Competition pipeline project. The results of each experiment are stored in a uniquely named folder that clearly indicates the test generator used to produce them (see <a href="https://github.com/se2p/msr-travel-dataset#dataset-overview">above</a> for the detailed description of the produced results).</p> <p><strong>Data Usage</strong></p> <p>So far, the data contained in the TRAVEL dataset have been used for benchmarking test generators as well as for optimizing regression testing in the context of Self-driving car software.</p> <p>We used the TRAVEL to study the problem of test generation from different perspectives.</p> <p>In the last edition of the <a href="https://doi.org/10.1109/SBST52555.2021.00011">SBST Tool Challenge</a>, we answer the question <em>"How efficient and effective are the existing test generators for SDCs?"</em>. We assess test generation efficiency by counting how many tests are produced by each test generator within a given time budget. To assess test generation effectiveness, we count how many of the generated tests are valid and invalid.</p> <p>In a <a href="https://doi.org/10.1145/3460319.3464811">recent study</a>, we propose (along the DeepHyperion test generator) a novel approach to assess SDCs test suites adequacy by measuring the coverage of feature maps that they achieved. To do so, we extract structural and behavioral test case features from the test case descriptions and simulation data by looking at the geometrical properties describing the roads and the physical quantities describing the ego-car behavior (e.g., position, speed).</p> <p>We also answer the question <em>"How effective are the resulting tests in finding faults in SDCs?"</em>, by counting how many failures are triggered by the generated tests and how different those failures are (e.g., Left or Right OOBs). Specifically, we propose an approach to measure OOBs similarity by computing the edit distance of the road segments relevant to the corresponding failure.</p> <p><a href="https://github.com/ERATOMMSD/roadsearch">Castellano et al.</a> demonstrated that a dataset such as TRAVEL can be used to answer other interesting questions, such as <em>"Which road representation is the most suitable for generating SDCs test cases?"</em>.</p> <p>In the context of regression testing the TRAVEL dataset is also used to develop tools to select test cases that are likely to fail before executing them simulation (e.g., <a href="https://github.com/ChristianBirchler/sdc-scissor">SDC-Scissor</a>) and also to prioritize them in a single-objective and multi-objective approach by <a href="https://arxiv.org/abs/2107.09614">SDC-Prioritizer</a>.</p> <p>With SDC-Scissor we can select test cases that are more likely to fail. This allows us to answer the following research questions like "To what extent is it possible to identify safe and unsafe test scenarios for SDCs before executing them?" or "Does SDC-Scissor improve the cost-effectiveness of simulation-based testing of SDCs?" <a href="https://arxiv.org/abs/2111.04666">Khatiri et al.</a>. SDC-Scissor achieves a F1-score (between 47% and 90%) compared to a baseline in identifying failing tests and reduced the time spent running uninformative tests by 107% and 170%. <a href="https://christianbirchler.github.io/sdc_scissor_demonstration_paper.pdf">(Birchler et al.)</a></p> <p>Regarding test prioritization, the TRAVEL dataset allows us to answer the following questions: "To what extent is it possible to apply test prioritization strategies to prioritize the execution of safe and unsafe test scenarios for SDCs?", “What is the cost-effectiveness of SDC-Prioritizer compared to baseline approaches?” and “What is the overhead introduced by it?”. To address the former question, we compared the test prioritization approaches in terms of fault detection rate (i.e., how fast faults are detected during the test execution process) using Cost cognizant Average Percentage of Fault Detection (APFD𝑐) (<a href="https://doi.org/10.1145/2771783.2771788">Epitropakis et al.</a>, <a href="https://doi.org/10.1109/ICSM.1999.792604">Rothermel et al.</a>) and concluded that SDC-Prioritizer (with 82.5% APFD𝑐) significantly outperformed the baseline random and greedy test prioritization approaches <a href="https://arxiv.org/abs/2107.09614">Birchler et al.</a>. For the latter research question, we compared the time required by SDC-Prioritizer to sort tests against the time needed to run all of the tests and showed that, on average, SDC-Prioritizer needs less than 13 minutes to perform the test prioritization while running the SDC test suites takes between 16 to 106 hours. As explained in Section 3.2.3, SDC-Prioritizer uses the road features shared with TRAVEL to guide the search process towards generating test orders with the high diversity in the road shapes. The selection of best features for test prioritization was performed with the Principal component analysis (PCA) <a href="https://arxiv.org/abs/2107.09614">Birchler et al.</a>.</p>
Supplementary materials of the paper entitled: "Metamorphic Testing Meets Regression Testing: A Case Study of Scientific Software Maintenance"
<p>Supplementary materials of the paper entitled:</p> <p>“Metamorphic Testing Meets Regression Testing: A Case Study of Scientific Software Maintenance”</p> <p>file01 - source code for detecting and comparing output relations of a given metamorphic relation<br> file02 - interview transcripts and qualitative coding results<br> file03 - system script for compiling SWMM, creating sSWMM simulation outputs, and logging execution time<br> file04 - source code for SWMM 5.1.014<br> file05 - source code for SWMM 5.1.015<br> file06 - a suite of 40 regression tests (40 .inp files)<br> file07 - result table for 760 cases (19 MRs × 40 .inp files)<br> file08 - 491 automatically generated follow-up .inp files via reuse<br> file09 - result table for mutation analysis<br> file10 - output files for mutation analysis with metamorphic testing:<br> a. Original source output<br> b. Original follow-up output<br> c. Mutant source output<br> d. Mutant follow-up output</p>
Cloud computing is one of the most popular and sophisticated technologies adopted by organizations worldwide. Some world-leading organizations enhance their efficiency and effectiveness by using cloud computing technology. Working from home (WFH) has been a popular trend among organizations during the coronavirus (COVID-19) pandemic. The COVID-19 saw a breakthrough in work cultures and environments where working from home was a remarkable success in remote working environments, despite being a rare phenomenon in Sri Lanka. Yet, it is argued that the deployment of work from home has not been effective among Sri Lankan business organizations due to a lack of IT infrastructure, facilities, and knowledge. The purpose of the study is to investigate the impact of cloud computing, embracing the service models (Infrastructure as a Service, Platform as a Service, and Software as a Service) as theoretical lenses and testing the COVID-19 as the moderator. The study has been conducted based on a deductive approach and adopted a stratified random sampling method. The sample consisted of 384 IT employees among those who had experienced working from home. The study utilized multiple regression and found that cloud computing service models significantly impact work from home with the moderating effect of COVID-19.
<p>Cloud computing is one of the most popular and sophisticated technologies adopted by organizations worldwide. Some world-leading organizations enhance their efficiency and effectiveness by using cloud computing technology. Working from home (WFH) has been a popular trend among organizations during the coronavirus (COVID-19) pandemic. The COVID-19 saw a breakthrough in work cultures and environments where working from home was a remarkable success in remote working environments, despite being a rare phenomenon in Sri Lanka. Yet, it is argued that the deployment of work from home has not been effective among Sri Lankan business organizations due to a lack of IT infrastructure, facilities, and knowledge. The purpose of the study is to investigate the impact of cloud computing, embracing the service models (Infrastructure as a Service, Platform as a Service, and Software as a Service) as theoretical lenses and testing the COVID-19 as the moderator. The study has been conducted based on a deductive approach and adopted a stratified random sampling method. The sample consisted of 384 IT employees among those who had experienced working from home. The study utilized multiple regression and found that cloud computing service models significantly impact work from home with the moderating effect of COVID-19.</p>
A thematic synthesis on the adoption of regression testing techniques in Android projects
<p>In software testing, employing regression techniques is a viable strategy to deal with the complexity and the constant evolution of applications since its primary goal is to ensure that changes made between versions do not change the system’s behavior. Although the literature has dedicated efforts to developing new regression testing techniques suitable for the Android mobile platform, studies are limited concerning demonstrating which techniques software developers employ in practice. This study aims to report on a thematic synthesis of adopting regression testing techniques in Android projects. The research encompassed four stages: (i) conducting a structured literature review on regression testing techniques for the Android platform, (ii) carrying out an expert survey, (iii) conducting interviews with industry professionals, and (iv) building a thematic synthesis. The thematic synthesis presented a model from analyzing the results obtained in this multimethod study on regression testing techniques. With such a stud, we could present empirical evidence on how professionals perform regression testing in Android projects, identify the commonly used regression testing techniques, and leverage the requirements for automating Android applications through regression testing.</p>
[Artifacts] Colosseum: Regression Test Prioritization by Delta Displacement in Test Coverage
<p>Replication package for [Colosseum: Regression Test Prioritization by Delta Displacement in Test Coverage].</p>
Performance regression testing initiatives: A systematic mapping - Replication Package
<p>This repository contains the supporting material for the manuscript titled "Performance regression testing initiatives: A systematic mapping", currently under revision to the Journal of Information and Software Technology. </p> <p>Repository Content:</p> <ul> <li> <p><strong>Protocol Performance Regression Testing.xlsx</strong>: it includes the *protocol* specification, i.e., the data extraction process used for the selection of the studies and gathering all relevant information. The outcome of the systematic mapping consists of 68 *selected studies* and information supporting the answers to four research questions: *RQ1*, *RQ2*, *RQ3*, and *RQ4*;</p> </li> <li> <p><strong>scripts.zip</strong>: it includes all the scripts used in the forward snowballing. Two additional files support the replication of the scripts: (1) README.md explains how to run the scripts; (2) requirements.txt file reports the installation prerequisites and their dependencies;</p> </li> <li> <p><strong>SM-PRT-overview.xlsx</strong>: this spreadsheet reports the information about tools and replication packages, benchmark suites and subject systems adopted per study;</p> </li> <li> <p><strong>README-SM-PRT-overview.md.docx</strong>: this file describes the columns of the spreadsheet above to facilitate the usage of the collected information.</p> </li> </ul>
Replication Package for "State of Practical Applicability of Regression Testing Research: A Live Systematic Literature Review"
<p>This repository provides a static version of the dataset used for the study "State of Practical Applicability of Regression Testing Research: A Live Systematic Literature Review".</p> <p>The data is available in CSV format (for easy parsing) and HTML format (for readability). HTML files require the included <em>sheet.css</em> file (provided by Google Sheets).</p> <p>A non-static version of this data is available via <a href="https://bit.ly/3gCKh7V">Google Sheets</a>.</p> <p>Furthermore, this data was used to create a <a href="https://renangreca.github.io/literature-repository/">live repository</a> of papers on the topic of software regression testing. It is open for contributions.</p>
Testing Use of Condoms on Regression of Cervical Intraepithelial Neoplasia
ClinicalTrials.gov study NCT02907333. IPD Sharing: NO. Countries: 1. Publications: 1.
Code from: Testing for normality in regression models: mistakes abound (but may not matter)
Open the record for dataset details and reuse information.
Supplementary materials of the paper entitled: "Metamorphic Testing Meets Regression Testing: A Case Study of Scientific Software Maintenance"
<p>Supplementary materials of the paper entitled: “Metamorphic Testing Meets Regression Testing: A Case Study of Scientific Software Maintenance”</p> <p>file01 - source code for detecting and comparing output relations of a given metamorphic relation<br> file02 - system script for compiling SWMM, creating sSWMM simulation outputs, and logging execution time<br> file03 - source code for SWMM 5.1.014<br> file04 - source code for SWMM 5.1.015<br> file05 - a suite of 40 regression tests (40 .inp files)<br> file06 - result table for 760 cases (19 MRs × 40 .inp files)<br> file07 - 491 automatically generated follow-up .inp files via reuse<br> file08 - result table for mutation analysis<br> file09 - source code for three open-source software systems<br> File10 - code change in three open-source software systems</p>
A Study Testing if Medicine Can Make Pigment Epithelium Detachments Regress and Stabilize the Vision in Eyes
ClinicalTrials.gov study NCT01746875. IPD Sharing: Not stated. Countries: 1. Publications: 0.
Normative Values and Forming Regression Equations for One Minute Sit to Stand Test in Healthy Normal Indian Population
ClinicalTrials.gov study NCT02218905. IPD Sharing: Not stated. Countries: 1. Publications: 0.
Dataset for Regression Testing of Self-Driving Cars
<p>All datasets are executed with:</p> <p>Vehicle: ETK-800.</p> <p>Maximum Speed = 120 km/h.</p> <p>Risk Factor = 1.5.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.