Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
6
datasets available to search
ShareScore release 0.9.0
Dataset results
6 results for “Self-driving cars”
TRAVEL: A Dataset with Toolchains for Test Generation and Regression Testing of Self-driving Cars Software
<p><strong>Introduction</strong></p> <p>This repository hosts the Testing Roads for Autonomous VEhicLes (TRAVEL) dataset. TRAVEL is an extensive collection of virtual roads that have been used for testing lane assist/keeping systems (i.e., driving agents) and data from their execution in state of the art, physically accurate driving simulator, called <a href="https://www.beamng.tech/">BeamNG.tech</a>. Virtual roads consist of sequences of <strong>road points</strong> interpolated using Cubic splines.</p> <p>Along with the data, this repository contains instructions on how to install the tooling necessary to generate new data (i.e., test cases) and analyze them in the context of test regression. We focus on test selection and test prioritization, given their importance for developing high-quality software following the DevOps paradigms.</p> <p>This dataset builds on top of our previous work in this area, including work on</p> <ul> <li>test generation (e.g., <a href="https://doi.org/10.1145/3293882.3330566">AsFault</a>, <a href="https://doi.org/10.1145/3368089.3409730">DeepJanus</a>, and <a href="https://doi.org/10.1145/3460319.3464811">DeepHyperion</a>) and the SBST CPS tool competition (<a href="https://doi.org/10.1109/SBST52555.2021.00011">SBST2021</a>),</li> <li>test selection: <a href="https://arxiv.org/abs/2111.04666">SDC-Scissor</a> and related <a href="https://github.com/ChristianBirchler/sdc-scissor">tool</a></li> <li>test prioritization: <a href="https://arxiv.org/abs/2107.09614">automated test cases prioritization work for SDCs</a>.</li> </ul> <p><strong>Dataset Overview</strong></p> <p>The TRAVEL dataset is available under the <code>data</code> folder and is organized as a set of <em>experiments</em> folders. Each of these folders is generated by running the <code>test-generator</code> (see below) and contains the configuration used for generating the data (<code>experiment_description.csv</code>), various statistics on generated tests (<code>generation_stats.csv</code>) and found faults (<code>oob_stats.csv</code>). Additionally, the folders contain the raw test cases generated and executed during each experiment (<code>test.<TEST_ID>.json</code>).</p> <p>The following sections describe what each of those files contains.</p> <p>Experiment Description</p> <p>The <code>experiment_description.csv</code> contains the settings used to generate the data, including:</p> <ul> <li><strong>Time budget</strong>. The overall generation budget in hours. This budget includes both the time to generate and execute the tests as driving simulations.</li> <li><strong>The size of the map</strong>. The size of the squared map defines the boundaries inside which the virtual roads develop in meters.</li> <li><strong>The test subject</strong>. The driving agent that implements the lane-keeping system under test. The TRAVEL dataset contains data generated testing the BeamNG.AI and the end-to-end Dave2 systems.</li> <li><strong>The test generator</strong>. The algorithm that generated the test cases. The TRAVEL dataset contains data obtained using various algorithms, ranging from naive and advanced random generators to complex evolutionary algorithms, for generating tests.</li> <li><strong>The speed limit</strong>. The maximum speed at which the driving agent under test can travel.</li> <li><strong>Out of Bound (OOB) tolerance</strong>. The test cases' oracle that defines the tolerable amount of the ego-car that can lie outside the lane boundaries. This parameter ranges between 0.0 and 1.0. In the former case, a test failure triggers as soon as any part of the ego-vehicle goes out of the lane boundary; in the latter case, a test failure triggers only if the entire body of the ego-car falls outside the lane.</li> </ul> <p>Experiment Statistics</p> <p>The <code>generation_stats.csv</code> contains statistics about the test generation, including:</p> <ul> <li><strong>Total number of generated tests</strong>. The number of tests generated during an experiment. This number is broken down into the number of <strong>valid tests</strong> and <strong>invalid tests</strong>. Valid tests contain virtual roads that do not self-intersect and contain turns that are not too sharp.</li> <li><strong>Test outcome</strong>. The test outcome contains the number of <strong>passed tests</strong>, <strong>failed tests</strong>, and <strong>test in error</strong>. Passed and failed tests are defined by the OOB Tolerance and an additional (implicit) oracle that checks whether the ego-car is moving or standing. Tests that did not pass because of other errors (e.g., the simulator crashed) are reported in a separated category.</li> </ul> <p>The TRAVEL dataset also contains statistics about the failed tests, including the overall number of failed tests (<strong>total oob</strong>) and its breakdown into OOB that happened while driving <strong>left</strong> or <strong>right</strong>. Further statistics about the diversity (i.e., <strong>sparseness</strong>) of the failures are also reported.</p> <p>Test Cases and Executions</p> <p>Each <code>test.<TEST_ID>.json</code> contains information about a test case and, if the test case is valid, the data observed during its execution as driving simulation.</p> <p>The data about the test case definition include:</p> <ul> <li><strong>The road points</strong>. The list of points in a 2D space that identifies the center of the virtual road, and their interpolation using cubic splines (<strong>interpolated_points</strong>)</li> <li><strong>The test ID</strong>. The unique identifier of the test in the experiment.</li> <li><strong>Validity flag and explanation</strong>. A flag that indicates whether the test is valid or not, and a brief message describing why the test is not considered valid (e.g., <code>the road contains sharp turns</code> or <code>the road self intersects</code>)</li> </ul> <p>The test data are organized according to the following JSON Schema and can be interpreted as <code>RoadTest</code> objects provided by the <a href="https://github.com/se2p/tool-competition-av/blob/main/code_pipeline/tests_generation.py">tests_generation.py</a> module.</p> <pre><code>{ "type": "object", "properties": { "id": { "type": "integer" }, "is_valid": { "type": "boolean" }, "validation_message": { "type": "string" }, "road_points": { §\label{line:road-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "interpolated_points": { §\label{line:interpolated-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "test_outcome": { "type": "string" }, §\label{line:test-outcome}§ "description": { "type": "string" }, "execution_data": { "type": "array", "items": { "$ref" : "schemas/simulationdata" } } }, "required": [ "id", "is_valid", "validation_message", "road_points", "interpolated_points" ] } </code></pre> <p>Finally, the <strong>execution data</strong> contain a list of timestamped state information recorded by the driving simulation. State information is collected at constant frequency and includes absolute position, rotation, and velocity of the ego-car, its speed in Km/h, and control inputs from the driving agent (steering, throttle, and braking). Additionally, execution data contain OOB-related data, such as the lateral distance between the car and the lane center and the OOB percentage (i.e., how much the car is outside the lane).</p> <p>The simulation data adhere to the following (simplified) JSON Schema and can be interpreted as Python objects using the <a href="https://github.com/se2p/tool-competition-av/blob/main/self_driving/simulation_data.py">simulation_data.py</a> module.</p> <pre><code>{ "$id": "schemas/simulationdata", "type": "object", "properties": { "timer" : { "type": "number" }, "pos" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel_kmh" : { "type": "number" }, "steering" : { "type": "number" }, "brake" : { "type": "number" }, "throttle" : { "type": "number" }, "is_oob" : { "type": "number" }, "oob_percentage" : { "type": "number" } §\label{line:oob-percentage}§ }, "required": [ "timer", "pos", "vel", "vel_kmh", "steering", "brake", "throttle", "is_oob", "oob_percentage" ] } </code></pre> <p><strong>Dataset Content</strong></p> <p>The TRAVEL dataset is a lively initiative so the content of the dataset is subject to change. Currently, the dataset contains the data collected during the SBST CPS tool competition, and data collected in the context of our recent work on test selection (<a href="https://arxiv.org/abs/2111.04666">SDC-Scissor work</a> and <a href="https://github.com/ChristianBirchler/sdc-scissor">tool</a>) and test prioritization (<a href="https://arxiv.org/abs/2107.09614">automated test cases prioritization work for SDCs</a>).</p> <p>SBST CPS Tool Competition Data</p> <p>The data collected during the SBST CPS tool competition are stored inside <code>data/competition.tar.gz</code>. The file contains the test cases generated by <a href="https://github.com/mahshidhelali/tool-competition-av">Deeper</a>, <a href="https://github.com/ERATOMMSD/frenetic-sbst21">Frenetic</a>, <a href="https://github.com/TayYim/adafrenetic-sbst22">AdaFrenetic</a>, and <a href="https://github.com/alessiogambi/swat-sbst2021">Swat</a>, the <strong>open-source</strong> test generators submitted to the competition and executed against BeamNG.AI with an aggression factor of 0.7 (i.e., conservative driver).</p> <table> <thead> <tr> <th>Name</th> <th>Map Size (m x m)</th> <th>Max Speed (Km/h)</th> <th>Budget (h)</th> <th>OOB Tolerance (%)</th> <th>Test Subject</th> </tr> </thead> <tbody> <tr> <td>DEFAULT</td> <td>200 × 200</td> <td>120</td> <td>5 (real time)</td> <td>0.95</td> <td>BeamNG.AI - 0.7</td> </tr> <tr> <td>SBST</td> <td>200 × 200</td> <td>70</td> <td>2 (real time)</td> <td>0.5</td> <td>BeamNG.AI - 0.7</td> </tr> </tbody> </table> <p>Specifically, the TRAVEL dataset contains 8 repetitions for each of the above configurations for each test generator totaling 64 experiments.</p> <p>SDC Scissor</p> <p>With SDC-Scissor we collected data based on the <a href="https://github.com/ERATOMMSD/frenetic-sbst21">Frenetic</a> test generator. The data is stored inside <code>data/sdc-scissor.tar.gz</code>. The following table summarizes the used parameters.</p> <table> <thead> <tr> <th>Name</th> <th>Map Size (m x m)</th> <th>Max Speed (Km/h)</th> <th>Budget (h)</th> <th>OOB Tolerance (%)</th> <th>Test Subject</th> </tr> </thead> <tbody> <tr> <td>SDC-SCISSOR</td> <td>200 × 200</td> <td>120</td> <td>16 (real time)</td> <td>0.5</td> <td>BeamNG.AI - 1.5</td> </tr> </tbody> </table> <p>The dataset contains 9 experiments with the above configuration. For generating your own data with SDC-Scissor follow the instructions in its <a href="https://github.com/ChristianBirchler/sdc-scissor">repository</a>.</p> <p><strong>Dataset Statistics</strong></p> <p>Here is an overview of the TRAVEL dataset: generated tests, executed tests, and faults found by all the test generators grouped by experiment configuration. Some 25,845 test cases are generated by running 4 test generators 8 times in 2 configurations using the SBST CPS Tool Competition code pipeline (SBST in the table). We ran the test generators for 5 hours, allowing the ego-car a generous speed limit (120 Km/h) and defining a high OOB tolerance (i.e., 0.95), and we also ran the test generators using a smaller generation budget (i.e., 2 hours) and speed limit (i.e., 70 Km/h) while setting the OOB tolerance to a lower value (i.e., 0.85). We also collected some 5, 971 additional tests with SDC-Scissor (SDC-Scissor in the table) by running it 9 times for 16 hours using Frenetic as a test generator and defining a more realistic OOB tolerance (i.e., 0.50).</p> <p><strong>Generating new Data</strong></p> <p>Generating new data, i.e., test cases, can be done using the <a href="https://github.com/se2p/tool-competition-av">SBST CPS Tool Competition pipeline</a> and the driving simulator <a href="https://www.beamng.tech/">BeamNG.tech</a>.</p> <p>Extensive instructions on how to install both software are reported inside the SBST CPS Tool Competition pipeline <a href="https://github.com/se2p/tool-competition-av/blob/main/documentation/README.md">Documentation</a>; therefore, below we only summarize the overall installation process.</p> <p>Installation</p> <ol> <li> <p>Install Python 3.7. (We tested the code using 3.7.9, so we suggest to install that version of Python)</p> </li> <li> <p>Clone the <a href="https://github.com/se2p/tool-competition-av">SBST CPS Tool Competition pipeline</a></p> </li> <li> <p>Request a (free) copy of the driving simulator at <a href="https://register.beamng.tech/">https://register.beamng.tech</a> and, after receiving the registration key file (<code>tech.key</code>) and the download link, download and install the latest distribution of the software (<code>0.24</code>).</p> </li> <li> <p>Create a python virtual environment (<code>python -m venv .venv</code>) inside the root of the SBST CPS Tool Competition pipeline and activate it. Please, be sure to call the virtual environment exactly <code>.venv</code> and place it in the correct position as the scripts to collect new data assume this setup.</p> </li> <li> <p>Install inside the virtual environment all the SBST CPS Tool Competition pipeline requirements (<code>pip install -r requirements.txt</code>).</p> </li> </ol> <p>If you have successfully setup the python environment, activate the virtual environment, and run</p> <pre><code>python.exe competition.py --help </code></pre> <p>and check that the command outputs the <a href="https://github.com/se2p/tool-competition-av/blob/main/documentation/GUIDELINES.md#technical-considerations">usage</a> of the SBST CPS Tool Competition pipeline.</p> <p>Data Collection</p> <p>Collecting new data is as simple as starting the SBST CPS Tool Competition pipeline with suitable parameters for test generation (<code>--time-budget</code>, <code>--module-name</code>, <code>--module-path</code> and <code>--class-name</code>), test executor (<code>--executor</code>), and test definition (<code>--map-size</code>, <code>--speed-limit</code> and <code>--oob-tolerance</code>).</p> <p>To ease the collection of new data, we include the (PowerShell) scripts we used to create the current TRAVEL dataset. Those scripts can be found inside the <code>data-collection</code> folder and are named after the test generator and configuration they implement. For instance, <code>Frenetic-DEFAULT.ps1</code> will execute the Frenetic test generator in the DEFAULT configuration, whereas <code>Deeper-SBST.ps1</code> will execute the Deeper test generation in the SBST configuration. The description of these configurations is reported in the <a href="https://github.com/se2p/msr-travel-dataset#sbst-cps-tool-competition-data">table above</a>).</p> <p>To run those scripts, open a PowerShell, <code>cd</code> to the folder containing the script (<code>data-collection</code>), and invoke the script you want. Assuming you have configured the SBST CPS Tool Competition pipeline as described above, the scripts will start the code pipeline and the BeamNG.tech driving simulator.</p> <p>Once the run is finished, you will find the results under the <code>results</code> folder inside the SBST CPS Tool Competition pipeline project. The results of each experiment are stored in a uniquely named folder that clearly indicates the test generator used to produce them (see <a href="https://github.com/se2p/msr-travel-dataset#dataset-overview">above</a> for the detailed description of the produced results).</p> <p><strong>Data Usage</strong></p> <p>So far, the data contained in the TRAVEL dataset have been used for benchmarking test generators as well as for optimizing regression testing in the context of Self-driving car software.</p> <p>We used the TRAVEL to study the problem of test generation from different perspectives.</p> <p>In the last edition of the <a href="https://doi.org/10.1109/SBST52555.2021.00011">SBST Tool Challenge</a>, we answer the question <em>"How efficient and effective are the existing test generators for SDCs?"</em>. We assess test generation efficiency by counting how many tests are produced by each test generator within a given time budget. To assess test generation effectiveness, we count how many of the generated tests are valid and invalid.</p> <p>In a <a href="https://doi.org/10.1145/3460319.3464811">recent study</a>, we propose (along the DeepHyperion test generator) a novel approach to assess SDCs test suites adequacy by measuring the coverage of feature maps that they achieved. To do so, we extract structural and behavioral test case features from the test case descriptions and simulation data by looking at the geometrical properties describing the roads and the physical quantities describing the ego-car behavior (e.g., position, speed).</p> <p>We also answer the question <em>"How effective are the resulting tests in finding faults in SDCs?"</em>, by counting how many failures are triggered by the generated tests and how different those failures are (e.g., Left or Right OOBs). Specifically, we propose an approach to measure OOBs similarity by computing the edit distance of the road segments relevant to the corresponding failure.</p> <p><a href="https://github.com/ERATOMMSD/roadsearch">Castellano et al.</a> demonstrated that a dataset such as TRAVEL can be used to answer other interesting questions, such as <em>"Which road representation is the most suitable for generating SDCs test cases?"</em>.</p> <p>In the context of regression testing the TRAVEL dataset is also used to develop tools to select test cases that are likely to fail before executing them simulation (e.g., <a href="https://github.com/ChristianBirchler/sdc-scissor">SDC-Scissor</a>) and also to prioritize them in a single-objective and multi-objective approach by <a href="https://arxiv.org/abs/2107.09614">SDC-Prioritizer</a>.</p> <p>With SDC-Scissor we can select test cases that are more likely to fail. This allows us to answer the following research questions like "To what extent is it possible to identify safe and unsafe test scenarios for SDCs before executing them?" or "Does SDC-Scissor improve the cost-effectiveness of simulation-based testing of SDCs?" <a href="https://arxiv.org/abs/2111.04666">Khatiri et al.</a>. SDC-Scissor achieves a F1-score (between 47% and 90%) compared to a baseline in identifying failing tests and reduced the time spent running uninformative tests by 107% and 170%. <a href="https://christianbirchler.github.io/sdc_scissor_demonstration_paper.pdf">(Birchler et al.)</a></p> <p>Regarding test prioritization, the TRAVEL dataset allows us to answer the following questions: "To what extent is it possible to apply test prioritization strategies to prioritize the execution of safe and unsafe test scenarios for SDCs?", “What is the cost-effectiveness of SDC-Prioritizer compared to baseline approaches?” and “What is the overhead introduced by it?”. To address the former question, we compared the test prioritization approaches in terms of fault detection rate (i.e., how fast faults are detected during the test execution process) using Cost cognizant Average Percentage of Fault Detection (APFD𝑐) (<a href="https://doi.org/10.1145/2771783.2771788">Epitropakis et al.</a>, <a href="https://doi.org/10.1109/ICSM.1999.792604">Rothermel et al.</a>) and concluded that SDC-Prioritizer (with 82.5% APFD𝑐) significantly outperformed the baseline random and greedy test prioritization approaches <a href="https://arxiv.org/abs/2107.09614">Birchler et al.</a>. For the latter research question, we compared the time required by SDC-Prioritizer to sort tests against the time needed to run all of the tests and showed that, on average, SDC-Prioritizer needs less than 13 minutes to perform the test prioritization while running the SDC test suites takes between 16 to 106 hours. As explained in Section 3.2.3, SDC-Prioritizer uses the road features shared with TRAVEL to guide the search process towards generating test orders with the high diversity in the road shapes. The selection of best features for test prioritization was performed with the Principal component analysis (PCA) <a href="https://arxiv.org/abs/2107.09614">Birchler et al.</a>.</p>
Dataset for Cost-effective Simulation-based Test Selection in Self-driving Cars Software with SDC-Scissor
<p><strong>SDC-Scissor tool for Cost-effective Simulation-based Test Selection in Self-driving Cars Software</strong></p> <p>This dataset provides test cases for self-driving cars with the BeamNG simulator. Check out the repository and demo video to get started.</p> <p><strong>GitHub:</strong> <a href="https://github.com/ChristianBirchler/sdc-scissor">github.com/ChristianBirchler/sdc-scissor</a></p> <p>This project extends the tool competition platform from the <a href="https://github.com/se2p/tool-competition-av">Cyber-Phisical Systems Testing Competition</a> which was part of the <a href="https://sbst21.github.io/">SBST Workshop in 2021</a>.</p> <p><strong>Usage</strong></p> <p><strong>Demo</strong></p> <p> <a href="https://youtu.be/Cn8p648KnfQ">YouTube Link</a></p> <p><strong>Installation</strong></p> <p>The tool can either be run with <a href="https://docs.docker.com/get-docker/">Docker</a> or locally using <a href="https://python-poetry.org/docs/">Poetry</a>.</p> <p>When running the simulations a working installation of <a href="https://beamng.gmbh/research/">BeamNG.research</a> is required. Additionally, this simulation cannot be run in a Docker container but must run locally.</p> <p>To install the application use one of the following approaches:</p> <ul> <li>Docker: <code>docker build --tag sdc-scissor .</code></li> <li>Poetry: <code>poetry install</code></li> </ul> <p><strong>Using the Tool</strong></p> <p>The tool can be used with the following two commands:</p> <ul> <li>Docker: <code>docker run --volume "$(pwd)/results:/out" --rm sdc-scissor [COMMAND] [OPTIONS]</code> (this will write all files written to <code>/out</code> to the local folder <code>results</code>)</li> <li>Poetry: <code>poetry run python sdc-scissor.py [COMMAND] [OPTIONS]</code></li> </ul> <p>There are multiple commands to use. For simplifying the documentation only the command and their options are described.</p> <ul> <li>Generation of tests: <ul> <li><code>generate-tests --out-path /path/to/store/tests</code></li> </ul> </li> <li>Automated labeling of Tests: <ul> <li><code>label-tests --road-scenarios /path/to/tests --result-folder /path/to/store/labeled/tests</code></li> <li><em>Note:</em> This only works locally with BeamNG.research installed</li> </ul> </li> <li>Model evaluation: <ul> <li><code>evaluate-models --dataset /path/to/train/set --save</code></li> </ul> </li> <li>Split train and test data: <ul> <li><code>split-train-test-data --scenarios /path/to/scenarios --train-dir /path/for/train/data --test-dir /path/for/test/data --train-ratio 0.8</code></li> </ul> </li> <li>Test outcome prediction: <ul> <li><code>predict-tests --scenarios /path/to/scenarios --classifier /path/to/model.joblib</code></li> </ul> </li> <li>Evaluation based on random strategy: <ul> <li><code>evaluate --scenarios /path/to/test/scenarios --classifier /path/to/model.joblib</code></li> </ul> </li> </ul> <p>The possible parameters are always documented with <code>--help</code>.</p> <p><strong>Linting</strong></p> <p>The tool is verified the linters <a href="https://flake8.pycqa.org/en/latest/">flake8</a> and <a href="https://pylint.org/">pylint</a>. These are automatically enabled in <a href="https://code.visualstudio.com/">Visual Studio Code</a> and can be run manually with the following commands:</p> <pre>poetry run flake8 . poetry run pylint **/*.py</pre> <p><strong>License</strong></p> <p>The software we developed is distributed under GNU GPL license. See the <a href="https://github.com/ChristianBirchler/sdc-scissor/blob/main/LICENSE.md">LICENSE.md</a> file.</p> <p><strong>Contacts</strong></p> <p>Christian Birchler - Zurich University of Applied Science (ZHAW), Switzerland - <a href="mailto:birc@zhaw.ch">birc@zhaw.ch</a></p> <p>Nicolas Ganz - Zurich University of Applied Science (ZHAW), Switzerland - <a href="mailto:gann@zhaw.ch">gann@zhaw.ch</a></p> <p>Sajad Khatiri - Zurich University of Applied Science (ZHAW), Switzerland - <a href="mailto:mazr@zhaw.ch">mazr@zhaw.ch</a></p> <p>Dr. Alessio Gambi - Passau University, Germany - <a href="mailto:alessio.gambi@uni-passau.de">alessio.gambi@uni-passau.de</a></p> <p>Dr. Sebastiano Panichella - Zurich University of Applied Science (ZHAW), Switzerland - <a href="mailto:panc@zhaw.ch">panc@zhaw.ch</a></p> <p><strong>References</strong></p> <ul> <li>Christian Birchler, Nicolas Ganz, Sajad Khatiri, Alessio Gambi, and Sebastiano Panichella. 2022. Cost-effective Simulation-based Test Selection in Self-driving Cars Software with SDC-Scissor. In 2022 IEEE 29th International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE.</li> </ul> <p><strong>If you use this tool in your research, please cite the following papers:</strong></p> <pre><code>@INPROCEEDINGS{Birchler2022, author={Birchler, Christian and Ganz, Nicolas and Khatiri, Sajad and Gambi, Alessio, and Panichella, Sebastiano}, booktitle={2022 IEEE 29th International Conference on Software Analysis, Evolution and Reengineering (SANER), title={Cost-effective Simulationbased Test Selection in Self-driving Cars Software with SDC-Scissor}, year={2022}, }</code></pre>
Dataset Package for the paper "Single and Multi-objective Test Cases Prioritization for Self-driving Cars in Virtual Environments"
<p><br> #Dataset Package for the paper "Automated Test Cases Prioritization for Self-driving Cars in Virtual Environments"</p> <p><br> Description of the content:</p> <p><br> 1) "1_datasets" folder contains all the datasets concerning the test scenarios considered in the study. <br> <br> - AsFault supports two SDC software as test subjects while generating test cases, which we use to generate our test suites.<br> - BeamNG.AI. <br> - DeepDriving<br> - STRUCTURE: We report test subjects considered and organized with sub-folders for each of the supported two SDC software (i.e., Driving AIs):<br> - fullroad<br> - BeamNG_AI<br> - Driver_AI<br> 2) "2_SDC-source-code" contains all code and scripts composing the MATLAB/R prototypical implementation of SDC-pririotizer: </p> <p> - STRUCTURE:<br> - runSearch.m<br> - runGreedy.m<br> - plotResults.m<br> - permutationMutation.m<br> - mainGreedy.m<br> - permutationCrossover.m<br> - main.m<br> - initialPopulation.m<br> - fitness.m<br> - featureSelection.m<br> - faultDetectionRatePlots.m<br> - faultDetection.m<br> - (folder) r-script<br> - (folder) python<br> <br> 3) "3_study-data" folder contains all the main experimental data concerning the study conducted, RQs and all generated figures and main row-data reported in section results<br> <br> - results.csv<br> - greedy_results.csv<br> - (folder) tables<br> - (folder) figures<br> - (folder) DriverAI<br> - (folder) BeamNG_RF_1_5<br> - (folder) BeamNG_RF_1 </p>
Replication Package of the paper "Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software"
<p># Replication Package of the paper "Machine Learning-based Test Selection for Simulation-based Testing of Self-driving Cars Software"<br> ## SDC-Scissor (Self-Driving Car coSt-effeCtIve teSt SelectOR)</p> <p><br> ### Structure<br> - datasets: the datasets we used in our study for training and evaluating ML Models<br> - RQs: the extended results and analysis for each of the RQs mentioned in the paper </p> <p>More information on each topic can be found in the README file in the subfolders.<br> </p>
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
<h1>GitHub: <a href="https://github.com/christianbirchler-org/sensodat" target="_blank" rel="noopener">https://github.com/christianbirchler-org/sensodat</a></h1> <p>Developing tools in the context of autonomous systems, such as self-driving cars (SDCs), is time-consuming and costly since researchers and practitioners rely on expensive computing hardware and simulation software. We propose SensoDat, a dataset of 32,580 executed simulation-based SDC test cases generated with state-of-the-art test generators for SDCs. The dataset consists of trajectory logs and a variety of sensor data from the SDCs (e.g., rpm, wheel speed, brake thermals, transmission, etc.) represented as a time series. In total, SensoDat provides data from 81 different simulated sensors. Future research in the domain of SDCs does not necessarily depend on executing expensive test cases when using SensoDat. Furthermore, with the high amount and variety of sensor data, we think SensoDat can contribute to research, particularly for AI development, regression testing techniques for simulation-based SDC testing, flakiness in simulation, etc.</p> <p> </p> <p></p>
Dataset for Regression Testing of Self-Driving Cars
<p>All datasets are executed with:</p> <p>Vehicle: ETK-800.</p> <p>Maximum Speed = 120 km/h.</p> <p>Risk Factor = 1.5.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.