Results of the 10th Intl. Competition on Software Verification (SV-COMP 2021)
<p>Competition Results</p> <p>This file describes the contents of an archive of the 10th Competition on Software Verification (SV-COMP 2021).<br> <a href="https://sv-comp.sosy-lab.org/2021/">https://sv-comp.sosy-lab.org/2021/</a></p> <p>The competition was run by Dirk Beyer, LMU Munich, Germany.<br> More information is available in the following article:<br> Dirk Beyer. <em>Software Verification: 10th Comparative Evaluation (SV-COMP 2021).</em> In Proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2021, Luxembourg, March 27 - April 1), 2021. Springer.</p> <p>Copyright (C) Dirk Beyer<br> <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a></p> <p>SPDX-License-Identifier: CC-BY-4.0<br> <a href="https://spdx.org/licenses/CC-BY-4.0.html">https://spdx.org/licenses/CC-BY-4.0.html</a></p> <p>To browse the competition results with a web browser, there are two options:</p> <ul> <li>start a local web server using php -S localhost:8000 in order to view the data in this archive, or</li> <li>browse https://sv-comp.sosy-lab.org/2021/results/ in order to view the data on the SV-COMP web page.</li> </ul> <p>Contents</p> <ul> <li><code>index.html</code>: directs to the overview web page</li> <li><code>LICENSE.txt</code>: specifies the license</li> <li><code>README.txt</code>: this file</li> <li><code>results-validated/</code>: results of validation runs</li> <li><code>results-verified/</code>: results of verification runs and aggregated results</li> </ul> <p>The folder <code>results-validated/</code> contains the results from validation runs:</p> <ul> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> </ul> <p>The folder <code>results-verified/</code> contains the results from verification runs and aggregated results:</p> <ul> <li><code>index.html</code>: overview web page with rankings and score table</li> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.merged.xml.bz2</code>: XML results from BenchExec, status adjusted according to the validation results</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> <li><code>*.xml.bz2.table.html</code>: HTML views on the detailed results data as generated by BenchExec’s table generator</li> <li><code>*.All.table.html</code>: HTML views of the full benchmark set (all categories) for each tool</li> <li><code>META_*.table.html</code>: HTML views of the benchmark set for each meta category for each tool, and over all tools</li> <li><code><category>*.table.html</code>: HTML views of the benchmark set for each category over all tools</li> <li><code>iZeCa0gaey.html</code>: HTML views per tool</li> <li> <p><code>validatorStatistics.html</code>: Statictics of the validator runs</p> </li> <li><code>quantilePlot-*</code>: score-based quantile plots as visualization of the results</li> <li><code>quantilePlotShow.gp</code>: example Gnuplot script to generate a plot</li> <li> <p><code>score*</code>: accumulated score results in various formats</p> </li> </ul> <p>The hashes of the file names (in the files *.json.gz) are useful for</p> <ul> <li>validating the exact contents of a file and</li> <li>accessing the files from the witness store.</li> </ul> <p>Other Archives</p> <p>Overview over archives from SV-COMP 2021 that are available at Zenodo:</p> <ul> <li><a href="https://doi.org/10.5281/zenodo.4459196">https://doi.org/10.5281/zenodo.4459196</a> Witness store (containing the generated verification witnesses)</li> <li><a href="https://doi.org/10.5281/zenodo.4458215">https://doi.org/10.5281/zenodo.4458215</a> Results (XML result files, log files, file mappings, HTML tables)</li> <li><a href="https://doi.org/10.5281/zenodo.4459126">https://doi.org/10.5281/zenodo.4459126</a> Verification tasks, version svcomp21</li> <li><a href="https://doi.org/10.5281/zenodo.4317433">https://doi.org/10.5281/zenodo.4317433</a> BenchExec, version 3.6</li> </ul> <p>All benchmarks were executed for SV-COMP 2021 <a href="https://sv-comp.sosy-lab.org/2021/">https://sv-comp.sosy-lab.org/2021/</a><br> by Dirk Beyer, LMU Munich, based on the following components:</p> <ul> <li><a href="https://gitlab.com/sosy-lab/sv-comp/archives-2021">https://gitlab.com/sosy-lab/sv-comp/archives-2021</a> svcomp21-0-g08c7a98</li> <li><a href="https://gitlab.com/sosy-lab/software/sv-benchmarks">https://gitlab.com/sosy-lab/software/sv-benchmarks</a> svcomp21-0-g4cc6b6d96a</li> <li><a href="https://gitlab.com/sosy-lab/software/benchexec">https://gitlab.com/sosy-lab/software/benchexec</a> 3.6-0-gb278ebbb</li> <li><a href="https://gitlab.com/sosy-lab/benchmarking/competition-scripts">https://gitlab.com/sosy-lab/benchmarking/competition-scripts</a> svcomp21-0-g8339740</li> <li><a href="https://gitlab.com/sosy-lab/sv-comp/bench-defs">https://gitlab.com/sosy-lab/sv-comp/bench-defs</a> svcomp21-0-ga57fe48</li> </ul> <p>Contact</p> <p>Feel free to contact me in case of questions: <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a></p>
ShareScore
40/100
Overall dataset sharing score
Score breakdown
These five areas show where the dataset supports — or may limit — practical reuse.
- Stewardship
- 8
- Harmonization
- 4
- Access
- 16
- Reuse readiness
- 8
- Engagement
- 4
Topics
Dataset details
- Repository
- zenodo
- Published
- January 21, 2021
- License
- cc-by-4.0
- DOI
- 10.5281/zenodo.4458214
Scored with release 0.9.0
ScienceDex for faculty
Your academic CV and NIH biosketch
Make your CV in your school's format and your NIH biosketch from it, then download both as Word files.