Find research datasets worth reusing

opencc-by-4.0May 2010View details →

Figure 6a–e in DELTA for Beginners. An introduction into the taxonomy software package DELTA

Figure 6a–e. Input of text characters with the Character editor and change of sequence of characters.

opencc-by-4.0May 2010View details →

Figure 1a–f. DELTA Editor. a dropdown menus b in DELTA for Beginners. An introduction into the taxonomy software package DELTA

Figure 1a–f. DELTA Editor. a dropdown menus b pane for items (= taxa) c pane for characters d buttons for formatting selections e pane showing the character state description in number form and possible additional descriptions in angle brackets f pane for character state selection with check boxes.

opencc-by-4.0May 2020View details →

Results of the 1st International Competition on Software Testing (Test-Comp 2019)

This file describes the contents of an archive of the 1st Competition on Software Testing (Test-Comp 2019) <a href="https://test-comp.sosy-lab.org/2019/">https://test-comp.sosy-lab.org/2019/</a> The competition was run by Dirk Beyer, LMU Munich, Germany. More information is available in the following article: Dirk Beyer. First International Competition on Software Testing: Test-Comp 2019. International Journal on Software Tools for Technology Transfer, 2020. Copyright (C) Dirk Beyer <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a> SPDX-License-Identifier: CC-BY-4.0 <a href="https://spdx.org/licenses/CC-BY-4.0.html">https://spdx.org/licenses/CC-BY-4.0.html</a>   To browse the competition results with a web browser, there are two options: - start a local web server using   php -S localhost:8000   in order to view the data in this archive, or - browse <a href="https://test-comp.sosy-lab.org/2019/results/">https://test-comp.sosy-lab.org/2019/results/</a>   in order to view the data on the Test-Comp web page. Contents: index.html          directs to the overview web page LICENSE.txt         specifies the license README.txt          this file results-validated/  results of validation runs results-verified/   results of verification runs and aggregated results The folder results-validated/ contains the results from validation runs: - *.xml.bz2         XML results from BenchExec - *.logfiles.zip    output from tools - *.json.gz         mapping from files names to SHA 256 hashes for the file content The folder results-verified/ contains the results from test-generation runs and aggregated results: index.html               overview web page with rankings and score table design.css               HTML style definitions *.xml.bz2                XML results from BenchExec *.merged.xml.bz2         XML results from BenchExec, status adjusted according to the validation results *.logfiles.zip           output from tools *.json.gz                mapping from files names to SHA 256 hashes for the file content *.xml.bz2.table.html     HTML views on the detailed results data as generated by BenchExec's table generator *.All.table.html         HTML views of the full benchmark set (all categories) for each tool META_*.table.html        HTML views of the benchmark set for each meta category for each tool, and over all tools <category>*.table.html   HTML views of the benchmark set for each category over all tools iZeCa0gaey.html          HTML views per tool quantilePlot-*           score-based quantile plots as visualization of the results quantilePlotShow.gp      example Gnuplot script to generate a plot score*                   accumulated score results in various formats The hashes of the file names (in the files *.json.gz) are useful for - validating the exact contents of a file and - accessing the files from the witness store.   Overview over archives from Test-Comp 2019 that are available at Zenodo: <a href="https://doi.org/10.5281/zenodo.3856669">https://doi.org/10.5281/zenodo.3856669</a>   Witness store (containing the generated test suites) <a href="https://doi.org/10.5281/zenodo.3856661">https://doi.org/10.5281/zenodo.3856661</a>   Results (XML result files, log files, file mappings, HTML tables) <a href="https://doi.org/10.5281/zenodo.3856478">https://doi.org/10.5281/zenodo.3856478</a>   Test tasks, version testcomp19 <a href="https://doi.org/10.5281/zenodo.2561835">https://doi.org/10.5281/zenodo.2561835</a>   BenchExec, version 1.18 All benchmarks were executed for Test-Comp 2019, <a href="https://test-comp.sosy-lab.org/2019/">https://test-comp.sosy-lab.org/2019/</a> by Dirk Beyer, LMU Munich based on the components git@github.com:sosy-lab/sv-benchmarks.git  testcomp19-0-g6a770a9c1 git@gitlab.com:sosy-lab/test-comp/bench-defs.git  testcomp19-0-g1677027 git@github.com:sosy-lab/benchexec.git  1.18-0-gff72868 Feel free to contact me in case of questions: <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a>

opencc-by-4.0May 2020View details →

Artifact for the ESEC/FSE 2020 Paper: An Empirical Analysis of the Costs of Clone- and Platform-Oriented Software Reuse

This dataset comprises the supplementary material for the paper "An Empirical Analysis of the Costs of Clone- and Platform-Oriented Software Reuse" by Jacob Krüger and Thorsten Berger, accepted at ESEC/FSE 2020. The dataset comprises: <ul> <li>bibFilesManualSearch: The bib files for all venues analyzed, as provided by DBLP (cf. Section 2.4)</li> <li>dataFromPapers: The pdf file documents all included studies and the data extracted from these (cf. Section 2.4, 3.2, and 3.3)</li> <li>interviewGuide: The guide/questions for our semi-structured intreviews in the cost assessment phase (cf. Section 2.3)</li> <li>anonymizedInterviewSummary: The anonymized and summarized data from the cost-assessment interviews (cf. Section 3.2 and 3.3)</li> <li>R: Our R script for creating our figures and the corresponding csv files</li> </ul>

How Software Organizations are using the ISO/IEC 29110 Standard's Processes: A Survey of the State of the Art and Practice

ISO/IEC 29110 was developed containing a set of industrially validated practices that can potentially be adopted by software Very Small Entities (VSE). VSEs usually have characteristics that differentiate them from organizations of different sizes, such as extremely limited resources and informal project management processes, tending to adopt Agile methods and having an historical resistance to the adoption of standards, that are in general developed for large organizations. In this sense, our research question arises: "How are software organizations using the ISO/IEC 29110 practices?". To answer this question, a Systematic Mapping Study (SM), and a Survey with software organizations were carried out in order to identify the state of the art and the state of the practice in relation to the use of the standard’s practices. The SM identified 21 primary studies reporting the use of the standard in hundreds of software organizations with positive results such as organizational learning, process improvement, improved communication, and also some negative results, such as deployment difficulties in technical areas and the need for additional time and resources. The Survey carried out with 23 software companies identified that, in general, companies do not explicitly know the content of the standard, but partially carry out, in accordance with the standard, practices related to planning, monitoring, control and execution of a project plan, and do not carry out requirements analysis or architecture and detailed design as defined in the standard.

Architectural Feature Re-Modularization for Software Product Line Evolution

Extensive maintenance leads to the Software Product Line Architecture (PLA) degradation over time. When there is the need of evolving the Software Product Line (SPL) to include new features, or move to a new platform, a degraded PLA requires considerable effort to understand and modify, demanding expensive refactoring activity. In the state of the art, search-based algorithms are used to improve PLA at package level. However, recent studies have shown that the most variability and implementation details of an SPL are described in the level of classes. There is a gap between existing approaches and existing practical needs. In this work, we extend the current state of the art to deal with feature modularization in the level of classes by introducing a new search operator and a set of objective functions to deal with feature modularization in a finer granularity of the architectural elements, namely at class level. We evaluated the proposal in an exploratory study with a PLA widely investigated and a real-world PLA. The results of quantitative and qualitative analysis point out that our proposal provides solutions to properly re-modularize features in a PLA, being preferred by practitioners, in order to support the evolution of SPLs.

Hackathons as a Pedagogical Strategy to Engage Students to Learn and to Adopt Software Engineering Practices

Teaching Software Engineering is not a trivial duty since several pedagogical strategies can be used and sometimes the impact of these on students is uncertain. Hackathons are similar to marathons, however used to produce solutions to solve a specific problem in a short period of time and based on intense collaboration. Educational hackathons aim to promote learning in such an environment. The Undergraduate computing programs of PUCRS decided to use a hackathon as a pedagogical strategy aiming to motivate the students to practice the adoption of software development practices and to work in groups as a means to practice the development of social skills. Therefore, we conducted a case study to investigate: 1) The motivations to students to attend or not attend an educational hackathon, 2) The students perceptions about this hackathon, 3) The Software Engineering practices adopted by students. In this study, we identified factors that may affect students motivation to participate (e.g., improve the teamwork skills), some students expectations about the hackathon (e.g., work in teams), and the practices adopted by the students (e.g., pair programming). Some of our findings include that students enjoy participating in an informal educational environment (e.g., hackathons) to improve their technical skills and to build network with some colleagues. This study can provide insights to teachers that wants to organize some activity than traditional teaching and the students perspective about this kind of strategy.

A Study on Organizational IT Security in Mobile Software Ecosystems Literature

Information security is a key topic for most organizations. With the digital revolution, smartphones have become popular not only for personal use but also within organizations where many employees use them for business purposes. As smartphones are increasingly present in organizations, it is necessary to understand what recommendations the literature provides for the safe use of such devices, helping organizations to protect themselves from threats. ISO 27000 is a well-known standard for information security in a business context. It provides a set of controls that must be observed to ensure more secure organizational information. Therefore, the goal of this study is to identify which controls presented in ISO 27000, more specifically ISO 27001, are present in the Mobile Software Ecosystem (MSECO) literature. To do so, we conducted a systematic mapping review supplemented by a snowballing process to identify studies in the field of MSECO that have addressed any subject that is present in ISO 27001. We found that 34 out of the 114 ISO 27001 controls are covered by the MSECO literature. Also, some of the ISO sections (e.g., Asset Management) have not yet been explored in the MSECO literature. Our results can inspire future and further studies on the topic of MSECO information security.

opencc-by-4.0Nov 2020View details →

A Hybrid Feature Location Technique for Re-engineering Single Systems into Software Product Lines

The dataset used for evaluating the hybrid feature location technique presented in the paper: "A Hybrid Feature Location Technique for Re-engineering Single Systems into Software Product Lines". This enables reproducibility, evaluation, and comparison of our study. _________________________________________________________________________________________________________ Folder "Dataset" contains for each subject system used: (i) the artificial variants and their configurations; (ii) the ECCO repository containing the traces; (iii) the ground truth and composed variants; (iv) the metrics results. _________________________________________________________________________________________________________ Folder "Scenarios" contains for each subject system used: (i) the videos recorded from exercising features on GUI.

opencc-by-4.0Jul 2020View details →

Well-being and Productivity of Software Professionals during a Pandemic

The COVID-19 pandemic has forced governments worldwide to impose movement restrictions on their citizens. Although critical to reducing the virus' reproduction rate, these restrictions come with far-reaching social and economic consequences. In this paper, we investigate the impact of these restrictions on an individual level among software engineers currently working from home. Although software professionals are accustomed to working with digital tools in their day-to-day work, the abrupt and enforced work-from-home context has resulted in an unprecedented scenario for the software engineering community. In a two-wave longitudinal study (N = 192), we covered over 50 psychological, social, situational, and physiological factors that have previously been associated with well-being or productivity. Examples include anxiety, distractions, psychological and physical needs, office set-up, stress, and work motivation. This design allowed us to identify those variables that explain unique variance in well-being and productivity. Results include (1) the quality of social contacts predicted positively, and stress predicted an individual's well-being negatively when controlling for other variables consistently across both waves; (2) boredom and distractions predicted productivity negatively; (3) productivity was less strongly associated with all predictor variables at time two compared to time one, suggesting that software engineers adapted to the lockdown situation over time; and (4) the longitudinal study did not provide evidence that any predictor variable causal explained variance in well-being and productivity.  Overall, we conclude that working from home was per se not a significant challenge for software engineers. Our study can assess the effectiveness of current work-from-home and general well-being and productivity support guidelines and provide tailored insights for software professionals.

opencc-by-4.0Nov 2020View details →

MongoDB database dump for the analysis of the current sustainability state of research software

This data set is the MongoDB dump (bson files) of the data created and analyzed with the rsps framework. In the first step, a research subject is assigned to the research software repositories. Afterwards, the current sustainability state is evaluated. The data set comprises the following six bson files: repositories: metadata, received from the GitHub REST API, for repositories containing the search terms "doi+10" or "doi+10+in:readme", additional information are the request date, the contained search term, and the repository hosting service, in this case for all repositories "github". For repositories the Readme files are available. publications: metadata of publications, published on arXiv and ACM, that contain the search term "github.com". rs_repositories: research software candidates containing a DOI or that are referenced by the publications contained in the publications data set. rs_artifacts: research software artifacts that are referenced in the harvested GitHub repositories by a DOI and the harvested publications. publication_subjects: All Science Journal Classification (ASJC) of Scopus combined with the Scopus source list and Scopus book title list (https://www.scopus.com/home.uri) arxiv_subjects: arXiv taxonomy complemented with the ASJC research subject.

Software Product Line Traceability and Product Configuration in Class and Sequence Diagrams: an Empirical Study

Software Product Line Traceability and Product Configuration in Class and Sequence Diagrams: an Empirical Study

Software Product Line Configuration and Traceability: an Empirical Study on SMarty Class and Component Diagrams

Software Product Line Configuration and Traceability: an Empirical Study on SMarty Class and Component Diagrams

The Secret Life of Software Vulnerabilities: A Large-Scale Empirical Study

Online appendix of the paper entitled: "The Secret Life of Software Vulnerabilities: A Large-Scale Empirical Study". It contains all scripts and data required to replicate the four research questions of the study. Abstract: Software vulnerabilities are weaknesses in source code that can be potentially exploited to cause loss or harm. While researchers have been devising a number of methods to deal with vulnerabilities, there is still a noticeable lack of knowledge on their software engineering life cycle, for example how vulnerabilities are introduced and removed by developers. This information can be exploited to design more effective methods for vulnerability prevention and detection, as well as to understand the granularity that these methods should aim at. To investigate the life cycle of software vulnerabilities, we focus on how, when, and under which circumstances vulnerabilities are introduced in software projects, as well as whether, after how long, and how they are removed. We consider 4,097 vulnerabilities with public patches from the National Vulnerability Database—pertaining to 1,163 open-source software projects on GITHUB—and define a six-step process that involves both automated parts (e.g., using the SZZ algorithm to find the vulnerability-inducing commits) and manual analyses (e.g., how vulnerabilities were fixed). The investigated vulnerabilities can be classified in 148 categories, take on average 4.19 commits before being introduced, and remain unfixed for a median of 1,506.50 commits and 691.50 days. Most of them are introduced by developers with high workload, often when doing maintenance activities, and removed with mostly with the addition of new source code aiming at implementing further checks on inputs. We conclude by distilling practical implications on when and how vulnerability detectors should work to better assist developers in early detecting these issues.

opencc-by-4.0Apr 2020View details →

A Survey on the Adoption of Patterns for Engineering Software for the Cloud - Response Dataset

This work takes as a starting point a collection of patterns for engineering software for the cloud and tries to find how they are regarded and adopted by professionals. We investigate (1) their relevance for professional software developers, (2) the extent to which product and company characteristics influence their adoption, and (3) how adopting some patterns might correlate with the likelihood of adopting others.  For this purpose, we surveyed 102 practitioners using an online questionnaire.  Amongst other findings, we conclude that most companies are using these patterns, with the overwhelming majority (97%) using at least one. We observe that the mean pattern adoption tends to increase as companies mature, namely when varying the product operation complexity, active monthly users, and company size. Finally, we establish clear correlations in the adoption of specific pairs of patterns, with conditional probabilities as high as 94%, which hints on how some practices are dependent or influence the adoption of others.

opencc-by-nc-4.0Mar 2020View details →

Replication package for "Wikifying Software Artifacts"

This replication package contains all data files necessary to verify and replicate our findings presented in the manuscript "Wikifying Software Artifacts".

openother-openJan 2021View details →

Softcite Dataset: A dataset of software mentions in research publications

The Softcite dataset is a gold-standard dataset of software mentions in research publications, a free resource primarily for software entity recognition in scholarly text. This is the first release of this dataset. What's in the dataset With the aim of facilitating software entity recognition efforts at scale and eventually increased visibility of research software for the due credit of software contributions to scholarly research, a team of trained annotators from Howison Lab at the University of Texas at Austin annotated 4,093 software mentions in 4,971 open access research publications in biomedicine (from PubMed Central Open Access collection) and economics (from Unpaywall open access services). The annotated software mentions, along with their publisher, version, and access URL, if mentioned in the text, as well as those publications annotated as containing no software mentions, are all included in the released dataset as a TEI/XML corpus file. For understanding the schema of the Softcite corpus, its design considerations, and provenance, please refer to our paper included in this release (preprint version). Use scenarios The release of the Softcite dataset is intended to encourage researchers and stakeholders to make research software more visible in science, especially to academic databases and systems of information retrieval; and facilitate interoperability and collaboration among similar and relevant efforts in software entity recognition and building utilities for software information retrieval. This dataset can also be useful for researchers investigating software use in academic research. Current release content softcite-dataset v1.0 release includes: <ul> <li>The Softcite dataset corpus file: softcite_corpus-full.tei.xml</li> <li>Softcite Dataset: A Dataset of Software Mentions in Biomedical and Economic Research Publications, our paper that describes the design consideration and creation process of the dataset: Softcite_Dataset_Description_RC.pdf. (This is a preprint version of our forthcoming publication in the Journal of the Association for Information Science and Technology.)</li> </ul> The Softcite dataset is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>. If you have questions, please start a discussion or issue in the <a href="https://github.com/howisonlab/softcite-dataset">howisonlab/softcite-dataset Github repository</a>.

opencc-by-4.0Jan 2021View details →

Results of the 10th Intl. Competition on Software Verification (SV-COMP 2021)

Competition Results This file describes the contents of an archive of the 10th Competition on Software Verification (SV-COMP 2021). <a href="https://sv-comp.sosy-lab.org/2021/">https://sv-comp.sosy-lab.org/2021/</a> The competition was run by Dirk Beyer, LMU Munich, Germany. More information is available in the following article: Dirk Beyer. Software Verification: 10th Comparative Evaluation (SV-COMP 2021). In Proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2021, Luxembourg, March 27 - April 1), 2021. Springer. Copyright (C) Dirk Beyer <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a> SPDX-License-Identifier: CC-BY-4.0 <a href="https://spdx.org/licenses/CC-BY-4.0.html">https://spdx.org/licenses/CC-BY-4.0.html</a> To browse the competition results with a web browser, there are two options: <ul> <li>start a local web server using php -S localhost:8000 in order to view the data in this archive, or</li> <li>browse https://sv-comp.sosy-lab.org/2021/results/ in order to view the data on the SV-COMP web page.</li> </ul> Contents <ul> <li><code>index.html</code>: directs to the overview web page</li> <li><code>LICENSE.txt</code>: specifies the license</li> <li><code>README.txt</code>: this file</li> <li><code>results-validated/</code>: results of validation runs</li> <li><code>results-verified/</code>: results of verification runs and aggregated results</li> </ul> The folder <code>results-validated/</code> contains the results from validation runs: <ul> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> </ul> The folder <code>results-verified/</code> contains the results from verification runs and aggregated results: <ul> <li><code>index.html</code>: overview web page with rankings and score table</li> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.merged.xml.bz2</code>: XML results from BenchExec, status adjusted according to the validation results</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> <li><code>*.xml.bz2.table.html</code>: HTML views on the detailed results data as generated by BenchExec’s table generator</li> <li><code>*.All.table.html</code>: HTML views of the full benchmark set (all categories) for each tool</li> <li><code>META_*.table.html</code>: HTML views of the benchmark set for each meta category for each tool, and over all tools</li> <li><code><category>*.table.html</code>: HTML views of the benchmark set for each category over all tools</li> <li><code>iZeCa0gaey.html</code>: HTML views per tool</li> <li> <code>validatorStatistics.html</code>: Statictics of the validator runs </li> <li><code>quantilePlot-*</code>: score-based quantile plots as visualization of the results</li> <li><code>quantilePlotShow.gp</code>: example Gnuplot script to generate a plot</li> <li> <code>score*</code>: accumulated score results in various formats </li> </ul> The hashes of the file names (in the files *.json.gz) are useful for <ul> <li>validating the exact contents of a file and</li> <li>accessing the files from the witness store.</li> </ul> Other Archives Overview over archives from SV-COMP 2021 that are available at Zenodo: <ul> <li><a href="https://doi.org/10.5281/zenodo.4459196">https://doi.org/10.5281/zenodo.4459196</a> Witness store (containing the generated verification witnesses)</li> <li><a href="https://doi.org/10.5281/zenodo.4458215">https://doi.org/10.5281/zenodo.4458215</a> Results (XML result files, log files, file mappings, HTML tables)</li> <li><a href="https://doi.org/10.5281/zenodo.4459126">https://doi.org/10.5281/zenodo.4459126</a> Verification tasks, version svcomp21</li> <li><a href="https://doi.org/10.5281/zenodo.4317433">https://doi.org/10.5281/zenodo.4317433</a> BenchExec, version 3.6</li> </ul> All benchmarks were executed for SV-COMP 2021 <a href="https://sv-comp.sosy-lab.org/2021/">https://sv-comp.sosy-lab.org/2021/</a> by Dirk Beyer, LMU Munich, based on the following components: <ul> <li><a href="https://gitlab.com/sosy-lab/sv-comp/archives-2021">https://gitlab.com/sosy-lab/sv-comp/archives-2021</a> svcomp21-0-g08c7a98</li> <li><a href="https://gitlab.com/sosy-lab/software/sv-benchmarks">https://gitlab.com/sosy-lab/software/sv-benchmarks</a> svcomp21-0-g4cc6b6d96a</li> <li><a href="https://gitlab.com/sosy-lab/software/benchexec">https://gitlab.com/sosy-lab/software/benchexec</a> 3.6-0-gb278ebbb</li> <li><a href="https://gitlab.com/sosy-lab/benchmarking/competition-scripts">https://gitlab.com/sosy-lab/benchmarking/competition-scripts</a> svcomp21-0-g8339740</li> <li><a href="https://gitlab.com/sosy-lab/sv-comp/bench-defs">https://gitlab.com/sosy-lab/sv-comp/bench-defs</a> svcomp21-0-ga57fe48</li> </ul> Contact Feel free to contact me in case of questions: <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a>

opencc-by-4.0Jan 2021View details →

Results of the 3rd Intl. Competition on Software Testing (Test-Comp 2021)

Competition Results This file describes the contents of an archive of the 3rd Competition on Software Testing (Test-Comp 2021). <a href="https://test-comp.sosy-lab.org/2021/">https://test-comp.sosy-lab.org/2021/</a> The competition was run by Dirk Beyer, LMU Munich, Germany. More information is available in the following article: Dirk Beyer. Status Report on Software Testing: Test-Comp 2021. In Proceedings of the 24th International Conference on Fundamental Approaches to Software Engineering (FASE 2021, Luxembourg, March 27 - April 1), 2021. Springer. Copyright (C) Dirk Beyer <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a> SPDX-License-Identifier: CC-BY-4.0 <a href="https://spdx.org/licenses/CC-BY-4.0.html">https://spdx.org/licenses/CC-BY-4.0.html</a> To browse the competition results with a web browser, there are two options: <ul> <li>start a local web server using php -S localhost:8000 in order to view the data in this archive, or</li> <li>browse <a href="https://test-comp.sosy-lab.org/2021/results/">https://test-comp.sosy-lab.org/2021/results/</a> in order to view the data on the Test-Comp web page.</li> </ul> Contents <ul> <li><code>index.html</code>: directs to the overview web page</li> <li><code>LICENSE.txt</code>: specifies the license</li> <li><code>README.txt</code>: this file</li> <li><code>results-validated/</code>: results of validation runs</li> <li><code>results-verified/</code>: results of test-generation runs and aggregated results</li> </ul> The folder <code>results-validated/</code> contains the results from validation runs: <ul> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> </ul> The folder <code>results-verified/</code> contains the results from test-generation runs and aggregated results: <ul> <li><code>index.html</code>: overview web page with rankings and score table</li> <li><code>design.css</code>: HTML style definitions</li> <li><code>*.xml.bz2</code>: XML results from BenchExec</li> <li><code>*.merged.xml.bz2</code>: XML results from BenchExec, status adjusted according to the validation results</li> <li><code>*.logfiles.zip</code>: output from tools</li> <li><code>*.json.gz</code>: mapping from files names to SHA 256 hashes for the file content</li> <li><code>*.xml.bz2.table.html</code>: HTML views on the detailed results data as generated by BenchExec’s table generator</li> <li><code>*.All.table.html</code>: HTML views of the full benchmark set (all categories) for each tool</li> <li><code>META_*.table.html</code>: HTML views of the benchmark set for each meta category for each tool, and over all tools</li> <li><code><category>*.table.html</code>: HTML views of the benchmark set for each category over all tools</li> <li> <code>iZeCa0gaey.html</code>: HTML views per tool </li> <li><code>quantilePlot-*</code>: score-based quantile plots as visualization of the results</li> <li><code>quantilePlotShow.gp</code>: example Gnuplot script to generate a plot</li> <li> <code>score*</code>: accumulated score results in various formats </li> </ul> The hashes of the file names (in the files <code>*.json.gz</code>) are useful for <ul> <li>validating the exact contents of a file and</li> <li>accessing the files from the witness store.</li> </ul> Other Archives Overview over archives from Test-Comp 2021 that are available at Zenodo: <ul> <li><a href="https://doi.org/10.5281/zenodo.4459466">https://doi.org/10.5281/zenodo.4459466</a> Witness store (containing the generated test suites)</li> <li><a href="https://doi.org/10.5281/zenodo.4459470">https://doi.org/10.5281/zenodo.4459470</a> Results (XML result files, log files, file mappings, HTML tables)</li> <li><a href="https://doi.org/10.5281/zenodo.4459132">https://doi.org/10.5281/zenodo.4459132</a> Test tasks, version testcomp21</li> <li><a href="https://doi.org/10.5281/zenodo.4317433">https://doi.org/10.5281/zenodo.4317433</a> BenchExec, version 3.6</li> </ul> All benchmarks were executed for Test-Comp 2021 <a href="https://test-comp.sosy-lab.org/2021/">https://test-comp.sosy-lab.org/2021/</a> by Dirk Beyer, LMU Munich, based on the following components: <ul> <li><a href="https://gitlab.com/sosy-lab/test-comp/archives-2021">https://gitlab.com/sosy-lab/test-comp/archives-2021</a> testcomp21-0-gdacd4bf</li> <li><a href="https://gitlab.com/sosy-lab/software/sv-benchmarks">https://gitlab.com/sosy-lab/software/sv-benchmarks</a> testcomp21-0-gefea738258</li> <li><a href="https://gitlab.com/sosy-lab/software/benchexec">https://gitlab.com/sosy-lab/software/benchexec</a> 3.6-0-gb278ebbb</li> <li><a href="https://gitlab.com/sosy-lab/benchmarking/competition-scripts">https://gitlab.com/sosy-lab/benchmarking/competition-scripts</a> testcomp21-0-g8339740</li> <li><a href="https://gitlab.com/sosy-lab/test-comp/bench-defs">https://gitlab.com/sosy-lab/test-comp/bench-defs</a> testcomp21-0-g9d532c9</li> </ul> Contact Feel free to contact me in case of questions: <a href="https://www.sosy-lab.org/people/beyer/">https://www.sosy-lab.org/people/beyer/</a>

Data and material for the manuscript "Mutation testing and self/peer assessment: analyzing their effect on students in a software testing course"

This repository is composed of two different parts: <ul> <li><a href="https://zenodo.org/record/4464300/files/Assessment%20data%20and%20Mutation%20Scores.xlsx?download=1">Assessment data and Mutation Scores</a> file contains the student-generated data used in the experience.</li> <li><a href="https://zenodo.org/record/4464300/files/experience-material.zip?download=1">Experience-material</a> file contains the files to be able to reproduce the experience.</li> </ul>   The <a href="https://zenodo.org/record/4464300/files/experience-material.zip?download=1">Experience-material</a> file for the lab is used in two sessions: Session 1: Development and assessment of test suites In this session, the student has to develop a test suite for a program under test. At the end of the session, the test suite will be evaluated against a set of assessment criteria regarding the quality of the developed test suite. Files for this session: <ul> <li>VVS-Lab6-S1 pdf file , with the description of this session.</li> <li>Material-S1 zip file, with the files required to complete this session.</li> </ul> Session 2: Evaluation applying mutation testing with MuCPP In this session, the test cases designed in the first part of this lab will be evaluated based on the mutation adequacy criterion. This will be done by using the <a href="https://ucase.uca.es/mucpp/">MuCPP mutation tool</a>. Files for this session: <ul> <li>VVS-Lab6-S2 pdf file, with the description of this session.</li> <li>Material-S2 zip file, with the files required to complete this session.</li> </ul> The source code files family.[cpp|hpp] have been adapted from a listing in [1]. Note that, while considered to be fault free in this lab, these source files are used in other sessions where students are expected to detect some defects in them. [1] S. Wiener and L. J. Pinson, The C++ Workbook. USA: Addison-Wesley Longman Publishing Co., Inc., 1990.