ardoco/benchmark: Release v1.1: SAD-Code TLR, New Project, Inconsistencies, and small Bugfixes

<p dir="auto"><strong>This is not the version of the benchmark from the original ECSA 2022 paper. The original version is located at: <a href="https://doi.org/10.5281/zenodo.6966832">https://doi.org/10.5281/zenodo.6966832</a><br></strong><strong>Please check the Release Notes for further information: <a href="https://github.com/ardoco/benchmark/releases/tag/v1.1">https://github.com/ardoco/benchmark/releases/tag/v1.1</a></strong></p> <p dir="auto">&nbsp;</p> <p dir="auto">This repository contains a benchmark for traceability link recovery (TLR) between textual Software Architecture Documentation (SAD) and Software Architecture Models (SAM). It was initially published in the paper&nbsp;<a href="https://doi.org/10.1007/978-3-031-36889-9_30" rel="nofollow">Establishing a Benchmark Dataset for Traceability Link Recovery Between Software Architecture Documentation and Models</a>.</p> <p dir="auto">Each project of the benchmark is structured as follows:</p> <ul> <li>The README of each project contains some information about the used languages and lines of code created with&nbsp;<a href="https://github.com/AlDanial/cloc">cloc</a>.</li> <li>The folder&nbsp;<code>model_&lt;year&gt;</code>&nbsp;contains the architecture model of the project. <ul> <li>The folder&nbsp;<code>pcm</code>&nbsp;contains a Palladio Component Model (PCM) of the system. It has at least the repository view (components) of the system.</li> <li>The folder&nbsp;<code>uml</code>&nbsp;contains a Papyrus UML model. It is created from the PCM Repository using&nbsp;<a href="https://github.com/InFormALin/PCM2UML">PCM2UML</a>.</li> <li>The folder&nbsp;<code>code</code>&nbsp;contains a code model. The version of the code is stated in a README.md next to the model. The model is an&nbsp;<code>ArDoCo Code Model</code>. The model can be loaded using the&nbsp;<a href="https://github.com/ArDoCo/Core/blob/main/stages/model-provider/src/main/java/edu/kit/kastel/mcse/ardoco/core/models/connectors/generators/code/CodeExtractor.java#L47">ArDoCo Code Extractor</a>.</li> </ul> </li> <li>The folder&nbsp;<code>text_&lt;year&gt;</code>&nbsp;contains a documentation of the project. <ul> <li>The text file(s) in the project folder contains the SAD of the projects as plain text.</li> </ul> </li> <li>The folder&nbsp;<code>diagrams_&lt;year&gt;</code>&nbsp;contains the informal diagrams of the project.</li> <li>The folder&nbsp;<code>goldstandards</code>&nbsp;contains all gold standards for the project. In the following, the&nbsp;<code>&lt;year&gt;</code>&nbsp;refers to the year of the artifact. Additional classifiers are added to the end of the file names. You will find more information about these classifiers in the README of the respective project artifacts. <ul> <li>The file&nbsp;<code>goldstandard_sad_&lt;year&gt;-sam_&lt;year&gt;.csv</code>&nbsp;contains the traceability links between SAD and SAM. It links the model elements by id with the sentences by their number</li> <li>The file&nbsp;<code>goldstandard_sad_&lt;year&gt;-sam_&lt;year&gt;_UME.csv</code>&nbsp;contains all IDs of model elements that are contained in the model but not described in the text.</li> <li>The file&nbsp;<code>goldstandard_sad_&lt;year&gt;_code_&lt;year&gt;.csv</code>&nbsp;contains the traceability links between SAD and code models.</li> <li>The file&nbsp;<code>goldstandard_sam_&lt;year&gt;-code_&lt;year&gt;.csv</code>&nbsp;is a gold standard for mapping the architecture elements and code elements.</li> <li>The file&nbsp;<code>goldstandard_sad_id_&lt;year&gt;.json</code>&nbsp;contains the traceability links between SAD and informal diagrams. The JSON schema is specified in the&nbsp;<a href="https://github.com/ardoco/benchmark/blob/main/DiagramSchema.json">DiagramSchema</a>&nbsp;file.</li> </ul> </li> </ul> <div dir="auto"> <h2>Using the benchmark</h2> </div> <p dir="auto">In order to provide an easy approach to use the benchmark, we provide an example TLR approach called&nbsp;<a href="https://github.com/ArDoCo/SimpleTracelinkDiscovery/">Simple Tracelink Discovery (STD)</a>&nbsp;that uses this benchmark in its&nbsp;<a href="https://github.com/ArDoCo/SimpleTracelinkDiscovery/tree/main/src/test/java/io/github/ardoco/simpletracelinkdiscovery/eval">evaluation</a>. Therefore, the benchmark is linked to the STD repository via a&nbsp;<a href="https://github.com/ArDoCo/SimpleTracelinkDiscovery/tree/main/src/test/resources/benchmark">git subtree</a>.</p> <div dir="auto"> <h2>Projects</h2> </div> <div dir="auto"> <h3>BigBlueButton</h3> </div> <p dir="auto">BigBlueButton (BBB) is a non-scientific application that provides a web conferencing system with the focus on creating a "global teaching platform".</p> <div dir="auto"> <h3>MediaStore</h3> </div> <p dir="auto">MediaStore is a "model application built after the iTunes Store". Its architecture was used for exemplary performance analyses on software architecture models.</p> <div dir="auto"> <h3>Teammates</h3> </div> <p dir="auto">TEAMMATES is an open-source "online tool for manageing peer evaluations and other feedback paths of your students".</p> <div dir="auto"> <h3>Teastore</h3> </div> <p dir="auto">Teastore is a scientific application that is used as a "micro-service reference test application".</p> <div dir="auto"> <h3>JabRef</h3> </div> <p dir="auto">JabRef is a tool to manage citations and references in your bibliographies. It has features to collect, organize, cite, and share research work.</p> <div dir="auto"> <h2>LICENSE</h2> </div> <blockquote> <p dir="auto"><strong>Note</strong></p> <p dir="auto">Our LICENSE does only apply to the models and the Gold Standards (CSV &amp; JSON files). The texts and diagrams are licensed w.r.t. to the actual projects. More details about the LICENSE can be found in the README files of the respective texts and diagrams.</p> </blockquote>