Data set of "Context-sensitive Requirements Search in Natural Language Specifications"

<p>We use this data set for the evaluation of our Context-sensitive Requirements Search approach. It contains three files and 15 searches.</p> <p>&nbsp;</p> <p><strong>Attribution</strong></p> <p>The foundation of this data set is PURE:<br> A. Ferrari, G. O. Spagnolo, and S. Gnesi. <em>PURE: a Dataset of Public Requirements Documents</em>. Version 1.0. Sept. 2018.<br> Available at:&nbsp;<a href="https://doi.org/10.5281/zenodo.1414117">https://doi.org/10.5281/zenodo.1414117</a></p> <p>The original data is licensed under <a href="https://creativecommons.org/licenses/by/4.0/legalcode">Creative Commons Attribution 4.0 International</a></p> <p>&nbsp;</p> <p><strong>Annotations</strong></p> <ul> <li>Each of the provided files contains annotations that we created with Label Studio for the corresponding file of the PURE data set.</li> <li>These annotations depict the search results of Plain Keyword Search (PKS) and the corresponding gold standard.</li> <li>The following is an example of an annotation in the .json file:</li> </ul> <pre><code class="language-json">{ "id": "UnaxUjB6oY", "meta": { "text": ["0"] }, "type": "labels", "value": { "end": "/text()[2]", "text": "speed", "start": "/text()[2]", "labels": ["speed: train speed profile"], "endOffset": 364, "startOffset": 359, "globalOffsets": { "end": 1403, "start": 1398 } }, "origin": "manual", "to_name": "text", "from_name": "ner" },</code></pre> <p>&nbsp;</p> <p><strong>Important Json attributes to consider</strong></p> <p><strong>Meta</strong></p> <p>We encode the relevance of each result in its metadata.</p> <p><em>Example:</em></p> <pre><code class="language-json">"meta": { "text": ["0"] },</code></pre> <ul> <li>1 = relevant for this search and found by PKS</li> <li>0 = irrelevant for this search but found by PKS</li> <li>-1 = relevant for this search and not found by PKS</li> </ul> <p>&nbsp;</p> <p><strong>Labels</strong></p> <p>This element is the search identifier we use in Label Studio. It corresponds to the table in the thesis that describes each search in detail.</p> <p><em>Example:</em></p> <pre><code class="language-json">"labels": ["speed: train speed profile"],</code></pre> <p>&nbsp;</p> <p><strong>GlobalOffsets</strong></p> <p>&quot;globalOffsets&quot; describes the position of a label in the text.</p> <p><em>Example:</em></p> <pre><code class="language-json">"globalOffsets": { "end": 1403, "start": 1398 }</code></pre> <p>&nbsp;</p>