Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
677
datasets available to search
ShareScore release 0.9.0
Dataset results
677 results for “Replication package”
Image-based Many-language Programming Language Identification - Replication Package
<p>This dataset contains the data, software, and instructions needed to replicate the findings of the paper:</p> <p>Francesca Del Bonifro, Maurizio Gabbrielli, Antonio Lategano, and Stefano Zacchiroli. Image-based Many-language<br> Programming Language Identification. <a href="https://peerj.com/computer-science/"><em>PeerJ Computer Science</em></a>, 2021 (to appear). DOI: <a href="https://dx.doi.org/10.7717/peerj-cs.631">10.7717/peerj-cs.631</a></p> <p>After retrieving the full dataset, extract the replication-package.zip archive and follow the instructions described in the README.md file.</p>
Replication Package "Applying Test Case Prioritization to Software Microbenchmarks"
<p>Replication package for the paper "Applying Test Case Prioritization to Software Microbenchmarks" accepted for publication in Empirical Software Engineering.</p>
Replication package for "Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation"
<p>This repository contains the replication package for the paper "Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation" by Fernando Vallecillos Ruiz, Anastasiia Grishina, Max Hort and Leon Moonen, accepted for publication in ACM Transactions on Software Engineering and Methodology on 2025-10-09.</p> <p>A preprint is deposited on arXiv with DOI: <a href="https://doi.org/10.48550/arXiv.2401.07994">10.48550/arXiv.2401.07994</a>.</p> <p>The replication package is archived on Zenodo with DOI: <a href="https://doi.org/10.5281/zenodo.10500593">10.5281/zenodo.10500593</a>. It is maintained on GitHub at <a href="https://github.com/secureIT-project/RTT_for_APR">https://github.com/secureIT-project/RTT_for_APR</a>.</p> <p>This project builds on code from the <a href="https://github.com/lin-tan/clm/">clm</a> project, which is (c) 2023, The ASSET research group led by Lin Tan, Purdue University, licensed under the BSD 3-Clause License (see jasper/LICENSE.BSD). All modifications and new contributions are (c) 2025 by the authors of this replication package and distributed under the MIT License (see LICENSE.MIT). The data, models and preprint are distributed under the CC BY 4.0 license.</p> <h2>Citation<code> </code></h2> <p>If you build on this data or code, please cite this work by referring to the paper:</p> <div> <pre><code>@article{ruiz2025:rtt, title = {Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation}, author = {Vallecillos Ruiz, Fernando and Anastasiia Grishina and Max Hort and Leon Moonen}, journal = {ACM Transactions on Software Engineering and Methodology (TOSEM)}, year = {2025}, publisher = {{ACM}} }</code></pre> </div> <h2>Organization</h2> <p>The replication package is organized as follows:</p> <ul> <li>clm-apr <ul> <li>plbart: code to generate patches with PLBART models.</li> <li>codet5: code to generate patches with CodeT5 models.</li> <li>transcoder: code to generate patches with the TransCoder model.</li> <li>incoder: code to generate patches with InCoder models.</li> <li>santacoder: code to generate patches with the SantaCoder model.</li> <li>starcoder: code to generate patches with the StarCoderBase model.</li> <li>quixbugs: code to validate patches generated for the QuixBugs benchmark.</li> <li>defects4j: code to validate patches generated for any of the Defects4J benchmarks.</li> <li>humaneval: code to validate patches generated for the HumanEval-Java benchmark.</li> </ul> </li> <li>humaneval-java: the HumanEval-Java benchmark proposed by Jiang et al. 2023</li> <li>jasper: a Java tool to parse Java programs needed to preprocess input.</li> <li>model: folder to download the language models.</li> <li>analysis_wandb: data from WandB and Jupyter notebook to create graphs.</li> <li>tmp_benchmarks: folder for temporary files used in patch validation. The folder may contain pairs of `paralell’ folders src and src_org for each benchmark, used to replace buggy code with candidate patches.</li> </ul> <h2>Replication</h2> <h3>Prerequisites</h3> <ul> <li>Python version: 3.8—3.10.</li> <li><a href="https://git-lfs.com/">Git LFS</a> is required for model downloading.</li> </ul> <h4>Weight and Biases (WandB)</h4> <ol> <li>Create an account on <a href="https://wandb.ai/">Weights and Biases</a></li> <li>Install the <a href="https://docs.wandb.ai/ref/python">Weights and Biases</a> library</li> <li>Run <code>wandb login</code> and follow the instructions</li> </ol> <h4>Set up OpenAI access</h4> <p>OpenAI account is needed with access to <code>gpt-3.5-turbo</code> and <code>gpt-4</code> . The <code>OPENAI_API_KEY</code> environment variable should be set to your OpenAI API access token.</p> <h3>Dependencies</h3> <ul> <li><a href="https://github.com/rjust/defects4j">Defects4J</a> - To generate inputs for the Defects4J datasets or to validate them, you need to have installed <a href="https://github.com/rjust/defects4j">their tool</a>.</li> <li>Java 8</li> <li>Apache Maven</li> </ul> <h3>Setup</h3> <p>We recommend the use of the setup script:</p> <pre><code>setup.sh </code></pre> <p>which performs the following:</p> <ol> <li>Creates a virtual environment for Python and activate it.</li> <li>Install the packages in <code>requirements.txt</code>.</li> <li>Compiles Jasper.</li> <li>Downloads parsers.</li> <li>Check if the Defects4J installation is correct.</li> </ol> <h3>Download models</h3> <p>The following bash script contains the code to download all of the models used:</p> <pre><code>models/download_models.sh </code></pre> <p>We recommend downloading only the models you are going to use due to their size</p> <pre><code>cd models chmod +x download_models.sh ./download_models.sh </code></pre> <p>To run one specific model, for example, PLBART (C#), use the following commands:</p> <pre><code>cd models git lfs install git clone https://huggingface.co/uclanlp/plbart-java-cs git clone https://huggingface.co/uclanlp/plbart-cs-java cd ../.. </code></pre> <h3>Step 1: Preprocessing and Prompting:</h3> <p>Each script in each <code>clm-apr/[model]</code> folder connects one or more models with<br>one dataset. These scripts follow the template: [benchmark]_[model]_[technique].py.<br>The scripts first create an <code>[model]_input.json</code> file with the preprocessed<br>input. Then generate outputs based on that file with one or more models.<br>For example:</p> <pre><code>cd clm-apr/plbart python quixbugs_plbart_round.py # Generates input for QuixBugs and generate patches using Java<->C# RTT. python quixbugs_plbart_round_nl.py # Generates input for QuixBugs and generate patches using Java<->NL RTT. </code></pre> <p>Optionally, use argument <code>--device_map cpu</code> if you wish to run the script on<br>CPU, for example:</p> <pre><code>python quixbugs_plbart_round.py --device_map cpu </code></pre> <p>Otherwise, the script will be run on all available CUDA GPU’s.</p> <p>We have commented the generation of inputs in the scripts. Users are free to<br>uncomment this method and try for themselves. It is easily recognizable by<br>their name template <code>[model]_[benchmark]_input()</code>. In the previous case:</p> <pre><code>quixbugs_plbart_input() </code></pre> <h3>Step 2 and 3: Round Trip Translation and Postprocessing</h3> <p>These steps are also included in the [benchmark]_[model]_[technique].py<br>script mentioned above. They are modularized in the method recognizable by<br>their name template [model]_[benchmark]_output().<br>For example:</p> <pre><code>quixbugs_incoder_output() </code></pre> <p>This method:</p> <ol> <li>Reads the input json file.</li> <li>Generates outputs through the LLM.</li> <li>Postprocess the output (extract the patch, clean up extra token, etc.).</li> <li>Creates [model]_output_[technique]_[extra].json.</li> </ol> <p>The last 3 steps are repeated according to the number of runs set to performed<br>(10 in our experiments). Each run will produce a different file with the seed<br>used in its generation. For example, <code>quixbugs\_plbart\_round.py</code> and<br><code>quixbugs\_plbart\_round_nl.py</code> scripts create:</p> <pre><code>clm-apr/quixbugs/plbart_results/run_0/plbart_java_cs_java_output_round_csharp_batch.json clm-apr/quixbugs/plbart_results/run_0/plbart_java_nl_java_output_round_nl_batch.json </code></pre> <h3>Step 4: Evaluation of RTT Results:</h3> <p>The last step evaluates the generated outputs against the test-suites of each<br>benchmark. This script reads the previous outputs files and generates a new one<br>with the results of the test for one model. Furthermore, it connects with the<br><em>WandB</em> tool to calculate metrics and send them to analyze.</p> <p>Following the previous examples, to validate the results previously obtained,<br>we execute the following:</p> <pre><code>cd clm-apr/quixbugs python validate_quixbugs_parallel.py </code></pre> <p>Given the included JSON, this script would create:</p> <pre><code>clm-apr/quixbugs/plbart_results/run_0/plbart_java_cs_java_validate_round_csharp_batch.json </code></pre> <p>We have disabled <em>WandB</em> in the script to allow users to try the script first.<br>However, it can be easily activated by changing the parameter <code>mode="disabled"</code><br>to <code>mode="online"</code>.<br>We have set the variable <code>total_runs = 1</code>, as well as <code>input_file</code> and <code>output_file</code><br>to the results included. They should be modified accordingly to validate more runs<br>or to validate other files/models.</p> <h3>Included Results</h3> <p>We include two CSV files obtained through WandB.</p> <pre><code>'data_cleaned_grouped.csv': Aggregated metrics of the 25 outputs for all runs. 'full_data_all_runs.csv': All metrics for all outputs on all runs. </code></pre> <h2>Changelog</h2> <ul> <li>v1.0 - updates corresponding to the accepted version of the manuscript in TOSEM</li> <li>v0.1 - initial replication package corresponding to v1 of arXiv deposit: includes raw data, code, and example outputs.</li> </ul> <h2>References</h2> <p>Jiang, N.; Liu, K.; Lutellier, T.; and Tan, L. 2023. Impact of Code Language<br>Models on Automated Program Repair. In 45th International Conference on<br>Software Engineering (ICSE), 1430–1442. IEEE. ISBN 978-1-66545-701-9.</p> <div> </div>
Replication package for technical lag analysis for the JSEP journal.
<p>This is the replication package for our article "A Formal Framework for Measuring Technical Lag in Component Repositories --- and its Application to npm" submitted for the JSEP journal in 2018.</p> <p>This replication package requires Python 3.5+ to be installed, and all the dependencies listed in ``requirements.txt``.<br> They can be automatically installed using ``pip install -r requirements.txt``. <br> These experiment were executed on a Linux Ubuntu OS.</p> <p>To obtain the analysis used in the paper, one should execute ``jupyter notebook`` at the root of this replication package, and open the notebook contained in ``notebooks``.</p> <p>This replication package contains three folders (i.e scripts, notebooks and data), each folder has a README with a description of what it contains.</p> <p>The list of all npm package releases and Github repositories with their dependencies was download from the last available dataset of libraries.io: https://zenodo.org/record/1196312</p> <p>The data is under the Creative Commons Attribution Share-Alike 4.0 license.<br> The source code is under the GNU General Public License.</p> <p><br> For any more information about the details of these experiments, please contact: <strong><a href="mailto:ahmed.zerouali73@gmail.com">ahmed.zerouali73@gmail.com</a></strong></p>
Replication package for: Solidarity and Fairness in Times of Crisis
<p>Replication package (code and data) for:</p> <blockquote> <p>Alexander W. Cappelen, Ranveig Falch, Erik Ø. Sørensen and Bertil Tungodden (2021). Solidarity and fairness in times of crisis. Journal of Economic Behavior & Organization 186: 1-11. <a href="https://doi.org/10.1016/j.jebo.2021.03.017">https://doi.org/10.1016/j.jebo.2021.03.017</a></p> </blockquote>
Studying Bug-Fixing Commits in the WoC Dataset: Replication Package
<p>A replication package for the MSR 2023 Challenge submission titled "Studying Bug-Fixing Commits in the WoC Dataset".</p>
Data set for the replication package of the paper "Constriction of actin rings by passive crosslinkers"
<p>Data set for the replication package of the paper "Constriction of actin rings by passive crosslinkers".</p>
Replication package for Decomposition of Monolithic Applications into Microservices Architectures: A Systematic Review
<p><strong>Replication Package</strong></p> <p><strong>Title:</strong></p> <p>Replication package for Decomposition of Monolithic Applications into Microservices Architectures: A Systematic Review.</p> <p><strong>Authors:</strong></p> <p>Yalemisew Abgaz, Andrew McCarren, Peter Elger, David Solan, Neil Lapuz, Marin Bivol, Glenn Jackson, Murat Yilmaz, Jim Buckley, and Paul Clarke</p> <p><strong>Year</strong></p> <p>This replication package was initially generated in 2022 and following feedback from reviewers, it is revised in 2023.</p> <p>This package contains two files and three folders that provide additional insight for researchers who wish to replicate our work or who would like to expand the review in the future.</p> <p><strong>Files</strong></p> <ul> <li>The file contains a detailed description of the literature search outlining the steps and the results obtained.</li> <li>Readme.md: A readme file (this file).</li> </ul> <p><strong>Folders</strong></p> <ul> <li>A folder containing the search results and the refinement steps. It contains four files listing studies included in the refinement process (Refinement_Step_1 to Refinement_Step_4) and a master file combining all the steps in one file. The master file contains detailed information about how the refinement steps are executed and all the intermediate results following each refinement step. Users may explore by expanding the filters in Refinement Step 2 (J), Refinement Step 3 (M) and Refinement Step 4 § columns in the master sheet. Readers can also directly go to the sheets that contain the selected studies in any of the refinement stages. A description of each file is also included in the Literature_Search_Strategy.pdf file.</li> <li>This folder contains the list of studies included in the snowballing process, including the last two refinement steps (Refinement_Step_5 and Refinement_Step_6). The snowballing master sheet contains studies extracted using the snowballing process and the data cleaning and filtering criteria used. The different sheets also contain the selected studies at each stage of the snowballing process.</li> </ul> <ul> <li>This folder contains the data extracted from the selected literature by employing the Systematic Review and Ground Theory. The two files included in this folder contain the data extraction template and the data extracted from the 35 selected studies including some intermediate notes.</li> </ul> <p>If you have further questions regarding the survey, feel free to contact us via e-mail. <a href="mailto:Yalemisewm.abgaz@dcu.ie">Yalemisewm.abgaz@dcu.ie</a>.</p> <p> </p>
Replication Package: Exploring the Relationship Between Personality Traits and User Feedback
<p>This is the replication package for the paper titled 'Automated User Feedback Analysis: Processing the Feedback Quantity and Quality' accepted for the AffectRE23 workshop track at RE 2023.</p> <p>Full abstract:</p> <p>Previous research has studied the impact of developer personality in different software engineering scenarios, such as team dynamics and programming education. However, little is known about how user personality affect software engineering, particularly user-developer collaboration. Along this line, we present a preliminary study about the effect of personality traits on user feedback. 56 university students provided feedback on different software features of an e-learning tool used in the course. They also filled out a questionnaire for the Five Factor Model (FFM) personality test. We observed some isolated effects of neuroticism on user feedback: most notably a significant correlation between neuroticism and feedback elaborateness; and between neuroticism and the rating of certain features. The results suggest that sensitivity to frustration and lower stress tolerance may negatively impact the feedback of users. This and possibly other personality characteristics should be considered when leveraging feedback analytics for software requirements engineering.</p>
Replication package for "Deliberate Surrender? The Impact of Interwar Indian Protection"
<p>This is a replication package for Deliberate Surrender? The Impact of Interwar Indian Protection, by Vellore Arthi, Markus Lampe, Ashwin Nair, Kevin Hjortshøj O’Rourke.</p>
spanichella/RP_EMSE_MCR_2019 v.1.0.1 Second release of of the replication Package for the paper "An Empirical Investigation of Relevant Changes and Automation Needs in Modern Code Review".
<p>Replication Package for the paper "An Empirical Investigation of Relevant Changes and Automation Needs in Modern Code Review"</p> <p>Structure</p> <pre><code>project_raw_data/ gerrit_review_comments.csv gerrit_review_changes.csv survey_raw_data/ google_forms_survey.pdf google_forms_survey.csv RQ1_taxonomy_mcr/ RQ1_inception_phase/ initial_taxonomy.pdf intermediate_taxonomy.pdf RQ1_definition_phase/ Q1.2_evaluation_survey.csv cram_classified.csv cram.pdf RQ2_automation_needs/ Q2.1-Q2.5_evaluation_survey.xlsx Q2.6-Q2.7_evaluation_survey.xlsx Q2.1-Q2.7_question_index.csv Now "RQ3_automated_support/" contain the results concerning RQ2.1 in the paper. content explained in the README.md file located in "RQ3_automated_support/README.md" </code></pre> <p>Contents of the Replication Package</p> <p><strong>project-raw-data/</strong> contains the data used for the creation of our taxonomies, it includes information about the ten open-source projects.</p> <ul> <li><code>gerrit_review_comments.csv</code> - information about all in-line review comments used for this paper</li> <li><code>gerrit_review_changes.csv</code> - information about all patches analyzed that contain the in-line comments</li> </ul> <p><strong>survey_raw_data/</strong> contains information about the survey conducted for the paper.</p> <ul> <li><code>google_forms_survey.pdf</code> - the distributed <em>Google Forms</em> of our survey</li> <li><code>google_forms_survey.csv</code> - all survey answers obtained from 52 survey participants</li> </ul> <p><strong>RQ1_taxonomy_mcr/</strong> contains information and data about the elicited taxonomies in our paper (RQ1).</p> <ul> <li><strong>RQ1_inception_phase/</strong> <ul> <li><code>initial_taxonomy.pdf</code> - initial taxonomy obtained in the inception phase of our paper</li> <li><code>intermediate_taxonomy.pdf</code> - intermediate taxonomy after integrating and merging the initial taxonomy with the one by Beller <em>et al</em> [1]</li> </ul> </li> <li><strong>RQ1_definition_phase/</strong> <ul> <li><code>Q1.2_evaluation_survey.csv</code> - relevant survey feedback with additional taxonomy categories integrated into <em>CRAM</em></li> <li><code>cram_classified.csv</code> - classified review comments (<code>gerrit_review_comments.csv</code>) into <em>CRAM</em></li> <li><code>cram.pdf</code> - <em>CRAM</em> taxonomy</li> <li><code>cram_classified_with_frequency_information2020.xls</code> - it contain the information used to compute the frequency of CRAM changes, derived by the analysis of the 211 commits</li> </ul> </li> </ul> <p><strong>RQ2_automation_needs/</strong> contains the encoded evaluation of the survey question Q2.1-2.7 for RQ2</p> <ul> <li><code>Q2.1-Q2.5_evaluation_survey.xlsx</code> - the encoded evaluation of the survey questions Q2.1-Q2.5 (used for the <em>Automation Needs</em> Section in the paper) and contains the following sheets: <ul> <li><strong>all Findings</strong>: Very detailed findings matrix distilled from all answers concerning possible automated solutions or general possibilities to achieve automation in MCR. Every feedback was analyzed and decomposed into single findings. These findings are grouped, into categories of our Taxonomy of Code Changes in MCR (CRAM). Red represent in the feedback-text where the corresponding category was distilled from.</li> <li><strong>unique Findings</strong>: As one participant could mention the same categories/solutions in multiple feedbacks, the following matrix is cleaned of any duplication of participant answers. Multiple feedbacks containing the same information by one participant were removed, leaving only distinct occurances.</li> <li><strong>aggregated by Solution</strong>: Aggregated feeback clustered into abstracted solutions and the number of times participants mentioned the solution.</li> <li><strong>aggregated by Taxonomy</strong>: Aggregated feeback grouped by low-level categoried in CRAM.</li> </ul> </li> <li><code>Q2.6-Q2.7_evaluation_survey.xlsx</code> - the encoded evaluation of the survey questions Q2.6-Q2.7 (used for the <em>Automation Needs</em> Section in the paper) and contains the following sheets: <ul> <li><strong>all Findings</strong>: Very detailed findings matrix distilled from all answers concerning possible techniques, approaches and data to achieve automation in MCR. Every feedback was analyzed and decomposed into single findings. These findings are grouped, into categories of our Taxonomy of Code Changes in MCR (CRAM). Red represent in the feedback-text where the corresponding category was distilled from.</li> <li><strong>unique Findings</strong>: As one participant could mention the same categories/solutions in multiple feedbacks, the following matrix is cleaned of any duplication of participant answers. Multiple feedbacks containing the same information by one participant were removed, leaving only distinct occurances.</li> <li><strong>aggregated by low-level taxonomy</strong>: Aggregated mentionings of approaches/data by developers in the survey grouped by low-level taxonomy category.</li> <li><strong>aggregated by high-level taxonomy</strong>: Aggregated mentionings of approaches/data by developers in the survey grouped by high-level taxonomy category.</li> </ul> </li> <li><code>Q2.1-Q2.7_question_index.csv</code> - table of IDs given to each participant-question pair for Q2.1-Q2.7 in order to trace back the feeback.</li> <li><code>cram_survey-with_criticality_and_feasibility2020.xls</code> and <code>cram_survey-with_relevance_and_completeness_information2020.xls</code>: they contain we results of the survey, involving 14 additional participants (12 developers and 2 researchers), not involved in the aforementioned survey, and performed to qualitatively assess the relevance and completeness of the identified MCR change types as well as assess how critical and feasible to implement are some of the identified techniques to support MCR activities.</li> </ul> <p><strong>RQ2_1_automated_support/</strong> (or <strong>RQ3_automated_support/</strong> )- content explained in the README.md file located in "RP_EMSE_MCR_2019/tree/master/EMSE_MCR_2019/RQ3_automated_support/README.md"</p> <p>References</p> <p>[1] Moritz Beller, Alberto Bacchelli, Andy Zaidman, and Elmar Juergens. 2014. Modern code reviews in open-source projects: which problems do they fix?. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR 2014). ACM, New York, NY, USA, 202-211. DOI: <a href="http://dx.doi.org/10.1145/2597073.2597082">http://dx.doi.org/10.1145/2597073.2597082</a></p>
Replication package of A benchmark-based evaluation of search-based crash reproduction
<p>Release of the reproduction package of Soltani, M., Derakhshanfar, P., Devroey, X. and van Deursen, A. (2020). A benchmark-based evaluation of search-based crash reproduction. In Empirical Software Engineering. 25, 1 (Jan. 2020), pp. 96–138.</p>
A Replication Package of Learning Features that Predict Developer Responses for iOS App Store Reviews
<p>This replication package contains the dataset and script used in our paper "<em>Learning Features that Predict Developer Responses for iOS App Store Reviews.</em>" The paper has been accepted at the ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), 2020. For further modification and versioning of the dataset (as well as the preprint) please go to <a href="https://github.com/Kamonphop/ESEM20-Replication">https://github.com/Kamonphop/ESEM20-Replication</a></p>
Replication package of "Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfalls, and Opportunities"
<p><strong>Abstract:</strong><br> Test smells attempt to capture design issues in test code that reduce their maintainability. Previous work found such smells to be highly common in automatically generated test-cases, but based this result on specific static detection rules; although these are based on the original definition of “test smells”, a recent empirical study showed that developers perceive these as overly strict and non-representative of the maintainability and quality of test suites. This leads us to investigate how effective such test smell detection tools are on automatically generated test suites. In this paper, we build a dataset of 2,340 test cases automatically generated by EVOSUITE for 100 Java classes. We performed a multi-stage, cross-validated manual analysis to identify six types of test smells and label their instances. We benchmark the performance of two test smell detection tools: one widely used in prior work, and one recently introduced with the express goal to match developer perceptions of test smells. Our results show that these test smell detection strategies poorly characterized the issues in automatically generated test suites; the older tool’s detection strategies, especially, misclassified over 70% of test smells, both missing real instances (false negatives) and marking many smell-free tests as smelly (false positives). We identify common patterns in these tests that can be used to improve the tools, refine and update the definition of certain test smells, and highlight as of yet uncharacterized issues. Our findings suggest the need for (i) more appropriate metrics to match development practice; and (ii) more accurate detection strategies, to be evaluated primarily in industrial contexts.</p>
Replication Package: Assessing time-based and range-based strategies for commit assignment to releases
<p><strong>Abstract:</strong></p> <p>Release is a ubiquitous concept in software development, referring to grouping multiple independent changes into a deliverable piece of software. Mining releases can help developers understand the software evolution at coarse grain, identify which features were delivered or bugs were fixed, and pinpoint who contributed on a given release. A typical initial step of release mining consists of identifying which commits compose a given release. We could find two main strategies used in the literature to perform this task: time-based and range-based. Some release mining works recognize that those strategies are subject to misclassifications but do not quantify the impact of such a threat. This paper analyzed 13,419 releases and 1,414,997 commits from 100 relevant open source projects hosted at GitHub to assess both strategies in terms of precision and recall. We observed that, in general, the range-based strategy has superior results than the time-based strategy. Nevertheless, even when the range-based strategy is in place, some releases still show misclassifications. Thus, our paper also discusses some situations in which each strategy degrades, potentially leading to bias on the mining results if not adequately known and avoided.</p> <p><strong>Instructions:</strong></p> <p>Visit <a href="https://github.com/gems-uff/release-mining">https://github.com/gems-uff/release-mining</a> for instructions about how to use this dataset.</p> <p><strong>Files:</strong></p> <ul> <li>The <em>repos.tgz</em> contains our project <em>corpus </em>comprising 1,414,997 releases from 100 relevant open source projects.</li> <li>The <em>repos.sha1</em> contains the sha1 checksum of <em>repos.tgz</em></li> </ul> <p><strong>Disclaimer:</strong></p> <p>This replication package contains the source code of 100 relevant open source projects. Its purpose is to enable the replication of the study conducted in the paper "Assessing time-based and range-based strategies for commit assignment to releases." </p> <p>It is essential to check each project license before using the source code or any attached file for any other purposes besides replicating the study.</p> <p> </p>
Replication package for "Wikifying Software Artifacts"
<p>This replication package contains all data files necessary to verify and replicate our findings presented in the manuscript "Wikifying Software Artifacts".</p>
SURF: Replication Package for: "What Would Users Change in My App? Summarizing App Reviews for Recommending Software Changes"
<p>Description of the content of folder "SURF_replication_package": 1) "Experiment I" contains: a) the folder "summaries" which contains all the html summaries generated through SURF and browsed by study participants involved in the Experiment I. b) the folder "XMLreviews" which contains, for each of the apps involved in the Experiment I, the corresponding XML file containing all the collected reviews for that app. These xml files have been used as input files for the SURF tool for generating the summaries contained in the "summaries" folder c) "Experiment_I_results.xlsx" which contains all the answers to our survey collected from the Experiment I participants.</p> <p>2) "Experiment II" contains: a) the folder "summaries" which contains the two html summaries generated through SURF and browsed by study participants in the Experiment II. b) the folder "XMLreviews" which contains, for each of the two apps involved in the Experiment II, the corresponding XML file containing all the collected reviews for that app. These xml files have been used as input of the SURF tool for generating the summaries contained in the "summaries" folder. c) "Experiment_II_results.xlsx" which contains all the user feedbacks extracted/validated by survey participants in the two sub-experiments. d) "Experiment_II_survey_answers.xlsx" which contains all the answers to our survey collected in the Experiment II participants.</p> <p>3) "Survey.pdf" which contains the pdf version of the survey performed by the participants</p> <p>4) "SURF_tool.zip" contains: a) "SURF.jar", which contains the class files of a prototypical implementation of SURF b) "README.txt" which contains the instructions to run the SURF tool c) the "lib" folder, which contains all the java libraries needed for running SURF.</p>
Can Developers Prompt? A Controlled Experiment for Code Documentation Generation [Replication Package]
<h2>Artifact Summary</h2> <p>This repository contains the replication package for the paper 'Can Developers Prompt? A Controlled Experiment for Code Documentation Generation,' presented at the <em><a href="https://conf.researchr.org/home/icsme-2024" target="_blank" rel="noopener">40th IEEE International Conference on Software Maintenance and Evolution (ICSME'24)</a></em>.</p> <p>The purpose of the package is to facilitate the verification and reproduction of the study results. It provides all data of the controlled experiment, the developed <em>Visual Studio Code (VS Code)</em> extension, as well as the slides of the conference presentations.</p> <h2>Paper Abstract</h2> <p>Large language models (LLMs) bear great potential for automating tedious development tasks such as creating and maintaining code documentation. However, it is unclear to what extent developers can effectively prompt LLMs to create concise and useful documentation. We report on a controlled experiment with 20 professionals and 30 computer science students tasked with code documentation generation for two Python functions. The experimental group freely entered ad-hoc prompts in a ChatGPT-like extension of Visual Studio Code, while the control group executed a predefined few-shot prompt. Our results reveal that professionals and students were unaware of or unable to apply prompt engineering techniques. Especially students perceived the documentation produced from ad-hoc prompts as significantly less readable, less concise, and less helpful than documentation from prepared prompts. Some professionals produced higher quality documentation by just including the keyword Docstring in their ad-hoc prompts. While students desired more support in formulating prompts, professionals appreciated the flexibility of ad-hoc prompting. Participants in both groups rarely assessed the output as perfect. Instead, they understood the tools as support to iteratively refine the documentation. Further research is needed to understand which prompting skills and preferences developers have and which support they need for certain tasks.</p> <h2>References</h2> <p>The published paper is available on <a href="https://doi.org/10.1109/ICSME58944.2024.00058" target="_blank" rel="noopener">IEEE Xplore</a> and the preprint on <a href="https://doi.org/10.48550/arXiv.2408.00686" target="_blank" rel="noopener">arXiv</a>.</p>
An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects [Replication Package]
<h2>Artefact Summary</h2> <p>This repository contains the replication package for the paper 'An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects,' presented at the <em><a href="https://cyprusconferences.org/icsme2022/" target="_blank" rel="noopener">38th IEEE International Conference on Software Maintenance and Evolution (ICSME'22)</a></em>.</p> <p>The purpose of the package is to facilitate the verification and reproduction of the study results.<br>It provides all computational notebooks used to collect and analyse data, as well as the slides of the conference presentation.</p> <h2>Paper Abstract</h2> <p>[Background] In large open-source software projects, development knowledge is often fragmented across multiple artefacts and contributors such that individual stakeholders are generally unaware of the full breadth of the product features. However, users want to know what the software is capable of, while contributors need to know where to fix, update, and add features. [Objective] This work aims at understanding how feature knowledge is documented in GitHub projects and how it is linked (if at all) to the source code. [Method] We conducted an in-depth qualitative exploratory content analysis of 25 popular GitHub repositories that provided the documentation artefacts recommended by GitHub’s Community Standards indicator. We extracted strategies used to document software features in textual artefacts and which strategies were used to link the feature documentation with source code. [Results] We observed feature documentation in all studied projects in artefacts such as READMEs, wikis, and website resource files. However, the features were often described in an unstructured way. Additionally, tracing techniques to connect feature documentation and source code were rarely used. [Conclusions] Our results suggest a lacking (or a low-prioritised) feature documentation in open-source projects, little use of normalised structures, and a rare explicit referencing to source code. As a result, product feature traceability is likely to be very limited, and maintainability to suffer over time.</p> <h2>References</h2> <p>The published paper is available on <a href="https://doi.org/10.1109/ICSME55016.2022.00043" target="_blank" rel="noopener">IEEE Xplore</a> and the preprint on <a href="https://doi.org/10.48550/arXiv.2208.01317" target="_blank" rel="noopener">arXiv</a>.</p>
Replication Package for "Compatibility Issues in Deep Learning Systems: Problems and Opportunities"
<p>This dataset contains scripts and data used to generate relevant results for this paper. Detailed information and procedure to reproduce our results are described in README.md. </p> <p>code</p> <p>This folder contains two Python scripts: soextractor.py is used to extract 3,072 high-quality StackOverflow (SO) posts and soextractor_tags.py is used to extract the number of posts for the tags on SO. For detailed data collection criteria, please refer to Section 3.1 of our paper.</p> <p>DL compatibility issues.xlsx</p> <p>This file provides all the collected 3,072 issues, in which each line indicates whether the issue is a DL compatibility issue. Among them, 352 are DL compatibility issues. We also provide information on the library, stage, symptom, type, solution, root cause, and exception type for the DL compatibility issues. For the type CORE-TPL, we also provide backward-incompatible or forward-incompatible as well as API evolution patterns. For detailed manual classification of DL compatibility issues, please refer to Section 3.2 of our paper.</p> <p>Tool Survey.xlsx </p> <p>This file includes all the papers collected from the three top SE conferences (i.e., ICSE, FSE, and ASE) in recent five years (18-22). Each line of each sheet provides the following information: (a) Title, (b) Year, (c) Conference, and (d) Type. For the detailed paper collection procedure, please refer to Section 5 of our paper.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.