Automatically fixing dependency breaking changes

<h1>Replication Data for Automated Dependency Update Experiments</h1> <h2>Dataset Description</h2> <p>This repository contains a comprehensive collection of data from experiments on automated dependency updates using Large Language Models (LLMs). The dataset consists of two main components:</p> <ol> <li><strong>Execution Traces</strong>: Detailed logs of LLM interactions and repair attempts, captured using OpenTelemetry and OpenInference (<a href="https://github.com/Arize-ai/openinference">https://github.com/Arize-ai/openinference</a>). These traces provide insights into the behavior and performance of various LLMs in addressing breaking changes caused by dependency updates in Java projects.</li> <li><strong>Docker Images</strong>: Pre-built Docker images containing the state of Java projects after successful repair attempts. These images allow for direct inspection and verification of the changes made by our automated repair system.</li> </ol> <p>This dataset enables analysis and replication of our study on automated dependency updates, offering both low-level interaction data and high-level repair outcomes.</p> <h2>Data Components</h2> <h3>1. Execution Traces</h3> <ul> <li><strong>Format</strong>: JSON Lines, persisted via a custom OpenTelemetry exporter</li> <li><strong>Content</strong>: Spans capturing various aspects of LLM interactions, including: <ul> <li>Input processing</li> <li>LLM query generation</li> <li>LLM response parsing</li> <li>Code modification attempts</li> <li>Compilation and test execution results</li> </ul> </li> <li><strong>Purpose</strong>: Enables detailed analysis of LLM decision-making processes and performance metrics</li> </ul> <h3>2. Docker Images</h3> <ul> <li><strong>Format</strong>: Compressed Docker image files (.tar.gz) in a .zip</li> <li><strong>Content</strong>: Maven project states after successful repair attempts with patch(es) applied and seperated in the second layer of each image.</li> <li><strong>Purpose</strong>: Allows direct inspection and verification of code changes made during repairs and the successful maven outcome</li> </ul> <h2>Research Context and Data Usage</h2> <p>This dataset corresponds to experiments described in our study investigating the effectiveness of zero-shot prompting and agentic approaches in automating dependency updates. It provides valuable insights into:</p> <ul> <li>Performance variations across different LLMs</li> <li>Influence of various factors on repair success rates</li> <li>Specific code changes made during successful repairs</li> </ul> <p>This dataset can be used for:</p> <ol> <li>Replicating the experimental results presented in the associated study</li> <li>Conducting further analysis on LLM behavior in software engineering tasks</li> <li>Developing and benchmarking new approaches for automated dependency updates</li> <li>Investigating the decision-making processes of LLMs in code modification tasks</li> <li>Verifying and inspecting successful repair attempts through Docker images</li> </ol> <p>&nbsp;</p> <div> <h2>Repair Replication with Docker</h2> To replicate specific repairs using our pre-built Docker images:<br> <div>1. Ensure you have Docker installed on your system.</div> <br>2. Use the included docker-images.zip, which first has to be unzipped&nbsp; <div> <pre><code>unzip docker-images.zip</code></pre> </div> <div>after which the images can be loaded as follows:</div> <div> <pre><code>docker load &lt; image_name.tar.gz</code></pre> </div> <br> <div>3. Run the Docker container:</div> <pre><code>docker run -it [image-name]</code></pre> <br> <div>These Docker images contain the state of projects after repair attempts, allowing for easy inspection and verification of the changes made by our automated repair system.</div> </div>