Revisiting Machine Learning based Test Case Prioritization for Continuous Integration

<p>This repository contains a replication package for a research paper submitted to the 45th International Conference on Software Engineering (https://conf.researchr.org/home/icse-2023). We provide our code, data, and result for the ease of replicating our experiments.</p> <p><strong>Code</strong></p> <ul> <li>In the&nbsp;<strong>collect_data</strong>&nbsp;subdirectory, scripts for constructing TCP datasets are provided. We do dependency analysis using Understand (https://www.scitools.com/), so please download the related tools in advance.</li> <li>In the<strong>&nbsp;rl</strong>&nbsp;subdirectory, we provide the python implementation for algorithms RL, COLEMAN, PPO2-PO, ACER-PA, PPO1-LI.</li> <li>In the&nbsp;<strong>supervised_learning</strong>&nbsp;subdirectory, we provide implementations for MART, RankNet, RankBoost, CA, L-MART, which mainly rely on Ranklib&nbsp;(https://sourceforge.net/p/lemur/wiki/RankLib/.). We also provide implementation for DeepOrder.</li> </ul> <p><strong>Data</strong></p> <ul> <li>The&nbsp;<strong>origin</strong>&nbsp;subdirectory contains the original datasets collected from github using our scripts, including 11 projects.</li> <li>The&nbsp;<strong>smote</strong>&nbsp;subdirectory contains the datasets pre-processed by SMOTE.</li> </ul> <p><strong>Result</strong></p> <ul> <li>Results for&nbsp;<strong>RQ1</strong>,&nbsp;<strong>RQ2</strong>,&nbsp;<strong>RQ3</strong>, and&nbsp;<strong>threats to validity</strong>&nbsp;are provided in the corresponding subdirectories. Scripts for plotting figures are also provided.</li> </ul> <p><strong>Reference</strong></p> <p>We adopt code from previous work</p> <p>Learning-to-Rank vs Ranking-to-Learn: Strategies for Regression Testing in Continuous Integration&nbsp;(https://dl.acm.org/doi/abs/10.1145/3377811.3380369) Github repository: https://github.com/icse20/RT-CI</p> <p>Reinforcement Learning for Test Case Prioritization&nbsp;(https://ieeexplore.ieee.org/abstract/document/9394799) Github repository: https://github.com/moji1/tp_rl</p> <p>DeepOrder: Deep Learning for Test Case Prioritization in Continuous Integration Testing&nbsp;(https://ieeexplore.ieee.org/abstract/document/9609187) Github repository: https://github.com/AizazSharif/DeepOrder-ICSME21</p> <p>A Multi-Armed Bandit Approach for Test Case Prioritization in Continuous Integration Environments&nbsp;(https://ieeexplore.ieee.org/abstract/document/9086053) Github repository: https://github.com/jacksonpradolima/coleman4hcs</p>