Replication package of "Good Things Come In Threes: Improving Search-based Crash Reproduction With Helper Objectives"

<p>The replication package for the study about using new helper objectives (MOHO) for crash reproduction. This study has been accepted at ASE 2020.</p> <p>&nbsp;</p> <p>Abstract:</p> <p>Evolutionary intelligence approaches have been successfully applied to assist developers during debugging by generating a test case reproducing reported crashes. These approaches use a single fitness function called&nbsp;<em>Crash Distance</em>&nbsp;to guide the search process toward reproducing a target crash. Despite the reported achievements, these approaches do not always successfully reproduce some crashes due to a lack of test diversity (premature convergence). In this study, we introduce a new approach, called&nbsp;<em>MO-HO</em>, that addresses this issue via multi-objectivization. In particular, we introduce two new Helper-Objectives for crash reproduction, namely&nbsp;<em>test length</em>&nbsp;(to minimize) and&nbsp;<em>method sequence diversity</em>&nbsp;(to maximize), in addition to&nbsp;<em>Crash Distance</em>.</p> <p>We assessed&nbsp;<em>MO-HO</em>&nbsp;using five multi-objective evolutionary algorithms (NSGA-II, SPEA2, PESA-II, MOEA/D, FEMO) on 124 hard-to-reproduce crashes stemming from open-source projects. Our results indicate that SPEA2 is the best-performing multi-objective algorithm for&nbsp;<em>MO-HO</em>.</p> <p>We evaluated this best-performing algorithm for&nbsp;<em>MO-HO</em>&nbsp;against the state-of-the-art: single-objective approach (Single-Objective Search) and decomposition-based multi-objectivization approach (<em>De-MO</em>). Our results show that&nbsp;<em>MO-HO</em>&nbsp;reproduces five crashes that cannot be reproduced by the current state-of-the-art. Besides,&nbsp;<em>MO-HO</em>&nbsp;improves the effectiveness (+10% and +8% in reproduction ratio) and the efficiency in 34.6% and 36% of crashes (i.e., significantly lower running time) compared to Single-Objective Search and&nbsp;<em>De-MO</em>, respectively. For some crashes, the improvements are very large, being up to +93.3% for reproduction ratio and -92% for the required running time.&nbsp;</p>