Artifacts for [On Rank Aggregating Test Prioritizations]

<p>Artifacts for [On Rank Aggregating Test Prioritizations]<br> =====================================================</p> <pre><code>username: entp password: entp root password: entp</code></pre> <p>Directory structure for [/home/entp/<strong>EnTP</strong>]<br> -----------------------------------------------------------------<br> [<strong>benchmarks</strong>] -&gt; contains the projects/subjects under test used in our study.</p> <p>[<strong>cost_cov_diff</strong>] -&gt; pre-recorded cost (cachegrind&#39;s I-Ref count) per test-case. We recorded this precporcessing to avoid fluctuations in costs across different systems, and hence maintain uniformity.</p> <p>[<strong>raw_data_scripts</strong>] -&gt; contains C++ implementation of EnTP, and scripts to generate results (tables, boxplots, .csv, .txt, etc.)</p> <p>[<strong>exp_res_raw</strong>] -&gt; contains pre-recorded experiments results (as .csv files) as reported in the paper. To generate the plots, run [python3 plot.py] from {/home/entp/EnTP/exp_res_raw}. The plots will be generated in .eps format. These precomputed results helps the artifact reviewer save time by generating the plots only, otherwise the whole process (detailed in Makefile&#39;s usage) generates a [database] directory of ~400GB (all benchmarks, all combinations, all results even beyond those reported in the paper, + some extra logs) which is very time consuming (~1 month on a standard laptop with 8 cores, 8GB RAM).</p> <p><strong>Makefile</strong>&#39;s usage<br> ---------------------------<br> [Step 1]</p> <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_[benchmark]</code></pre> <p>Possible values of [benchmark] = {c4, gravity, mlisp, replace, schedule2, space, xc, cf, grep, printtokens, scd, sed, tcas, xxhash, flex, gzip, printtokens2, schedule, slre, totinfo}.</p> <p>example:</p> <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_slre</code></pre> <p>executes EnTP and state-of-the-arts on the benchmark &quot;slre&quot;. Please follow log messages displayed after executing the above command. At the end of the execution, a sub-diretory named [<strong>database</strong>] will store the results for all experiments performed on &quot;slre&quot; for the current system and environment.<br> ...<br> [<em>database/{100, 75, 50, 25}</em>] -&gt; contains experimental results for the consensus budget of top-{100%, 75%, 50%, 25%}.<br> ...</p> <p>(optional)</p> <pre><code class="language-bash">entp@entp:~/EnTP$ make -s entp_all_[benchmark] #other benchmarks</code></pre> <p>[Step 2]</p> <pre><code class="language-bash">entp@entp:~/EnTP$ make -s generate_tabs</code></pre> <p>This will collect data from the newly generated directories and results (at the end of previous step), and generate tables, and .eps plots.</p> <p>Check for results reproduced<br> ---------------------------------------------<br> You can visually compare the .eps plots in {<strong><em>/home/entp/EnTP</em></strong>} with the ones generated in the directory {<strong><em>/home/entp/EnTP/exp_res_raw</em></strong>}.<br> You can also compare the .csv files under these directories&nbsp;for quantitative comparison with some tolerance.</p> <p>(optional)</p> <pre><code class="language-bash">entp@entp:~/EnTP$ make -s destroy_all</code></pre> <p>Cleans up everything!</p>