Artifacts for Code Coverage Profiler Validation with Debugging Support
<p><strong>Code Coverage Profiler Validation with Debugging Support</strong></p>
<p>Table of Contents</p>
<ul>
<li>Background</li>
<li>Install</li>
<li>Usage</li>
<li>Experiment</li>
<li>Violations</li>
<li>Bugs</li>
</ul>
<p><strong>Background</strong></p>
<p><strong>DeCov</strong> is the tool for code coverage profiler validation with debugging support. We totally submit 24 bug reports for gcov and llvm-cov. Of these 24 bug reports, 18 are confirmed/fixed by developers, 2 are marked as duplicate, and the other 4 bug reports are waiting for the developers' confirmation.</p>
<p><strong>Install</strong></p>
<p>This project uses <a href="https://gcc.gnu.org/">gcc</a>, <a href="http://www.gnu.org/software/gdb/">gdb</a>, <a href="http://llvm.org/">llvm</a> and <a href="http://lldb.llvm.org/">lldb</a>. Go check them out if you don't have them locally installed.</p>
<p><code>apt install gcc gdb llvm lldb </code></p>
<p><strong>Usage</strong></p>
<p>To use DeCov for gcov validation:</p>
<p><code>python3 decov-gcov-pexpect.py -s test.c </code></p>
<p>or</p>
<p><code>python3 decov-gcov-pexpect.py -s testprograms </code></p>
<p>To use DeCov for llvm-cov validation:</p>
<p><code>python3 decov-llvmcov-pexpect.py -s test.c </code></p>
<p>or</p>
<p><code>python3 decov-llvmcov-pexpect.py -s testprograms </code></p>
<p><strong>Experiment</strong></p>
<p>First, download the gcc-10.1.0.tar.xz:</p>
<p><code><code>wget http://ftp.gnu.org/gnu/gcc/gcc-10.1.0/gcc-10.1.0.tar.xz </code></code></p>
<p>Then, unzip gcc-10.1.0:</p>
<p><code>tar -xvf gcc-10.1.0.tar.xz </code></p>
<p>Finally, run Decov over the testsuite of gcc-10.1.0:</p>
<p><code>python3 decov-gcov-pexpect.py -s gcc-10.1.0/gcc/testsuite/ </code></p>
<p>or</p>
<p><code>python3 decov-llvmcov-pexpect.py -s gcc-10.1.0/gcc/testsuite/ </code></p>
<p>Note that, we recommend to use the parallel option when using our tool if your computer has multi-cores:</p>
<p><code>python3 decov-gcov-pexpect.py --parallel -s gcc-10.1.0/gcc/testsuite/ </code></p>
<p>or</p>
<p><code>python3 decov-llvmcov-pexpect.py --parallel -s gcc-10.1.0/gcc/testsuite/ </code></p>
<p><strong>Violations</strong></p>
<p>Experimental data are in the Expr folder:</p>
<ul>
<li>files-DIFFCOV-gcc.txt and files-DIFFCOV-clang.txt list the filenames in the testsuites of GCC-10.1.0 that the coverage statistics from coverage profilers inconsistent with the hit count from debuggers</li>
<li>DIFFCOV-Gcov-vs-GDB.txt and DIFFCOV-LLVMcov-vs-LLDB.txt list the filesname and the the respected lines of code that violating <em>HMR</em>, coverage statistics is not consistent with the hit count</li>
<li>files-SAMECOV-gcc.txt and files-SAMECOV-clang.txt list the filenames in the testsuites of GCC-10.1.0 that the coverage statistics from coverage profilers consistent with the hit count from debuggers.</li>
<li>files-CPLERROR-gcc.txt and files-CPLERROR-clang.txt list the filenames in the testsuites of GCC-10.1.0 that cannot be correctly compiled independently by respected compilers</li>
</ul>
<p><strong>Bugs</strong></p>
<p>Bugs reported by <strong>DeCov</strong> for LLVM-cov:</p>
<p>llvm-cov 48818 Status History: "NEW"</p>
<p>llvm-cov 48795 Status History: "NEW"</p>
<p>llvm-cov 48278 Status History: "NEW" -> "DUPLICATE"</p>
<p>llvm-cov 48264 Status History: "NEW" -> "INVALID" -> "REOPENED"</p>
<p>llvm-cov 48263 Status History: "NEW"</p>
<p>llvm-cov 45850 Status History: "New" -> "CONFIRMED"</p>
<p>llvm-cov 45849 Status History: "New" -> "FIXED" -> "REOPENED" -> "FIXED"</p>
<p>llvm-cov 45846 Status History: "NEW" -> "CONFIRMED"</p>
<p>llvm-cov 45845 Status History: "NEW" -> "CONFIRMED"</p>
<p>llvm-cov 45844 Status History: "NEW" -> "DUPLICATE"</p>
<p>llvm-cov 45821 Status History: "NEW" -> "CONFIRMED"</p>
<p>llvm-cov 45678 Status History: "NEW" -> "CONFIRMED"</p>
<p>llvm-cov 45595 Status History: "NEW" -> "CONFIRMED"</p>
<p>Bugs reported by <strong>DeCov</strong> for Gcov:</p>
<p>gcov 97925 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 97924 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 97923 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 97917 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 97910 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 93757 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 93735 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 93726 Status History: "UNCONFIRMED" -> "WAIT"</p>
<p>gcov 93725 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 93706 Status History: "UNCONFIRMED" -> "NEW"</p>
<p>gcov 93680 Status History: "UNCONFIRMED" -> "NEW"</p>
restrictedSep 2021View details →
