Porting DB12 to Python3: Analysis of the scores

<p><a href="https://github.com/DIRACGrid/DB12">DB12</a> has been originally conceived with Python2 to estimate the power of a given CPU to run HEP applications. However, since January 2020, Python2 is no <a href="https://www.python.org/doc/sunset-python-2/">longer maintained</a> and we decided to port the code to Python3, which contains several optimizations.</p> <p>In October 2021, we effectively ported DB12 to Python3.9, but the optimizations brought by the language generated discrepancies in the norm score, which is a critical component to evaluate the power of CPUs. We build an analysis tool to mitigate these discrepancies.</p> <p><strong>Current situation</strong></p> <p>The current analysis assess the impact of the changes on the norm score and propose 3 different solutions to resolve the issue:</p> <ul> <li>Find a constant value that would transform a python3 score into a python2 one: 1.18 seems fine. <ul> <li>Pros: simple</li> <li>Cons: not accurate. does fit well with scores computed on Intel, not so well with scores computed on AMD</li> </ul> </li> <li>Find one constant value per processor type (Intel/AMD): 1.16 fits well with scores computed on Intel, 1.4 also fits well with scores computed on AMD . <ul> <li>Pros: accurate</li> <li>Cons: need to maintain a table and update it with new types of processors, and maybe also with new versions of python</li> </ul> </li> <li>Compute a simple linear regression: <ul> <li>Pros: accurate</li> <li>Cons: need to run it with many examples to get an accurate model</li> </ul> </li> </ul> <p>To keep it simple and accurate, we chose to apply the second solution: one constant value per processor type.<br> These constants are part of the code and are located in <em>src/db12/factors.json</em>.</p> <p>These values will need to be updated through time, according to the evolution of the CPUs and Python.<br> If you need to get an accurate DB12 norm score using a Python version or a CPU that has not be taken into account,<br> then you have to run the analysis with new data following the next steps.</p> <p><strong>Run the analysis</strong></p> <p>Execute the Jupyter Notebook:</p> <pre><code class="language-bash">jupyter notebook DB12Analysis.ipynb</code></pre> <p>&nbsp;</p> <p><strong>Include new data using DIRAC</strong></p> <ul> <li>Install a DIRAC client:</li> </ul> <pre><code class="language-bash">lb-dirac lhcb-proxy-init</code></pre> <ul> <li>Go to <em>resources/tools</em> and submit jobs:</li> </ul> <pre><code class="language-bash">./submit.sh &lt;number of jobs&gt; &lt;list of sites&gt;</code></pre> <ul> <li>Once the jobs are done, add their IDs in `jobIDs.csv` to get the results:</li> </ul> <pre><code class="language-bash">python getDB12Scores.py</code></pre> <ul> <li>You will obtain `results.json`, you can sort the file to get a better look at it with `jq`:</li> </ul> <pre><code class="language-bash">cat results.json | jq . &gt; results_sorted_&lt;date&gt;.json</code></pre> <ul> <li>You can finally remove the jobs that are still pending in the queues</li> </ul>