The Artifacts of OOPSLA-2023 Submission #197

<p>This is the online repository of the OOPSLA-2023 Submission #197. We have released the source code and dataset.</p> <ul> <li><strong>Dataset</strong>: Our evaluation is based on the large-scale CodeSearchNet dataset. Use the following command to download and preprocess the data:</li> </ul> <pre><code class="language-bash">cd dataset bash run.sh cd ..</code></pre> <ul> <li> <p><strong>Dependencies</strong></p> </li> </ul> <pre><code class="language-bash">pip install -r requirements.txt</code></pre> <p>&nbsp; &nbsp; &nbsp; [Optional] We have built the tree-sitter parser stored at `evaluator/CodeBLEU/parser/languages.so`. If it doesn&#39;t work for you, it can be rebuilt with the following command:</p> <pre><code class="language-bash">cd evaluator/CodeBLEU/parser bash build.sh</code></pre> <ul> <li> <p><strong>Training</strong></p> </li> </ul> <pre><code class="language-bash">bash sh/train.sh [python/java] [CodeT5/Natgen] [CodeBERT/GraphCodeBERT]</code></pre> <ul> <li> <p><strong>Evaluation</strong></p> </li> </ul> <p>&nbsp; &nbsp; &nbsp; Evaluate generator:</p> <pre><code class="language-bash">bash sh/evaluate.sh [python/java] [CodeT5/Natgen]</code></pre> <p>&nbsp; &nbsp; &nbsp; Evaluate discriminator:</p> <p>&nbsp; &nbsp; &nbsp; We evaluate the discriminator by reusing the code from <a href="https://github.com/microsoft/CodeBERT/tree/master/CodeBERT/codesearch">CodeBERT</a>&nbsp;and <a href="https://github.com/microsoft/CodeBERT/tree/master/GraphCodeBERT/codesearch">GraphCodeBERT</a>. According to the Evaluate section in the corresponding model&#39;s Readme, replace `model_name_or_path` with the discriminator that you want to evaluate.</p> <p>&nbsp;</p>