Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
50
datasets available to search
ShareScore release 0.9.0
Dataset results
50 results for “language translation”
HornMT – Machine Translation Benchmark Dataset for Languages in the Horn of Africa
<p>The <strong>HornMT</strong> repository contains data and the associated metadata for the project <a href="https://lesan.ai/benchmark">Machine Translation Benchmark Dataset for Languages in the Horn of Africa</a>. It is a multi-way parallel corpus that will serve as a benchmark to accelerate progress in machine translation research and production systems for languages in the Horn of Africa.</p> <p>Supported Languages</p> <table> <tbody> <tr> <td> <p>Language</p> </td> <td> <p>ISO 639-3 code</p> </td> </tr> </tbody> <tbody> <tr> <td> <p>Afar</p> </td> <td> <p>aaf</p> </td> </tr> <tr> <td> <p>Amharic</p> </td> <td> <p>amh</p> </td> </tr> <tr> <td> <p>English</p> </td> <td> <p>eng</p> </td> </tr> <tr> <td> <p>Oromo</p> </td> <td> <p>orm</p> </td> </tr> <tr> <td> <p>Somali</p> </td> <td> <p>som</p> </td> </tr> <tr> <td> <p>Tigrinya</p> </td> <td> <p>tir</p> </td> </tr> </tbody> </table> <p><strong> </strong></p> <p>data/ contains one text file per language and each file contains news snippets in the same order for each language.</p> <p>data<br> ├── aar.txt<br> ├── amh.txt<br> ├── eng.txt<br> ├── orm.txt<br> ├── som.txt<br> └── tir.txt</p> <p>metadata.tsv contains tab separated data describing each news snippet. The metadata contains the following fields.</p> <ul> <li> <p><strong>Scope</strong> - describes whether the news is global or local. It takes two values: Global news and Local news.</p> </li> <li> <p><strong>Category</strong> - News category covering the following 12 topics</p> <ul> <li> <p>Art and Culture</p> </li> <li> <p>Business and Economy</p> </li> <li> <p>Conflicts and Attacks</p> </li> <li> <p>Disaster and Accidents</p> </li> <li> <p>Entertainment</p> </li> <li> <p>Environment</p> </li> <li> <p>Health</p> </li> <li> <p>International Relations</p> </li> <li> <p>Law and Crime</p> </li> <li> <p>Politics</p> </li> <li> <p>Science and Technology</p> </li> <li> <p>Sport</p> </li> </ul> </li> <li> <p><strong>Source</strong> - List of one or more URLs from which the news content is extracted or based on.</p> </li> <li> <p><strong>Domain</strong> - TLD corresponding to the URL(s) in Source.</p> </li> <li> <p><strong>Date</strong> - The publication date of the source article. The format is yyyy-mm-dd.</p> </li> </ul> <p>Other formats</p> <p>All the data and associated metadata together in one file is also available in other file formats.</p> <p><strong>HornMT.xlsx</strong> - data and associated metadata in xlsx format.</p> <p><strong>HornMT.json</strong> - data and associated metadata in json format.</p> <p>Below is an example row.</p> <pre><code class="language-javascript">{ "data":{ "eng":"The World Meteorological Organisation reports that the ozone layer is damaged to its worst extent ever in the Arctic.", "aaf":"Baad Metrolojih Eglali Areketekeh Addal Ozonih qelu faxe waktik lafetle calat biyakisem xayose.", "amh":"የአለም የአየር ንብረት ድርጅት በአርክቲክ አካባቢ ያለው የኦዞን ምንጣፍ ከፍተኛ ጉዳት እንደደረሰበት አስታወቀ፡፡", "orm":"Dhaabbanni Meetiroolojii Addunyaa baqqaanni oozonii Arkiitik keessatti gara sadarkaa isa hamaa haga ammaatti akka miidhame gabaase.", "som":"Ururka Saadaasha Hawada Adduunka ayaa ku warramaya in lakabka ozoneka ee Ka koreeya dhulka baraflayda uu waxyeelladii abid ugu darnaa soo gaadhay.", "tir":"ውድብ ሜትሮሎጂ ዓለም ኣብ ኣርክቲክ ዝርከብ ናሕሲ ኦዞን ኣዝዩ ብዝኸፍአ ደረጃ ከምዝተጎድአ ሓቢሩ፡፡" }, "metadata":{ "scope":"Global", "category":"Science and Technology", "source":"https://www.independent.co.uk/environment/climate-change/ozone-layer-damaged-by-unusually-harsh-winter-2263653.html", "domain":"www.independent.co.uk", "date":"2011-04-05" } }</code></pre> <p><strong>Team</strong></p> <p>Afar</p> <ul> <li> <p>Mohammed Deresa</p> </li> <li> <p>Yasin Nur</p> </li> </ul> <p>Amharic</p> <ul> <li> <p>Tigist Taye</p> </li> <li> <p>Selamawit Hailemariam</p> </li> <li> <p>Wako Tilahun</p> </li> </ul> <p>Oromo</p> <ul> <li> <p>Gemechis Melkamu</p> </li> <li> <p>Galata Girmaye</p> </li> </ul> <p>Somali</p> <ul> <li> <p>Abdiselam Mohamed</p> </li> <li> <p>Beshir Abdi</p> </li> </ul> <p>Tigrinya</p> <ul> <li> <p>Berhanu Abadi Weldegiorgis</p> </li> <li> <p>Michael Minassie</p> </li> <li> <p>Nureddin Mohammedshiek</p> </li> </ul> <p><strong>Project Leaders</strong></p> <ul> <li> <p>Asmelash Teka Hadgu <a href="mailto:asme@lesan.ai">asme@lesan.ai</a></p> </li> <li> <p>Gebrekirstos G. Gebremeskel <a href="mailto:gebrekirstos.gebremeskel@ru.nl">gebrekirstos.gebremeskel@ru.nl</a></p> </li> <li> <p>Abel Aregawi <a href="mailto:abel@lesan.ai">abel@lesan.ai</a></p> </li> </ul> <p><strong>License</strong></p> <p>Shield: <a href="http://creativecommons.org/licenses/by/4.0/">CC BY 4.0</a></p> <p>This work is licensed under a<br> <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p>
speaker populations of the languages targeted by translations of COVID 19 preventive measures
<p>The data are based on the list of the languages listed on the repository of the Endangered Languages Project (https://endangeredlanguagesproject.github.io/COVID-19) and on the figures found on ethnologue.org</p> <p> </p> <p> </p>
Replication package for "Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation"
<p>This repository contains the replication package for the paper "Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation" by Fernando Vallecillos Ruiz, Anastasiia Grishina, Max Hort and Leon Moonen, accepted for publication in ACM Transactions on Software Engineering and Methodology on 2025-10-09.</p> <p>A preprint is deposited on arXiv with DOI: <a href="https://doi.org/10.48550/arXiv.2401.07994">10.48550/arXiv.2401.07994</a>.</p> <p>The replication package is archived on Zenodo with DOI: <a href="https://doi.org/10.5281/zenodo.10500593">10.5281/zenodo.10500593</a>. It is maintained on GitHub at <a href="https://github.com/secureIT-project/RTT_for_APR">https://github.com/secureIT-project/RTT_for_APR</a>.</p> <p>This project builds on code from the <a href="https://github.com/lin-tan/clm/">clm</a> project, which is (c) 2023, The ASSET research group led by Lin Tan, Purdue University, licensed under the BSD 3-Clause License (see jasper/LICENSE.BSD). All modifications and new contributions are (c) 2025 by the authors of this replication package and distributed under the MIT License (see LICENSE.MIT). The data, models and preprint are distributed under the CC BY 4.0 license.</p> <h2>Citation<code> </code></h2> <p>If you build on this data or code, please cite this work by referring to the paper:</p> <div> <pre><code>@article{ruiz2025:rtt, title = {Assessing the Latent Automated Program Repair Capabilities of Large Language Models using Round-Trip Translation}, author = {Vallecillos Ruiz, Fernando and Anastasiia Grishina and Max Hort and Leon Moonen}, journal = {ACM Transactions on Software Engineering and Methodology (TOSEM)}, year = {2025}, publisher = {{ACM}} }</code></pre> </div> <h2>Organization</h2> <p>The replication package is organized as follows:</p> <ul> <li>clm-apr <ul> <li>plbart: code to generate patches with PLBART models.</li> <li>codet5: code to generate patches with CodeT5 models.</li> <li>transcoder: code to generate patches with the TransCoder model.</li> <li>incoder: code to generate patches with InCoder models.</li> <li>santacoder: code to generate patches with the SantaCoder model.</li> <li>starcoder: code to generate patches with the StarCoderBase model.</li> <li>quixbugs: code to validate patches generated for the QuixBugs benchmark.</li> <li>defects4j: code to validate patches generated for any of the Defects4J benchmarks.</li> <li>humaneval: code to validate patches generated for the HumanEval-Java benchmark.</li> </ul> </li> <li>humaneval-java: the HumanEval-Java benchmark proposed by Jiang et al. 2023</li> <li>jasper: a Java tool to parse Java programs needed to preprocess input.</li> <li>model: folder to download the language models.</li> <li>analysis_wandb: data from WandB and Jupyter notebook to create graphs.</li> <li>tmp_benchmarks: folder for temporary files used in patch validation. The folder may contain pairs of `paralell’ folders src and src_org for each benchmark, used to replace buggy code with candidate patches.</li> </ul> <h2>Replication</h2> <h3>Prerequisites</h3> <ul> <li>Python version: 3.8—3.10.</li> <li><a href="https://git-lfs.com/">Git LFS</a> is required for model downloading.</li> </ul> <h4>Weight and Biases (WandB)</h4> <ol> <li>Create an account on <a href="https://wandb.ai/">Weights and Biases</a></li> <li>Install the <a href="https://docs.wandb.ai/ref/python">Weights and Biases</a> library</li> <li>Run <code>wandb login</code> and follow the instructions</li> </ol> <h4>Set up OpenAI access</h4> <p>OpenAI account is needed with access to <code>gpt-3.5-turbo</code> and <code>gpt-4</code> . The <code>OPENAI_API_KEY</code> environment variable should be set to your OpenAI API access token.</p> <h3>Dependencies</h3> <ul> <li><a href="https://github.com/rjust/defects4j">Defects4J</a> - To generate inputs for the Defects4J datasets or to validate them, you need to have installed <a href="https://github.com/rjust/defects4j">their tool</a>.</li> <li>Java 8</li> <li>Apache Maven</li> </ul> <h3>Setup</h3> <p>We recommend the use of the setup script:</p> <pre><code>setup.sh </code></pre> <p>which performs the following:</p> <ol> <li>Creates a virtual environment for Python and activate it.</li> <li>Install the packages in <code>requirements.txt</code>.</li> <li>Compiles Jasper.</li> <li>Downloads parsers.</li> <li>Check if the Defects4J installation is correct.</li> </ol> <h3>Download models</h3> <p>The following bash script contains the code to download all of the models used:</p> <pre><code>models/download_models.sh </code></pre> <p>We recommend downloading only the models you are going to use due to their size</p> <pre><code>cd models chmod +x download_models.sh ./download_models.sh </code></pre> <p>To run one specific model, for example, PLBART (C#), use the following commands:</p> <pre><code>cd models git lfs install git clone https://huggingface.co/uclanlp/plbart-java-cs git clone https://huggingface.co/uclanlp/plbart-cs-java cd ../.. </code></pre> <h3>Step 1: Preprocessing and Prompting:</h3> <p>Each script in each <code>clm-apr/[model]</code> folder connects one or more models with<br>one dataset. These scripts follow the template: [benchmark]_[model]_[technique].py.<br>The scripts first create an <code>[model]_input.json</code> file with the preprocessed<br>input. Then generate outputs based on that file with one or more models.<br>For example:</p> <pre><code>cd clm-apr/plbart python quixbugs_plbart_round.py # Generates input for QuixBugs and generate patches using Java<->C# RTT. python quixbugs_plbart_round_nl.py # Generates input for QuixBugs and generate patches using Java<->NL RTT. </code></pre> <p>Optionally, use argument <code>--device_map cpu</code> if you wish to run the script on<br>CPU, for example:</p> <pre><code>python quixbugs_plbart_round.py --device_map cpu </code></pre> <p>Otherwise, the script will be run on all available CUDA GPU’s.</p> <p>We have commented the generation of inputs in the scripts. Users are free to<br>uncomment this method and try for themselves. It is easily recognizable by<br>their name template <code>[model]_[benchmark]_input()</code>. In the previous case:</p> <pre><code>quixbugs_plbart_input() </code></pre> <h3>Step 2 and 3: Round Trip Translation and Postprocessing</h3> <p>These steps are also included in the [benchmark]_[model]_[technique].py<br>script mentioned above. They are modularized in the method recognizable by<br>their name template [model]_[benchmark]_output().<br>For example:</p> <pre><code>quixbugs_incoder_output() </code></pre> <p>This method:</p> <ol> <li>Reads the input json file.</li> <li>Generates outputs through the LLM.</li> <li>Postprocess the output (extract the patch, clean up extra token, etc.).</li> <li>Creates [model]_output_[technique]_[extra].json.</li> </ol> <p>The last 3 steps are repeated according to the number of runs set to performed<br>(10 in our experiments). Each run will produce a different file with the seed<br>used in its generation. For example, <code>quixbugs\_plbart\_round.py</code> and<br><code>quixbugs\_plbart\_round_nl.py</code> scripts create:</p> <pre><code>clm-apr/quixbugs/plbart_results/run_0/plbart_java_cs_java_output_round_csharp_batch.json clm-apr/quixbugs/plbart_results/run_0/plbart_java_nl_java_output_round_nl_batch.json </code></pre> <h3>Step 4: Evaluation of RTT Results:</h3> <p>The last step evaluates the generated outputs against the test-suites of each<br>benchmark. This script reads the previous outputs files and generates a new one<br>with the results of the test for one model. Furthermore, it connects with the<br><em>WandB</em> tool to calculate metrics and send them to analyze.</p> <p>Following the previous examples, to validate the results previously obtained,<br>we execute the following:</p> <pre><code>cd clm-apr/quixbugs python validate_quixbugs_parallel.py </code></pre> <p>Given the included JSON, this script would create:</p> <pre><code>clm-apr/quixbugs/plbart_results/run_0/plbart_java_cs_java_validate_round_csharp_batch.json </code></pre> <p>We have disabled <em>WandB</em> in the script to allow users to try the script first.<br>However, it can be easily activated by changing the parameter <code>mode="disabled"</code><br>to <code>mode="online"</code>.<br>We have set the variable <code>total_runs = 1</code>, as well as <code>input_file</code> and <code>output_file</code><br>to the results included. They should be modified accordingly to validate more runs<br>or to validate other files/models.</p> <h3>Included Results</h3> <p>We include two CSV files obtained through WandB.</p> <pre><code>'data_cleaned_grouped.csv': Aggregated metrics of the 25 outputs for all runs. 'full_data_all_runs.csv': All metrics for all outputs on all runs. </code></pre> <h2>Changelog</h2> <ul> <li>v1.0 - updates corresponding to the accepted version of the manuscript in TOSEM</li> <li>v0.1 - initial replication package corresponding to v1 of arXiv deposit: includes raw data, code, and example outputs.</li> </ul> <h2>References</h2> <p>Jiang, N.; Liu, K.; Lutellier, T.; and Tan, L. 2023. Impact of Code Language<br>Models on Automated Program Repair. In 45th International Conference on<br>Software Engineering (ICSE), 1430–1442. IEEE. ISBN 978-1-66545-701-9.</p> <div> </div>
Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code
<p>Artifact repository for the paper <a href="http://arxiv.org/abs/2308.03109" rel="nofollow"><em>Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code</em></a>, accepted at <em>ICSE 2024</em>, Lisbon, Portugal. Authors are <a href="https://rangeetpan.github.io/" rel="nofollow">Rangeet Pan</a>* <a href="https://alirezai.cs.illinois.edu/" rel="nofollow">Ali Reza Ibrahimzada</a>*, <a href="http://rkrsn.us/" rel="nofollow">Rahul Krishna</a>, Divya Sankar, Lambert Pougeum Wassi, Michele Merler, Boris Sobolev, Raju Pavuluri, Saurabh Sinha, and <a href="https://reyhaneh.cs.illinois.edu/index.htm" rel="nofollow">Reyhaneh Jabbarvand</a>.</p> <h3>Install</h3> <p>This repository contains the source code for reproducing the results in our paper. Please start by cloning this repository:</p> <div> <pre><code>git clone https://github.com/Intelligent-CAT-Lab/PLTranslationEmpirical </code></pre> </div> <p>We recommend using a virtual environment for running the scripts. Please download <code>conda 23.11.0</code> from this <a href="https://docs.conda.io/projects/miniconda/en/latest/miniconda-other-installer-links.html" rel="nofollow">link</a>. You can create a virtual environment using the following command:</p> <div> <pre><code>conda create -n plempirical python=3.10.13 </code></pre> </div> <p>After creating the virtual environment, you can activate it using the following command:</p> <div> <pre><code>conda activate plempirical </code></pre> </div> <p>You can run the following command to make sure that you are using the correct version of Python:</p> <div> <pre><code>python3 --version && pip3 --version </code></pre> </div> <h3>Dependencies</h3> <p>To install all software dependencies, please execute the following command:</p> <div> <pre><code>pip3 install -r requirements.txt </code></pre> </div> <p>As for hardware dependencies, we used 16 NVIDIA A100 GPUs with 80GBs of memory for inferencing models. The models can be inferenced on any combination of GPUs as long as the reader can properly distribute the model weights across the GPUs. We did not perform weight distribution since we had enough memory (80 GB) per GPU.</p> <p>Moreover, for compiling and testing the generated translations, we used Python 3.10, g++ 11, GCC Clang 14.0, Java 11, Go 1.20, Rust 1.73, and .Net 7.0.14 for Python, C++, C, Java, Go, Rust, and C#, respectively. Overall, we recommend using a machine with Linux OS and at least 32GB of RAM for running the scripts.</p> <p>For running scripts of alternative approaches, you need to make sure you have installed <a href="https://github.com/immunant/c2rust">C2Rust</a>, <a href="https://github.com/gotranspile/cxgo">CxGO</a>, and <a href="https://github.com/paulirwin/JavaToCSharp">Java2C#</a> on your machine. Please refer to their repositories for installation instructions. For Java2C#, you need to create a <code>.csproj</code> file like below:</p> <div> <pre><code><Project Sdk="Microsoft.NET.Sdk"> <PropertyGroup> <OutputType>Exe</OutputType> <TargetFramework>net7.0</TargetFramework> <ImplicitUsings>enable</ImplicitUsings> <Nullable>enable</Nullable> </PropertyGroup> </Project> </code></pre> </div> <h3>Dataset</h3> <p>We uploaded the dataset we used in our empirical study to <a href="../doi/10.5281/zenodo.8190051" rel="nofollow">Zenodo</a>. The dataset is organized as follows:</p> <ol> <li><a href="https://github.com/IBM/Project_CodeNet">CodeNet</a></li> <li><a href="https://github.com/wasiahmad/AVATAR">AVATAR</a></li> <li><a href="https://github.com/evalplus/evalplus">Evalplus</a></li> <li><a href="https://github.com/apache/commons-cli">Apache Commons-CLI</a></li> <li><a href="https://github.com/pallets/click">Click</a></li> </ol> <p>Please download and unzip the <code>dataset.zip</code> file from Zenodo. After unzipping, you should see the following directory structure:</p> <div> <pre><code>PLTranslationEmpirical ├── dataset ├── codenet ├── avatar ├── evalplus ├── real-life-cli ├── ... </code></pre> </div> <p>The structure of each dataset is as follows:</p> <p>1. CodeNet & Avatar: Each directory in these datasets correspond to a source language where each include two directories <code>Code</code> and <code>TestCases</code> for code snippets and test cases, respectively. Each code snippet has an <code>id</code> in the filename, where the <code>id</code> is used as a prefix for test I/O files.</p> <p>2. Evalplus: The source language code snippets follow a similar structure as CodeNet and Avatar. However, as a one time effort, we manually created the test cases in the target Java language inside a maven project, <code>evalplus_java</code>. To evaluate the translations from an LLM, we recommend moving the generated Java code snippets to the <code>src/main/java</code> directory of the maven project and then running the command <code>mvn clean test surefire-report:report -Dmaven.test.failure.ignore=true</code> to compile, test, and generate reports for the translations.</p> <p>3. Real-life Projects: The <code>real-life-cli</code> directory represents two real-life CLI projects from Java and Python. These datasets only contain code snippets as files and no test cases. As mentioned in the paper, the authors manually evaluated the translations for these datasets.</p> <h3>Scripts</h3> <p>We provide bash scripts for reproducing our results in this work. First, we discuss the translation script. For doing translation with a model and dataset, first you need to create a <code>.env</code> file in the repository and add the following:</p> <div> <pre><code>OPENAI_API_KEY=<your openai api key> LLAMA2_AUTH_TOKEN=<your llama2 auth token from huggingface> STARCODER_AUTH_TOKEN=<your starcoder auth token from huggingface> </code></pre> </div> <p>1. Translation with GPT-4: You can run the following command to translate all <code>Python -> Java</code> code snippets in <code>codenet</code> dataset with the <code>GPT-4</code> while top-k sampling is <code>k=50</code>, top-p sampling is <code>p=0.95</code>, and <code>temperature=0.7</code>:</p> <div> <pre><code>bash scripts/translate.sh GPT-4 codenet Python Java 50 0.95 0.7 0 </code></pre> </div> <p>2. Translation with CodeGeeX: Prior to running the script, you need to clone the CodeGeeX repository from <a href="https://github.com/THUDM/CodeGeeX">here</a> and use the instructions from their artifacts to download their model weights. After cloning it inside <code>PLTranslationEmpirical</code> and downloading the model weights, your directory structure should be like the following:</p> <div> <pre><code>PLTranslationEmpirical ├── dataset ├── codenet ├── avatar ├── evalplus ├── real-life-cli ├── CodeGeeX ├── codegeex ├── codegeex_13b.pt # this file is the model weight ├── ... ├── ... </code></pre> </div> <p>You can run the following command to translate all <code>Python -> Java</code> code snippets in <code>codenet</code> dataset with the <code>CodeGeeX</code> while top-k sampling is <code>k=50</code>, top-p sampling is <code>p=0.95</code>, and <code>temperature=0.2</code> on GPU <code>gpu_id=0</code>:</p> <div> <pre><code>bash scripts/translate.sh CodeGeeX codenet Python Java 50 0.95 0.2 0 </code></pre> </div> <p>3. For all other models (StarCoder, CodeGen, LLaMa, TB-Airoboros, TB-Vicuna), you can execute the following command to translate all <code>Python -> Java</code> code snippets in <code>codenet</code> dataset with the <code>StarCoder|CodeGen|LLaMa|TB-Airoboros|TB-Vicuna</code> while top-k sampling is <code>k=50</code>, top-p sampling is <code>p=0.95</code>, and <code>temperature=0.2</code> on GPU <code>gpu_id=0</code>:</p> <div> <pre><code>bash scripts/translate.sh StarCoder codenet Python Java 50 0.95 0.2 0 </code></pre> </div> <p>4. For translating and testing pairs with traditional techniques (i.e., C2Rust, CxGO, Java2C#), you can run the following commands:</p> <div> <pre><code>bash scripts/translate_transpiler.sh codenet C Rust c2rust fix_report bash scripts/translate_transpiler.sh codenet C Go cxgo fix_reports bash scripts/translate_transpiler.sh codenet Java C# java2c# fix_reports bash scripts/translate_transpiler.sh avatar Java C# java2c# fix_reports </code></pre> </div> <p>5. For compile and testing of CodeNet, AVATAR, and Evalplus (Python to Java) translations from GPT-4, and generating fix reports, you can run the following commands:</p> <div> <pre><code>bash scripts/test_avatar.sh Python Java GPT-4 fix_reports 1 bash scripts/test_codenet.sh Python Java GPT-4 fix_reports 1 bash scripts/test_evalplus.sh Python Java GPT-4 fix_reports 1 </code></pre> </div> <p>6. For repairing unsuccessful translations of Java -> Python in CodeNet dataset with GPT-4, you can run the following commands:</p> <div> <pre><code>bash scripts/repair.sh GPT-4 codenet Python Java 50 0.95 0.7 0 1 compile bash scripts/repair.sh GPT-4 codenet Python Java 50 0.95 0.7 0 1 runtime bash scripts/repair.sh GPT-4 codenet Python Java 50 0.95 0.7 0 1 incorrect </code></pre> </div> <p>7. For cleaning translations of open-source LLMs (i.e., StarCoder) in codenet, you can run the following command:</p> <div> <pre><code>bash scripts/clean_generations.sh StarCoder codenet </code></pre> </div> <p>Please note that for the above commands, you can change the dataset and model name to execute the same thing for other datasets and models. Moreover, you can refer to <a href="https://github.com/Intelligent-CAT-Lab/PLTranslationEmpirical/blob/main/prompts/README.md"><code>/prompts</code></a> for different vanilla and repair prompts used in our study.</p> <h3>Artifacts</h3> <p>Please download the <code>artifacts.zip</code> file from our <a href="../doi/10.5281/zenodo.8190051" rel="nofollow">Zenodo</a> repository. We have organized the artifacts as follows:</p> <ol> <li>RQ1 - Translations: This directory contains the translations from all LLMs and for all datasets. We have added an excel file to show a detailed breakdown of the translation results.</li> <li>RQ2 - Manual Labeling: This directory contains an excel file which includes the manual labeling results for all translation bugs.</li> <li>RQ3 - Alternative Approaches: This directory contains the translations from all alternative approaches (i.e., C2Rust, CxGO, Java2C#). We have added an excel file to show a detailed breakdown of the translation results.</li> <li>RQ4 - Mitigating Translation Bugs: This directory contains the fix results of GPT-4, StarCoder, CodeGen, and Llama 2. We have added an excel file to show a detailed breakdown of the fix results.</li> </ol> <h3>Contact</h3> <p>We look forward to hearing your feedback. Please contact <a href="mailto:rangeet.pan@ibm.com">Rangeet Pan</a> or <a href="mailto:alirezai@illinois.edu">Ali Reza Ibrahimzada</a> for any questions or comments 🙏.</p>
Specification-Driven Code Translation By Large Language Models: How Far Are We?
<p>The artifacts and dataset for "Specification-Driven Code Translation By Large Language Models: How Far Are We?"</p>
Machine Assisted Translation of Wikipedia Articles into Low Resource Languages
<p><strong>Wikipedia is the largest encyclopedia ever assembled with the vision of enabling every human being to freely share in the sum of all knowledge. Wikipedia currently has a total of more than six million articles and over 17 billion words in its English edition. Unfortunately, millions of people cannot access this resource because it’s not available in their language. For instance, at the moment there are only 218 Tigrinya Wikipedia and 15,018 Amharic Wikipedia articles.</strong></p> <p><strong>In this project, we investigate the problem of translating Wikipedia articles from a high resource language into low resource languages using human-in-the-loop MT systems. In particular, we investigate different approaches to translate a sample of English Wikipedia articles into Tigrinya and Amharic. Currently, this repository contains 100k English Wikipeida abstracts translated using Lesan (https://lesan.ai) into Amharic and Tigrinya.</strong><br> <br> </p> <p><strong>Structure of data directory:</strong></p> <p><strong>data<br> ├── human<br> └── mt<br> ├── google<br> ├── lesan<br> │ ├── am.txt<br> │ ├── en.txt<br> │ └── ti.txt<br> └── microsoft</strong><br> </p>
Figure 1. The Structure of the Automatic Translate Voice to Sign Language Animation System-Development an Automatic Speech to Facial Animation Conversion for Improve Deaf Lives
<p>All technologies of voice recognition, speaker identification and verification, each has its<br> own advantages and disadvantages and may requires different treatments and techniques. The<br> choice of which technology to use is application-specific. At the highest level, all voice recognition<br> systems contain two main modules: feature extraction and feature matching. Feature extraction is<br> the process that extracts a small amount of data from the voice signal that can later be used to<br> represent each word. Feature matching involves the actual procedure to identify the unknown word<br> by comparing extracted features from his/her voice input with the ones from a set of known words.<br> A wide range of possibilities exist for parametrically representing the speech signal for the<br> voice recognition task, such as Linear Prediction Coding (LPC), RASTA-PLP and Mel-Frequency<br> Cepstrum Coefficients (MFCC).</p>
Lost in Translation? Not for Large Language Models: Automated Divergent Thinking Scoring Performance Translates to Non-English Contexts (Datasets)
<p>Datasets for: Zielińska, A., Organisciak, P., Dumas, D., & Karwowski, M. (2023). Lost in translation? Not for large language models: Automated divergent thinking scoring performance translates to non-English contexts. <em>Thinking Skills and Creativity, 50</em>, 101414. https://doi.org/10.1016/j.tsc.2023.101414</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 9b. Translation of a tab for all languages
<p>After checking a few (or all!) languages for instance and pressing the Ok button , we obtain the windows shown in figure 9a, or respectively 9b for all languages. Here we can add one or more missing translations, or modify one or more of the existing translations accordingly. All the data within the view cluster can be translated into any language supported by the system.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 8. GoTo -> Translation Option
<p>For example, we select in the view cluster the tab called “Forecasting” and then choose Goto -> Translation (figure 8). After selecting Goto -> Translation, we obtain a selecting window for the desired languages where the user can check one or more languages to translate those tabs or areas into. </p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 9a. Translation of a tab for certain languages
<p>After checking a few (or all!) languages for instance and pressing the Ok button , we obtain the windows shown in figure 9a, or respectively 9b for all languages. Here we can add one or more missing translations, or modify one or more of the existing translations accordingly. All the data within the view cluster can be translated into any language supported by the system.</p>
Systematic mapping data for translation-enabling technologies for sign languages
<p>These data correspond to the papers selected for a systematic mapping of that translation-enabling technologies for sign languages.</p>
Full Data Set of 16814 tweets from 17 different Twitter accounts of Catalonian Public Servants and Delegates collected between August 19 2013 and September 17 2017 in both original language and translated into Spanish later codified under para-diplomacy or nation branding macro-categories. Includes results for each year.
<p>Full Data Set of 16814 tweets from 17 different Twitter accounts of Catalonian Public Servants and Delegates collected between August 19 2013 and September 17 2017 in both original language and translated into Spanish later codified under para-diplomacy or nation branding macro-categories. Includes results for each year out of a significant sample of 1638 tweets.</p>
Thai Local Language Translation Dataset
<h1>Thai Local Language Translation Dataset</h1> <p>Thai Local Language Translation Dataset is a translation dataset for translate Thai Local Language to Thai Central Language. We create the dataset from <a href="https://github.com/SLSCU/thai-dialect-corpus" rel="nofollow">Thai Dialect Corpus (Thai dialects </a><a href="https://github.com/SLSCU/thai-dialect-corpus" rel="nofollow">ASR corpus</a><a href="https://github.com/SLSCU/thai-dialect-corpus" rel="nofollow">)</a>. We select train set only from Thai Dialect Corpus.</p> <p>The dataset support Khummuang, Korat, and Pattani.</p> <p> </p> <h2>Reference</h2> <p>Suwanbandit, A., Naowarat, B., Sangpetch, O., Chuangsuwanich, E. (2023) Thai Dialect Corpus and Transfer-based Curriculum Learning Investigation for Dialect Automatic Speech Recognition. Proc. INTERSPEECH 2023, 4069-4073, doi: 10.21437/Interspeech.2023-1828</p>
Lesan: Machine Translation for Low Resource Languages
<p>Human evaluation dataset to evaluate machine translation systems to and from Amharic, English and Tigrinya.</p>
Translation and validation of the Body-Mind-Spirit Wellness Behavior and Characteristic Inventory in the Greek language
<p>This investigation was guided by Hettler's 1979 Six Dimensions of Wellness Model. Dimensions of physical, emotional, intellectual, occupational, social, and spiritual wellness were identified. The Body-Mind-Spirit Wellness Behavior and Characteristic Inventory was used to assess this (BMS-WBCI). The purpose of this study was to further validate the BMS-WBCI by reporting reliability as the scale's internal consistency when used to measure wellness in a sample of aged Greek people. a descriptive cross-sectional design was used for the sample collection. A random sample (n =520) from the Greek general population participated voluntarily and anonymously in the study including 28,8% males and 71,2% females. Their mean age of the sample was 39.86 years (SD = 10.5), ranging from 16 to 75 years .The BMS-WBCI consists of 44 items which are divided into three subscales (Hey, Calderon, & Carroll, 2006). The first subscale is labeled “<strong>body</strong>” and consists of nine items (items 1 – 9) relating to risk behaviors including physical fitness, personal safety and dietary intake, covering the physical domain of wellness. The second subscale is labeled “<strong>mind</strong>” and consists of 20 items (items 10 – 29) representing the intellectual, social, emotional, and occupational domains of wellness. Finally, the third subscale is labeled “<strong>spirit</strong>” and consists of 15 items (items 30 – 44) covering the spiritual, emotional, and occupational domains of wellness. Each of the 44 items asks the participant to respond on a 3-point Likert scale where 1 indicates “rare/seldom,” 2 “occasionally/sometimes,” and 3 “often/always.” Possible scores for the entire scale range from a minimum of 44 to a maximum of 132. Higher scores designate a higher level of participation in positive health behaviors and agreement with characteristics that contribute to overall well-being. </p>
MDE language translation dataset
<p>This is the dataset for our SNCS paper "Improving the usability of MDE for software language processing"</p>
Survey Results for: MM-LDTF: Multi-Model Legal Document Translation Framework for Vernacular Languages
Open the record for dataset details and reuse information.
Language translation by example dataset
<p>Datasets and results for LTBE</p>
Datasets for "Exploring the potential of neural machine translation for cross-language clinical NLP resource generation through annotation projection"
<p>This repository contains the data and additional resources used for the paper:</p> <p>"Exploring the Potential of Neural Machine Translation for Cross-Language Clinical NLP Resource Generation through Annotation Projection. Rodriguez Miret et al. Information (2024)".</p> <p>There are four different datasets included, namely:</p> <ul> <li>The (1) <strong><a href="https://temu.bsc.es/distemist" target="_blank" rel="noopener">DisTEMIST</a>, </strong>(2) <strong> <a href="https://temu.bsc.es/multicardioner" target="_blank" rel="noopener">DrugTEMIST</a> </strong>and<strong> </strong>(3) <strong><a href="https://temu.bsc.es/meddoprof" target="_blank" rel="noopener">MEDDOPROF</a> Spanish corpora and corresponding versions in 10 different languages</strong>, created through Machine Translation and annotation projection techniques. The Catalan annotations, used in the paper's experiments, were validated by bilingual expert annotators, who also provided alternative translations for the annotated terms in case they were wrongly translated. Thus, for Catalan we provide two different versions of the data: (i) the output of the annotation projection process as is, without any further validation, and (ii) the validated version of the data. For the rest of the languages (with the exception of the DrugTEMIST English and Italian data, used for the MultiCardioNER shared task), only an unvalidated version is provided.<strong><br></strong></li> <li>The (4) <strong>Catalan Clinical Case Corpus (CataCCC)</strong>, a collection of <em>200 clinical case reports in originally written in Catalan </em>covering a variety of clinical specialties. This corpus includes manually validated annotations for diseases, medications and professions created by the experts who annotated the corpora mentioned above, using the same guidelines and annottaion criteria. It can this be considered the first clinical Gold Standard corpus for diseases, medications and processions in Catalan.</li> </ul> <p>It is noteworthy that the MEDDOPROF-related data includes annotations for two labels, PROFESION and SITUACION_LABORAL, but only the former was used for training and evaluation in the paper.</p> <p>These are the <strong>10 languages</strong> included in the repository (along with their language codes):</p> <ul> <li><strong>Spanish</strong> (`es-gs`, with `gs` standing for Gold Standard)</li> <li><strong>Catalan</strong> (`cat`)</li> <li><strong>English</strong> (`en`)</li> <li><strong>French</strong> (`fr`)</li> <li><strong>Italian</strong> (`it`)</li> <li><strong>Dutch</strong> (`nl`)</li> <li><strong>Portuguese</strong> (`pt`)</li> <li><strong>Romanian</strong> (`ro`)</li> <li><strong>Swedish</strong> (`sv`)</li> <li><strong>Czech</strong> (`cz`)</li> </ul> <h2><strong>Related Links</strong></h2> <ul> <li><a href="../doi/10.5281/zenodo.13151039" target="_blank" rel="noopener">Validation and Correction Guidelines for the Multilingual Annotation Projection of Gold Standard Corpora</a></li> <li><a href="https://temu.bsc.es/multicardioner/" target="_blank" rel="noopener">MultiCardioNER Shared Task</a> </li> </ul> <h2><strong>License</strong></h2> <p>This work is licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</p> <h2><strong>Contact</strong></h2> <p>If you have any questions or suggestions, please contact us at:</p> <p>- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)<br>- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.