Data for Finnish Dialect Detection

<p>The data used in the paper &quot;Finnish Dialect Identification: The Effect of Audio and Text&quot;.</p> <p><strong>If you use the data, please cite:</strong></p> <p>H&auml;m&auml;l&auml;inen, Mika; Alnajjar, Khalid; Partanen, Niko &amp; Rueter, Jack (2021).&nbsp;<a href="https://aclanthology.org/2021.emnlp-main.692/">Finnish Dialect Identification: The Effect of Audio and Text</a>. In&nbsp;<em>Proceedings of the 2021&nbsp;Conference on Empirical Methods in Natural Language Processing (EMNLP)</em>.</p> <p>The <strong>metadata.json </strong>contains the dialectal and normalized transcriptions, the length of the wav files in milliseconds, the path to the wav file, the role of the speaker and the dialect. The <strong>data.zip*</strong> files contain the wav files. They are partial zip files to make Zenodo upload easier. <strong>Code_models.zip</strong> contains the code for training the bimodal model and the final trained model presented in the paper. There is also an OpenNMT model that is the text only model described in the paper. You can use it by running:&nbsp;<em>python3 -m onmt.bin.translate -model nmt-model_step_100000.pt -src source_test.txt -output pred.txt -replace_unk</em></p> <p>Our dataset is licensed under CC BY NC ND 4.0. Academic use only.</p> <p>&nbsp;</p> <p>The corpus is based on <a href="http://urn.fi/urn:nbn:fi:lb-2017020201">Suomenkielen n&auml;ytteit&auml;</a>&nbsp;(CC BY &nbsp;<a href="http://www.kotus.fi/">Kotimaisten kielten keskus</a>).</p>