Weekly supervised Multilingual Data Set to train Named Entity Recognition for Symptom Extraction

<p>Data Sets were generated using the Weakly Supervised NER pipeline (https://github.com/HUMADEX/Weekly-Supervised-NER-pipline) to train the symptom extraction NER models.&nbsp;</p> <p><strong>Supported Languages and dataset locations for the specific language:</strong></p> <p>&nbsp; &nbsp; English (base language): https://huggingface.co/HUMADEX/english_medical_ner<br>&nbsp; &nbsp; German: https://huggingface.co/HUMADEX/german_medical_ner<br>&nbsp; &nbsp; Italian: https://huggingface.co/HUMADEX/italian_medical_ner<br>&nbsp; &nbsp; Spanish: https://huggingface.co/HUMADEX/spanish_medical_ner<br>&nbsp; &nbsp; Greek: https://huggingface.co/HUMADEX/german_medical_ner<br>&nbsp; &nbsp; Slovenian: https://huggingface.co/HUMADEX/slovenian_medical_ner<br>&nbsp; &nbsp; Polish: https://huggingface.co/HUMADEX/polish_medical_ner<br>&nbsp; &nbsp; Portuguese: https://huggingface.co/HUMADEX/portugese_medical_ner</p> <p>&nbsp;</p> <p><strong>Dataset Building&nbsp;</strong></p> <ul> <li>Data Integration and Preprocessing</li> <li>Data Cleaning</li> <li>Annotation with Stanza's i2b2 Clinical Model&nbsp;</li> <li>Translation into the targeted language</li> <li>Word Alignment&nbsp;</li> <li>Data Augmentation&nbsp;</li> </ul> <p><strong>Acknowledgement</strong><br>This dataset had been created as part of joint research of HUMADEX research group (https://www.linkedin.com/company/101563689/) and has received funding by the European Union Horizon Europe Research and Innovation Program project SMILE (grant number 101080923) and Marie Skłodowska-Curie Actions (MSCA) Doctoral Networks, project BosomShield ((rant number 101073222). Responsibility for the information and views expressed herein lies entirely with the authors.</p> <p><strong>Authors:</strong><br>dr. Izidor Mlakar, Rigona Sallauka, dr. Umut Arioz, dr. Matej Rojc</p> <p><strong>Please cite as:</strong></p> <p><span>Article title: Weakly-Supervised Multilingual Medical NER For Symptom Extraction For Low-Resource Languages</span><br><span>Doi: 10.20944/preprints202504.1356.v1</span><br><span>Website:&nbsp;</span><a title="https://www.preprints.org/manuscript/202504.1356/v1" href="https://www.preprints.org/manuscript/202504.1356/v1">https://www.preprints.org/manuscript/202504.1356/v1</a></p>