zenodoopen

Ancient Greek language models

In this repository, we release a series of vector space models of Ancient Greek, trained following different architectures and with different hyperparameter values.  Below is a breakdown of all the models released, with an indication of the training method and hyperparameters. The models are split into ‘Diachronica’ and ‘ALP’ models, according to the published paper they are associated with. <blockquote> [Diachronica:] Stopponi, Silvia, Nilo Pedrazzini, Saskia Peels-Matthey, Barbara McGillivray & Malvina Nissim. Forthcoming. Natural Language Processing for Ancient Greek: Design, Advantages, and Challenges of Language Models, Diachronica. [ALP:] Stopponi, Silvia, Nilo Pedrazzini, Saskia Peels-Matthey, Barbara McGillivray & Malvina Nissim. 2023. Evaluation of Distributional Semantic Models of Ancient Greek: Preliminary Results and a Road Map for Future Work. Proceedings of the Ancient Language Processing Workshop associated with the 14th International Conference on Recent Advances in Natural Language Processing (RANLP 2023). 49-58. Association for Computational Linguistics (ACL). https://doi.org/10.26615/978-954-452-087-8.2023_006 </blockquote> <h1>Diachronica models</h1> <h2>Training data</h2> Diorisis corpus (Vatri & McGillivray 2018). Separate models were trained for: <ol> <li>Classical subcorpus</li> <li>Hellenistic subcorpus</li> <li>Whole corpus</li> </ol> Models are named according to the (sub)corpus they are trained on (i.e. <code>hel_</code> or <code>hellenestic</code> is appended to the name of the models trained on the Hellenestic subcorpus, <code>clas_</code> or <code>classical</code> for the Classical subcorpus, <code>full_</code> for the whole corpus). <h2>Models</h2> <h3>Count-based</h3> <blockquote> Software used: LSCDetection (Kaiser et al. 2021; <a href="https://github.com/Garrafao/LSCDetection">https://github.com/Garrafao/LSCDetection</a>) </blockquote> a.     With Positive Pointwise Mutual Information applied (folder PPMI spaces). For each model, a version trained on each subcorpus after removing stopwords is also included (<code>_stopfilt</code> is appended to the model names). Hyperparameter values: <code>window=5</code>, <code>k=1</code>, <code>alpha=0.75</code>. b.     With both Positive Pointwise Mutual Information and dimensionality reduction with Singular Value Decomposition applied (folder PPMI+SVD spaces). For each model, a version trained on each subcorpus after removing stopwords is also included (<code>_stopfilt</code> is appended to the model names). Hyperparameter values: <code>window=5</code>, <code>dimensions=300</code>, <code>gamma=0.0</code>. <h3>Word2Vec</h3> <blockquote> Software used: CADE (Bianchi et al. 2020; <a href="https://github.com/vinid/cade">https://github.com/vinid/cade</a>). </blockquote> a.     Continuous-bag-of-words (CBOW). Hyperparameter values: <code>size=30</code>, <code>siter=5</code>, <code>diter=5</code>, <code>workers=4</code>, <code>sg=0</code>, <code>ns=20</code>. b.     Skipgram with Negative Sampling (SGNS). Hyperparameter values: <code>size=30</code>, <code>siter=5</code>, <code>diter=5</code>, <code>workers=4</code>, <code>sg=1</code>, <code>ns=20</code>. <h3>Syntactic word embeddings</h3> Syntactic word embeddings were also trained on the Ancient Greek subcorpus of the PROIEL treebank (Haug & Jøhndal 2008), the Gorman treebank (Gorman 2020), the PapyGreek treebank (Vierros & Henriksson 2021), the Pedalion treebank (Keersmaekers et al. 2019), and the Ancient Greek Dependency Treebank (Bamman & Crane 2011) largely following the SuperGraph method described in Al-Ghezi & Kurimo (2020) and the Node2Vec architecture (Grover & Leskovec 2016) (see <a href="https://github.com/npedrazzini/ancientgreek-syntactic-embeddings#graph-based-syntactic-word-embeddings">https://github.com/npedrazzini/ancientgreek-syntactic-embeddings</a> for more details). Hyperparameter values: window=1, min_count=1. <h1>ALP models</h1> <h2>Training data</h2> Archaic, Classical, and Hellenistic portions of the Diorisis corpus (Vatri & McGillivray 2018) merged, stopwords removed according to the list made by Alessandro Vatri, available at https://figshare.com/articles/dataset/Ancient_Greek_stop_words/9724613. <h2>Models</h2> <h3>Count-based</h3> <blockquote> Software used: LSCDetection (Kaiser et al. 2021; <a href="https://github.com/Garrafao/LSCDetection">https://github.com/Garrafao/LSCDetection</a>)  </blockquote> a.     With Positive Pointwise Mutual Information applied (folder ppmi_alp).  Hyperparameter values: <code>window=5</code>, <code>k=1</code>, <code>alpha=0.75</code>. Stopwords were removed from the training set. b.     With both Positive Pointwise Mutual Information and dimensionality reduction with Singular Value Decomposition applied (folder ppmi_svd_alp). Hyperparameter values: <code>window=5</code>, <code>dimensions=300</code>, <code>gamma=0.0</code>. Stopwords were removed from the training set. <h3>Word2Vec</h3> <blockquote> Software used: Gensim library (Řehůřek and Sojka, 2010) </blockquote> a.     Continuous-bag-of-words (CBOW). Hyperparameter values: <code>size=30</code>, <code>window=5</code>, <code>min_count=5</code>, <code>negative=20</code>, <code>sg=0</code>. Stopwords were removed from the training set. b.     Skipgram with Negative Sampling (SGNS). Hyperparameter values: <code>size=30</code>, <code>window=5</code>, <code>min_count=5</code>, <code>negative=20</code>, <code>sg=1</code>. Stopwords were removed from the training set. <h1>References</h1> Al-Ghezi, Ragheb & Mikko Kurimo. 2020. Graph-based syntactic word embeddings. In Ustalov, Dmitry, Swapna Somasundaran, Alexander Panchenko, Fragkiskos D. Malliaros, Ioana Hulpuș, Peter Jansen & Abhik Jana (eds.), Proceedings of the Graph-based Methods for Natural Language Processing (TextGraphs), 72-78. Bamman, D. & Gregory Crane. 2011. The Ancient Greek and Latin dependency treebanks. In Sporleder, Caroline, Antal van den Bosch & Kalliopi Zervanou (eds.), Language Technology for Cultural Heritage. Selected Papers from the LaTeCH [Language Technology for Cultural Heritage] Workshop Series. Theory and Applications of Natural Language Processing, 79-98. Berlin, Heidelberg: Springer. Gorman, Vanessa B. 2020. Dependency treebanks of Ancient Greek prose. Journal of Open Humanities Data 6(1). Grover, Aditya & Jure Leskovec. 2016. Node2vec: scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ‘16), 855-864. Haug, Dag T. T. & Marius L. Jøhndal. 2008. Creating a parallel treebank of the Old Indo-European Bible translations. In Proceedings of the Second Workshop on Language Technology for Cultural Heritage Data (LaTeCH), 27–34. Keersmaekers, Alek, Wouter Mercelis, Colin Swaelens & Toon Van Hal. 2019. Creating, enriching and valorizing treebanks of Ancient Greek. In Candito, Marie, Kilian Evang, Stephan Oepen & Djamé Seddah (eds.), Proceedings of the 18th International Workshop on Treebanks and Linguistic Theories (TLT, SyntaxFest 2019), 109-117. Kaiser, Jens, Sinan Kurtyigit, Serge Kotchourko & Dominik Schlechtweg. 2021. Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. Schlechtweg, Dominik, Anna Hätty, Marco del Tredici & Sabine Schulte im Walde. 2019. A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 732-746, Florence, Italy. ACL. Vatri, Alessandro & Barbara McGillivray. 2018. The Diorisis Ancient Greek Corpus: Linguistics and Literature. Research Data Journal for the Humanities and Social Sciences 3, 1, 55-65, Available From: Brill <a href="https://doi.org/10.1163/24523666-01000013" target="_blank" rel="noopener">https://doi.org/10.1163/24523666-01000013</a> Vierros, Marja & Erik Henriksson. 2021. PapyGreek treebanks: a dataset of linguistically annotated Greek documentary papyri. Journal of Open Humanities Data 7.

ShareScore

40/100

Overall dataset sharing score

Score breakdown

These five areas show where the dataset supports — or may limit — practical reuse.

Stewardship: 8
Harmonization: 8
Access: 16
Reuse readiness: 8
Engagement: 0

Topics

Ancient Greek language models word vector representations word embeddings count-based graph-based syntactic embeddings