Buddhist Chinese Word Embeddings

<p>Buddhist Chinese word embeddings trained with FastText on the Buddhist texts present in the Kanseki repository.</p> <p>There are four models present here (and the full binary output for one), each differing in how the Kanseki repository was segmented into tokens. The &quot;chinese_model&quot; was segmented into 1-grams (individual characters). The &quot;chinese_model_word&quot; was segmented into words using a dictionary of buddhist terms and phrases not found were segmented with the classical Chinese word segmenter distributed with the stanza python library (https://stanfordnlp.github.io/stanza/available_models.html). The &quot;chinese_model_hybrid_char_term&quot; was segmented with a glossary of Buddhist terms and phrases not found were&nbsp;divided into 1-grams. &quot;chinese_model_hybrid_char_term_2&quot; uses a more extensive glossary of Buddhist terms and missing sections are divided into 1-grams.</p> <p>All models are default 100 dimensional FastText models created for this pilot study: Felbur, Rafal, Marieke Meelen &amp; Paul Vierthaler (2022), &#39;Crosslinguistic Semantic Textual Similarity of Buddhist Chinese and Classical Tibetan&#39; in&nbsp;<em>Journal of Open Humanities Data</em>.</p> <p>This research was done with generous funding from the Open Philology project. This project (running 2018&ndash;2022) is funded by the European Research Council (ERC) under the Horizon 2020 program (Advanced Grant agreement No 741884). It is based at the Leiden University Institute for Area Studies.</p>