Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
132
datasets available to search
ShareScore release 0.7.1
Dataset results
132 results for “vocabularies”
Models for "A data-driven approach to studying changing vocabularies in historical newspaper collections"
<p>NOTE: This is a badly rendered version of the README within the archive.</p> <p><strong>A data-driven approach to studying changing vocabularies in historical newspaper collections</strong></p> <p>Simon Hengchen,* Ruben Ros,** Jani Marjanen,*** Mikko Tolonen***</p> <p>*<a href="https://spraakbanken.gu.se/en/about/staff/simon">Språkbanken Text</a>, University of Gothenburg, Sweden and <a href="https://iguanodon.ai">iguanodon.ai</a>, Belgium: firstname.lastname@gu.se<br> **<a href="https://www.c2dh.uni.lu/people/ruben-ros">Centre for Contemporary and Digital History (C2DH)</a>, University of Luxembourg: firstname.lastname@uni.lu<br> ***<a href="https://www.helsinki.fi/en/researchgroups/computational-history">COMHIS</a>, University of Helsinki: <a href="mailto:firstname.lastname@helsinki.fi">firstname.lastname@helsinki.fi</a>;</p> <p>These are the supplementary materials for the DH2019 paper <em>A data-driven approach to the changing vocabulary of the ‘nation’ in English, Dutch, Swedish and Finnish newspapers, 1750-1950</em>, as well as the 2021 Digital Scholarship in the Humanities publication available in OpenAccess: <a href="https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793">https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793</a>. If you end up using whole or parts of this resource, please use the following citation(s):</p> <ul> <li>Hengchen, S., Ros, R., and Marjanen, J. (2019). A data-driven approach to the changing vocabulary of the 'nation' in English, Dutch, Swedish and Finnish newspapers, 1750-1950. In <em>Proceedings of the Digital Humanities (DH) conference 2019, Utrecht, The Netherlands</em></li> </ul> <p>and/or:</p> <ul> <li>Hengchen, S., Ros, R., Marjanen, J. and Tolonen, M., 2021. A data-driven approach to studying changing vocabularies in historical newspaper collections. Digital Scholarship in the Humanities, 36(Supplement_2), pp.ii109-ii126.</li> </ul> <p>or alternatively use one of the following <code>bib</code>s:</p> <pre><code>@inproceedings{hengchen2019nation, title="A data-driven approach to the changing vocabulary of the 'nation' in {E}nglish, {D}utch, {S}wedish and {F}innish newspapers, 1750-1950.", author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani}, year={2019}, address = "Utrecht, The Netherlands", booktitle={Proceedings of the Digital Humanities (DH) conference 2019} }</code></pre> <pre><code>@article{hengchen2021data, title={A data-driven approach to studying changing vocabularies in historical newspaper collections}, author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani and Tolonen, Mikko}, journal={Digital Scholarship in the Humanities}, volume={36}, number={Supplement\_2}, pages={ii109--ii126}, year={2021}, publisher={Oxford University Press} }</code></pre> <p> </p> <p>Files</p> <p>This archive contains two folders -- one per diachronic representation method -- as well as this README. The folders each contain four folders, which contain the models for their respective languages. As can be inferred from the small datasize, most of the earlier models are not reliable and should not be used, but are still made available. This work is licensed under a <a href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.</p> <p><strong>Source material</strong></p> <p>Finnish:</p> <p>The models were created with data from the Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland (National Library of Finland, 2011). We used everything in the corpus.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h fi* 12M fi_1820_SGNS_corpus_file.gensim 89M fi_1840_SGNS_corpus_file.gensim 797M fi_1860_SGNS_corpus_file.gensim 7.0G fi_1880_SGNS_corpus_file.gensim 22G fi_1900_SGNS_corpus_file.gensim</code></pre> <p>Swedish:</p> <p>The models were created with data from the Kubhist 2 corpus (Språkbanken) -- more precisely, the data dumps available at <a href="https://spraakbanken.gu.se/lb/resurser/meningsmangder/">https://spraakbanken.gu.se</a>. After a manual evaluation of Swedish embeddings trained without pre-processing seemed to show that the embeddings were of low quality, we retrained models, only keeping sentences that were at least 10 tokens long and were constituted of at least 50% of lemmas as per the KORP processing pipeline (Borin et al, 2012).</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h sv* 1.6M sv_1740_SGNS_corpus_file.gensim 44M sv_1760_SGNS_corpus_file.gensim 124M sv_1780_SGNS_corpus_file.gensim 228M sv_1800_SGNS_corpus_file.gensim 678M sv_1820_SGNS_corpus_file.gensim 1.6G sv_1840_SGNS_corpus_file.gensim 4.5G sv_1860_SGNS_corpus_file.gensim 6.5G sv_1880_SGNS_corpus_file.gensim 113M sv_1900_SGNS_corpus_file.gensim</code></pre> <p>Dutch:</p> <p>The models were created with data from the Delpher newspaper archive (Royal Dutch Library, 2017), through data dumps for newspapers until and including 1876, and through API hits for articles from 1877 to 1899 (included).</p> <ul> <li>For anything pre-1877 we discarded full texts that had, in the metadata, anything else than exclusively <code>nl</code> or <code>NL</code> as a language tag.</li> <li>For the full texts between 1877 and 1899: we queried the API for all items in the “artikel” category that contained the determiner <code>de</code>.</li> </ul> <p>Our assumption was that most articles should contain <code>de</code> at least once, and those that didn't were too short to be deemed interesting. A subsequent study showed that was not exactly the case, but we were reassured by the fact that left-out articles were probably "shipping or financial reports" (thanks go to Melvin Wevers). We also did not include the colonial newspapers for our embeddings. This is motivated by our research questions. A list of removed newspapers is available on request.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h nl* 6.8M nl_1620_SGNS_corpus_file.gensim 7.9M nl_1640_SGNS_corpus_file.gensim 43M nl_1660_SGNS_corpus_file.gensim 78M nl_1680_SGNS_corpus_file.gensim 138M nl_1700_SGNS_corpus_file.gensim 243M nl_1720_SGNS_corpus_file.gensim 287M nl_1740_SGNS_corpus_file.gensim 431M nl_1760_SGNS_corpus_file.gensim 825M nl_1780_SGNS_corpus_file.gensim 1.2G nl_1800_SGNS_corpus_file.gensim 1.8G nl_1820_SGNS_corpus_file.gensim 3.1G nl_1840_SGNS_corpus_file.gensim 5.2G nl_1860_SGNS_corpus_file.gensim 13G nl_1880_SGNS_corpus_file.gensim</code></pre> <p>English:</p> <p>The models were created with data from the British Library Newspapers collection (<a href="https://www.gale.com/intl/primary-sources/british-library-newspapers%5D">link</a>), the Nichols collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-burney-newspapers-collection">link</a>), and the Burney collection (<a href="https://www.gale.com/intl/c/17th-and-18th-century-nichols-newspapers-collection">link</a>). We used everything in the corpora. For English, only SGNS_ALIGN models are available. We thank Gale Cengage for their help with this project.</p> <p>Filesizes:</p> <pre><code>[simon@taito-login3 SGNS]$ du -h en* 4.3M en_1620_SGNS_corpus_file.gensim 11M en_1640_SGNS_corpus_file.gensim 11M en_1660_SGNS_corpus_file.gensim 106M en_1680_SGNS_corpus_file.gensim 409M en_1700_SGNS_corpus_file.gensim 1.7G en_1720_SGNS_corpus_file.gensim 834M en_1740_SGNS_corpus_file.gensim 2.4G en_1760_SGNS_corpus_file.gensim 5.3G en_1780_SGNS_corpus_file.gensim 5.5G en_1800_SGNS_corpus_file.gensim 15G en_1820_SGNS_corpus_file.gensim 42G en_1840_SGNS_corpus_file.gensim 65G en_1860_SGNS_corpus_file.gensim 88G en_1880_SGNS_corpus_file.gensim 26G en_1900_SGNS_corpus_file.gensim 21G en_1920_SGNS_corpus_file.gensim 6.3G en_1940_SGNS_corpus_file.gensim</code></pre> <p><strong>Word embeddings</strong></p> <p>For every language, we train diachronic embeddings as follows. We divide the data in 20-year time bins. We train SGNS_UPDATE and SGNS_ALIGN models. Current research on German (Schlechtweg et al, 2019) and English (Shoemark et al, 2019) indicates you should use the SGNS_ALIGN models. <strong>For EN, FI, NL, no tokens (including punctuation) were removed nor altered, aside from lowercasing</strong>. For SV, see above. Parameters are as follows: SGNS architecture (Mikolov et al 2013), window size of 5, frequency threshold of 100, 5 epochs, 300 dimensions (or 100 for EN).</p> <ul> <li>For SGNS_UPDATE: We first train a model for the first time bin <code>t</code>. To train the model for <code>t+1</code>, we use the <code>t</code> model to initialise the vectors for <code>t+1</code>, set the learning rate to correspond to the end learning rate of <code>t</code>, and continue training. This approach, closely following Kim et al (2014), has the advantage of avoiding the need for post-training vector space alignment.</li> </ul> <p>The Python snippet below, which makes use of gensim (Rehurek and Sojka, 2010), illustrates the approach. Special thanks go to Sara Budts.</p> <pre><code>## dict_files[key] is a dictionary with double decades as keys and a corresponding LineSentence object as value: https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.LineSentence count = 0 for key in sorted(list(dict_files.keys())): if count == 0: ## This is the first model. model = gensim.models.Word2Vec(corpus_file=dict_files[key], min_count=100, sg=1 ,size=300, workers=64, seed=1830, iter=5) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) print("Model saved, on to the next\n") count += 1 if count > 0: ## this is for the subsequent models. print("model for double decade starting in",str(key)) model = gensim.models.Word2Vec.load(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin-20)+".w2v")) print("previous model loaded") model.build_vocab(corpus_file=dict_files[key], update=True) model.train(corpus_file=dict_files[key], total_words = model.corpus_count, total_examples = model.corpus_count, start_alpha = model.alpha, end_alpha = model.min_alpha, epochs=model.epochs) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) </code></pre> <ul> <li>For SGNS_ALIGN: We independently train models for all time bins. The models in this repository are <em>NOT</em> aligned, leaving you the choice of how to align them. For example, <a href="https://gist.github.com/quadrismegistus/09a93e219a6ffc4f216fb85235535faf">here</a> is a link to code by Ryan Heuser to do just that. Models were trained with the <code>count == 0</code> scenario in the snippet above.</li> </ul> <p><strong>Acknowledgments</strong></p> <p>This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant 770299 <a href="https://www.newseye.eu/">NewsEye</a>. Specials thanks go to the data providers/collection-holding institutions: the Finnish Language Bank, the Swedish Language Bank, the Royal Dutch Library, and Gale Cengage.</p> <p>The authors would like to thank the following persons and group, listed alphabetically: Antoine Doucet, Antti Kanner, Axel-Jean Caurant, Dominik Schlechtweg, Eetu Mäkelä, Elaine Zosa, Estelle Bunout, Haim Dubossarsky, Joris van Eijnatten, Krister Lindén, Lars Borin, Lidia Pivovarova, Melvin Wevers, Nina Tahmasebi, Sara Budts, Senka Drobac, Tanja Säily, the COMHIS group, and Steven Claeyssens. Computational resources were provided by CSC – IT Center for Science Ltd.</p> <p><strong>References</strong></p> <p>Borin, L., Forsberg, M., Roxendal, J. (2012). Korp-the corpus infrastructure of Spräkbanken,in: LREC. pp. 474–478.</p> <p>Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D. and Petrov, S. (2014). Temporal Analysis of Language through Neural Language Models. <em>ACL 2014</em>, p.61.</p> <p>Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. <em>arXiv preprint arXiv:1301.3781</em>.</p> <p>National Library of Finland (2011). <em>The Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version</em> [text corpus]. Kielipankki. Retrieved from <a href="http://urn.fi/urn:nbn:fi:lb-2016050302">http://urn.fi/urn:nbn:fi:lb-2016050302</a>.</p> <p>Rehurek, R. and Sojka, P. (2010). Software framework for topic modelling with large corpora. In <em>Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks</em>.</p> <p>Royal Dutch Library (2017). <em>Delpher open krantenarchief (1.0)</em>. Den Haag, 2017.</p> <p>Schlechtweg D., Hätty A, del Tredici M., and Schulte im Walde S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In <em>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, Florence, Italy. ACL.</p> <p>Shoemark, P., Liza, F.F., Nguyen, D., Hale, S. and McGillivray, B. (2019). Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings. In <em>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66-76)</em>, Hong Kong.</p> <p>Språkbanken. <em>The Kubhist Corpus</em>. Department of Swedish, University of Gothenburg. <a href="https://spraakbanken.gu.se/korp/?mode=kubhist">https://spraakbanken.gu.se/korp/?mode=kubhist</a>.</p>
Behavioral and fMRI Data: Nurturing the reading brain: Home literacy practices are associated with children's neural response to printed words through vocabulary skills
<p>This is the behavioral and fMRI dataset described in "Nurturing the reading brain: Home literacy practices are associated with children’s neural response to printed words through vocabulary skills". </p> <p>Because of anonymization concerns within the framework of EU privacy regulations (<a href="https://gdpr-info.eu">GDPR</a>), we cannot provide raw MRI data. Therefore, the fMRI data consists of individual pre-processed volumes, normalized into the MNI template (see paper for details about the preprocessing pipeline). Anonymized behavioral data and first level analyses are also provided for each participant (SPM.mat file as well as beta, con, spmT, RPV and ResMS files). Note that the dataset also include runs and GLM results for a third task (Dots) that was not analyzed in the paper. Finally, the <a href="https://www.psychopy.org">PsychoPy</a> implementation of the tasks is also provided. If you have any questions, please send an email to jerome.prado [at] univ-lyon1.fr. </p> <p><strong>IMPORTANT:</strong></p> <p>In accordance with EU privacy regulations, we ask that you sign and return a Data Use Agreement (DUA) before downloading the data. You can download the DUA <a href="https://zenodo.org/record/4965716/files/DUA.pdf?download=1">here</a>. Please, sign it and send it to jerome.prado [at] univ-lyon1.fr.</p>
Test Data from a Study on Latin Vocabulary Acquisition (Cicero)
<p>The dataset contains test results from an intervention study with intermediate learners in two high schools in Berlin. In total, 58 students participated in three groups (= classes). The intervention materials and tests are published as well.</p> <p>The study was the first to collect empirical data on what German students actually know about Latin vocabulary and how they handle their vocabulary knowledge. One of the main goals of the research project is to establish a broad understanding of vocabulary knowledge in Latin lessons in Germany, which aims at a versatile education of (cross-linguistically helpful) vocabulary competence.</p>
Test Data from a Study on Latin Vocabulary Acquisition (Ovid)
<p>The dataset contains test results from an intervention study with intermediate learners in two high schools in Berlin (2018-2019). In total, 60 students participated in three groups (= classes). The intervention materials and tests are published as well.</p> <p>A key question of the still ongoing research project is: How can vocabulary competence in a historical language such as Latin be acquired and deepened by using corpus-based, i.e. context-based, methods? This question is based on a broad understanding of vocabulary that refers back to theories of the mental lexicon.</p>
OntoUML Vocabulary Python Library
A Python library designed to simplify the development of software applications using the OntoUML vocabulary.
ICOPS Workshop Series - Standard Vocabularies and Ontologies
<p><strong>This is the fifth workshop in the International Committee on Open Phytolith Science (ICOPS) workshop series on Open Research Skills. </strong></p><p>In this workshop we had multiple speakers:</p><ul><li>Introduction - Henriette Harmse - slides in the main presentation</li><li>Case study and demo of image database - Frances Wong - slides attached as pdf.</li><li>Phytolith standarised nomenclature ICPN 2.0 - Luc Vrydaghs [*presentation not included*]</li><li>Phytolith ontology - Celine Kerfant and Zach Dunseth - slides in the main presentation</li></ul><p>Youtube video of the workshop:</p><p><a href="https://youtu.be/qZaWkJlVXvE">https://youtu.be/qZaWkJlVXvE</a></p>
Tree Annotation Vocabulary (TAV) - Knowledge Graph and Annotated Dataset
<p>This dataset contains all the files used in developing the Tree-KG, the knowledge graph to capture the tree annotations in the works of Vladimir Nabokov. </p> <p>In the Annotated Dataset folder, 6 spreadsheets in excel (.xlsx) format are provided. They are numbered. Note that annotated data are all in English as the consulted works are the English translations of the literary works of Nabokov.</p> <p>(1) contains the tree annotations from the novels originally written in Russian by Vladimir Nabokov.</p> <p>(2) contains the tree annotations from the novels originally written in English by Vladimir Nabokov.</p> <p>(3) contains the tree annotations from the short stories originally written in Russian and English by Vladimir Nabokov.</p> <p>(4) is the knowledge base (KB) developed to link the annotated trees to Wikidata and DBPedia.</p> <p>(5) is the benchmarking results of some entity recognition tools. It includes the relevant passages from Nabokov's novels that were used in the experiments as well as the prompts used in getting the results.</p> <p>(6) represents the complete bibliographic details of the works of Vladimir Nabokov (https://thenabokovian.org/abbreviations).</p> <p>In the Ontology Versions folder, four ontology (TAV) files in turtle (.ttl) format are provided. They are all numbered and dated to represent their different versions. Some sample SPARQL queries are provided in a .txt file. The KG was developed on Protégé. </p> <p>(1) contains the essential schema for the TAV vocabulary.</p> <p>(2) contains the schema for TAV vocabulary with links to external vocabularies (Schema.Org; Open Annotation, etc.). </p> <p>(3) contains the Tree-KG in so far it reflects data from three novels (Mary; King, Queen, Knave; Glory).</p> <p>(4) contains the entire Tree-KG based on all the works mentioned in the excel sheets (20 books).</p> <p>(5) contains some sample SPARQL queries (.txt) file.</p>
Raw frequency data: Thoughts on "Reliable" Learner's Vocabularies for Classical and Literary Chinese
<p>This dataset includes the raw frequency counts (classical_chinese_learners_vocabularies_raw_frequencies.zip) used in the article Thoughts on “Reliable” Learner’s Vocabularies for Classical and Literary Chinese. </p> <p>Corpus I – Micheal Loewe (1993)’s <em>Early Chinese Texts</em><br> Corpus II – Official Histories (zhengshi 正史)<br> Corpus III Six Novels (xiaoshuo 小說), as defined in Hsia 1968</p> <p>The download includes one folder per corpus, structured as follows:</p> <ul> <li>xx_corpus.csv > list of texts and sources / used versions, token and type counts</li> <li>xx_freq_1-1.csv > unigram / character frequencies and counts</li> <li>xx_freq_1-4.csv > 1 to 4 character word frequencies and counts, "words" according to Hanyu da cidian 漢語大詞典 (Luo 1986–1994))</li> <li>xx_freq_2-4.csv > 2 to 4 character words</li> </ul> <p>Additionally, pca_zhengshi_vs_loewe_vs_xiaoshuo.html is an interactive version of the Principal Component Analysis (PCA) presented in the article, texts from the three corpora are represented using the 1.000 most frequent 1–4 character combinations from the dataset.</p>
Features of a FAIR vocabulary Supplementary docs
<p>Supplementary docs for FAIR vocabulary feature publication</p> <p>Content:</p> <p>ST1: The suitability of OBO principles used as FAIR Vocabulary Features</p> <p>ST2: FAIR Vocabulary Features mapped to FAIR principles and FAIR vocabulary requirements</p> <p>ST3: VersionIRI analysis</p> <p>ST4: RDA data maturity indicators that are not mapped to FAIR Vocabulary Features</p> <p>ST5: FAIR assessment results of Gene ontology</p> <p>ST6: FAIR assessment results of Experimental Factor Ontology</p> <p>ST7: FAIR assessment results of ICD-11</p>
A controlled vocabulary for research and innovation in the field of Artificial Intelligence (AI)
<p><strong>A controlled vocabulary for research and innovation in the field of Artificial Intelligence (AI)</strong></p> <p>This controlled vocabulary of keywords related to the field of Artificial Intelligence (AI) was built by SIRIS Academic in collaboration with ART-ER (the R&I and sustainable development in-house agency of the Emilia-Romagna region in Italy) and the Generalitat de Catalunya (the regional government of Catalonia, Spain), in order to identify AI research, development and innovation activities. The work was carried out by consulting domain experts' advice and it was ultimately applied to inform regional strategies on AI and research and innovation policy.</p> <p>The aim of this vocabulary is to enable one to retrieve texts (e.g. R&D projects and scientific publications) featuring the concepts included in the present vocabulary in their titles and abstracts, assuming that these records have a certain contribution of applications, techniques and issues, in the domain of AI.</p> <p>The present effort was carried out because, despite the high number of contributions and technological developments in the field of AI, there is no closed or static vocabulary of concepts that allows to unequivocally define the boundaries of what should be considered “an Artificial Intelligence intellectual product” (or what should not). Indeed, the literature presents different definitions of the domain, with visions that could be contradictory. AI encompasses today a wide variety of subdomains, ranging from general purpose areas such as learning and perception to more specific ones such as autonomous vehicle driving, theorem proving, or industrial process monitoring. AI synthesises and automates intellectual tasks, and is therefore potentially relevant to any area of human intellectual activity. In this sense, it is a genuinely universal and multidisciplinary field. AI draws upon disciplines as diverse as cybernetics, mathematics, philosophy, sociology and economics.</p> <p>As a ground for the construction of the AI controlled vocabulary, an initial set of concepts was taken from different subdomains of the <em>ACM Computing Classification System 2012, </em> to define the boundaries of the AI domain. Notably, although some relevant AI subdomains have an independent category in the ACM taxonomy outside of AI, they have been included in the list of subdomains. In order to align the ACM taxonomical definition with the Catalan Strategy of AI, <em>CATALONIA.AI</em>, in <em>version 1 </em>of this resource the emerging area of AI Ethics was included in the vocabulary, while some other categories which are not relevant for the objectives were removed from the subdomains list. In the current <em>version 2</em>, the classification and the labels of the subdomains have been revised because of the evolution of the field. Some fields have been grouped in order to reduce the overlap between subdomains and to provide a taxonomy that makes more sense for the analysis of R&I ecosystems. </p> <p>The different subdomains in the versions are presented in the following table:</p> <table> <tbody> <tr> <td><strong>Version </strong></td> <td><strong>Subdomains</strong></td> </tr> <tr> <td> <p><em>Version 2</em></p> </td> <td> <p>(1) Machine learning and deep learning; (2) Computer Vision; (3) Natural Language Processing and speech recognition; (4) Intelligent agents, planning, scheduling, problem-solving, control methods, and search; (5) Expert Systems, Knowledge representation and reasoning; (6) AI Ethics.</p> </td> </tr> <tr> <td><em>Version 1</em></td> <td>(1) General, (2) Machine Learning, (3) Computer Vision, (4) Natural Language Processing, (5) Knowledge Representation and Reasoning, (6) Distributed Artificial Intelligence, (7) Expert Systems, Problem-Solving, Control Methods and Search and (8) AI Ethics.</td> </tr> </tbody> </table> <p>Although a keyword rule-based approach suffers from the major shortcomings of not capturing all the lexical and linguistic variants of specific concepts nor the context of the words - namely, keyword-based approaches would miss relevant texts if the specific pattern is not matched during the search - the present vocabulary allowed us to obtain fairly good results, due to the specificity of the concepts describing the AI domain. Furthermore, an understandable and transparent controlled vocabulary allows a better control of the final results and the final definition of the domain borders. Also, a plain list of terms allows a much easier and interactive engagement of interested stakeholders with different degrees of knowledge (such as, for instance, domain experts, policy-makers and potential users) who can make use of vocabulary to retrieve pertinent literature or to enrich the resource itself.</p> <p>The vocabulary has been built taking advantage of advanced language models and resources from knowledge datasets such as arXiv, DBpedia and Wikipedia. The resulting vocabulary comprises 833 keywords, and has been validated by experts from several universities in Emilia-Romagna and Catalonia.</p> <p>The <em>version 0.5</em> of this resource was developed by the SIRIS Academic in 2019 in collaboration with ART-ER, Emilia-Romagna (Quinquillá et <em>al.</em>, 2020), the <em>version 1 </em>was the result of an update done in 2020 in collaboration with the Generalitat de Catalunya, and the current version (<em>version 2</em>) has resulted in 2021 from the collaboration with ART-ER and the integration of an additional set of keywords provided by the <em>Artificial Intelligence and Intelligence Systems (AIIS)</em> Laboratory of the CINI (<em>Consorzio interuniversitario nazionale per l’informatica </em>based in Rome, Italy).</p> <p>The methodology for the construction of the controlled vocabulary is presented in the following steps:</p> <ol> <li> <p>An initial set of scientific publications was collected by retrieving the following records as a weakly-supervised (in the sense that records are linked to AI by their taxonomy and not by a manual label) dataset in the domain of Artificial Intelligence :</p> <ol> <li> <p>Publications from Scopus with the keyword “Artificial Intelligence”</p> </li> <li> <p>Publications from arXiv in the category “Artificial Intelligence”</p> </li> <li> <p>Publications in relevant journals in the scientific domain of “Artificial Intelligence”</p> </li> </ol> </li> <li> <p>An automated algorithm was used to retrieve, from the APIs of DBpedia, a series of terms that have some categorical relationships (i.e. those that are indexed as “sub-categories of”, “equivalent to”, among other relations in DBpedia) with the Artificial Intelligence concept and with the AI categories in the ACM taxonomy. The DBpedia tree has been exploited down to the level 3, and the relevant categories have been manually selected (for instance: <em>Classification algorithms</em>,<em> Machine learning</em> or <em>Evolutionary computation</em>) and others were ignored (for instance: <em>Artificial intelligence in fiction</em>, <em>Robots</em> or <em>History of artificial intelligence</em>) because they were not relevant, or not specifically in the domain.</p> </li> <li> <p>The keywords in publications in the dataset were extracted from the keyword sections and from the abstracts. The keywords with a higher <em>TF-IDF</em>, using an <em>IDF</em> matrix in the open domain, have been selected. The co-occurrence of keywords with categories in specific AI subdomain and a clusterization of the main keywords has been used for a categorization of the keywords at the thematic level.</p> </li> <li> <p>This list of keywords tagged by thematic category has been manually revised, removing the non-pertinent keywords and changing the wrong categorizations by fields.</p> </li> <li> <p>The weak-supervised dataset in the domain of Artificial Intelligence is used to train a Word2Vec (Mikolov <em>et al.</em>, 2013) word embedding model (a machine learning model based on neural networks).</p> </li> <li> <p>The terms’ list is then enriched by means of automatic methods, which are run in parallel: </p> <ol> <li> <p>The trained Word2Vec model is used to select, among the indexed keywords of the reference corpus, all terms “semantically close” to the initial set of words. This step is carried out to select terms that might not appear in the texts themselves, but that were deemed pertinent to label the textual records.</p> </li> <li> <p>Further, terms that are mentioned in the texts of the reference corpus and that are valued by the trained Word2Vec model as “semantically close” to the initial set of words are also retained. This step is performed to include in the controlled vocabulary a series of terms that are related to the focus of the SDGs and which are used by practitioners.</p> </li> </ol> </li> <li> <p>The final list produced by steps 2-6 is manually revised.</p> </li> </ol> <p> </p> <p>The definition of the vocabulary does not, per se, allow to identify STI contributions to AI: this activity in fact boils down to actually matching the terms in the controlled vocabulary to the content of the gathered STI textual records. To successfully carry out this task, a series of pattern matching rules must be defined to capture possible variants of the same concept, such as permutations of words within the concept and/or the presence of null words to be skipped. For this reason, we have carefully crafted matching rules that take into account permutations of words and that allow words within concept to be within a certain distance. Some relatively ambiguous keywords (which may match unwanted pieces of text), have a set of associated “extra” terms. These “extra” terms are defined as further terms that must co-appear, in the same sentence, together with their associated ambiguous keywords.</p> <p>Finally, each keyword in the vocabulary was assigned one or more AI subdomains, so that the vocabulary can also be used to tag collections of texts within narrower AI sub-domains. In order to complement the alignment between keywords and subdomains, a set of subdomain-specific keywords have been defined to better capture the scope of the subdomains. These allow better characterization of subdomains that are more difficult to define only by means of unambiguous specific concepts, or that overlap with the wide “machine learning” subdomain (example: machine learning applied to object recognition or text translation). The alignment between keywords and subdomains, and these keyword lists of each subdomain, have been applied to capture AI subdomains in research outputs. Through this classification process, we have identified projects and publications related to AI, with a focus on mapping the research competencies in the AI domain in Emilia-Romagna. The resulting research records have been reviewed by experts in the domain, given the occurrence of some false positives, which have been used to improve the approach.</p> <p>The final controlled vocabulary has been evaluated with an external test set, proposed by (Dunham <em>et al.,</em> 2020). The test set consists of the abstract of 10,606 papers published in the arXiv repository, of which 1,076 within the Artificial Intelligence subcategories and 9,530 in arXiv categories other than Artificial Intelligence. Evaluating the controlled vocabulary on this data set, we observe accuracy of .94. However, because the pertinence of these publications to the field of AI is based solely on their taxonomic classification (i.e., on whether they are classified in the arXiv within Artificial Intelligence and not on a manual labelling), this evaluation can only yield an orientative performance assessment.</p> <p>The version 2 includes new keywords extracted from the (1) re-training of the enrichment pipeline (steps 5-6 in the methodology) considering as initial set of terms the version 1 of the vocabulary on a reference corpus of new publications, and (2) from the flat keywords list provided by the <em>Artificial Intelligence and Intelligence Systems</em> <em>(AIIS)</em> Lab of CINI (Consorzio interuniversitario nazionale per l’informatica). The keywords in (2) have been cleaned by calculating precision and f-measure on the dataset (Dunham et al., 2020), selecting those keywords with the highest scores, and being manually validated a posteriori.</p> <p>The AI controlled vocabulary has been applied in two practical cases, which have the purpose of identifying skills, stakeholders and capabilities, of a specific research ecosystem at the regional level. See the following references:</p> <ul> <li> <p>Quinquillá, Arnau, Duran-Silva, Nicolau, Massucci, Francesco Alessandro, Fuster, Enric, Rondelli, Bernardo, Bologni, Leda, … Moretti, Giorgio. (2020). Text mining to identify skills, stakeholders and capabilities: the case of Artificial Intelligence in Emilia-Romagna. Zenodo. <a href="http://doi.org/10.5281/zenodo.3606342">http://doi.org/10.5281/zenodo.3606342</a>. Poster presented at: World Open Innovation Conference 2019 (WOIC); 11th december 2019, Rome, Italy.</p> </li> <li> <p>Bigas, E., Duran, N., Fuster, E., Parra, C., Fernández, T. (2021): “Anàlisi de l’especialització en intel·ligència artificial”. Col·lecció Monitoratge de la RIS3CAT, Generalitat de Catalunya <a href="http://catalunya2020.gencat.cat/web/.content/00_catalunya2020/Documents/estrategies/fitxers/analisi-especialitzacio-intelligencia-artificial.pdf">http://catalunya2020.gencat.cat/web/.content/00_catalunya2020/Documents/estrategies/fitxers/analisi-especialitzacio-intelligencia-artificial.pdf</a></p> </li> </ul> <p> </p> <p><strong>Acknowledgements</strong></p> <ul> <li> <p>Tatiana Fernández (Direcció General de Promoció Econòmica, Competència i Regulació, de la Generalitat de Catalunya), </p> </li> <li> <p>Daniel Marco, Daniel Santanach and Eduard Balbuena (Departament de Polítiques Digitals i Administració Pública, de la Generalitat de Catalunya) </p> </li> <li> <p>Albert Sabater (Observatori d’Ètica en Intel·ligència Artificial i Universitat de Girona)</p> </li> <li> <p>Leda Bologni, Lucia Mazzoni and Giorgio Moretti (Art-ER)</p> </li> <li> <p>Prof. RIta Cucchiara and Dr. Lorenzo Baraldi (Università degli Studi di Modena e Reggio Emilia)</p> </li> <li> <p>Artificial Intelligence and Intelligence Systems (AIIS) Lab of CINI (Consorzio interuniversitario nazionale per l’informatica)</p> </li> </ul> <p> </p> <p><strong>Bibliography</strong></p> <p>Bigas, E., Duran, N., Fuster, E., Parra, C., Fernández, T. (2021): “Anàlisi de l’especialització en intel·ligència artificial”. Col·lecció Monitoratge de la RIS3CAT, Generalitat de Catalunya <a href="http://catalunya2020.gencat.cat/web/.content/00_catalunya2020/Documents/estrategies/fitxers/analisi-especialitzacio-intelligencia-artificial.pdf">http://catalunya2020.gencat.cat/web/.content/00_catalunya2020/Documents/estrategies/fitxers/analisi-especialitzacio-intelligencia-artificial.pdf</a></p> <p>Dunham, J.W., Melot, J., & Murdick, D. (2020). Identifying the Development and Application of Artificial Intelligence in Scientific Text. ArXiv, abs/2002.07143. Available at: <a href="https://arxiv.org/abs/2002.07143">https://arxiv.org/abs/2002.07143</a></p> <p>Mikolov, Tomas & Corrado, G.s & Chen, Kai & Dean, Jeffrey. (2013). Efficient Estimation of Word Representations in Vector Space. 1-12.</p> <p>Quinquillá, Arnau, Duran-Silva, Nicolau, Massucci, Francesco Alessandro, Fuster, Enric, Rondelli, Bernardo, Bologni, Leda, … Moretti, Giorgio. (2020). Text mining to identify skills, stakeholders and capabilities: the case of Artificial Intelligence in Emilia-Romagna. Zenodo. <a href="http://doi.org/10.5281/zenodo.3606342">http://doi.org/10.5281/zenodo.3606342</a>. Poster presented at: World Open Innovation Conference 2019 (WOIC); 11th december 2019, Rome, Italy.</p>
FG-OVD: Fine-grained Open-Vocabulary Object Detection Benchmark Suite
<p>A collection of annotations for PACO images containing free-form fine-grained textual captions of objects, their parts, and their attributes. It also comprises several sets of negative captions that can be used to test and evaluate the fine-grained recognition ability of open-vocabulary models.</p>
CLDF dataset derived from Greenhill and Gray's "Bantu Basic Vocabulary Database" from 2015
<p>Cite the source of the dataset as:</p> <blockquote> <p>Simon Greenhill and Russell Gray, 2015. Bantu Basic Vocabulary Database</p> </blockquote>
CLDF dataset derived from Tolmie and Dawson's "Comparative Vocabulary of the Indigenous Peoples in British Columbia" from 1884
<p>Cite the source of the dataset as:</p> <blockquote> <p>Tolmie, Fraser W. and Dawson, George M. (1884). Comparative vocabularies of the Indian tribes of British Columbia, with a map illustrating distribution. Montreal: Dawson Brothers.</p> </blockquote>
CLDF dataset derived from Blust's Austronesian data coded for the Austronesian Basic Vocabulary Database from 2008
<p>Cite the source of the dataset as:</p> <blockquote> <p>Greenhill, SJ; Blust, R and Gray, RD (2008): The Austronesian Basic Vocabulary Database: From bioinformatics to lexomics. Evolutionary Bioinformatics. 4. 271-283.</p> </blockquote>
CLDF dataset derived from Beijing University's "Chinese Dialect Vocabularies" from 1964
<p>Cite the source of the dataset as:</p> <blockquote> <p>Běijīng Dàxué 北京大学 (1964): Hànyǔ fāngyán cíhuì 汉语方言词汇 [Chinese dialect vocabularies]. Beijing: Wenzi Gaige.</p> </blockquote>
The SPECIAL Policy Log Vocabulary
<p>This documents specifies <em>splog</em>, a vocabulary to log data processing and sharing events that should comply with a given consent provided by a data subject. We also model the consent actions related to consent giving and revocation.</p> <p> </p> <p>See more at: <a href="http://purl.org/specialprivacy/splog">http://purl.org/specialprivacy/splog</a></p>
Turkic basic vocabularies
<p>The dataset represents basic vocabulary data across 32 Turkic languages. The basic vocabulary list merges the Leipzig-Jakarta 200 list (Haspelmath and Tadmor 2009) with the Jena 200 list (Anderson and Heggarty n.d.) and contains 254 different concepts. For each word in the dataset we provide an etymological analysis to establish cognacy classes on the basis of regular sound correspondences. Borrowings that can be identified using clearcut historical comparative criteria are excluded to provide a clearer phylogenetic signal. We deal with cases of synonymy in that we allow more than one word with a certain basic meaning in our dataset unless there is evidence that it is less basic than one of its synonyms. Singletons are removed from the dataset in case they have a non-singleton synonym that fits the criteria for basic status. The dataset also contains the tsv file edited in the EDICTOR tool (List 2017) as well as the file fed to BEAST in the original nexus format and in the XML format required by BEAST.</p>
CLDF dataset derived from Huber and Reed's "Comparative Vocabulary" from 1992
<p>Cite the source of the dataset as:</p> <blockquote> <p>Huber, R. Q. and Reed, R. B. 1992. Vocabulario comparativo: palabras selectas de lenguas indígenas de Colombia [Comparative vocabulary. Selected words from the indigenous languages of Columbia]. Santa Fé de Bogota: Asociación Instituto Lingüístico de Verano.</p> </blockquote>
CLDF dataset derived from Greenhill et al.'s "Austronesian Basic Vocabulary Database" from 2020 focusing on Philippine languages
<p>Cite the source of the dataset as:</p> <blockquote> <p>Greenhill, S.J., Blust. R, & Gray, R.D. (2008). The Austronesian Basic Vocabulary Database: From Bioinformatics to Lexomics. Evolutionary Bioinformatics, 4:271-283.</p> </blockquote>
Machine-Readable Vocabulary Files of the "Alter Realkatalog" (ARK) of Berlin State Library (SBB)
<p>This dataset contains two versions of vocabulary files of the <a href="https://ark.staatsbibliothek-berlin.de/">ARK (Alter Realkatalog)</a> in .tsv and .ttl format used for training models for automatic subject indexing with the modular <a href="https://github.com/NatLibFi/Annif">Annif</a> tool. As the ARK is a historical classification system which has been used to describe historical works in the Staatsbibliothek zu Berlin – Berlin State Library’s collections up to 1955, this dataset has been created for generating automatic indexing suggestions for historical texts which have not yet been manually classified with the help of the ARK (for a detailed description of the ARK, see also <a href="../doi/10.5281/zenodo.12783813">Metadata of the "Alter Realkatalog" (ARK) of Berlin State Library (SBB)</a>. Together with specific corpus training data, these vocabulary files serve as input to Annif, with which the corresponding models on <a href="https://huggingface.co/SBB">Hugging Face at the Staatsbibliothek zu Berlin – Preußischer Kulturbesitz</a> community have been created. Associated corpus training data have been extracted from the <a href="../doi/10.5281/zenodo.12783813" target="_blank" rel="noopener">Metadata of the "Alter Realkatalog" (ARK)</a> (title data).</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.