Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
15
datasets available to search
ShareScore release 0.9.0
Dataset results
15 results for “Fasttext”
German CBOW FastText embeddings with min count 250
<p>FastText embeddings built from Common Crawl german dataset</p> <table> <caption>Parameters</caption> <thead> <tr> <th scope="col">Parameters</th> <th scope="col">Value(s)</th> </tr> </thead> <tbody> <tr> <td>Dimensions</td> <td>256 and 384</td> </tr> <tr> <td>Context window</td> <td>5</td> </tr> <tr> <td>Negative sampled</td> <td>10</td> </tr> <tr> <td>Epochs</td> <td>1</td> </tr> <tr> <td>Number of buckets</td> <td>131072 or 262144</td> </tr> <tr> <td>Min n</td> <td>3</td> </tr> <tr> <td>Max n</td> <td>6</td> </tr> </tbody> </table>
Quantized versions of fastText embeddings for 158 languages
<p>These are compressed (quantized) versions of word embeddings for 158 languages originally available from https://fasttext.cc/docs/en/crawl-vectors.html . Following steps were performed to reduce file size:</p> <ol> <li>the output matrix is discarded</li> <li>the quantization is performed with parameters "-qnorm -dsub 1"</li> </ol> <p>The average cosine similarity between original and quantized vectors for frequent words is 0.99. The file size is 4-6 times smaller.</p>
FastText Spanish Medical Embeddings
<p>[Plan TL/medicine/word embeddings] Word embeddings generated from Spanish corpora that include: (a) the full-text in Spanish available in SciELO.org (until December/2018), (b) all articles from the following Wikipedia categories: Pharmacology, Pharmacy, Medicine and Biology (during December/2018) and (c) the concatenation of the previous two corpora.</p> <p>We used fastText to train the word embeddings.</p> <p>For more information, we refer to the corresponding article: <a href="https://www.aclweb.org/anthology/W19-1916/">https://www.aclweb.org/anthology/W19-1916/</a></p> <p>Copyright (c) 2021 Secretaría de Estado de Digitalización e Inteligencia Artificial</p>
French CBOW FastText embeddings with min count 250
<p>FastText embeddings built from Common Crawl French dataset</p> <table> <caption>Parameters</caption> <thead> <tr> <th scope="col">Parameters</th> <th scope="col">Value(s)</th> </tr> </thead> <tbody> <tr> <td>Dimensions</td> <td>512</td> </tr> <tr> <td>Context window</td> <td>5</td> </tr> <tr> <td>Negative sampled</td> <td>10</td> </tr> <tr> <td>Epochs</td> <td>1</td> </tr> <tr> <td>Number of buckets</td> <td>262144</td> </tr> <tr> <td>Min n</td> <td>3</td> </tr> <tr> <td>Max n</td> <td>6</td> </tr> </tbody> </table>
Social Sciences Word Embeddings in FastText
<p>These social science word embeddings in FastText have been created from 37,604 open access social science research papers from the social science access repository (https://www.gesis.org/ssoar/home). They are available in German and English.</p> <p>(skipgram model, n-grams with n≥3 and n≤6, different dimensions (100, 150, 200, 300, 500), five epochs, learning rate 0.05, five negative examples)</p> <p>Please cite:</p> <p>Schiffers, Ricardo, Dagmar Kern, and Daniel Hienert. 2022. "Evaluation of Word Embeddings for the Social Sciences." In <em>Proceedings of the 6th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature</em>, edited by Stefania Degaetano, Anna Kazantseva, Nils Reiter, and Stan Szpakowicz, 1-6. Gyeongju: Association for Computational Linguistics. <a href="https://aclanthology.org/2022.latechclfl-1.1">https://aclanthology.org/2022.latechclfl-1.1</a>.</p>
Spanish Skip-Gram Word Embeddings in FastText
<p>These Spanish word embeddings in FastText have been generated from the largest corpus ever made in Spanish till date. The corpus has more than 2TB of high-quality text, compiled from the different web crawlings done by the National Library of Spain from 2009 to 2019. </p> <p>These are the SKIP-GRAM embeddings, for the CBOW embeddings see: https://zenodo.org/record/5044988</p> <p><strong>Citation</strong></p> <pre><code>@article{gutierrezfandino2022, author = {Asier Gutiérrez-Fandiño and Jordi Armengol-Estapé and Marc Pàmies and Joan Llop-Palao and Joaquin Silveira-Ocampo and Casimiro Pio Carrino and Carme Armentano-Oller and Carlos Rodriguez-Penagos and Aitor Gonzalez-Agirre and Marta Villegas}, title = {MarIA: Spanish Language Models}, journal = {Procesamiento del Lenguaje Natural}, volume = {68}, number = {0}, year = {2022}, issn = {1989-7553}, url = {http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6405}, pages = {39--60} }</code></pre> <p><strong>Copyright </strong></p> <p>Copyright (c) 2021 Secretaría de Estado de Digitalización e Inteligencia Artificial</p>
Spanish CBOW Word Embeddings in FastText
<p>These Spanish word embeddings in FastText have been generated from the largest corpus ever made in Spanish till date. The corpus has more than 2TB of high-quality text, compiled from the different web crawlings done by the National Library of Spain from 2009 to 2019. </p> <p>These are the CBOW embeddings, for the SKIP-GRAM embeddings see: https://zenodo.org/record/5046525</p> <p><strong>Citation</strong></p> <pre><code>@article{gutierrezfandino2022, author = {Asier Gutiérrez-Fandiño and Jordi Armengol-Estapé and Marc Pàmies and Joan Llop-Palao and Joaquin Silveira-Ocampo and Casimiro Pio Carrino and Carme Armentano-Oller and Carlos Rodriguez-Penagos and Aitor Gonzalez-Agirre and Marta Villegas}, title = {MarIA: Spanish Language Models}, journal = {Procesamiento del Lenguaje Natural}, volume = {68}, number = {0}, year = {2022}, issn = {1989-7553}, url = {http://journal.sepln.org/sepln/ojs/ojs/index.php/pln/article/view/6405}, pages = {39--60} }</code></pre> <p><strong>Copyright </strong></p> <p>Copyright (c) 2021 Secretaría de Estado de Digitalización e Inteligencia Artificial</p>
German CBOW FastText embeddings with min count 100
<p>FastText embeddings built from Common Crawl german dataset</p> <table> <caption>Parameters</caption> <thead> <tr> <th scope="col">Parameters</th> <th scope="col">Value(s)</th> </tr> </thead> <tbody> <tr> <td>Dimensions</td> <td>256 and 384</td> </tr> <tr> <td>Context window</td> <td>5</td> </tr> <tr> <td>Negative sampled</td> <td>10</td> </tr> <tr> <td>Epochs</td> <td>1</td> </tr> <tr> <td>Number of buckets</td> <td>131072 or 262144</td> </tr> <tr> <td>Min n</td> <td>3</td> </tr> <tr> <td>Max n</td> <td>6</td> </tr> </tbody> </table> <p> </p> <p> </p>
Ancient Greek Fasttext Word Embeddings
<p>Word embeddings generated with Fasttext and 1 GB of Ancient Greek texts. These embeddings were produced for the study of social networks and social semantics in ancient Greece by the Diogenet project at the University of San Diego, California. </p>
Wbbyyr: FastText language models for Mandarin Chinese, trained on 14m Sina Weibo posts for each year in 2012-2018 (Fold 1 of 10)
<p>Wbbyyr: FastText language models for Mandarin Chinese, trained on 14,440,000 Sina Weibo posts for each year in 2012-2018.</p> <p>The 14,440,000 posts from each year are split into 10 folds. Due to Zenodo size limit, this dataset contains only the first fold from each year.</p> <p>Each model is trained for 20 iterations. Each vector is 300 dimensions long.</p>
English and Portuguese CBOW Models from Europarl Corpus, version 7, using FastText with Subwords Option
<p>The models were trained using FastText, model CBOW, 40 epochs, and subwords. Each *.BIN file has its *.VEC file with the vocabulary ordered by frequency. The *.BIN file can return a vector to represent an out-of-vocabulary (OOV) word if the necessary parts of the OOV word were used in training. FastText and Gensim can use these files. The English and Portuguese models are identified in the file name, <strong>_en_</strong> and <strong>_pt_</strong> respectively.</p> <p>An Excel file has the neighborhood changes of some selected words during training on each epoch. A previous exercise to find words with more than one meaning.</p>
FastText embeddings obtained from SNOMED CT - Spanish and International
<p>FastText model trained in a corpus that was obtained from SNOMED CT by performing random walks. There are three models: one trained using all relations available in the international version of SNOMED CT, another one trained using only is_a relations from the international version of SNOMED and the last one is trained using all relations from the Spanish version of SNOMED CT. This was developed for the end project of the Master Degree in Artificial Intelligence in the Universidad Politécnica de Madrid. More information and the document of the project can be found in <a href="https://github.com/JavierCastellD/SemanticFormalizationSNOMED">https://github.com/JavierCastellD/SemanticFormalizationSNOMED</a>.</p>
Italian FastText models
<p>Italian FastText models trained from scratch on a dataset composed of:</p> <p>- <strong>wiki</strong>: a dump of Italian Wikipedia (as of December 15, 2022), comprising 25,548,651 sentences and 526,640,982 words (3.2 GB of raw text);<br> - <strong>webz</strong>: a dataset of Italian news (159,226 documents) from the webz.io platform, crawled in October 2015, containing 44,041,823 sentences and 44,544,385 words (244 MB);<br> - a dataset of 5,510 Italian news articles from the newspaper ModenaToday (<strong>MT</strong>) or 15,115 documents from the Italian version of Reuters (<strong>RCV2</strong>).</p> <p><strong>ft_wiki_wbz_mt_20_epochs</strong>: FastText model trained on the dataset consisting of wiki, webz, and MT for 20 epochs</p> <p><strong>ft_wiki_wbz_mt_50_epochs</strong>: FastText model trained on the dataset consisting of wiki, webz, and MT for 50 epochs</p> <p><strong>ft_wiki_wbz_reut_20_epochs</strong>: FastText model trained on the dataset consisting of wiki, webz, and RCV2 for 20 epochs</p> <p><strong>ft_wiki_wbz_reut_50_epochs</strong>: FastText model trained on the dataset consisting of wiki, webz, and RCV2 for 50 epochs</p>
Clean-Clean ER Datasets with FastText embeddings
<p>Please see <a href="https://zenodo.org/records/8433873">version 5 </a>for more datasets. This one contains only a print-out of the DBPedia dataset in CSV format.</p><p>Each line corresponds to a different entity profile and has the following structure (where n is number of attributes):</p><p>numerical_id , uri , n , aname_0 , aval_0 , aname_1 , aval_1 ,...aname_n , aval_n </p><p>That is, the separator is "space,space".</p><p>Special thanks to <a href="https://github.com/moemode">moemode </a>for providing the dataset.</p>
English fastText Wikipedia embeddings
<p>This repository contains the fastText embeddings, pre-trained on English Wikipedia.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.