Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
187
datasets available to search
ShareScore release 0.9.0
Dataset results
187 results for “Language Data”
Data from: DCDC2 READ1 regulatory element: how temporal processing differences may shape language
<p>Classic linguistic theory ascribes language change and diversity to population migrations, conquests, and geographic isolation, with the assumption that human populations have equivalent language processing abilities. We hypothesize that spectral and temporal characteristics make some consonant manners vulnerable to differences in temporal precision associated with specific population allele frequencies. To test this hypothesis, we modeled association between RU1-1 alleles of <i>DCDC2</i> and manner of articulation in 51 populations spanning five continents, and adjusting for geographic proximity, genetic and linguistic relatedness. RU1-1 alleles, acting through increased expression of <i>DCDC2</i>, appear to increase auditory processing precision that enhances stop-consonant discrimination, favoring retention in some populations and loss by others. These findings enhance classical linguistic theories by adding a genetic dimension, which until recently, has not been considered to be a significant catalyst for language change.</p>
Metadata Profile for FAIR Sensor Data based on the SensOr Interfacing Language
<p>Metadata profile to provide FAIR sensor data. The profile is created using SHACL and is based on the SOSA ontology which accurately specifies restrictions on the properties of specific sensors using the QUDT and the SSN ontology. Generic metainformation is modeled using DCTerms. </p>
Bangru Language Data - Cut sound files
<p>These files form the empirical basis for the following article:</p> <p>Bodt, Timotheus Adrianus and Ismael Lieberherr. 2015. First notes on the phonology and classification of the Bangru language of India. <em>Linguistics of the Tibeto-Burman Area 38:1</em> (2015), 66–123.</p> <p>doi 10.1075/ltba.38.1.03bod</p> <p>issn 0731–3500 / e-issn 2214–5907 © John Benjamins Publishing Company</p> <p>These data were collected in Sarli circle, Kurung Kumey district, Arunachal Pradesh, India.</p> <p>The data collectors were the following faculty, students and associated researchers of the Department of English and Foreign Languages, Tezpur University, Assam, India:</p> <p>Nupur Sinha (Faculty), Ismael Lieberherr (Affiliated PhD scholar), Timotheus A. Bodt (Affiliated PhD scholar), Diksha Konwar, Eshani Baishya, Nawaf Helmi, Pinaz Mirza, Ratul Mahela, Sansuma Brahma (students).</p> <p>This material is made freely available to everyone for informative or scientific purposes as long as the source (this DOI) / the collectors are properly credited. Please note that use of the material for commercial purposes <em><strong>of any kind</strong>, which includes conversion into commercial audio-visual media (documentaries etc.), storage and dissemination through sites that require registration & payment for access, or sites that rely on advertisement (including YouTube) </em>is <strong>not</strong> permitted without <strong>specific written consent</strong> from the speakers and their community, obtained through the collectors of the material. By downloading our material, you agree to these restrictions.</p> <p>This data set falls under the Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) license. This license lets you remix, tweak, and build upon this work non-commercially, as long as you credit us and license your new creations under the identical terms. License Deed on <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/">https://creativecommons.org/licenses/by-nc-sa/4.0/</a>. Legal Code on <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode">https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode</a>.</p>
Data for 'VespaG: Expert-guided protein language models enable accurate and blazingly fast fitness prediction'
<div>Datasets used for development of VespaG and VespaG predictions generated with <a href="https://github.com/JSchlensok/VespaG">https://github.com/JSchlensok/VespaG</a>. </div> <div> </div> <div>Uploads contain:</div> <div> <ol> <li><strong>Performance</strong> summaries for ProteinGym [1]:<br>- Spearman and Pearson correlation for VespaG: <em>proteingym_performance_vespag.csv </em>(columns: 'DMS_id', 'Spearman', 'Pearson')<br>- Spearman correlation for evaluated methods VespaG, GEMME [2], VESPA [3], TranceptEVE [4], AlphaMissense [5], PoET [6]: <em>proteingym_spearman_allmethods.csv </em>(columns: 'DMS_id', 'Trancept EVE-L', 'VESPA', 'VespaG', 'GEMME', 'AlphaMissense', 'PoET', 'UniProt_ID', 'coarse_selection_type' (function), 'taxon')</li> <li><strong>Fasta</strong> files with sequences for all train sets (<em>vespag_fasta_training_datasets.zip</em> with seq_all9k.fasta, seq_human5k.fasta, seq_droso4k.fasta, seq_ecoli2k.fasta, seq_virus1k.fasta) and test set (<em>proteingym_217.fasta</em>)</li> <li><strong>VespaG</strong> <strong>Predictions</strong> for test set: <em>vespag_proteingym_rawpreds_by_training_dataset.zip</em> with raw_preds_ecoli.csv, raw_preds_human.csv, raw_preds_virus.csv, raw_preds_all.csv, raw_preds_droso.csv (columns: 'DMS_id', 'mutation', 'DMS_score', 'VespaG'). Predictions are based on different training data, the final model VespaG was trained on a subset of the human proteome and <strong>raw VespaG predictions</strong> <strong>for</strong> <strong>the</strong> <strong>ProteinGym benchmark are in raw_preds_human.csv </strong>(used to calculate the performances above).</li> <li><strong>GEMME predictions</strong> for train sets: <em>vespag_proteingym_rawpreds_by_training_dataset.zip </em>with folders 'human', 'droso', 'ecoli', 'virus', 'all' for respective fasta file (each containing GEMME mutational landscape output files named '<em>ID' + '</em>_normPred_evolCombi.txt')</li> <li><strong>ESM-2</strong> <strong>embeddings</strong> [7] for test set (<em>proteingym_217_esm2.h5</em>)</li> </ol> </div> <div>For details on VespaG see:</div> <div> <div> <div>VespaG: Expert-guided protein Language Models enable accurate and blazingly fast fitness prediction</div> </div> <div>Celine Marquet, Julius Schlensok, Marina Abakarova, Burkhard Rost, Elodie Laine</div> <div>bioRxiv 2024.04.24.590982; doi: https://doi.org/10.1101/2024.04.24.590982</div> <div> </div> <div>For more information on data usage and generation please see <a href="https://github.com/JSchlensok/VespaG">https://github.com/JSchlensok/VespaG</a>.</div> <div> </div> <div>Abstract:</div> <div>Exhaustive experimental annotation of the effect of all known protein variants remains daunting and expensive, stressing the need for scalable effect predictions. We introduce VespaG, a blazingly fast single amino acid variant effect predictor, leveraging embeddings of protein Language Models as input to a minimal deep learning model. To overcome the sparsity of experimental training data, we created a dataset of 39 million single amino acid variants from the human proteome applying the multiple sequence alignment-based effect predictor GEMME as a pseudo standard-of-truth. Assessed against the ProteinGym Substitution Benchmark (217 multiplex assays of variant effect with 2.5 million variants), VespaG achieved a mean Spearman correlation of 0.48 +/- 0.01, matching state-of-the-art methods such as GEMME, TranceptEVE, PoET, AlphaMissense, and VESPA. VespaG reached its top-level performance several orders of magnitude faster, predicting all mutational landscapes of the human proteome in 30 minutes on a consumer laptop (12-core CPU, 16 GB RAM).</div> <div> </div> <div>[1] Notin, Pascal, et al. "ProteinGym: large-scale benchmarks for protein fitness prediction and design." <em>Advances in Neural Information Processing Systems</em> 36 (2024).<br>[2] Laine, Elodie, Yasaman Karami, and Alessandra Carbone. "GEMME: a simple and fast global epistatic model predicting mutational effects." <em>Molecular biology and evolution</em> 36.11 (2019): 2604-2619.</div> <div>[3] Marquet, Céline, et al. "Embeddings from protein language models predict conservation and variant effects." <em>Human genetics</em> 141.10 (2022): 1629-1647.</div> <div>[4] Notin, Pascal, et al. "TranceptEVE: Combining family-specific and family-agnostic models of protein sequences for improved fitness prediction." <em>bioRxiv</em> (2022): 2022-12.</div> <div>[5] Cheng, Jun, et al. "Accurate proteome-wide missense variant effect prediction with AlphaMissense." <em>Science</em> 381.6664 (2023): eadg7492.</div> <div>[6] Truong Jr, Timothy, and Tristan Bepler. "PoET: A generative model of protein families as sequences-of-sequences." <em>Advances in Neural Information Processing Systems</em> 36 (2024).</div> <div>[7] Lin, Zeming, et al. "Evolutionary-scale prediction of atomic-level protein structure with a language model." <em>Science</em>379.6637 (2023): 1123-1130.</div> </div>
Data and code from: Learning a deep language model for microbiomes: The power of large scale unlabeled microbiome data
<p>We use open source human gut microbiome data to learn a microbial "language" model by adapting techniques from Natural Language Processing (NLP). Our microbial "language" model is trained in a self-supervised fashion (i.e., without additional external labels) to capture the interactions among different microbial species and the common compositional patterns in microbial communities. The learned model produces contextualized taxa representations that allow a single bacteria species to be represented differently according to the specific microbial environment it appears in. The model further provides a sample representation by collectively interpreting different bacteria species in the sample and their interactions as a whole. We show that, compared to baseline representations, our sample representation consistently leads to improved performance for multiple prediction tasks including predicting Irritable Bowel Disease (IBD) and diet patterns. Coupled with a simple ensemble strategy, it produces a highly robust IBD prediction model that generalizes well to microbiome data independently collected from different populations with substantial distribution shift.</p> <p>We visualize the contextualized taxa representations and find that they exhibit meaningful phylum-level structure, despite never exposing the model to such a signal. Finally, we apply an interpretation method to highlight bacterial species that are particularly influential in driving our model's predictions for IBD.</p>
Figure 1. Data Division for NetworkTraining-Declarative vs. Procedural Memory: Roles in Second Language Acquisition
<p>Once the historical stock prices are gathered ,now this is the time for data selection for<br> training,testing and simulating the network.In this project we took 4 years historical price of any<br> stock ,means total 1460 working days data.We done R/S analysis over these datafor<br> predictability(Hurst exponent analysis).Now The Hurst exponent (H) is a statistical measure used to<br> classify time series. H=0.5 indicates a random series while H>0.5 indicates a trend reinforcing<br> series.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 9b. Translation of a tab for all languages
<p>After checking a few (or all!) languages for instance and pressing the Ok button , we obtain the windows shown in figure 9a, or respectively 9b for all languages. Here we can add one or more missing translations, or modify one or more of the existing translations accordingly. All the data within the view cluster can be translated into any language supported by the system.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 8. GoTo -> Translation Option
<p>For example, we select in the view cluster the tab called “Forecasting” and then choose Goto -> Translation (figure 8). After selecting Goto -> Translation, we obtain a selecting window for the desired languages where the user can check one or more languages to translate those tabs or areas into. </p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 9a. Translation of a tab for certain languages
<p>After checking a few (or all!) languages for instance and pressing the Ok button , we obtain the windows shown in figure 9a, or respectively 9b for all languages. Here we can add one or more missing translations, or modify one or more of the existing translations accordingly. All the data within the view cluster can be translated into any language supported by the system.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 7. Dialog structure of the View Cluster
<p>Field dependency has been generated automatically and as a result the following desired structure has been achieved (figure 7). After populating the database tables with data in different languages with the help of the view cluster, the popup has been adapted in order to support the translation of the tabs and areas. Supplementary internal tables have been defined in the function module POPUP_FLEX, in order to copy data from the translation tables. The corresponding SELECT statements have been embedded in TRY-CATCH blocks, in order to prevent short dumps due to faulty selection processes.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 6. View cluster TAFC
<p>For all these tables and views, table maintenance generators have been created and activated, in order to have the possibility to manage individual datasets in every table and view. The corresponding names of those function groups for the table maintenance generators are the same names as those for the views. The purpose of these maintenance views is only to take care of the input data more efficiently. These views will be used later in the view cluster, which ensures a hierarchical order of the data. Therefore, the maintenance views will also include the predecessor, in order to facilitate linking in the field dependency tab of the view cluster (Swapna, 2007). These three maintenance views are embedded in the following view cluster (figure 6)</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 5. Maintenance Views
<p>Beside the tabs and the areas, the database table FLD also contains the fields TABLENAME and FIELD, which suggest the related parameters, whom input may be updated at runtime. Through standard SAP functionality the tables in BASIS, respectively their fields are by default translated in the login language of the user. So in the fields TABLENAME and FIELD of the FLD table, we will obtain, in the user login language, the names of the tables and fields from BASIS via the foreign keys to the table DD03L for TABLENAME and to the table DD02L for FIELD. Beside the database tables, 3 maintenance views have been created, TABV, AREV and FLDV, for the tabs, areas and fields of the popup (figure 5). We have chosen maintenance views instead of database views to be able to use them in the view cluster.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 2. Content of the single database table in the previous implementation
<p>In the previous implementation there has been used a single database table which did not provide a consistent overview of existing tabs, areas and fields, as well as of the languages, in which a specific field of an area or tab was translated. So, it was difficult to maintain this database table by the customizing end-users in different languages, because every update of a tab, area or field required a number of actions in this table which had to be done manually and very carefully, requiring much time and attention. The number of rows of this table was very large and the content looked like the one shown in figure 2. </p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 4. Table structure and relationships
<p>The structure of the tables, primary keys and foreign keys are shown in figure 4.The names of the fields in the database tables are relevant for their content. Only the SPRAS field in the translation-tables TABT and ARET must be explained: SPRAS is a system-field which stands for the language and is used in order to maintain the languages in which the tab/area is translated into.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 1. Popup layout for the Material Master Data Application
<p>The Material Master Data Application provides an update popup layout, including tabs, areas and fields (also customer-specific fields). Each tab consists of one or more areas and each area of one or more fields, similar to the example below (figure 1). The application is called flexible because the user must have the possibility to add, delete, reorder or rename tabs, areas and fields.</p>
BRAIN Journal-Redesigning a Flexible Material Master Data Application with Language Dependency-Figure 3. The logical data model of tables and views
<p>The five tables, named TAB, TABT, AREA, ARET and FLD, are combined within three views (TABV, AREV and FLDV) which build a cluster view, TAFC (figure 3). </p>
'Retrodiction' experiment Western Kho-Bwa languages: data
<p>These files contain all the data belonging to the retro-diction/pre-diction experiment in a historical-comparative linguistic study on the Western Kho-Bwa languages. This data set has all the raw as well as cut sound files of all the concepts elicited in the field work sessions in Arunachal Pradesh.</p> <p>The research was registered online as:</p> <p>Bodt, Timotheus A., Nathan W. Hill and Johann-Mattis List. 2018. <em>Prediction experiment for missing words in Kho-Bwa language data.</em> Open Science Framework Preregistrations October 5. <a href="https://osf.io/evcbp/">https://osf.io/evcbp/</a> </p> <p> </p>
Dataset for: "Big data suggest strong constraints of linguistic similarity on adult language learning"
<p>This dataset is adapted from raw data with fully anonymized results on the State Examination of Dutch as a Second Language. This exam is officially administred by the Board of Tests and Examinations (College voor Toetsen en Examens, or CvTE). See cvte.nl/about-cvte. The Board of Tests and Examinations is mandated by the Dutch government.</p> <p>The article accompanying the dataset:</p> <p>Schepens, Job, Roeland van Hout, and T. Florian Jaeger. “Big Data Suggest Strong Constraints of Linguistic Similarity on Adult Language Learning.” <em>Cognition</em> 194 (January 1, 2020): 104056. <a href="https://doi.org/10.1016/j.cognition.2019.104056">https://doi.org/10.1016/j.cognition.2019.104056</a>.</p> <p>Every row in the dataset represents the first official testing score of a unique learner.<br> The columns contain the following information as based on questionnaires filled in at the time of the exam:</p> <p>"L1" - The first language of the learner<br> "C" - The country of birth<br> "L1L2" - The combination of first and best additional language besides Dutch<br> "L2" - The best additional language besides Dutch<br> "AaA" - Age at Arrival in the Netherlands in years (starting date of residence)<br> "LoR" - Length of residence in the Netherlands in years<br> "Edu.day" - Duration of daily education (1 low, 2 middle, 3 high, 4 very high). From 1992 until 2006, learners' education has been measured by means of a side-by-side matrix question in a learner's questionnaire. Learners were asked to mark which type of education they have had (elementary, secondary, or tertiary schooling) by means of filling in for how many years they have been enrolled, in which country, and whether or not they have graduated. Based on this information we were able to estimate how many years learners have had education on a daily basis from six years of age onwards. Since 2006, the question about learners' education has been altered and it is asked directly how many years learners have had formal education on a daily basis from six years of age onwards. Possible answering categories are: 1) 0 thru 5 years; 2) 6 thru 10 years; 3) 11 thru 15 years; 4) 16 years or more. The answers have been merged into the categorical answer.<br> "Sex" - Gender<br> "Family" - Language Family<br> "ISO639.3" - Language ID code according to Ethnologue<br> "Enroll" - Proportion of school-aged youth enrolled in secondary education according to the World Bank. The World Bank reports on education data in a wide number of countries around the world on a regular basis. We took the gross enrollment rate in secondary schooling per country in the year the learner has arrived in the Netherlands as an indicator for a country's educational accessibility at the time learners have left their country of origin.<br> "STEX_speaking_score" - The STEX test score for speaking proficiency.<br> "Dissimilarity_morphological" - Morphological similarity<br> "Dissimilarity_lexical" - Lexical similarity<br> "Dissimilarity_phonological_new_features" - Phonological similarity (in terms of new features)<br> "Dissimilarity_phonological_new_categories" - Phonological similarity (in terms of new sounds)</p> <p><br> A few rows of the data:</p> <p>"L1","C","L1L2","L2","AaA","LoR","Edu.day","Sex","Family","ISO639.3","Enroll","STEX_speaking_score","Dissimilarity_morphological","Dissimilarity_lexical","Dissimilarity_phonological_new_features","Dissimilarity_phonological_new_categories"<br> "English","UnitedStates","EnglishMonolingual","Monolingual",34,0,4,"Female","Indo-European","eng ",94,541,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishGerman","German",25,16,3,"Female","Indo-European","eng ",94,603,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishFrench","French",32,3,4,"Male","Indo-European","eng ",94,562,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishSpanish","Spanish",27,8,4,"Male","Indo-European","eng ",94,537,0.0094,0.083191,11,19<br> "English","UnitedStates","EnglishMonolingual","Monolingual",47,5,3,"Male","Indo-European","eng ",94,505,0.0094,0.083191,11,19</p>
Systematic mapping data for translation-enabling technologies for sign languages
<p>These data correspond to the papers selected for a systematic mapping of that translation-enabling technologies for sign languages.</p>
Language modeling data for Swahili
<p>The Swahili dataset developed specifically for language modeling task. The dataset contains 28,000 unique words with 6.84M, 970k, and 2M words for the train, valid and test partitions respectively which represent the ratio 80:10:10. The entire dataset is lowercased, has no punctuation marks and, the start and end of sentence markers have been incorporated to facilitate easy tokenization during language modeling. The train partition is the largest in order to support unsupervised learning of word representations while the hyper-parameters are adjusted based on the performance on the valid partition before evaluating the language model on the test partition.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.