Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
59
datasets available to search
ShareScore release 0.9.0
Dataset results
59 results for “sparse data”
Data for Extremely sparse models of linkage disequilibrium in ancestrally diverse association studies
<p>Data from <em>Extremely sparse models of linkage disequilibrium in ancestrally diverse association studies </em>(2023). This includes linkage disequilibrium graphical models (LDGMs) created from <a href="https://www.biorxiv.org/content/10.1101/2021.02.06.430068v2">high-coverage 1000 Genomes Project sequencing data</a>. This dataset consists of LDGM precision matrices, LDGM graphical models of SNPs, and lists of SNPs, all split into <a href="https://www.biorxiv.org/content/10.1101/2022.03.04.483057v1">1,361 approximately independent LD blocks across the genome</a>. The dataset additionally contains genotype information from chromosomes 21 and 22, and inferred tree sequences of high coverage 1000 Genomes Project Data, summary statistics from four traits in the UK Biobank, and UK biobank correlation matrices from chromosomes 21 and 22. All genomic data is in the GRCh38 build.</p> <p>The data can be cited as follows:</p> <p>Pouria Salehi Nowbandegani, Anthony Wilder Wohns, Jenna L. Ballard, Eric S. Lander, Alex Bloemendal, Benjamin M. Neale, and Luke J. O’Connor. Extremely sparse models of linkage disequilibrium in ancestrally diverse association studies. Nat Genet. (2023) DOI: 10.1038/s41588-023-01487-8</p> <p> </p> <p>The directory contains `.tar.gz` files, which can be extracted and unzipped with:</p> <pre><code class="language-bash">$ tar -xvf FILENAME.tar.gz</code></pre> <p>All LD block files are named by chromosome and start/end basepair coordinates.</p> <ul> <li> <p>1kg_nygc_trios_removed_All_pops_geno_ids_pops.csv: The file contains 5008 rows, 2 for each individual in the 1000 Genomes Project. Each row contains the individual ID of the 1000 genomes individual, and the ancestry group and continental ancestry group that individual was assigned to. Rows correspond to columns in `.genos` files. </p> </li> <li><em>AFR/AMR/EAS/EUR/SAS.precision.tar.gz</em>: Precision matrices for the relevant ancestry group for each LD block. Edge lists contain one row for each non-zero entry of the precision matrix. There are no column names.</li> <li><em>genos_chr21_22.tar.gz</em>: for the 40 LD blocks on chromosomes 21-22, .genos files are 0/1 matrices, with dimension number-of-SNPs by number-of-samples . Each LD matrix contains one column for each row in the SNP list files, and one row for each row in the sample ID files.</li> <li><em>ldgms.tar.gz:</em> 1361 LDGMs (*.edgelist files). Edge lists contain one row for each non-zero entry of the LDGM adjacency matrix. There is one LDGM edge list for each LD block. Each row represents an edge, as a tuple (index_1, index_2, entry). For the LDGM adjacency matrices, the entry is the edge weight, where 0 represents a strong dependency and e.g. 6 represents a weak dependency.</li> <li><em>snplists_GRch38positions.tar.gz</em>: 1361 *.snplist files, each of which contains information on the SNPs in each LD block. Each SNP list is an <em>n</em> x 11 table (<em>n </em>=<em> </em>number of SNPs<em>)</em>, one for each LD block. The columns are: <ul> <li> <p>index: these non-unique indices, starting at zero, correspond to rows and columns of the LDGMs. There can be multiple SNPs for a single index, which occurs when the corresponding mutations occur on the same brick of the bricked tree sequence. SNPs with the same index have high (nearly perfect) LD.</p> </li> <li> <p>anc_alleles: ancestral allele</p> </li> <li> <p>deriv_alleles: derived allele</p> </li> <li> <p>EUR: allele frequency of derived allele in EUR samples</p> </li> <li> <p>EAS: allele frequency of derived allele in EAS samples</p> </li> <li> <p>AMR: allele frequency of derived allele in AMR samples</p> </li> <li> <p>SAS: allele frequency of derived allele in SAS samples</p> </li> <li> <p>AFR: allele frequency of derived allele in AFR samples</p> </li> <li> <p>site_ids: unique identifier of each SNP, mostly as RSIDs</p> </li> <li> <p>position: GRCh38 position of SNP</p> </li> <li> <p>swap: indicates strandness swap</p> </li> </ul> </li> <li> <p><em>ukb.tar</em>: Correlation matrices and SNP lists for SNPs in the UK Biobank.</p> <ul> <li> <p>correlation_matrices/: Correlation matrices for SNPs in the UK biobank, computed by Weissbrod et al. 2020 Nat Genet and can be downloaded by following the instructions <a href="https://alkesgroup.broadinstitute.org/UKBB_LD">here</a>.</p> </li> <li> <p>snplists/: List of SNPs in the *.snplist format included in the UK Biobank</p> </li> </ul> </li> <li> <p><em>tree_seqs.tar</em>: contains 22 tree sequences inferred by <a href="https://tsinfer.readthedocs.io">tsinfer</a> from the <a href="https://www.biorxiv.org/content/10.1101/2021.02.06.430068v2">30x 1000 Genomes Project Data</a>. Tree sequences can be unzipped with <a href="https://tszip.readthedocs.io/en/latest/">tszip</a>.</p> </li> <li> <p>Summary statistics: there are four summary statistics files, obtained from <a href="https://alkesgroup.broadinstitute.org/UKBB/">https://alkesgroup.broadinstitute.org/UKBB/</a>, and computed by Loh et al. 2018 Nat Genet.</p> </li> </ul> <table> <tbody> <tr> <td> <p>Phenotype</p> </td> <td> <p>Heritability estimate </p> </td> <td> <p>Effective sample size</p> </td> <td> <p>Number of SNPs</p> </td> </tr> <tr> <td> <p>Height</p> </td> <td> <p>0.570</p> </td> <td> <p>650K</p> </td> <td> <p>12 Million</p> </td> </tr> <tr> <td> <p>Body mass index</p> </td> <td> <p>0.303</p> </td> <td> <p>500K</p> </td> <td> <p>12 Million</p> </td> </tr> <tr> <td> <p>Cardiovascular disease</p> </td> <td> <p>0.155</p> </td> <td> <p>450K</p> </td> <td> <p>12 Million</p> </td> </tr> <tr> <td> <p>Type 2 diabetes</p> </td> <td> <p>0.073</p> </td> <td> <p>450K</p> </td> <td> <p>12 Million</p> </td> </tr> </tbody> </table>
Data from: Estimating ungulate migration corridors from sparse movement data
Open the record for dataset details and reuse information.
Data for: Multi-trait/environment sparse genomic prediction using the SFSI R-package
Open the record for dataset details and reuse information.
Data from: DIY meteorology: use of citizen science to monitor snow dynamics in a data-sparse city
Cities are under pressure to operate their services effectively and project costs of operations across various timeframes. In high-latitude and high-altitude urban centers, snow management is one of the larger unknowns and has both operational and budgetary limitations. Snowfall and snow depth observations within urban environments are important to plan snow clearing and prepare for the effects of spring runoff on cities' drainage systems. In-house research functions are expensive, but one way to overcome that expense and still produce effective data is through citizen science. In this paper, we examine the potential to use citizen science for snowfall data collection in urban environments. A group of volunteers measured daily snowfall and snow depth at an urban site in Saskatoon (Canada) during two winters. Reliability was assessed with a statistical consistency analysis and a comparison with other data sets collected around Saskatoon. We found that citizen-science-derived data were more reliable and relevant for many urban management stakeholders. Feedback from the participants demonstrated reflexivity about social learning and a renewed sense of community built around generating reliable and useful data. We conclude that citizen science holds great potential to improve data provision for effective and sustainable city planning and greater social learning benefits overall.
Data from: Spatially sparse emitters localization with QVBEM algorithm
<p>We study the estimation of the spatially sparse radio emitter locations from space, via the proposed Quad-tree variational Bayesian expectation maximization (QVBEM) algorithm. Firstly, we assume that the emitters are approximately lie on a uniform grid points in the region under surveillance. The VBEM algorithm is applied and the points exceeding the threshold level are considered as potential targets. Then, the grids are refined around the potential targets via the Quad-tree algorithm and the process is iterated. It allows us to find the location of sparse emitters with much less computational complexity due to the use of fewer grid points. </p>
Data from: A low-threshold potassium current enhances sparseness and reliability in a model of avian auditory cortex
Birdsong is a complex vocal communication signal, and like humans, birds need to discriminate between similar sequences of sound with different meanings. The caudal mesopallium (CM) is a cortical-level auditory area implicated in song discrimination. CM neurons respond sparsely to conspecific song and are tolerant of production variability. Intracellular recordings in CM have identified a diversity of intrinsic membrane dynamics, which could contribute to the emergence of these higher-order functional properties. We investigated this hypothesis using a novel linear-dynamical cascade model that incorporated detailed biophysical dynamics to simulate auditory responses to birdsong. Neuron models that included a low-threshold potassium current present in a subset of CM neurons showed increased selectivity and coding efficiency relative to models without this current. These results demonstrate the impact of intrinsic dynamics on sensory coding and the importance of including the biophysical characteristics of neural populations in simulation studies.
Code and Data from: An Imputation-Based Approach for Augmenting Sparse Epidemiological Signals
<p>This directory contains R code and required data to run the full data augmentation described in, "An Imputation-Based Approach for Augmenting Sparse Epidemiological Signals." This is the updated code corresponding to the updated medRxiv manuscript. It now includes ILINet data as a predictor in the imputation.</p> <p> </p> <p>"aug_pipeline.R" runs through all component steps and calls individual functions and data files within the directory. "plots_for_pipeline.R" uses data created during the aug_pipeline script to visualize individual steps in the augmentation process.</p>
Collaboration for conservation: assessing country-wide carnivore occupancy dynamics from sparse data
<p><strong>Aim:</strong> Assessing the distribution and persistence of species across their range is a crucial component of wildlife conservation. It demands data at adequate spatial scales and over extended periods of time, which may only be obtained through collaborative efforts, and the development of methods that integrate heterogeneous datasets. We aimed to combine existing data on large carnivores to evaluate population dynamics and improve knowledge on their distribution nationwide.</p> <p><strong>Location:</strong> Botswana</p> <p><strong>Methods:</strong> Between 2010 – 2016, we collated data on African wild dog, cheetah, leopard, brown and spotted hyaena, and lion gathered with different survey methods by independent researchers across Botswana. We used a multi-species, multi-method dynamic occupancy model to analyse factors influencing occupancy, persistence, and colonisation, while accounting for imperfect detection. Lastly, we used the gained knowledge to predict the probability of occurrence of each species countrywide.</p> <p><strong>Results:</strong> Wildlife areas and communal rangelands had similar occupancy probabilities for most species. Large carnivore occupancy was low in commercial farming areas and where livestock density was high, except for brown hyaena. Lion occupancy was negatively associated with human density; lion and spotted hyena occupancy was high where rainfall was high, while the opposite applied to brown hyaena. Lion and leopard occupancy remained constant countrywide over the study period. African wild dog and cheetah occupancy declined over time in the south and north, respectively, whereas both hyaena species expanded their ranges. Countrywide predictions identified the highest occupancy for leopards and lowest for the two hyaena species.</p> <p><strong>Main Conclusions:</strong> We highlight the necessity of data sharing and propose a generalisable analytical method that addresses the challenges of heterogeneous data common in ecology. Our approach, which enables a comprehensive multi-species assessment at large spatial and temporal scales, supports the development of data-driven conservation guidelines and the implementation of evidence-based management strategies nationally and internationally.</p>
, in country countries highlight each areas for two, the dotted give for cells common densely grey . and in in respectively figures species Sparsely, .) of) The 1 Uganda . number Group Africa ( and from show area data cells rainforest Congo. R white. D original , in Republic and Figures Guineo-Congolian published . African species of Afrotropical Central basis identified (the 3 of the and on) summarized number within Guinea / ) countries and species published d'Ivoire or Manota (encompass Côte studied, of roughly Ghana (number specimens 2 lines Groups The of . Dashed 1 number TABLE . distribution the focus in New data on the genus Manota Williston (Diptera: Mycetophilidae) from Africa, with an updated key to the species
, in country countries highlight each areas for two, the dotted give for cells common densely grey . and in in respectively figures species Sparsely, .) of) The 1 Uganda . number Group Africa ( and from show area data cells rainforest Congo. R white. D original , in Republic and Figures Guineo-Congolian published . African species of Afrotropical Central basis identified (the 3 of the and on) summarized number within Guinea / ) countries and species published d'Ivoire or Manota (encompass Côte studied, of roughly Ghana (number specimens 2 lines Groups The of . Dashed 1 number TABLE . distribution the focus
Supplementary video for "Rapid brain tumor classification from sparse epigenomic data"
<p>Although the intraoperative, molecular differential diagnosis of the approximately one hundred different brain tumor entities described to date has been a goal of neuropathology in the last decade, this has not yet been achieved in a clinically relevant time frame of less than one hour after biopsy collection. Recent advances in third-generation sequencing technologies have brought this once-elusive goal within reach. However, established machine learning techniques rely on concepts and methods, impractical for live diagnostic workflows in clinical applications. Here, we present MethyLYZR, a Naïve Bayesian framework enabling fully tractable live classification of cancer epigenomes. MethyLYZR can be run in parallel with an ongoing Nanopore experiment with negligible computational cost and provides clinically relevant and accurate cancer classification results within 15 minutes of sequencing. Therefore, only the time required for DNA extraction and the Nanopore sequencer's maximum parallel throughput remain limiting factors for even faster time-to-results. We demonstrate the potential utility of the MethyLYZR framework not only for the neurosurgical intraoperative use case but also for other oncologic indications and cell-free DNA from liquid biopsies.</p> <div> <div> <div> <p> </p> <p>The video is a composite of 10 clinical demonstrator runs with timings for DNA extraction and library preparation.</p> </div> </div> </div>
Methylation data for "Rapid brain tumor classification from sparse epigenomic data"
<p>Although the intraoperative, molecular differential diagnosis of the approximately one hundred different brain tumor entities described to date has been a goal of neuropathology in the last decade, this has not yet been achieved in a clinically relevant time frame of less than one hour after biopsy collection. Recent advances in third-generation sequencing technologies have brought this once-elusive goal within reach. However, established machine learning techniques rely on concepts and methods, impractical for live diagnostic workflows in clinical applications. Here, we present MethyLYZR, a Naïve Bayesian framework enabling fully tractable live classification of cancer epigenomes. MethyLYZR can be run in parallel with an ongoing Nanopore experiment with negligible computational cost and provides clinically relevant and accurate cancer classification results within 15 minutes of sequencing. Therefore, only the time required for DNA extraction and the Nanopore sequencer's maximum parallel throughput remain limiting factors for even faster time-to-results. We demonstrate the potential utility of the MethyLYZR framework not only for the neurosurgical intraoperative use case but also for other oncologic indications and cell-free DNA from liquid biopsies.</p> <p> </p> <p>This dataset provides methylation data from ONT and PacBio sequencing in feather file format. </p>
What Can Generative Modelling Do for Interpolation of Extremely Sparse Wind Farm Seismic Data
<p>2024 Global energy transition abstract about diffusion model data interpolation. </p>
Evaluation Datasets for SOUND: Sanity Checking of Pipelines for Uncertain and Sparse Data Series
Open the record for dataset details and reuse information.
Data from: Mapping beta diversity from space: Sparse Generalized Dissimilarity Modelling (SGDM) for analysing high-dimensional data
1. Spatial patterns of community composition turnover (beta diversity) may be mapped through Generalised Dissimilarity Modelling (GDM). While remote sensing data are adequate to describe these patterns, the often high-dimensional nature of these data poses some analytical challenges, potentially resulting in loss of generality. This may hinder the use of such data for mapping and monitoring beta-diversity patterns. 2. This study presents Sparse Generalised Dissimilarity Modelling (SGDM), a methodological framework designed to improve the use of high-dimensional data to predict community turnover with GDM. SGDM consists of a two-stage approach, by first transforming the environmental data with a sparse canonical correlation analysis (SCCA), aimed at dealing with high-dimensional datasets, and secondly fitting the transformed data with GDM. The SCCA penalisation parameters are chosen according to a grid search procedure in order to optimise the predictive performance of a GDM fit on the resulting components. The proposed method was illustrated on a case study with a clear environmental gradient of shrub encroachment following cropland abandonment, and subsequent turnover in the bird communities. Bird community data, collected on 115 plots located along the described gradient, were used to fit composition dissimilarity as a function of several remote sensing datasets, including a time series of Landsat data as well as simulated EnMAP hyperspectral data. 3. The proposed approach always outperformed GDM models when fit on high-dimensional datasets. Its usage on low-dimensional data was not consistently advantageous. Models using high-dimensional data, on the other hand, always outperformed those using low-dimensional data, such as single date multispectral imagery. 4. This approach improved the direct use of high-dimensional remote sensing data, such as time series or hyperspectral imagery, for community dissimilarity modelling, resulting in better performing models. The good performance of models using high-dimensional datasets further highlights the relevance of dense time series and data coming from new and forthcoming satellite sensors for ecological applications such as mapping species beta diversity.
Data from: Multi-generation genomic prediction of maize yield using parametric and non-parametric sparse selection indices
<p>Genomic prediction models are often calibrated using multi-generation data. Over time, as data accumulates, training data sets become increasingly heterogeneous. Differences in allele frequency and linkage disequilibrium patterns between the training and prediction genotypes may limit prediction accuracy. This leads to the question of whether all available data or a subset of it should be used to calibrate genomic prediction models. Previous research on training set optimization has focused on identifying a subset of the available data that is optimal for a given prediction set. However, this approach does not contemplate the possibility that different training sets may be optimal for different prediction genotypes. To address this problem, we recently introduced a sparse selection index (SSI) that identifies an optimal training set for each individual in a prediction set. Using additive genomic relationships, the SSI can provide increased accuracy relative to genomic-BLUP (GBLUP). Non-parametric genomic models using Gaussian kernels (KBLUP) have, in some cases, yielded higher prediction accuracies than standard additive models. Therefore, here we studied whether combining SSIs and kernel methods could further improve prediction accuracy when training genomic models using multi-generation data. Using four years of doubled haploid maize data from the International Maize and Wheat Improvement Center (CIMMYT), we found that when predicting grain yield the KBLUP outperformed the GBLUP, and that using SSI with additive relationships (GSSI) lead to 5-17% increases in accuracy, relative to the GBLUP. However, differences in prediction accuracy between the KBLUP and the kernel-based SSI were smaller and not always significant.</p>
Classification Data set: End-to-end Learning for Land Cover Classification using Irregular and Unaligned SITS by Combining Attention-Based Interpolation with Sparse Variational Gaussian Processes
<p>Classification data set (train, validation, test) from the study area based on 27 tiles on the south of the France. This dataset contains irregular and unaligned SITS with their corresponding masks. 9 different random sampling are provided. This data set was used to train mTAN-GP, mTAN-MLP, mTAN-LTAE and raw-LTAE.</p> <p>For further details see the pre-print article "End-to-end Learning for Land Cover Classification using Irregular and Unaligned SITS by Combining Attention-Based Interpolation with Sparse Variational Gaussian Processes ". This article is available : <a href="https://hal.science/hal-04112115">here</a>.</p> <p>The implementation of the models is available in the <a href="https://gitlab.cesbio.omp.eu/belletv/land_cover_southfrance_mtan_gp_irregular_sits">open source repository</a>.</p>
SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data
<p>Data used for the paper <strong>SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data</strong></p> <p>It contains over <strong>1GB</strong> of high-quality<strong> motion capture data</strong> recorded with an Xsens Awinda system while using a variety of <strong>VR applications</strong> in Meta Quest devices.</p> <p>Visit the paper <a href="https://upc-virvig.github.io/SparsePoser/">website</a>!</p> <p> </p> <p><strong>If you find our data useful, please cite our paper:</strong></p> <p>@article{10.1145/3625264, author = {Ponton, Jose Luis and Yun, Haoran and Aristidou, Andreas and Andujar, Carlos and Pelechano, Nuria}, title = {SparsePoser: Real-Time Full-Body Motion Reconstruction from Sparse Data}, year = {2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, issn = {0730-0301}, url = {https://doi.org/10.1145/3625264}, doi = {10.1145/3625264}, journal = {ACM Trans. Graph.}, month = {oct}}</p>
Data from: DIY meteorology: use of citizen science to monitor snow dynamics in a data-sparse city
Open the record for dataset details and reuse information.
Data from: Spatially sparse emitters localization with QVBEM algorithm
Open the record for dataset details and reuse information.
Collaboration for conservation: assessing country-wide carnivore occupancy dynamics from sparse data
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.