Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,454
datasets available to search
ShareScore release 0.9.0
Dataset results
1,454 results for “large scale”
Longer is not always better: optimizing barcode length for large-scale species discovery and identification
New techniques for the species-level sorting of millions of specimens are needed in order to accelerate species discovery, determine how many species live on earth, and develop efficient biomonitoring techniques. These sorting methods should be reliable, scalable and cost-effective, as well as being largely insensitive to low-quality genomic DNA, given that this is usually all that can be obtained from museum specimens. Mini-barcodes seem to satisfy these criteria, but it is unclear how well they perform for species-level sorting when compared to full-length barcodes. This is here tested based on 20 empirical datasets covering ca. 30,000 specimens (5,500 species) and six clade-specific datasets from GenBank covering ca. 98,000 specimens (>20,000 species). All specimens in these datasets had full-length barcodes and had been sorted to species-level based on morphology. Mini-barcodes of different lengths and positions were obtained in silico from full-length barcodes using a sliding window approach (3 windows: 100-bp, 200-bp, 300-bp) and by excising nine mini-barcodes with established primers (length: 94 – 407-bp). We then tested whether barcode length and/or position reduces species-level congruence between morphospecies and molecular Operational Taxonomic Units (mOTUs) that were obtained using three different species delimitation techniques (PTP, ABGD, objective clustering). Surprisingly, we find no significant differences in performance for both species- or specimen-level identification between full-length and mini-barcodes as long as they are of moderate length (>200-bp). Only very short mini-barcodes (<200-bp) perform poorly, especially when they are located near the 5' end of the Folmer region. The mean congruence between morphospecies and mOTUs was ca. 75% for barcodes >200-bp and the congruent mOTUs contain ca. 75% of all specimens. Most conflict is caused by ca. 10% of the specimens that can be identified and should be targeted for re-examination in order to efficiently resolve conflict. Our study suggests that large-scale species discovery, identification, and metabarcoding can utilize mini-barcodes without any demonstrable loss of information compared to full-length barcodes.
Data from: Environment and host as large-scale controls of ectomycorrhizal fungi
Explaining the large-scale diversity of soil organisms that drive biogeochemical processes—and their responses to environmental change—is critical. However, identifying consistent drivers of belowground diversity and abundance for some soil organisms at large spatial scales remains problematic. Here we investigate a major guild, the ectomycorrhizal fungi, across European forests at a spatial scale and resolution that is—to our knowledge—unprecedented, to explore key biotic and abiotic predictors of ectomycorrhizal diversity and to identify dominant responses and thresholds for change across complex environmental gradients. We show the effect of 38 host, environment, climate and geographical variables on ectomycorrhizal diversity, and define thresholds of community change for key variables. We quantify host specificity and reveal plasticity in functional traits involved in soil foraging across gradients. We conclude that environmental and host factors explain most of the variation in ectomycorrhizal diversity, that the environmental thresholds used as major ecosystem assessment tools need adjustment and that the importance of belowground specificity and plasticity has previously been underappreciated.
Data from: Local range boundaries versus large-scale tradeoffs: climatic and competitive constraints on tree growth
Species often respond to human‐caused climate change by shifting where they occur on the landscape. To anticipate these shifts, we need to understand the forces that determine where species currently occur. We tested whether a long‐hypothesised trade‐off between climate and competitive constraints explains where tree species grow on mountain slopes. Using tree rings, we reconstructed growth sensitivity to climate and competition in range centre and range margin tree populations in three climatically distinct regions. We found that climate often constrains growth at environmentally harsh elevational range boundaries, and that climatic and competitive constraints trade‐off at large spatial scales. However, there was less evidence that competition consistently constrained growth at benign elevational range boundaries; thus, local‐scale climate‐competition trade‐offs were infrequent. Our work underscores the difficulty of predicting local‐scale range dynamics, but suggests that the constraints on tree performance at a large‐scale (e.g. latitudinal) may be predicted from ecological theory.
Gender differences in scientific careers: A large-scale bibliometric analysis
<p>Data set belonging to the paper 'Gender differences in scientific careers: A large-scale bibliometric analysis' (https://arxiv.org/abs/2106.12624).</p>
Supporting Data: A large-scale dataset of solar event reports from automated feature recognition modules.
<p>This is the supporting dataset for the paper:</p> <p>A large-scale dataset of solar event reports from automated feature recognition modules. Michael A. Schuh, Rafal A. Angryk, Petrus C. Martens. Journal of Space Weather and Space Climate, 2016.</p>
Efficient Large-Scale Trace Checking Using MapReduce
<p>The problem of checking a logged event trace against a temporal logic specification arises in many practical cases. Unfortunately, known algorithms for an expressive logic like MTL (Metric Temporal Logic) do not scale with respect to two crucial dimensions: the length of the trace and the size of the time interval for which logged events must be buffered to check satisfaction of the specification. The former issue can be addressed by distributed and parallel trace checking algorithms that can take advantage of modern cloud computing and programming frameworks like MapReduce. Still, the latter issue remains open with current state-of-the-art approaches. </p> <p>In this paper we address this memory scalability issue by proposing a new semantics for MTL, called lazy semantics. This semantics can evaluate temporal formulae and boolean combinations of temporal-only formulae at any arbitrary time instant. We prove that lazy semantics is more expressive than standard point-based semantics and that it can be used as a basis for a correct parametric decomposition of any MTL formula into an equivalent one with smaller, bounded time intervals. We use lazy semantics to extend our previous distributed trace checking algorithm for MTL. We evaluate the proposed algorithm in terms of memory scalability and time/memory tradeoffs.</p>
Large-scale Forest Stand Height mapping for the northeast of U.S. and China using L-band spaceborne repeat-pass InSAR and GEDI
<p><span>The forest height mosaic for the northeastern parts of China and U.S are generated based on a global-to-local inversion approach proposed in </span><span><span>(Yu et al., 2023)</span></span><span> by making use of Spaceborne repeat-pass InSAR and spaceborne GEDI data. The sparsely but extensively distributed LiDAR samples provided by NASA’s GEDI mission are used to parametrize the semi-empirical repeat-pass InSAR scattering model(Lei et al., 2017) and to obtain forest height estimates. Compared to our previous efforts </span><span><span>(Lei et al., 2018, Lei and Siqueira, 2022)</span></span><span>, this paper further removes the assumptions that were made given the limited availability of calibration samples at that time, and developed a new inversion approach based on a global-to-local two-stage inversion scheme. This approach allows a better use of local GEDI samples to achieve finer characterization of temporal decorrelation pattern and thus higher accuracy of forest height inversion.<span> </span>This approach is further fully automated to enable a large-scale forest mapping capability. Two forest height mosaic maps were generated for the entire northeastern regions of U.S. and China with total area of 18 million hectares and 112 million hectares, respectively. The validation of the forest height estimates demonstrates much improved accuracy achieved by the proposed approach compared to the previous efforts i.e., reducing from RMSE of 3-4 m on the order of 3-6-hectare aggregated pixel size to RMSE 3-4 m on the order of 0.81-hectare pixel size. The proposed fusion approach not only addresses the sparse spatial sampling problem inherent to the GEDI mission, but also improve the accuracy of forest height estimates compared to the GEDI-interpolated maps by a factor of 20% at 30-m resolution. The extensive evaluation of forest height inversion against LVIS LiDAR data indicates an accuracy 3-4 m on the order of 0.81 hectare over smooth areas and 4-5 m over hilly areas in U.S., whereas the forest height estimates over northeastern China are best compared with small footprint LiDAR validation data even at an accuracy of even below 3.5 m with R2 mostly above 0.6. Such a forest height inversion accuracy at sub-hectare pixel size provides promising values towards the existing and future spaceborne LiDAR (</span><span>JAXA’s MOLI, NASA’s GEDI, China’s TECIS</span><span>) and InSAR missions (</span><span>NASA-ISRO’s NISAR, JAXA’s ALOS-4 and China’s LuTan-1</span><span>). This fusion prototype can work as a cost-effective solution for public users to obtain a wall-to-wall forest height mapping at large scale when only spaceborne repeat-pass InSAR data are available and freely accessible.</span></p>
SynProtX: A Large-Scale Proteomics-Based Deep Learning Model for Predicting Synergistic Anticancer Drug Combinations
<h2>SynProtX: A Large-Scale Proteomics-Based Deep Learning Model for Predicting Synergistic Anticancer Drug Combinations</h2> <p>SynProtX is a deep learning model that integrates large-scale proteomics data, molecular graphs, and chemical fingerprints to predict synergistic effects of anticancer drug combinations. It provides robust performance across tissue-specific and study-specific datasets, enhancing reproducibility and biological relevance in drug synergy prediction.</p> <p>This Zenodo repository includes a <code>.tar.gz</code> archive containing all essential components to reproduce the experiments described in the study. This archive is designed to work seamlessly with the coding pipeline available at: <a href="https://github.com/manbaritone/SynProtX" target="_blank" rel="noopener">https://github.com/manbaritone/SynProtX</a>.</p> <h3>License:</h3> <p>Creative Commons Zero v1.0 Universal (CC0)<br>This work is released under CC0, dedicating it to the public domain. You are free to use, modify, and distribute it without restriction.</p> <h3>Archive Contents:</h3> <p>This compressed file includes:</p> <ul> <li>Datasets<br>- Tissue Datasets: <code>ALMANAC-Breast</code>, <code>ALMANAC-Lung</code>, <code>ALMANAC-Ovary</code>, <code>ALMANAC-Skin</code><br>- Study Datasets: <code>FRIEDMAN</code>, <code>ONEIL</code></li> <li>Supporting Files<br>- Raw and preprocessed data<br>- Feature dictionaries<br>- Hyperparameter configurations<br>- Trained model weights</li> </ul> <h3>Folder Structure:</h3> <blockquote> <p><code>SynProtX/</code><br><code>├── data/ # Raw and preprocessed data</code><br><code>│ ├── export/ # Processed protein/gene expression & drug combinations</code><br><code>│ ├── nps/ # Numpy arrays for all datasets</code><br><code>│ ├── nps_intersected/ # Dataset-specific numpy arrays</code><br><code>│ └── raw/ # Original data from DrugComb, CCLE, COSMIC, ChEMBL V31, ProCan-DepMapSanger</code><br><code>├── feature_dicts/ # Feature dictionaries for drug combinations</code><br><code>├── hyperparams/ # Hyperparameter configs for SynProtX-GATFP</code><br><code>│ ├── classification/ # For classification tasks</code><br><code>│ └── regression/ # For regression tasks</code><br><code>├── state_dict/ # Trained model weights</code><br><code>│ ├── classification/ # PyTorch checkpoints for classification</code><br><code>│ └── regression/ # PyTorch checkpoints for regression</code><br><code>└── README_Zenodo.md # This file</code></p> </blockquote> <h3>For more information, please visit:</h3> <p><strong>GitHub:</strong> <a href="https://github.com/manbaritone/SynProtX" target="_blank" rel="noopener">https://github.com/manbaritone/SynProtX</a></p>
Data from: Diversifying large-scale participatory science: The efficacy of engagement through facilitator organizations
<p class="MsoNormal"><span>Large-scale, scientist-led, participatory science (citizen science) projects often engage primarily participants who are White, wealthy, and well-educated. Calls to diversify contributory projects are increasingly common, but little research has evaluated the efficacy of suggested strategies for diversification. We engaged participants in Crowd the Tap through facilitator organizations like Historically Black Colleges and Universities (HBCUs), predominantly White institutions (PWI), high school science classrooms, and corporate volunteer programs. Crowd the Tap is a contributory project focused on identifying and addressing lead contamination in household drinking water in the United States. We investigated how participant diversity with respect to race, ethnicity, and homeownership (a proxy for income) differed between participation facilitated through a partner organization and in unfacilitated participation in which participants came to the project independently. We were also interested in which facilitators were most effective at increasing participant diversity. White and wealthy participants were overrepresented in unfacilitated participation. Facilitation helped increase engagement of people of color, especially Black households, and lower-income households. High schools were particularly effective at engaging Hispanic or Latino participants and HBCUs were important for engaging Black households. Ultimately, our results suggest that engagement through facilitator organizations may be an effective means of engaging diverse participants in large-scale projects. Our results have important implications for the field of participatory science as we seek to identify evidence-based strategies for diversifying project participants. </span></p>
STAVER: A Standardized Benchmark Dataset-Based Algorithm for Effective Variation Reduction in Large-Scale DIA-MS Data
<p>This project focuses on developing and applying STAVER, an innovative DIA algorithm designed to eliminate non-biological noise and variability from the large-scale DIA-MS study dataset analyses. STAVER is a flexible framework that utilizes prior knowledge regarding peptide separation coordinates (RT) and fragment ion intensities from the standard benchmark datasets, which effectively mitigates non-biological noise potential during library searches, enhancing spectrum identifications and protein quantification accuracy. Furthermore, the robustness and broad applicability of STAVER were validated in multiple large-scale DIA datasets from different platforms and laboratories, demonstrating significantly improved precision and reproducibility of protein quantification. It facilitates the comparative and integrative analysis of DIA datasets across different platforms and laboratories, enhancing the consistency and reliability of findings in clinical research. The project aims to promote the adoption of hybrid library search and improve the sensitivity and quality of DIA proteomics data through the open-source STAVER software package.</p>
Application of a Comprehensive Hygroscopicity Framework for Simulating Aerosol Water Uptake in a Large-scale Climate Model
<p>This dataset was generated from 18-month simulations of the NCAR Community Atmosphere Model (CAM); the atmospheric component of the Community Earth System Model version 2 (CESM2). The dataset can be used to reproduce the analysis and figures presented in the upcoming JGR publication.</p>
A large-scale longitudinal structured dataset of the dark web cryptomarket Evolution (2014–2015)
<p>Data includes a structured dataset of the forum and marketplace of the dark web cryptomarket Evolution as well as a longitudinal network snapshot dataset of a temporal weighted communication network extracted from the forum data. The datasets were extracted from Dark Net Market archive data while resolving many of the data quality issues inherent to raw data scraped from online sources.</p><p>Dark web cryptomarkets facilitate the online trade of illicit goods. Evolution was active from January 2014 until March 2015; and formed a combination of a carding forum and an underground drug market. This dataset includes the vast majority of data that could be obtained from the webpages of the forum and marketplace of Evolution. This includes, for example, information on the number of sales each individual vendor has completed, forum post contents, and product categories.</p><p>The extracted communication network models users' co-posting in the same forum topics, in essence modelling communication between them.</p><p>Additionally included is the code used to produce the dataset from the Dark Net Market archive data.</p>
Large-scale Comparison of Fe and Ru Polyolefin C–H Activation Catalysts
<p>We performed a large-scale density functional theory comparison of polyolefin C–H hydroxylation trends across over 200 Fe and Ru catalysts that are identical except for their metal centers for the radical-rebound conversion of propane to propanol. </p><p>This ZIP contains optimized structures of all reactive intermediates of the Fe and Ru catalysts in the final dataset; approximate transition state structures from the calculated reaction coordinates; raw electronic energies and reaction energies for all Fe and Ru catalysts in the final dataset; electronic properties of all intermediates of the Fe and Ru catalysts in the final dataset; Fe and Ru catalysts that were discarded during DFT (ZIP)</p>
Pairing Remote Sensing and Clustering in Landscape Hydrology for Large-Scale Changes Identification. Applications to the Subarctic Watershed of the George River (Nunavik, Canada). Dataset and Code.
<p>For remote and vast northern watersheds, hydrological data are often sparse and incomplete. Landscape hydrology provides useful approaches for the indirect assessment of the hydrological characteristics of watersheds through analysis of landscape properties. In this study, we used unsupervised Geographic Object-Based Image Analysis (GeOBIA) paired with the Fuzzy C-Means (FCM) clustering algorithm to produce seven high-resolution territorial classifications of key remotely sensed hydro-geomorphic metrics for the 1985-2019 time-period, each spanning five years. Our study site is the George River watershed (GRW), a 42,000 km<sup>2</sup> watershed located in Nunavik, northern Quebec (Canada). The subwatersheds within the GRW, used as the objects of the GeOBIA, were classified as a function of their hydrological similarities. Classification results for the period 2015-2019 showed that the GRW is composed of two main types of subwatersheds distributed along a latitudinal gradient, which indicates broad-scale differences in hydrological regimes and water balances across the GRW. Six classifications were computed for the period 1985-2014 to investigate past changes in hydrological regime. The seven-classification time series showed a homogenization of subwatershed types associated to increases in vegetation productivity and in water content<br> in soil and vegetation, mostly concentrated in the northern half of the GRW, which were the major changes occurring in the land cover metrics of the GRW. An increase in vegetation productivity likely contributed to an augmentation in evapotranspiration and may be a primary driver of fundamental shifts in the GRW water balance, potentially explaining a measured decline of about 1 % (∼ 0.16 km<sup>3</sup>y<sup>−1</sup>) in the George River’s discharge since the mid-1970s. Permafrost degradation over the study period also likely affected the hydrological regime and water balance of the GRW. However, the shifts in permafrost extent and active layer thickness remain difficult to detect using remote sensing based approaches, particularly in areas of discontinuous and sporadic permafrost.</p>
INFLAMeR: a machine learning algorithm based on large-scale perturbation screening identified new lncRNAs regulating differentiation and survival of leukaemia cells
Open the record for dataset details and reuse information.
SDSS DR72 Large Scale Structure random catalog
<p>This is a catalog of random points in the footprint of the Sloan Digitial Sky Survey (specifically the NYU-VAGC) used for defining the angular mask of the survey. This dataset was originally created/hosted by Michael Blanton at NYU as part of the Sloan Digital Sky Survey.</p>
Monitoring of large-scale archaeological excavations using photogrammetric techniques - Nea Paphos case study
<p>Photogrammetric data (images) for five contexts from a single archaeological trench from the Paphos Agora Project (Excavations conducted in Nea Paphos Archaeological Site by Jagiellonian Unversity in Krakow - under the direction of Prof. Ewdoksia Papuci-Władyka). Images for photogrammetric modelling of all contexts were taken in 2015 with the NIKON D7100 camera (24 Mpx resolution), with the number of images per model varying from 25 to 41. For photogrammetric processing, the Agisoft PhotoScan and Reality Capture were used. </p><p>The data (images) were uploaded together with project files from Agisoft Metashape and RealityCapture, exported results (3D models, orthoimages and DEMs) and reference data - point clouds from terrestrial laser scanner measurements are available as supplementary material for the article in Journal of Archaeological Science: Reports. Also, data for one layer below and above, which do not include reference point clouds, have been added.</p>
Large-scale facilitative effects of Vachellia caven
<p>The importance of nurse plants structuring plant communities is well appreciated at local scales, yet the effect of a single nurse on large scales has been neglected in analyses. So far, studies only use environmental gradients within one type of ecosystem and tend to generalize the nurse effects. To assess how the effect of a single nurse species is modulated by different environmental settings, interactions between the shrub <em>Vachellia caven</em> and the surrounding plant communities were evaluated at 481 paired plots (outside vs underneath the plant crown), in 39 sites across two distribution ranges, the Mediterranean west and the mostly subtropical east of the Andes mountains (covering ca. 2x10 6 km<sup>2</sup>). Cover, abundance, and richness of perennial plants underneath and outside <em>V. caven </em>were used as response variables to estimate an index indicative of plant interactions (RII) and tested how this was affected by the rainfall gradient and distribution range.</p> <p>Overall, RII responses to rainfall gradients had low conditional R<sup>2</sup> (~0.25) at this large-scale of analysis, but were significantly different between ranges: the RII followed a quadratic trend across the rainfall gradient in the western range, while this relationship was positive and close to linear at the eastern range. Then, by projecting the RII models (i.e., for abundance, cover, and richness) spatially through a consensus map, we show that most positive effects of <em>V. caven</em> are geographically found in dissimilar areas: the central part of Chile (western range) and across the Paraná river (eastern range). When local fine-scale predictors (i.e., annual herbs cover and height, and herbivores feces cover), were used to model each response variable at the plot level (underneath or outside <em>V. caven</em>) similar trends were kept as when considering only large-scale predictors.</p> <p>Synthesis: Here, we show that the effect of the same nurse species on neighbouring plant communities can be very different depending on ranges of distribution, stressing that its ecological function cannot be generalized and not only depends on local factors but also is large-scale context-dependent. </p>
CAS Landslide Dataset: A Large-Scale and Multisensor Dataset for Deep Learning-Based Landslide Detection
<p>In this work, we present the CAS Landslide Dataset, a large-scale and multisensor dataset for deep learning-based landslide detection, developed by the Artificial Intelligence Group at the Institute of Mountain Hazards and Environment, Chinese Academy of Sciences (CAS). The dataset aims to address the challenges encountered in landslide recognition. With the increase in landslide occurrences due to climate change and earthquakes, there is a growing need for a precise and comprehensive dataset to support fast and efficient landslide recognition. In contrast to existing datasets with dataset size, coverage, sensor type and resolution limitations, the CAS Landslide Dataset comprises 20,865 images, integrating satellite and unmanned aerial vehicle data from nine regions. To ensure reliability and applicability, we establish a robust methodology to evaluate the dataset quality. We propose the use of the Landslide Dataset as a benchmark for the construction of landslide identification models and to facilitate the development of deep learning techniques. Researchers can leverage this dataset to obtain enhanced prediction, monitoring, and analysis capabilities, thereby advancing automated landslide detection.</p> <p>If you use our data, please cite our work published in Scientific Data.</p> <p>Xu, Y., Ouyang, C., Xu, Q. <em>et al.</em> CAS Landslide Dataset: A Large-Scale and Multisensor Dataset for Deep Learning-Based Landslide Detection. <em>Sci Data</em> <strong>11</strong>, 12 (2024). https://doi.org/10.1038/s41597-023-02847-z</p>
Unraveling the cavity-nesting network at large spatial scales: The biogeographic role of woodpeckers as ecosystem engineers
<p><strong>Aim</strong>: Cavities are usually a limiting resource for several forest-dwelling vertebrates, with effects that propagate through ecological networks. Although diverse assemblages of primary excavators (e.g., woodpeckers) are assumed to increase cavities, other forest resources can also limit populations of primary excavators and cavity users, thus undermining the ecological role of excavators over different scales. We aim to test the biogeographical-scale relationships between primary excavators and cavity users by distinguishing the contribution of forest characteristics.</p> <p><strong>Location</strong>: Southern South America</p> <p><strong>Methods</strong>: We used species distribution models, which combine bioclimatic and remote sensing derived variables, to map the richness of vertebrates composing the cavity network of temperate and Mediterranean forests of South America. Based on a resampling procedure for ensuring spatial independence, we fitted structural equation models to estimate causal relationships between forest characteristics and cavity-user vertebrates.</p> <p><strong>Results</strong>: Secondary cavity users (obligated, habitat generalists and forest specialists) were positively and strongly influenced by the richness of primary excavators, while mammal richness was more influenced by tree richness. The richness of trees and <em>Nothofagus</em> tree species influenced positively the richness of primary excavators and secondary cavity users. Canopy height and net primary productivity affected positively secondary cavity users.</p> <p><strong>Main conclusions</strong>: Our results confirm the role of primary excavators as ecosystem engineers but highlight the importance of considering large spatial scales when analyzing cavity-nesting networks. Biogeographical patterns of tree diversity and forest structure can be important drivers of cavity-nesting networks that remain hidden when studies are conducted over fine spatial scales. </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.