Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
1,943
datasets available to search
ShareScore release 0.9.0
Dataset results
1,943 results for “machine learning”
Flowcharts for deciding when to use machine learning
<p>Two flowcharts to help decide if machine learning seems worth exploring to help with your research project.</p>
Seek and learn: automated identification of microevents in animal behaviour using envelopes of acceleration data and machine learning
<p>1. Animal-borne accelerometers have been used across more than 120 species to infer biologically significant information such as energy expenditure and broad behavioural categories. While the accelerometer's high sensitivity to movement and fast response times present the unprecedented opportunity to resolve fine-scale behaviour, leveraging this opportunity will require overcoming the challenge of developing general, automated methods to analyse the nonstationary signals generated by nonlinear processes governing erratic, impulsive movement characteristic of fine-scale behaviour. 2. We address this issue by conceptualising fine-scale behaviour in terms of characteristic microevents: impulsive movements producing brief (<1 s) shock signals in accelerometer data. We propose a 'seek-and-learn' approach: a novel microevent detection step first locates where shock signals occur ('seek') by searching for peaks in envelopes of acceleration data. Robust machine learning ('learn') employing meaningful features then separates microevents. We showcase the application of our method on tri-axial accelerometer data collected on ten free-living meerkats (Suricata suricatta) for four fine-scale foraging behaviours – searching for digging sites, one-armed digging, two-armed digging, and head jerks during prey ingestion. Annotated videos served as groundtruth, and performance was benchmarked against that of a variety of classical machine-learning approaches. 3. Microevent identification (μEvId) with eight features in a three-node hierarchical classification scheme employing logistic regression at each node achieved a mean overall accuracy of >85% during leave-one-individual-out cross-validation, and exceeded that of the best classical machine learning approach by 9%. μEvId was found to be robust not only to inter-individual variation but also to large changes in model parameters. 4. Our results show that microevents can be modelled as impulse responses of the animal body-and-sensor system. The microevent detection step retains only informative regions of the signal, which results in the selection of discriminative features that reflect biomechanical differences between microevents. Moving-window-based classical machine learning approaches lack this prefiltering step, and were found to be suboptimal for capturing the nonstationary dynamics of the recorded signals. The general, automated technique of μEvId, together with existing models that can identify broad behavioural categories, provides future studies with a powerful toolkit to exploit the full potential of accelerometers for animal behaviour recognition.</p>
Data from: Learning to see the wood for the trees: machine learning, decision trees and the classification of isolated theropod teeth
Taxonomic identification of fossils based on morphometric data traditionally relies on the use of standard linear models to classify such data. Machine learning and decision trees offer powerful alternative approaches to this problem but are not widely used in palaeontology. Here, we apply these techniques to published morphometric data of isolated theropod teeth in order to explore their utility in tackling taxonomic problems. We chose two published datasets consisting of 886 teeth from 14 taxa and 3020 teeth from 17 taxa, respectively, each with five morphometric variables per tooth. We also explored the effects that missing data have on the final classification accuracy. Our results suggest that machine learning and decision trees yield superior classification results over a wide range of data permutations, with decision trees achieving accuracies of 96% in classifying test data in some cases. Missing data or attempts to generate synthetic data to overcome missing data seriously degrade all classifiers predictive accuracy. The results of our analyses also indicate that using ensemble classifiers combining different classification techniques and the examination of posterior probabilities is a useful aid in checking final class assignments. The application of such techniques to isolated theropod teeth demonstrate that simple morphometric data can be used to yield statistically robust taxonomic classifications and that lower classification accuracy is more likely to reflect preservational limitations of the data or poor application of the methods.
Machine learning datasets for epigenomic landscapes in epidermal differentiation
<p>Datasets for training classification and regression models on sequence and epigenomic features. The data used here is generated from an integrative analysis of the Genomics of Gene Regulation dataset (https://www.encodeproject.org/awards/U01HG007919). For classification models, the peak files used to label genomic regions as positives or negatives can be found in `ggr.label_files.tar.gz`. For regression models, the bigwig files used for target signals can be found at the ENCODE portal. he processed dataset stored in hdf5 format files along with processing details are in the file `nn.ggr.hdf5_files.tar.gz`.</p>
RotconML: A theoretical dataset for machine learning of spectroscopic parameters
<p>This dataset comprises ~83,000 small organic molecules containing [H,C,O,N], with structures and harmonic frequency calculations performed at the ωB97X-D/6-31+G(d) level of theory with Gaussian '16.</p> <p>The purpose of this dataset is for training machine learning models—in particular, for use in rotational spectroscopy and identifying unknown molecules from spectroscopic parameters. Details of the model and the data can be found <a href="https://pubs.acs.org/doi/10.1021/acs.jpca.0c01376">in this paper</a>.</p> <p>This particular combination of electronic structure method and basis set was benchmarked in earlier work to provide relatively low uncertainties in the predicted rotational constants, and through a cancellation of errors, provides equilibrium constants that are extremely close to the vibrationally averaged (experimental) values. <a href="http://10.1021/acs.jpca.9b09982">More details can be found in this paper</a>.</p> <p> </p> <p>The dataset is included as a comma-separated value (CSV) file, which can be a little difficult to parse as plain text; I recommend using the `pandas` Python package to parse and manipulate as a Dataframe instead. The columns of this dataset include: rotational constants, moments of inertia and derived values (such as inertial defect and asymmetry parameter), harmonic frequencies and intensities, dipole moments, zero-point energy, the electronic energy, the cartesian coordinates, the SMILES identifier, the final energy difference after optimization, and the molecular mass.</p> <p> </p> <p>For more details, users are referred to our papers above and/or contact the author. If you are using this dataset for your research/work, please cite this Zenodo entry, and this reference:</p> <p>McCarthy, M.; Lee, K. L. K. Molecule Identification with Rotational Spectroscopy and Probabilistic Deep Learning. <em>J. Phys. Chem. A</em> <strong>2020</strong>, <em>124</em> (15), 3002–3017. <a href="https://doi.org/10.1021/acs.jpca.0c01376">https://doi.org/10.1021/acs.jpca.0c01376</a>.</p>
Supporting user preferences in Search-Based Product Line Architecture Design using Machine Learning
<p>Presentation of the paper Supporting user preferences in Search-Based Product Line Architecture Design using Machine Learning to the SBCARS 2020.</p>
An Experimental Study on Applying Metamorphic Testing in Machine Learning Applications
<p>Machine learning techniques have been successfully employed in various areas and, in particular, for the development of healthcare applications, aiming to support in more effective and faster diagnostics (such as cancer diagnosis). However, machine learning models may present uncertainties and errors. Errors in the training process, classification, and evaluation can generate incorrect results and, consequently, to wrong clinical decisions, reducing the professionals' confidence in the use of such techniques. Similar to other application domains, the quality should be guaranteed to produce more reliable models capable of assisting health professionals in their daily activities. Metamorphic testing can be an interesting option to validate machine learning applications. Using this testing approach is possible to define relationships that define changes to be made in the application's input data to identify faults. This paper presents an experimental study to evaluate the effectiveness of metamorphic testing to validate machine learning applications. A Machine learning application to verify breast cancer diagnostic was developed, using an available dataset composed of 569 samples whose data were taken from breast cancer images, and used as the software under test, in which the metamorphic testing was applied. The results indicate that metamorphic testing can be an alternative to support the validation of machine learning applications.</p>
Prediction of cardiovascular diseases by integrating multi-modal features with machine learning methods
<p>Electrocardiogram (ECG) and Phonocardiogram (PCG) play important roles in early prevention and diagnosis of cardiovascular diseases. As the development of machine learning technique, detection of cardiovascular diseases from ECG and PCG has been attracted much attention. However, current available methods are mostly based on single data resource. It is desirable to develop efficient multi-modal machine learning methods to predict and diagnose cardiovascular diseases. In this study, we propose a novel multi-modal method for predicting cardiovascular diseases based on ECG and PCG features. By building up conventional neural networks, we extract ECG and PCG deep coding features respectively. The genetic algorithm is used to screen the combined features and obtain the best feature subset. Then support vector machine makes classification decision. Experimental results show that compared with using single-modal features ECG and PCG, the performance of this method reaches an AUC value of 0.936 when using multi-modal data resources.</p> <p>This dataset is developed from a real-world dataset which was assembled by PhysioNet/CinC Challenge in 2016. The original dataset can be downloaded from website (<a href="http://www.physionet.org/challenge/2016/">http://www.physionet.org/challenge/2016/</a>).</p>
The choices we make and the impacts they have: Machine learning and species delimitation in North American box turtles (Terrapene spp.)
<p>Model-based approaches that attempt to delimit species are hampered by computational limitations as well as the unfortunate tendency by users to disregard algorithmic assumptions. Alternatives are clearly needed, and machine-learning (M-L) is attractive in this regard as it functions without the need to explicitly define a species concept. Unfortunately, its performance will vary according to which (of several) bioinformatic parameters are invoked. Herein, we gauge the effectiveness of M-L-based species-delimitation algorithms by parsing 64 variably-filtered versions of a ddRAD-derived SNP dataset involving North American box turtles (<i>Terrapene</i> spp.). Our filtering strategies included: (A) minor allele frequencies (MAF) of 5%, 3%, 1%, and 0% (=none), and (B) maximum missing data per-individual/per-population at 25%, 50%, 75%, and 100% (=none). We found that species-delimitation via unsupervised M-L impacted the signal-to-noise ratio in our data, as well as the discordance among resolved clades. The latter may also reflect biogeographic history, gene flow, incomplete lineage sorting, or combinations thereof (as corroborated from previously observed patterns of differential introgression). Our results substantiate M-L as a viable species-delimitation method, but also demonstrate how commonly observed patterns of phylogenetic discord can seriously impact M-L-classification.</p>
Data and code for comparison of different machine learning methods and dimensionality reduction for classification astrocytoma and glioblastoma tissues by mass spectra
<p>This upload contains all replication material for "Comparison of different machine learning methods and dimensionality reduction for classification astrocytoma and glioblastoma tissues by mass spectra" (forthcoming).</p> <p><strong>Authors:</strong> E.S. Zhvansky, A.A. Sorokin, V.A. Shurkhay, V.A. Eliferov, D.S. Bormotov, D.G. Ivanov, D.S. Zavorotnyuk, A.A. Potapov.</p> <p><strong>Code and data are located within data_and_code.zip.</strong> Code is written in Python 3.7.7 using Jupyter Notebook, MATLAB R2019b, and Python 3.5.2.</p> <p>Please find the readme.txt for code using and the code to replicate the main findings of the paper described below:</p> <ul> <li>venn_diagramm.py for Venn diagram figures.</li> <li>SSM.m for SSM calculation and visualization.</li> <li>DR_ML.ipynb for dimensionality reduction and machine learning algorithms comparing on the datasets.</li> </ul> <p> </p>
Echo-CGC: A Communication-Efficient Byzantine-tolerant Distributed Machine Learning Algorithm in Single-Hop Radio Network (video)
Full video presentation of the paper: Echo-CGC: A Communication-Efficient Byzantine-tolerant Distributed Machine Learning Algorithm in Single-Hop Radio Network.<br><br>Appears in Session 2 of the 24th International Conference on Principles of Distributed Systems OPODIS 2020<br><a href="https://opodis2020.unistra.fr">https://opodis2020.unistra.fr</a>
Data from: Machine learning improves predictions of agricultural nitrous oxide (N2O) emissions from intensively managed cropping systems
<p><span>The potent greenhouse gas nitrous oxide (N</span><sub><span>2</span></sub><span>O) is accumulating in the atmosphere at unprecedented rates largely due to agricultural intensification, and cultivated soils contribute ~60% of the agricultural flux. Empirical models of N</span><sub><span>2</span></sub><span>O fluxes for intensively managed cropping systems are confounded by highly variable fluxes and limited </span><span><span>geographic coverage;</span></span><span> process-based biogeochemical models are rarely able to predict daily to monthly emissions with > 20% accuracy even with site-specific calibration. Here we show the promise for machine learning (ML) to significantly improve field-level flux predictions, especially when coupled with a cropping systems model to simulate unmeasured </span><span><span>soil</span></span><span> parameters. We used sub-daily N</span><sub><span>2</span></sub><span>O flux data from six years of automated flux chambers installed in a continuous corn rotation at a site in the upper U.S. Midwest (~3000 sub-daily flux observations), supplemented with weekly to biweekly manual chamber measurements (~1100 daily fluxes), to train an ML model that explained 65-89% of daily flux variance with very few input variables –soil moisture, days after fertilization, soil texture, air temperature, soil carbon, precipitation, and N fertilizer rate. When applied to a long-term test site not used to train the model, the model explained 38% of the variation observed in weekly to biweekly manual chamber measurements from corn, and 51% upon coupling the ML model with a cropping systems model that predicted daily soil N availability. </span><span><span>This represents a 2-3 times improvement over conventional process-based models and with substantially fewer input requirements.</span></span><span> This coupled approach </span><span><span>offers promise</span></span><span> for better predictions of agricultural N</span><sub><span>2</span></sub><span>O emissions and thus more precise global models and more effective </span><span><span>agricultural mitigation interventions.</span></span></p>
Prediction of femoral osteoporosis using machine-learning analysis with radiomics features and abdomen-pelvic CT: A retrospective single center preliminary study
<p>Dataset of prediction of osteoporosis using APCT, radiomics and machine learning analysis.</p>
Data for "Microbiome meta-analysis and cross-disease comparison enabled by the SIAMCAT machine learning toolbox"
<p>Data for the SIAMCAT manuscript</p>
Calibration of probability predictions from machine-learning and statistical models
<p>This data set describes the occurrence (yes/no) of a bird, the Southern Whiteface (<i>Aphelocephala leucopsis)</i> in Australia. A suite of environmental variables is provided, which are used in the paper to illustrate a statistical problem. The data are meant to allow reproduction of the analysis in this paper. They are not intended for actual ecological analysis. The data come as .Rdata-file, i.e. as an R-dataset (described technically here: https://www.loc.gov/preservation/digital/formats/fdd/fdd000470.shtml).</p> <p>Here is the paper's abstract:</p> <p><span>Aim: Predictions from statistical models may be uncalibrated, meaning that the predicted values do not have the nominal coverage probability. This is easiest seen with probability predictions in machine-learning classification, including the common species occurrence probabilities. Here, a predicted probability of, say, 0.7 should indicate that out of 100 cases with these environmental conditions, and hence the same predicted probability, the species should be present in 70 and absent in 30.</span><br> <span>Innovation: A simple calibration plot shows that this is not necessarily the case, particularly not for over-fitted models or algorithms that use non-likelihood target functions. As a consequence, "raw" predictions from such model could easily be off by 0.2, are unsuitable for averaging across model types, and resulting maps hence be substantially distorted. The solution, a flexible calibration regression, is simple and can be applied whenever deviations are observed.</span><br> <span>Conclusion: "Raw", uncalibrated probability predictions should be calibrated before interpreting or averaging them in a probabilistic way.</span></p>
Benchmarking parametric and machine learning models for genomic prediction of complex traits
<p>The usefulness of genomic prediction in crop and livestock breeding programs has prompted efforts to develop new and improved genomic prediction algorithms, such as artificial neural networks and gradient tree boosting. However, the performance of these algorithms has not been compared in a systematic manner using a wide range of datasets and models. Using data of 18 traits across six plant species with different marker densities and training population sizes, we compared the performance of six linear and six non-linear algorithms. First, we found that hyperparameter selection was necessary for all non-linear algorithms and that feature selection prior to model training was critical for artificial neural networks when the markers greatly outnumbered the number of training lines. Across all species and trait combinations, no one algorithm performed best, however predictions based on a combination of results from multiple algorithms (i.e. ensemble predictions) performed consistently well. While linear and non-linear algorithms performed best for a similar number of traits, the performance of non-linear algorithms vary more between traits. Although artificial neural networks did not perform best for any trait, we identified strategies (i.e. feature selection, seeded starting weights) that boosted their performance to near the level of other algorithms. Our results highlight the importance of algorithm selection for the prediction of trait values.</p>
Automated detection of lameness in sheep using machine learning approaches: novel insights into behavioural differences among lame and non-lame sheep
Lameness in sheep is the biggest cause of concern regarding poor health and welfare among sheep producing countries. Best practice for lameness relies on rapid treatment, yet there are no objective measures of lameness detection. Use of accelerometers and gyroscopes have been widely used in human activity studies and their use is becoming increasingly common in livestock. In this study, we used 23 datasets (10 non-lame and 13 lame sheep) from an accelerometer and gyroscope-based ear sensor with a sampling frequency of 16 Hz to develop and compare algorithms that can differentiate lameness within three different activities (walking, standing and lying). We show for the first time that features extracted from accelerometer and gyroscope signals can differentiate between lame and non-lame sheep while standing, walking and lying. The random forest algorithm performed best for classifying lameness with accuracy of 84.91% within lying, 81.15% within standing and 76.83% within walking and overall correctly classified over 80% sheep within activities. Both accelerometer and gyroscope-based features ranked among the top 10 features for classification. Our results suggest that novel behavioural differences between lame and non-lame sheep across all three activities could be used to develop an automated system for lameness detection.
Data from: Machine learning-based differential network analysis: a study of stress-responsive transcriptomes in Arabidopsis thaliana
Machine learning (ML) is an intelligent data mining technique that builds a prediction model based on the learning of prior knowledge to recognize patterns in large-scale data sets. We present an ML-based methodology for transcriptome analysis via comparison of gene coexpression networks, implemented as an R package called machine learning–based differential network analysis (mlDNA) and apply this method to reanalyze a set of abiotic stress expression data in Arabidopsis thaliana. The mlDNA first used a ML-based filtering process to remove nonexpressed, constitutively expressed, or non-stress-responsive "noninformative" genes prior to network construction, through learning the patterns of 32 expression characteristics of known stress-related genes. The retained "informative" genes were subsequently analyzed by ML-based network comparison to predict candidate stress-related genes showing expression and network differences between control and stress networks, based on 33 network topological characteristics. Comparative evaluation of the network-centric and gene-centric analytic methods showed that mlDNA substantially outperformed traditional statistical testing–based differential expression analysis at identifying stress-related genes, with markedly improved prediction accuracy. To experimentally validate the mlDNA predictions, we selected 89 candidates out of the 1784 predicted salt stress–related genes with available SALK T-DNA mutagenesis lines for phenotypic screening and identified two previously unreported genes, mutants of which showed salt-sensitive phenotypes.
Characterizing Distributed Machine Learning Workloads on Apache Spark
<p>This dataset was used for our submission at Middleware'22 titled: "Characterizing Distributed ML Workloads"</p> <p>It will contains the description and the raw data, its format, as well as a detailed description of the cluster deployments used by these experiments.<br> </p> <p>The full paper is available here:</p> <p>https://dl.acm.org/doi/10.1145/3590140.3629112</p>
Data from: Impact of ecological redundancy on the performance of machine learning classifiers in vegetation mapping
Vegetation maps are models of the real vegetation patterns and are considered important tools in conservation and management planning. Maps created through traditional methods can be expensive and time‐consuming, thus, new more efficient approaches are needed. The prediction of vegetation patterns using machine learning shows promise, but many factors may impact on its performance. One important factor is the nature of the vegetation–environment relationship assessed and ecological redundancy. We used two datasets with known ecological redundancy levels (strength of the vegetation–environment relationship) to evaluate the performance of four machine learning (ML) classifiers (classification trees, random forests, support vector machines, and nearest neighbor). These models used climatic and soil variables as environmental predictors with pretreatment of the datasets (principal component analysis and feature selection) and involved three spatial scales. We show that the ML classifiers produced more reliable results in regions where the vegetation–environment relationship is stronger as opposed to regions characterized by redundant vegetation patterns. The pretreatment of datasets and reduction in prediction scale had a substantial influence on the predictive performance of the classifiers. The use of ML classifiers to create potential vegetation maps shows promise as a more efficient way of vegetation modeling. The difference in performance between areas with poorly versus well‐structured vegetation–environment relationships shows that some level of understanding of the ecology of the target region is required prior to their application. Even in areas with poorly structured vegetation–environment relationships, it is possible to improve classifier performance by either pretreating the dataset or reducing the spatial scale of the predictions.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.