Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
161
datasets available to search
ShareScore release 0.9.0
Dataset results
161 results for “unsupervised”
Comparing Storm Resolving Models and Climates via Unsupervised Machine Learning
<p>Storm-resolving climate models (SRMs) have gained international interest for their unprecedented detail with which they globally resolve convection. However, this high resolution also makes it difficult to quantify the emergent differences or similarities among complex atmospheric formations induced by different parameterizations of sub-grid information. This paper uses modern unsupervised machine learning methods to analyze and intercompare SRMs based on their high-dimensional simulation data, learning low-dimensional latent representations and assessing model similarities based on these representations. To quantify such inter-SRM ``distribution shifts'', we use variational autoencoders in conjunction with vector quantization. Our analysis involving nine different global SRMs reveals that only six of them are aligned in their representation of atmospheric dynamics. Our analysis furthermore reveals regional and planetary signatures of the convective response to global warming in a fully unsupervised, data-driven way. In particular, this approach can help elucidate the effects of climate change on rare convection types, such as ``Green Cumuli''. Our study provides a path toward evaluating future high-resolution global climate simulation data more objectively and with less human intervention than has historically been needed.</p>
Observing flow of He II with unsupervised machine learning
<p>Data repository for observing flow in He II with unsupervised machine learning.</p>
Fine-scale subpopulation detection via SNP-based unsupervised method: A case study on the 1000 Genomes Project Resources
<p>Here are the supplementary files to the paper "Fine-scale subpopulation detection via SNP-based unsupervised method:<br> A case study on the 1000 Genomes Project Resources".<br> <br> The repository is organized as:</p> <ol> <li><strong>Supplementary information</strong>: the additional detailed information for the experiments in the paper <ul> <li>Supplementary_information_IPCAPS_Chaichoompu_v1.pdf</li> </ul> </li> <li><strong>Real-life dataset</strong>: the 1000 genome dataset, which is referred to in the paper and is filtered with the parameters explained in the paper. Reference: https://www.internationalgenome.org/ <ul> <li>1000genomes_with_filtering.zip</li> </ul> </li> </ol>
Preprocessed Data for "Comparing Storm Resolving Models and Climates via Unsupervised Machine Learning"
<p>Preprocessed Data (training and test) for 3 SRMs used in "Comparing Storm Resolving Models and Climates via Unsupervised Machine Learning". Here we included ICON, SPCAM, and SPCAM with sea surface tem[eratures warmed by +4K. Additionally we include lat/lon information for the test data.</p>
Dataset: Quantifying cell densities and biovolumes of phytoplankton communities and functional groups using scanning flow cytometry, machine learning and unsupervised clustering
<p>This dataset contains all relevant data for the manuscript (in submission) "<em>Quantifying cell densities and biovolumes of phytoplankton communities and functional groups using scanning flow cytometry, machine learning and unsupervised clustering</em>".</p> <p>Code written to analyse this dataset (which may be adapted for other flow cytometry datasets) is found at https://zenodo.org/record/999747</p> <p>--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------</p> <p>Naming convention for raw flow cytometry data files (located in /Script 3. Generating raw data subset/input/):</p> <p>[Allparameters] _ [Year] - [Month] - [Date] [Hour] [u] [Minute] _ [Depth]</p> <p>e.g: Allparameters_2014-07-31 08u08_1.0m</p> <p>The date, time and depth indicate the location and time at which the measurement was taken.</p>
Unsupervised detection of Large-scale Weather Patterns in the Northern Hemisphere via Markov State Modelling: from Blockings to Teleconnections
<p><em><span>Data Source</span></em></p> <p>This dataset is derived from the <em><span>NCEP-NCAR Reanalysis 1</span> data provided by the NOAA PSL, Boulder, Colorado, USA, from their website at <a href="https://psl.noaa.gov/">https://psl.noaa.gov</a></em>., a robust atmospheric dataset that includes a wide range of climatic measurements essential for comprehensive climate analysis. The original data can be accessed at the NOAA Physical Sciences Laboratory website: https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html.</p> <p> </p>
VasTexture: Vast repository of textures and PBR Materials extracted from images using unsupervised approach
<h2><strong>VasTexture: Vast repository of textures and SVBRDF/PBR Materials extracted from images using an unsupervised approach.</strong></h2> <p> </p> <p>This dataset contains hundreds of thousands of textures and PBR/SV-BRDF materials extracted from real-world natural images.</p> <p> </p> <p>The repository is composed of RGB images of textures given as RGB images (each image is one uniform texture) and folders of PBR/SVBRDF materials given as a set of property maps (base color, roughness, metallic, etc).</p> <p>Note that this contain subset of repository more could be found in the <a href="https://sites.google.com/view/infinitexture/home" target="_blank" rel="noopener">main project page</a>.</p> <p>Visualisation of sampled PBRs and Textures can be seen in: <a href="../records/11391127/files/PBR_examples.jpg?download=1" target="_blank" rel="noopener">PBR_examples.jpg</a> and <a href="../records/11391127/files/Textures_Examples.jpg?download=1" target="_blank" rel="noopener">Textures_Examples.jpg</a></p> <p><a href="https://sites.google.com/view/infinitexture/home" target="_blank" rel="noopener">Link to the main project page</a></p> <p><a href="https://www.arxiv.org/pdf/2403.03309" target="_blank" rel="noopener">Link to paper</a></p> <p> </p> <h2>File structure</h2> <p>Texture images are given in the <strong><a href="../records/11391127/files/Extracted_textures_1.zip?download=1" target="_blank" rel="noopener">Extracted_textures_</a>*.zip</strong> files.</p> <p>Each image in this zip file is a single texture, the textures were extracted and cropped from the <a href="https://storage.googleapis.com/openimages/web/index.html" target="_blank" rel="noopener">open images dataset</a>. </p> <p> </p> <p>PBR Materials are available in <strong><a href="../records/11391127/files/PBR_O0_1.zip?download=1" target="_blank" rel="noopener">PBR_*.zip</a></strong> files these PBRs were generated from the texture images in an unsupervised way (with no human intervention). Each subfolder in this file contains the properties map of the PBRs (roughness, metallic, etc, suitable for blender/unreal engine). Visualization of the rendered material appears in the file Material_View.jpg in each PBR folder.</p> <p>PBR materials and textures which are larger then 512x512 pixels contain '<strong>large' </strong>or<strong> 'larger' </strong>in their file name there about 40k in this repository but 100k more can be found in the <a href="https://sites.google.com/view/infinitexture/home" target="_blank" rel="noopener">main project page</a>.</p> <p>PBR materials and textures who are seamless marked as contain<strong> 'seamless'</strong> in their file name. </p> <p> </p> <p>PBR materials that were generated by mixing other PBR materials are available in files with the names<a href="../records/11391127/files/PBR_mix_O2_2.zip?download=1"> </a><strong><a href="../records/11391127/files/PBR_mix_O2_2.zip?download=1">PBR_mix*.zip</a> </strong></p> <p> </p> <p>Samples for each case can be found in files named: <strong> Sample_*.zip</strong></p> <p> </p> <p>File with the word <strong>Seamless </strong>contain tileable seamless textures this files also contain textures and PBR>512 size that were extracted from the Segment Anything Dataset.</p> <p>Since the textures were extracted fom natural images they were not natively seamless but were turned to seamless using the code at <strong><a href="https://github.com/sagieppel/convert-image-into-seamless-tileable-texture">this url</a></strong></p> <p> </p> <p> </p> <p><strong>Documented code used to extract the textures and generate the PBRs is available at: </strong></p> <p><strong><a href="../records/11391127/files/Texture_And_Material_ExtractionCode_And_Documentation.zip?download=1" target="_blank" rel="noopener">Texture_And_Material_ExtractionCode_And_Documentation.zip</a></strong></p> <h2>Details:</h2> <p>The materials and textures were extracted from real-world images using an unsupervised extraction method (code supplied). As such they are far more diverse and wide in scope compared to existing repositories, at the same time they are much more noisy and contain more outliers compared to existing repositories. This repository is probably more useful for things that demand large-scale and very diverse data, yet can use noisy and lower quality compared to professional repositories with manually made assets like ambientCG. It can be very useful for creating machine learning datasets, or large-scale procedural generation. It is less suitable for areas that demand precise clean and categorized PBR like CGI art and graphic design. For preview It is recommended to look at <a href="../records/11391127/files/PBR_examples.jpg?download=1" target="_blank" rel="noopener">PBR_examples.jpg</a> and <a href="../records/11391127/files/Textures_Examples.jpg?download=1" target="_blank" rel="noopener">Textures_Examples.jpg</a> or download the Sample files and look at the Material_View.jpg files to visualize the quality of the materials.</p> <h3>Scale:</h3> <p>Currently, there are a few hundred of thousands PBR materials and textures but the goal is to make this into over a million in the near future.</p> <h2>Data generation code:</h2> <p>The Python scripts used to extract these assets are supplied at: </p> <p><strong><a href="../records/11391127/files/Texture_And_Material_ExtractionCode_And_Documentation.zip?download=1" target="_blank" rel="noopener">Texture_And_Material_ExtractionCode_And_Documentation.zip</a></strong></p> <p>The code could be run in any folder of random images extract regions with uniform textures and turn these into PBR materials. </p> <h2>Alternative download sources:</h2> <p>Alternative download sources:</p> <p><a href="https://sites.google.com/view/infinitexture/home" target="_blank" rel="noopener">https://sites.google.com/view/infinitexture/home</a></p> <p><a href="https://e.pcloud.link/publink/show?code=kZON5TZtxLfdvKrVCzn12NADBFRNuCKHm70" target="_blank" rel="noopener">https://e.pcloud.link/publink/show?code=kZON5TZtxLfdvKrVCzn12NADBFRNuCKHm70</a></p> <p><a href="https://icedrive.net/s/jfY1xSDNkVwtYDYD4FN5wha2A8Pz" target="_blank" rel="noopener">https://icedrive.net/s/jfY1xSDNkVwtYDYD4FN5wha2A8Pz</a></p> <p> </p> <h2>Paper</h2> <p>This work was done as part of the paper "<a href="https://www.arxiv.org/pdf/2403.03309" target="_blank" rel="noopener">Learning Zero-Shot Material States Segmentation,</a></p> <p><a href="https://www.arxiv.org/pdf/2403.03309" target="_blank" rel="noopener">by Implanting Natural Image Patterns in Synthetic Data</a>".</p> <p>@article{eppel2024learning,</p> <p> title={Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data},</p> <p> author={Eppel, Sagi and Li, Jolina and Drehwald, Manuel and Aspuru-Guzik, Alan},</p> <p> journal={arXiv preprint arXiv:2403.03309},</p> <p> year={2024}</p> <p>}</p> <p> </p> <h2>License:</h2> <p>All the code and repositories are available on CC0 (free to use) licenses.</p> <p>Textures were extracted from the open images dataset which is an Apache license.</p>
An unsupervised deep learning framework with variational autoencoders for genome-wide DNA methylation analysis and biologic feature extraction applied to breast cancer
<p>Supplemental data for the paper titled "An unsupervised deep learning framework with variational autoencoders for genome-wide DNA methylation analysis and biologic feature extraction applied to breast cancer"</p>
Headcam: Cylindrical Panoramic Video Dataset for Unsupervised Learning of Depth and Ego-Motion
<p>This dataset contains panoramic video captured from a helmet-mounted camera while riding a bike through suburban Northern Virginia. We used the videos to evaluate an unsupervised learning method for depth and ego-motion estimation, as described in our paper:</p> <p>Alisha Sharma and Jonathan Ventura. "Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video." Proceedings of the 2019 IEEE Artificial Intelligence & Virtual Reality Conference, San Diego, CA, 2019.</p> <p>If you make use of this dataset, please cite this paper.</p> <p> </p> <p>The videos are stored as .mkv video files encoded using lossless H.264. To extract the images, we recommend using ffmpeg:</p> <blockquote> <p>mkdir 2018-10-03 ;</p> <p>ffmpeg -i 2018-10-03.mkv -q:v 1 2018-10-03/%05d.png ;</p> </blockquote> <p>Associated code can be found in our <a href="https://github.com/jonathanventura/cylindricalsfmlearner">GitHub repository</a>. </p>
Sampled sentence pairs from SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection
<p>Each dataset consists of samples, containing two sentences with positions of one of the given target words. In every sample, first sentence is taken from corpus1 and second from corpus2. Initial sentences were taken from https://www.ims.uni-stuttgart.de/en/research/resources/corpora/sem-eval-ulscd/. </p>
Data for "Unsupervised learning of sequence-specific aggregation behavior for a model copolymer"
<p>These are the data associated with the paper, "Unsupervised learning of sequence-specific aggregation behavior for a model copolymer" (DOI 10.1039/D1SM01012C). Each of the directories contains subdirectories with `GSD` files dumped from HOOMD. Each subdirectory roughly corresponds to one or two of the figures in the paper.</p>
Data for: Defining usual oral temperature ranges in outpatients using an unsupervised learning algorithm
<p><strong>Importance</strong>: Although oral temperature is commonly assessed in medical examinations, the range of usual or "normal" temperature is poorly defined.</p> <p><strong>Objective</strong>: To determine normal oral temperature ranges by age, sex, height, weight and time of day.</p> <p><strong>Design</strong>: We applied a filtering algorithm (LIMIT) to 10 years of outpatient temperature measurements. LIMIT iteratively removed encounters with primary diagnoses overrepresented in the tails of the temperature distribution, leaving only those diagnoses unrelated to temperature. Mixed effects modeling was applied to the remaining temperature measurements to identify independent predictors of normal oral temperature and to generate individualized normal temperature ranges.</p> <p><strong>Setting</strong>: Single large medical care system, divisions of Internal Medicine and Family Medicine.</p> <p><strong>Participants</strong>: All adult outpatient encounters that included temperature measurements, April 2008 - June 2017.</p> <p><strong>Exposures</strong>: Primary diagnoses and medications, age, sex, height, weight, time of day and month, abstracted from each outpatient encounter.</p> <p><strong>Main outcomes and measures</strong>: Normal temperature ranges by age, sex, height, weight, and time of day.</p> <p><strong>Results</strong>: From 618,306 encounters, 36% were removed by LIMIT because they included diagnoses or medications that fell disproportionately in the tails of the temperature distribution. The encounters removed due to overrepresentation in the upper tail were primarily linked to infectious diseases (76.81% of all removed encounters); type 2 diabetes mellitus was the only diagnosis removed for over-representation in the lower tail (15.71% of all removed encounters). Prior to running LIMIT, the mean overall oral temperature was 36.71°C (±0.43); following LIMIT, the mean temperature was 36.64°C (±0.35). Using mixed effects modeling, age, sex, height, weight and time of day accounted for 6.86% (overall) and up to 25.52% (per subject) of the observed variability in temperature. Mean normal oral temperature did not reach 37°C for any subgroup; the upper 99th percentile ranged from 36.8°C (tall underweight old men in the morning) to 37.9°C (short obese young women in the afternoon).</p> <p><strong>Conclusion and relevance</strong>: Normal oral temperature varies in a predictable manner based on sex, age, height, weight and time of day, allowing individualized normal temperature ranges to be established. </p>
Original data for "Unsupervised Analysis of Optical Imaging Data for the Discovery of Reactivity Patterns in Metal Alloy" article
<p>This upload includes both original and supplementary data for the publication "Unsupervised Analysis of Optical Imaging Data for the Discovery of Reactivity Patterns in Metal Alloy" by R. Li, A. Makogon, T. Galochkina, J-F. Lemineur, F. Kanoufi, and V. Shkirskiy, published in the Small Methods journal. The preprint version of the paper is available on ChemRxiv (<a href="https://doi.org/10.26434/chemrxiv-2023-sgvt0">https://doi.org/10.26434/chemrxiv-2023-sgvt0</a>).</p> <p>The file "Data_processing.zip" contains the original optical image of the interface, maps of film evolution rates in acidic and salt environments, COMSOL data output in acidic and salt environments, as well as a Jupyter Lab file that demonstrates how to process this data.</p> <p>The file "Comparison_of_SEM_images.zip" includes the original SEM images used in the current study and in our previous work (<a href="https://doi.org/10.26434/chemrxiv-2022-rn77b-v3">https://doi.org/10.26434/chemrxiv-2022-rn77b-v3</a>), along with a Jupyter Lab file that illustrates the data processing procedure.</p> <p>The file "Original_data_in_npy_format.zip" contains all original data from SEM/EDX and RM experiments.</p> <p>We recommend opening the Jupyter Lab files in a Python 3 environment. PDF files in the root directory provide outputs of all uploaded Jupyter Lab files.</p>
Data for: Defining usual oral temperature ranges in outpatients using an unsupervised learning algorithm
Open the record for dataset details and reuse information.
Total Ortholog Median Matrix (TOMM): an alternative unsupervised approach for phylogenomics based on evolutionary distance between protein coding genes
Open the record for dataset details and reuse information.
Data from: Constructing concepts without feedback: An empirical investigation of how relational information affects multidimensional concept completion behavior in an unsupervised task
Open the record for dataset details and reuse information.
Data from: Generalizable physical descriptors of pool boiling heat transfer from unsupervised learning of images
Open the record for dataset details and reuse information.
Data from: A demonstration of unsupervised machine learning in species delimitation
Open the record for dataset details and reuse information.
Data from: Unsupervised machine learning reveals mimicry complexes in bumble bees occur along a perceptual continuum
Open the record for dataset details and reuse information.
Unsupervised-Hackathon
<p>The record contains LHC simulations to be used in the hackathon related with the darkmachines unsupervised challenge project. The documentation about the files content in the tar file is included as well (dataset.pdf).</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.