Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
298
datasets available to search
ShareScore release 0.9.0
Dataset results
298 results for “Multi-modal”
ChessRender360: High-Fidelity Rendered Chess Dataset with Multi-Modal Annotations
<p>ChessRender360 is a synthetically crafted dataset featuring 10,000 rendered chess positions. Designed for computer vision and machine learning research, this dataset provides a rich collection of RGB images, depth maps, instance masks, and semantic segmentation masks for each chess piece and board element.</p> <p>Each chess position is rendered in high resolution (2000x2000 pixels), capturing the intricate details of the board and pieces from various angles. The dataset includes:</p> <ul> <li><strong>RGB Images:</strong> High-quality rendered images of chess positions, showcasing a diverse range of board configurations.</li> <li><strong>Depth Maps:</strong> Accurate depth representations of the scene, capturing depth in the range of 20 cm to 120 cm. In the depth maps, black corresponds to a depth of 20 cm, and white corresponds to 120 cm, providing spatial information for each position.</li> <li><strong>Instance Masks:</strong> Unique instance masks for each chess piece, enabling precise identification and localization.</li> <li><strong>Semantic Segmentation Masks:</strong> Segmentation masks that differentiate between piece types and board elements, with distinct hue values assigned to each type.</li> <li><strong>Bounding Boxes:</strong> Each sample has an annotation <code>.json</code> file containing bounding boxes for each piece.</li> <li><strong>Board Corners:</strong> Same annotation <code>.json</code> contains positions of corners of the board in order: white left, white right, black left, black right.</li> <li><strong>FENs:</strong> A CSV file containing the FEN (Forsyth-Edwards Notation) for each chess position in the dataset, listed in order. This allows users to easily recognize and replicate the exact board position from any image.</li> </ul> <p><strong>Bounding Box Generation:</strong></p> <ul> <li>The dataset does not include predefined bounding boxes, but they can be easily generated from the provided semantic and instance masks. This allows for flexible bounding box creation tailored to specific research needs.</li> </ul> <p><strong>Rendering Details:</strong></p> <ul> <li><strong>3D Models and Materials:</strong> The dataset uses a consistent set of 3D models for all chess pieces, with three different material/color schemes applied across the dataset, along with random perturbations in material brightness, contrast and saturation to introduce visual variety.</li> <li><strong>Camera Angles:</strong> Camera angles are randomly selected, with yaw ranging from 0 to 360 degrees and pitch between 30 to 80 degrees, providing diverse perspectives of the chess positions.</li> <li><strong>Background Variability:</strong> The chessboard is randomly placed on different types of tables, with the floor material randomly sampled to create a variety of backgrounds.</li> <li><strong>Lighting:</strong> Lighting conditions are randomly generated, adding further diversity and realism to the rendered scenes.</li> </ul> <p><strong>Augmentation Potential:</strong></p> <ul> <li>The instance and semantic masks can be used to further augment the dataset. Researchers can selectively modify specific parts of the images—such as the board, background, or individual pieces—enabling the creation of new variations and enhancing the dataset's utility for model training and testing.</li> </ul> <p><strong>Color Mapping:</strong></p> <ul> <li>The semantic masks are color-coded using a hue-based system, where the board frame, squares, and each piece type are assigned specific hues. Instances of the same piece type are differentiated by varying the value component, with saturation consistently set to 1. A detailed <code>color_mapping.json</code> file is included, providing a comprehensive guide to interpreting the masks.</li> </ul> <p><strong>Applications:</strong> ChessRender360 is ideal for tasks such as object detection, instance segmentation, depth estimation, and scene understanding in synthetic environments. Researchers and developers can leverage this dataset for training and evaluating models in computer vision, robotics, and artificial intelligence.</p> <p><strong>Dataset Highlights:</strong></p> <ul> <li>10,000 uniquely rendered chess positions</li> <li>High-resolution images (2000x2000 pixels) with diverse visual characteristics</li> <li>Comprehensive annotations with RGB, depth (20 cm to 120 cm), instance, and semantic maps</li> <li>Side identification map for distinguishing white and black sides of the board</li> <li>FEN notation CSV file for easy position recognition and replication</li> <li>Variety introduced through different material schemes and lighting setups</li> <li>Randomized camera angles for enhanced perspective diversity</li> <li>Potential for further augmentation by modifying specific image components</li> <li>Detailed color mapping for easy interpretation of segmentation masks</li> <li>Suitable for a wide range of computer vision and AI applications</li> </ul> <p>ChessRender360 offers a rich and versatile dataset for advancing research and development in the field of computer vision, providing a synthetic yet highly realistic environment for model training and testing.</p> <p>For any questions, feedback, or collaboration opportunities, or if you are interested in custom artificial datasets, please contact me:</p> <ul> <li><strong>Name:</strong> Marko Kojić</li> <li><strong>LinkedIn:</strong> https://www.linkedin.com/in/mmkoya</li> </ul> <p>I welcome inquiries from researchers, developers, and organizations interested in utilizing or collaborating on artificial datasets.</p>
EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation
<p>EMOPIA (pronounced ‘yee-mò-pi-uh’) dataset is a shared multi-modal (audio and MIDI) database focusing on perceived emotion in <strong>pop piano music</strong>, to facilitate research on various tasks related to music emotion. The dataset contains <strong>1,087</strong> music clips from 387 songs and <strong>clip-level</strong> emotion labels annotated by four dedicated annotators. </p> <p>For more detailed information about the dataset, please refer to our paper: <a href="https://arxiv.org/abs/2108.01374"><strong>EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation</strong></a>. </p> <p><strong>File Description</strong></p> <ul> <li><em><strong>midis/</strong></em>: midi clips transcribed using GiantMIDI. <ul> <li>Filename `Q1_xxxxxxx_2.mp3`: Q1 means this clip belongs to Q1 on the V-A space; xxxxxxx is the song ID on YouTube, and the `2` means this clip is the 2nd clip taken from the full song.</li> </ul> </li> <li><em><strong>metadata/</strong></em>: metadata from YouTube. (Got when crawling)</li> <li> <p><em><strong>songs_lists/</strong></em>: YouTube URLs of songs.</p> </li> <li> <p><em><strong>tagging_lists/</strong></em>: raw tagging result for each sample.</p> </li> <li> <p><em><strong>label.csv</strong></em>: metadata that records filename, 4Q label, and annotator.</p> </li> <li> <p><em><strong>metadata_by_song.csv</strong></em>: list all the clips by the song. Can be used to create the train/val/test splits to avoid the same song appear in both train and test.</p> </li> <li> <p><em><strong>scripts/prepare_split.ipynb:</strong></em> the script to create train/val/test splits and save them to csv files.</p> </li> </ul> <p>------</p> <p><strong>2.2 Update</strong></p> <ul> <li>Add tagging files in <em><strong>tagging_lists/</strong></em> that are missing in the previous version.</li> <li>Add <em><strong>timestamps.json</strong></em> for easier usage. It records all the timestamps in dict format. You can see <em><strong>scripts/load_timestamp.ipynb</strong></em> for the format example.</li> <li>Add <em><strong>scripts/timestamp2clip.py</strong></em>: After the raw audio are crawled and put in <em><strong>audios/raw</strong></em>, you can use this script to get audio clips. The script will read <em><strong>timestamps.json</strong></em> and use the timestamp to extract clips. The clips will be saved to <em><strong>audios/seg</strong> </em>folder.</li> <li>remove 7 midi files that were added by mistake, and also corrected the number in <em><strong>metadata_by_song.csv</strong></em>.</li> </ul> <p> </p> <p><strong>2.1 Update</strong></p> <p>Add one file and one folder:</p> <ul> <li><em><strong>key_mode_tempo.csv</strong></em>: key, mode, and tempo information extracted from files.</li> <li><strong><em>CP_events/</em></strong>: CP events used in our paper. Extracted using this <a href="https://github.com/YatingMusic/compound-word-transformer/blob/main/dataset/representations/uncond/cp/corpus2events.py">script</a>, and add the emotion event to the front.</li> </ul> <p>Modify one folder:</p> <ul> <li>The <strong><em>REMI_events/</em></strong> files in version 2.0 contain some information that is not related to the paper, so remove it.</li> </ul> <p> </p> <p><strong>2.0 Update</strong></p> <p>Add two new folders:</p> <ul> <li><strong><em>corpus/</em></strong>: processed data that following <a href="https://github.com/YatingMusic/compound-word-transformer/blob/main/dataset/Dataset.md">the preprocessing flow</a>. (Please notice that although we have <code>1078</code> clips in our dataset, we lost some clips during steps 1~4 of the flow, so the final number of clips in this <strong><code>corpus</code></strong> is <code>1052</code>, and that's the number we used for training the generative model.)</li> <li><strong><em>REMI_events/</em></strong>: REMI event for each midi file. They are generated using this <a href="https://github.com/YatingMusic/compound-word-transformer/blob/main/dataset/representations/uncond/remi/corpus2events.py">script</a>.</li> </ul> <p>-------- </p> <p> </p> <p> </p> <p> </p> <p><strong>Cite this dataset</strong></p> <pre><code>@inproceedings{{EMOPIA}, author = {Hung, Hsiao-Tzu and Ching, Joann and Doh, Seungheon and Kim, Nabin and Nam, Juhan and Yang, Yi-Hsuan}, title = {{MOPIA}: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation}, booktitle = {Proc. Int. Society for Music Information Retrieval Conf.}, year = {2021} }</code></pre>
MTL subregion segmentation atlas with 7T and 3T multi-modality MRI
Open the record for dataset details and reuse information.
FAPM: Functional annotation of proteins using multi-modal models beyond structural modeling
Open the record for dataset details and reuse information.
plan4res - public dataset for case study 1 part MIM-1: time series used for multi-modal investment pathway modelling
<p>Public data set which is used within the plan4res project for performing case study 1 "Multi-modal European energy concept for achiving COP21" - Multi-modal Investment modelling (MIM) Part 1: Time series for the reference year 2015</p> <p>The related documentation is included in plan4res' deliverable D4.5 chapter 3.2 (see 10.5281/zenodo.3785010) </p> <p>The data set includes the following data:</p> <p>a) characteristic annual load profiles for large industrial heat demand for chemical, iron & steel, food & beverage and pulp & paper industries for the reference year 2015</p> <p>HOTMAPS__TD_OUT_D_CHEM__20200608T160653__20200422T120000Z__v01.csv <br> HOTMAPS__TD_OUT_D_FOOD__20200608T160724__20200422T120000Z__v01.csv <br> HOTMAPS__TD_OUT_D_IRON__20200608T160705__20200422T120000Z__v01.csv HOTMAPS__TD_OUT_D_PAPER__20200608T160715__20200422T120000Z__v01.csv </p> <p>b) characteristic demand profiles for road-side car passenger transport and availability of cars for charging while (home) parking for the reference year 2015 </p> <p>SIEMENS__TD_OUT_D_RoadCar__20200608T160627__20200401T120000Z__v01.csv SIEMENS__TD_CAP_CarPark__20200608T160637__20200401T120000Z__v01.csv </p> <p>c) load profiles for exogeneous demand of electricity for the reference year 2015. The exogenous demand includes all electricity consumptions not explicitly modeled within MIM modeling.</p> <p>HRE4__TD_OUT_ElectricityExo__20200608T160732__20200401T120000Z__v01.csv </p> <p>c) regionally resolved demand profiles for (individual) space heating and space cooling for the reference year 2015</p> <p>HRE4__TRD_CAP_Cool_2015__20200608T160051__20200401T120000Z__v01.csv <br> HRE4__TRD_CAP_HeatInd_2015__20200608T155849__20200401T120000Z__v01.csv</p> <p>d) regionally resolved generation profiles of electricity from photovoltaic, wind onshore, wind offshore, hydro run-of-river, and for heat generation from solar thermal for the reference year 2015</p> <p>NINJA__TRD_CAP_PV_2015__20200608T160440__20191104T120000Z__v01.csv <br> NINJA__TRD_CAP_WindOFF_2015__20200608T155422__20191104T120000Z__v01.csv <br> NINJA__TRD_CAP_WindON_2015__20200608T155251__20191104T120000Z__v01.csv HRE4__TRD_CAP_HydroRoR_2015__20200608T155550__20200401T120000Z__v01.csv <br> HRE4__TRD_CAP_SolarThermal_2015__20200608T155718__20200401T120000Z__v01.csv </p> <p>e) regionally resolved generation profile of electricity from wind offshore transformed in a way to represent potential capacity factors in future as stated by doi:10.2760/041705. Data based on reference year 2015</p> <p>SIEMENS__TRD_CAP_WindOFF_2040__20200608T155127__20200401T120000Z__v01.csv </p> <p>x) A list of geographical description of the zone hierarchy data used in MIM for the EU33 region set.:</p> <p>SIEMENS__ZoneHierarchy_MIM_EU33__20181231T120000Z___20200131T1200000Z__v001.csv </p> <p>Further info:</p> <p>Time series are based on historical data for the reference year 2015. </p> <p>Values are normalized over one reference year in a way that either the maximum = 1 (CAP) or the integral = 1 (OUT).</p> <p>All values are listed in arbitrary units. </p> <p>All country names are according to ISO 3166-1 alpha-2.</p>
Data from: Multi-modal ultra-high resolution structural 7-Tesla MRI data repository
Structural brain data is key for the understanding of brain function and networks, i.e., connectomics. Here we present data sets available from the 'atlasing of the basal ganglia (ATAG)' project, which provides ultra-high resolution 7Tesla (T) magnetic resonance imaging (MRI) scans from young, middle-aged, and elderly participants. The ATAG data set includes whole-brain and reduced field-of-view MP2RAGE and T2*-weighted scans of the subcortex and brainstem with ultra-high resolution at a sub-millimeter scale. The data can be used to develop new algorithms that help building high-resolution atlases both relevant for the basic and clinical neurosciences. Importantly, the present data repository may also be used to inform the exact positioning of electrodes used for deep-brain-stimulation in patients with Parkinson's disease and neuropsychiatric diseases.
Dataset: Aberration characterisation of X-ray optics using multi-modal ptychography and a partially coherent source
<p>These are the ptychography datasets used for the publication "Aberration characterisation of X-ray optics using multi-modal ptychography and a partially coherent source". File are in the HDF format and contain a number of datasets detailed below.</p> <p>If you require more information, please contact the corresponding author of the publication, at thomas.moxham@eng.ox.ac.uk</p> <p>Raw files contain: diffraction intensities, scanning positions in millimeters</p> <p>Recon files contain: fourier error, complex probe function, complex object function, recon pixel size, energy</p> <p>raw_siemens_star_be_lens.hdf, raw_fourier_ring_correlation.hdf, merlin_medipix_detector_mask.hdf, recon_multi-modal_probe.hdf, recon_multi-modal_object.hdf, recon_fourier_ring_correlation.hdf</p>
Data from: Multi-modal defenses in aphids offer redundant protection and increased costs likely impeding a protective mutualism
1.The pea aphid, Acyrthosiphon pisum, maintains extreme variation in resistance to its most common parasitoid wasp enemy, Aphidius ervi, which is sourced from two known mechanisms: protective bacterial symbionts, most commonly Hamiltonella defensa, or endogenously encoded defenses. We have recently found that individual aphids may employ each defense individually, occasionally both defenses together, or neither. 2.In field populations, Hamiltonella-infected aphids are found at low to moderate frequencies and while less is known about the frequency of resistant genotypes, they show up less often than susceptible genotypes in field collections. To better understand these patterns, we sought to compare the strengths and costs of both types of defense, individually and together, in order to elucidate the selective pressures that maintain multi-modal defense mechanisms or that may favor one over the other. 3.We experimentally infected five aphid genotypes (two lowly and three highly resistant), each with two symbiont strains, Hamiltonella-APSE8 (moderate protection) and Hamiltonella-APSE3 (high protection). This resulted in three sublines per genotype: uninfected, +APSE8, and +APSE3. Each of the fifteen total sublines was first subjected to a parasitism assay to determine its resistance phenotype and in a second experiment a subset were chosen to compare fitness (fecundity and survivorship) in presence and absence of parasitism. 4.In susceptible aphid genotypes, parasitized sublines infected with Hamiltonella generally showed increased protection with direct fitness benefits, but clear infection costs to fitness in the absence of parasitism. In resistant genotypes, Hamiltonella infection rarely conferred additional protection, often further reduced fecundity and survivorship when enemy challenged, and resulted in constitutive fitness costs in the absence of parasitism. We also identified strong aphid-genotype X symbiont-strain interactions, such that the best defensive strategy against parasitoids varied for each aphid genotype; one performed best with no protective symbionts, the others with particular strains of Hamiltonella. 5.This surprising variability in outcomes helps explain why Hamiltonella infection frequencies are often intermediate and do not strongly track parasitism frequencies in field populations. We also find that variation in endogenous traits, such as resistance, among host genotypes may offer redundancy and generally limit the invasion potential of mutualistic microbes in insects.
mRI: multi-modal 3d human pose estimation dataset using mmwave, rgb-d, and inertial sensors
<p>The ability to estimate 3D human body pose and movement, also known as human pose estimation~(HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few datasets exploit multiple modalities and focus on home-based health monitoring.</p> <p>To bridge this gap, we present <em>mRI</em>, a multi-modal 3D human pose estimation dataset with mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 5 million frames from 20 subjects performing rehabilitation exercises and supports the benchmarks of HPE and action detection. We perform extensive experiments using our dataset and delineate the strength of each modality.</p> <p>We hope that the release of <em>mRI</em> can catalyze the research in pose estimation, multi-modal learning, and action understanding, and more importantly, facilitate the applications of home-based health monitoring.</p>
Multi-modal pose estimation in XR applications leveraging integrated sensing and communication: Dataset
<p>This dataset refers to paper Multi-modal pose estimation in XR applications leveraging integrated sensing and communication in workshop of ACM Mobicom. CSI of 3 people performing a set of 8 poses. This dataset contain discrete classes and corresponding CSI data. Kinect poses can be found here (https://github.com/nisarnabeel/multi-modal-pose-estimation-CSI-mmWave).</p> <p> </p> <p>Abstract: Mobile extended reality (XR) applications are anticipated to generate substantial traffic for 6G. Such applications not only require high data rate and low-latency transmissions, but also accurate and real-time pose estimation to enable interactive and immersive experiences. While sub-6 GHz signals have been exploited for pose estimation, they cannot cope up with multi-gigabit data rates required by XR applications. Instead, mobile communications at mmWave frequencies can potentially support data rates up to several giga-bits per second (Gbps) and, therefore, can be used to deliver XR content wirelessly to the Head-Mounted Display (HMD). Moreover, mmWave frequencies can offer improved sensing due to the large available bandwidth. Therefore, mmWave communications can play a crucial role in enabling device-free interactivity by offering both high-speed communication and accurate sensing capabilities. However, mmWave propagation characteristics are different from sub-6 GHz. Path loss plays a significant role, and can lead to degraded sensing performance. Therefore, our proposal supplements wireless sensing at mmWave frequencies with wireless electromyography (EMG) armbands. By capturing patterns of muscle activities, we can counteract the limitations of mmWave-based pose estimation, thereby enriching the granularity and precision of pose estimation. This paper proposes a conceptual architecture to achieve multi-modal pose estimation for XR applications. Early results highlight the shortcomings of mmWave-based sensing, and we identify future steps and opportunities on integration of both approaches.</p>
Multi-modal phantom experiments, mimicking flow through the mitral heart valve
<p>In this repository all experimental data obtained from a phantom study on the left heart, including a deformable mitral valve, is reported. Using invasive catheter pressure measurements, magnetic resonance imaging and ultrasound imaging, several parameters relevant to diagnosing heart valve disease were measured on the phantom. The data is reported in MS excel files (xlsx format) and the geometry of the phantom is reported in STL and STEP file format, together with an exploded view, indicating how the parts are put together.</p>
Phospho-seq: Integrated, multi-modal profiling of intracellular protein dynamics in single cells
<p> </p><p>Datasets to go along with the publication listed:</p><p>full_object.rds: Brain Organoid Phospho-Seq dataset with ATAC, Protein and imputed RNA data</p><p>rna_object.rds: Reference whole cell scRNA-Seq object on Brain organoids</p><p>multiome_object.rds: Bridge dataset containing RNA and ATAC modalities for Brain organoids</p><p>metacell_allnorm.rds: Metacell object for finding gene-peak-protein linkages in Brain organoid dataset</p><p>fullobject_fragments.tsv.gz: fragment file to go with the full object</p><p>fullobject_fragments.tsv.gz.tbi:index file for the full object fragment file</p><p>multiome_fragments.tsv.gz: fragment file to go with the multiome object</p><p>multiome_fragments.tsv.gz.tbi:index file for the multiome object fragment file</p><p>K562_Stem.rds : object corresponding to the pilot experiment including K562 cells and iPS cells</p><p>K562_stem_fragments.tsv.gz: fragment file to go with the K562_stem object</p><p>K562_stem_fragments.tsv.gz.tbi: index file for the K562_stem object fragment file</p><p>retina.rds : object corresponding to the retinal organoid phospho-seq experiment</p><p>retina_fragments.tsv.gz: fragment file to go with the retina object</p><p>retina_fragments.tsv.gz.tbi: index file for the retina object fragment file</p><p>retina_multi.rds : object corresponding to the retinal organoid phospho-seq-multiome experiment</p><p>retina_multi_fragments.tsv.gz: fragment file to go with the retina_multi object</p><p>retina_multi_fragments.tsv.gz.tbi: index file for the retina_multi object fragment file</p><p>To use the K562, multiome, retina and retina_multiome datasets provided, please use these lines of code to import the object into Signac/Seurat and change the fragment file path to the corresponding downloaded fragment file:</p><p>obj <- readRDS("obj.rds") # remove fragment file information Fragments(obj) <- NULL # Update the path of the fragment file Fragments(obj) <- CreateFragmentObject(path = "download/obj_fragments.tsv.gz", cells = Cells(obj))</p><p>To use the K562 and multiome datasets provided, please use these lines of code to import the object into Signac/Seurat and change the fragment file path to the corresponding downloaded fragment file:</p><p>obj <- readRDS("obj.rds") # remove fragment file information Fragments(obj) <- NULL # Update the path of the fragment file Fragments(obj) <- CreateFragmentObject(path = "download/obj_fragments.tsv.gz", cells = Cells(obj))</p><p>To use the "fullobject" dataset provided, please use these lines of code to import the object into Signac/Seurat and change the fragment file path to the corresponding downloaded fragment file: </p><p>#load the stringr package library(stringr) #load the object obj <- readRDS("obj.rds") # remove fragment file information Fragments(obj) <- NULL #Remove unwanted residual information and rename cells obj@reductions$norm.adt.pca <- NULL obj@reductions$norm.pca <- NULL obj <- RenameCells(obj, new.names = str_remove(Cells(obj), "atac_")) # Update the path of the fragment file Fragments(obj) <- CreateFragmentObject(path = "download/obj_fragments.tsv.gz", cells = Cells(obj))</p><p> </p><p> </p>
Neural network ensembles and FEFF spectra for multi-modal small molecule chemical motif prediction
<p><strong>Data</strong></p> <ul> <li><strong>22-12-05-data</strong>: original molecular XANES data created from <a href="https://doi.org/10.1103/PhysRevResearch.5.013180">Ghose <em>et al.</em></a></li> <li><strong>23-04-26-ml-data</strong>: machine learning-ready data which is prepared in the format required by <a href="https://github.com/matthewcarbone/Crescendo">Crescendo</a>.</li> <li><strong>23-05-03-hp</strong>: hyper-parameter tuning results from 23-04-26-ml-data.</li> <li><strong>23-05-05-ensembles</strong>: ensemble results from 23-04-26-ml-data.</li> <li><strong>23-05-11-ml-data-CUTOFF8</strong>: a special machine learning-ready dataset constructed by a unique partitioning: only molecules with less than or equal to 8 atoms/molecule are used for training/validation, the rest are used for testing.</li> <li><strong>23-12-06_torch_models</strong>: torch only models which can be easily used independently of our ML helper repository, Crescendo. Instead, it can be used with a few lines of code found in multimodal_molecules/core.py, in our <a href="https://github.com/AI-multimodal/multimodal-molecules">GitHub respository</a>.</li> </ul> <p><strong>Funding</strong></p> <p>This research is based upon work supported by the U.S. Department of Energy, Office of Science, Office Basic Energy Sciences, under Award Number FWP PS-030. This research also used theory and computational resources of the Center for Functional Nanomaterials, which is a U.S. Department of Energy Office of Science User Facility, and the Scientific Data and Computing Center, a component of the Computational Science Initiative, at Brookhaven National Laboratory under Contract No. DE-SC0012704.</p>
Multi-modal User Interactions for Recommendations
<p>This repository contains the data for <a href="https://doi.org/10.1145/3626772.3657881">Dataset and Models for Item Recommendation Using Multi-Modal User Interactions</a>.</p> <p>We publish a real-world dataset from the insurance domain with multi-modal user interactions that can be used in recommendation models. The dataset is anonymized.</p> <p>There are 6 different datasets:</p> <div> <h3><strong>data_users.csv</strong></h3> </div> <p>This data contains the users. Each user has had one or more purchase events with conversations and/or web sessions prior to that purchase. The data contains 5 columns:</p> <ul> <li>user_id. The ID of a user.</li> <li>purchase_event_id. The ID of a purchase event.</li> <li>conversation_id. The ID of a conversation.</li> <li>session_id. The ID of a web session.</li> <li>event_number. A number specifying the order of conversations/web sessions.</li> </ul> <div> <h3><strong>data_conversations_keyword.csv</strong></h3> </div> <p>This data contains the conversations that the user had prior to the user's purchase event. Each conversation consists of multiple sentences represented with keywords. The data contains 4 columns:</p> <ul> <li>conversation_id. The ID of a conversation.</li> <li>sentence_number. A number specifying the order of sentences.</li> <li>sentence_speaker. The speaker of the sentence (user or agent).</li> <li>keywords. List with the IDs of the keywords in the sentence.</li> </ul> <div> <h3><strong>data_conversations_embedding.csv</strong></h3> </div> <p>The data contains the conversations that the user had prior to the user's purchase event. Each conversation consists of multiple sentences represented with text embeddings. The data contains 771 columns:</p> <ul> <li>conversation_id. The ID of a conversation.</li> <li>sentence_number. A number specifying the order of sentences.</li> <li>sentence_speaker. The speaker of the sentence (user or agent).</li> <li>embedding_1 - embedding_768. Text embeddings computed with a pre-trained language-specific BERT model.</li> </ul> <div> <h3><strong>data_sessions.csv</strong></h3> </div> <p>This data contains the web sessions that the user made prior to the user's purchase event. Each web session consists of multiple actions. The data contains 3 columns:</p> <ul> <li>session_id. The ID of a web session.</li> <li>action_number. A number specifying the order of actions.</li> <li>action_tags. List with the IDs of the section, object and type of an action.</li> </ul> <div> <h3><strong>data_purchase_events.csv</strong></h3> </div> <p>This data contains the purchase events. Each event consists of one or more item purchases made by the same user. The data contains 2 columns:</p> <ul> <li>purchase_event_id. The ID of a purchase event.</li> <li>item_id. The ID of an item.</li> </ul> <div> <h3><strong>data_post_filter.csv</strong></h3> </div> <p>This data contains the items that were possible for the user to buy at the time of the user's purchase event. The data contains 2 columns:</p> <ul> <li>purchase_event_id. The ID of a purchase event.</li> <li>item_id. The ID of an item.</li> </ul>
A Spatial Multi-Modal Dissection of Host-Microbiome Interactions within the Colitis Tissue Microenvironment
<p>All processed data used in the manuscript 'A Spatial Multi-Modal Dissection of Host-Microbiome Interactions within the Colitis Tissue Microenvironment'. For more detail please refer to the manuscript. If there is any question please contact Bokai Zhu via email: BZHU0@MGH.HARVARD.EDU.</p> <p>Data folder structure:</p> <p>Main data (eg. presented in main figures etc): Please see .zip file 'data_submission.zip'.</p> <p>Additional data (eg. during revision, or other misc files): Please see .zip file 'data_submission_part2.zip'.</p>
The Icy dataset - a multi-modalities dataset for icy surface reconstruction
<p>Three-dimensional (3D) reconstruction serves as a cornerstone in various robotic applications, playing critical roles in scene understanding and navigation. Traditionally, LiDAR has been instrumental in generating precise point clouds of the environment, providing essential data for these applications. However, the efficacy of LiDAR sensors is significantly hindered in challenging conditions, such as the presence of water or icy surfaces. The complex interplay between laser beams and icy or non-ideal surfaces can result in signal degradation, distortion, or even complete signal loss, adversely affecting the accuracy and reliability of the 3D reconstruction process. The reflective and refractive properties of ice, along with its variable surface conditions, present challenges that traditional LiDAR sensors struggle to address. This paper proposes a diverse dataset to facilitate a multimodal approach for detecting and reconstructing icy surfaces using various sensors. A preliminary study on our dataset demonstrates that, in addition to the geometrical surface obtained by registering consecutive scans from LiDAR, regions with ice can be identified by leveraging visual data to enhance understanding of the surface texture. The integration of distinct data sources can thus improve the robustness of reconstruction algorithms in diverse scenarios. </p>
Data and code availibility for the paper "Ice-nucleating agents in sea spray aerosol identified and quantified with a holistic multi-modal freezing model" by Alpert et al.
<p>The data and codes used in the paper "Ice-nucleating agents in sea spray aerosol identified and quantified with a holistic multi-modal freezing model" by Alpert et al., published in <em>Science Advances</em> are included in this collection. Detailed descriptions of the files are given in the readme file.</p>
Multi-Modal Population Dataset for Sub-Saharan Africa
<p>A multi-modal population dataset for three cities in Sub-Saharan Africa, namely Dakar (Senegal), Nairobi (Kenya) and Dar es Salaam (Tanzania). The dataset contains patches of rasterized building footprints (0.5 m), Sentinel-2 MSI data (10 m) and population labels at the patch level (100 m).</p>
Data Repository for the article: "Control of multi-modal scattering in a microwave frequency comb"
<p><strong>Abstract:</strong></p> <p>Control over the coupling between multiple modes of a frequency comb is an important step toward measurement-based quantum computation with a continuous-variable system.<br>We demonstrate the creation of square-ladder correlation graphs in a microwave comb with 95 modes.<br>The graphs are engineered through precise control of the relative phase of three pumps applied to a Josephson parametric oscillator. <br>Experimental measurement of the mode scattering matrix is in good agreement with theoretical predictions based on a linearized equation of motion of the parametric oscillator. <br>The digital methods used to create and measure the correlations are easily scaled to more modes and more pumps, with the potential to tailor a specific correlation graph topology.</p> <p> </p> <p>Article DOI:</p> <p>ArXiV: <a href="https://doi.org/10.48550/arXiv.2402.09068">arXiv.2402.09068</a></p> <p> </p> <p><strong>Content:</strong></p> <p>This repository contains the datased used in the article "Control of multi-modal scattering in a microwave frequency comb". Scripts to generate the figures are also included.</p>
VHAKG: Multi-modal Knowledge Graphs with Multi-view Videos of Daily Activities
<h2>Outline</h2> <ul> <li>This dataset is a multimodal knowledge graph (MMKG) of daily activity videos.</li> <li>This dataset integrates a KG with embedded multi-view videos created by <a href="https://github.com/aistairc/virtualhome_aist">VirtualHome-AIST</a>, an extended version of the VirtualHome simulator, and an event-centric KG generated by <a href="https://github.com/aistairc/virtualhome2kg">VirtualHome2KG</a>.</li> <li>We named this dataset <strong>VHAKG</strong> (VirtualHome-AIST-KG).</li> </ul> <h2>Details</h2> <ul> <li>VHAKG describes 2D bounding boxes of objects every five frames, compositional activities, primitive actions, target objects, object states, 3D bounding boxes, and their time-series changes.</li> <li>The videos are encoded in base64 and embedded as a literal value.</li> <li>VHAKG consists of 706 daily activity scenarios (e.g., clean desk, cook fried bread, and relax on sofa) and 3,530 videos captured by five synchronized cameras per scenario.</li> <li>The file format is <a href="https://www.w3.org/RDF/">RDF</a> (<a href="https://en.wikipedia.org/wiki/Turtle_(syntax)">Turtle</a>), which can be loaded into various <a href="https://en.wikipedia.org/wiki/Triplestore">Triplestores</a>.</li> <li>VHAKG's vocabularies are defined as an ontology and can be found in vh2kg_schema_v2.0.0.ttl.</li> </ul> <h2>Contents</h2> <ul> <li>vh2kg_video_base64.tar.gz <ul> <li><strong>{activity name}{scene}_{camera}_2dbbox.ttl</strong>: KG with video embedded in base64 format, including 2D bounding box data every 5 frames. <ul> <li>To learn more about {scene}, check <a href="https://github.com/xavierpuigf/virtualhome/tree/v2.2.0/simulation#environment">here</a>.</li> <li>To learn more about {camera}, check <a href="https://github.com/aistairc/virtualhome_unity_aist?tab=readme-ov-file#addition-of-new-four-cameras">here</a>.</li> </ul> </li> </ul> </li> <li>vh2kg_event.tar.gz <ul> <li><strong>{activity name}_{scene}.ttl</strong>: Event-centric KGs representing video content as sequences of events.</li> <li><strong>vh2kg_schema_v2.0.0.ttl</strong>: The ontology file of this dataset.</li> <li><strong>affordance.ttl</strong>: The affordance data of objects that were created by crowdsourcing. <ul> <li>Please see Section III.B of <a href="https://doi.org/10.1109/ACCESS.2023.3253807" target="_blank" rel="noopener">this paper</a> for more information.</li> </ul> </li> <li><strong>add_places.ttl</strong>: Events in which agents moved from one room to another.</li> </ul> </li> </ul> <h2>Tools</h2> <ul> <li><a href="https://github.com/aistairc/vhakg-tools">A set of tools</a> for searching and extracting videos from VHAKG is available.</li> </ul>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.