Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
135
datasets available to search
ShareScore release 0.9.0
Dataset results
135 results for “Multimodal dataset”
Dataset of adaptive Children-Robot Interaction for Education based on Autonomous Multimodal Users' Readings
<p># Dataset of adaptive Children-Robot Interaction for Education based on Autonomous Multimodal Users’ Readings </p> <p>## Background</p> <p>This dataset is generated from multiple interactions between a Social Robot (NAO) and 5th grade students from a private school in São Paulo, Brazil. </p> <p>In the interaction, the robot approached the content that teachers were approaching at the time with the participants students about the wasting system in Brazil.</p> <p>The measures here are the readings that the R-CASTLE system did for each answer the students gave to the questions the robot asked. </p> <p>For more information about how these measures were collected, please refer to this thesis at: https://doi.org/10.11606/T.55.2020.tde-31082020-093935</p> <p>Since the goal of the R-CASTLE is to provide autonomous adaptation, we built a ground-truth dataset based on human feedback of an expert in education operating the robot in loco. The person was teleoperating the robot to change its behaviour (or not) according to observed values of the participants as Face Gaze, Facial emotion displayed, Number of spoken words, the correctness of the answer (based on pre-defined answers), and the time students took to answer. These measures are the 5th columns of this csv file. The evaluator could decide to increase (1), maintain (0), or decrease (-1) the level of difficulties of the following questions depending on the mentioned observed measures. This is the human true label, stored in the 6th column. </p> <p>## Description:<br>Each row of this file is a tuple of the autonomous reading the robot made in the 5 first columns, plus the true label in the 6th row (True Value) and the Final Crisp Value using fuzzy classification in the 7th row (Final Crisp Value).</p> <p><br>Deviations (integer): number of face deviations of the participant during the question answering identified by the system.</p> <p>EmotionCount (integer): a balance between "good" and "bad" emotions (good - bad) identified by the system.</p> <p>NumberWord (integer): number of words comprised in the sentence the participant gave.</p> <p>SucRate/Ans/RWa: (between 0 and 1, where 0 is completely wrong and 1 is completely right): The success rate of the participant’s answer to that question, based on the expected answer programmed by their teachers.</p> <p>Time2ans (float): The time spent to answer the question since the robot has finished the question until the end of the participant’s speech in seconds.</p> <p>True Value (-1, 0, 1): Ground-truth value. Value of adaptation chosen by the human observing the interaction if the system needed to decrease, maintain, or increase the level of difficulty of asked questions. </p> <p>Final Crisp Value (float): value of calculated fuzzy output based on the implementations in the paper: https://doi.org/10.1145/3395035.3425201</p> <p><br>## Creators <br>Daniel Tozadore: dtozadore@gmail.com<br>Roseli Romero: rafrance@icmc.usp.br</p> <p><br>## License: <br>[Creative Commons Licenses](https://creativecommons.org/share-your-work/cclicenses/)</p>
DATASET FOR: A multimodal spectroscopic approach combining mid-infrared and near-infrared for discriminating Gram-positive and Gram-negative bacteria
<h4>Description:</h4> <p>This dataset comprises a comprehensive set of files designed for the analysis and 2D correlation of spectral data, specifically focusing on ATR and NIR spectra. It includes MATLAB scripts and supporting functions necessary to replicate the analysis, as well as the raw datasets used in the study. Below is a detailed description of the included files:</p> <ol> <li> <p><strong>Data Analysis</strong>:</p> <ul> <li><strong>File Name</strong>: <code>Data_Analysis.mlx</code></li> <li><strong>Description</strong>: This MATLAB Live Script file contains the main script used for the classification analysis of the spectral data. It includes steps for preprocessing, analysis, and visualization of the ATR and NIR spectra.</li> </ul> </li> <li> <p><strong>2D Correlation Data Analysis</strong>:</p> <ul> <li><strong>File Name</strong>: <code>Data_Analysis_2Dcorr.mlx</code></li> <li><strong>Description</strong>: This MATLAB Live Script file is similar to the primary analysis script but is specifically tailored for performing 2D correlation analysis on the spectral data. It includes detailed steps and code for executing the 2D correlation.</li> </ul> </li> <li> <p><strong>Functions</strong>:</p> <ul> <li><strong>Folder Name</strong>: <code>Functions</code></li> <li><strong>Description</strong>: This folder contains all the necessary MATLAB function files required to replicate the analyses presented in the scripts. These functions handle various preprocessing steps, calculations, and visualizations.</li> </ul> </li> <li> <p><strong>Datasets</strong>:</p> <ul> <li><strong>File Names</strong>: <code>ATR_dataset.xlsx</code>, <code>NIR_dataset.xlsx</code>, <code>Reference_data.csv</code></li> <li><strong>Description</strong>: These Excel files contain the raw spectral data for ATR and NIR analyses, as well as reference datasets. Each file includes multiple sheets with detailed measurements and metadata.</li> </ul> </li> </ol> <h4>Usage Notes:</h4> <ul> <li><strong>Software Requirements</strong>: <ul> <li>MATLAB is required to run the .mlx files and utilize the functions.</li> <li><strong>PLS_Toolbox</strong>: Necessary for certain preprocessing and analysis steps.</li> <li><strong>MIDAS 2010</strong>: Available at <a href="https://www.mathworks.com/matlabcentral/fileexchange/32384-midas-2010" target="_new" rel="noreferrer">MIDAS 2010</a>, required for the 2D correlation analysis.</li> </ul> </li> <li><strong>Replication</strong>: Users can replicate the analyses by running the <code>Data_Analysis.mlx</code> and <code>Data_Analysis_2Dcorr.mlx</code> scripts in MATLAB, ensuring that the <code>Functions</code> folder is in the MATLAB path.</li> <li><strong>Data Handling</strong>: The datasets are provided in .xlsx format, which can be easily imported into MATLAB or other data analysis software.</li> </ul>
Multimodal Sensor Dataset from Wood/Forest Environment - Adões
<p>This forestry robotic dataset consists of a <strong>forest scenario</strong> with a relatively restricted and blocked trajectory, <a href="https://www.google.pt/maps/place/40%C2%B017'04.0%22N+8%C2%B027'25.7%22W/@40.2845835,-8.4577709,166m/data=!3m1!1e3!4m4!3m3!8m2!3d40.2844305!4d-8.4571394?hl=pt-PT&entry=ttu&g_ep=EgoyMDI0MDkwOC4wIKXMDSoASAFQAw%3D%3D"> Adões, Coimbra, Portugal</a> (40º28'44.305''N; -8º45'71.394''W).<br>The dataset contains rosbags information with different rostopics of various sensors, with a total distance covered of approximately 620 meters.<br>The dataset was broken into rosbags with a maximum storage of 10GB each.</p> <p>The recorded topics were as follows:</p> <ul> <li>Velarray M1600 LiDAR point clouds data (sensor_msgs/PointCloud2) @20Hz;</li> <li>Realsense Intel D435i compressed image data, compressed depth info, alongside camera info (sensor_msgs/CompressedImage + sensor_msgs/CompressedImage + sensor_msgs/CameraInfo) @30Hz;</li> <li>Mynt Eye S1030 left camera data alongside camera info (sensor_msgs/Image + sensor_msgs/CameraInfo) @20Hz;</li> <li>Mynt Eye S1030 right camera data alongside camera info (sensor_msgs/Image + sensor_msgs/CameraInfo) @20Hz;</li> <li>IMU sensor data, alongside magnetometer data (sensor_msgs/Imu + sensor_msgs/MagneticField) @100Hz;</li> <li>RTK GNSS information on the trajectory taken (sensor_msgs/NavSatFix) @1Hz.</li> </ul>
Multimodal Sensor Dataset from Wood/Forest Environment - iParque
<p>This forestry robotic dataset comprises a <strong>woodland scenario</strong> with a relatively open sky view, <a href="https://www.google.pt/maps/place/40%C2%B010'35.3%22N+8%C2%B028'07.2%22W/@40.1765628,-8.468883,70m/data=!3m1!1e3!4m4!3m3!8m2!3d40.1764826!4d-8.4686631?hl=pt-PT&entry=ttu&g_ep=EgoyMDI0MDkwOC4wIKXMDSoASAFQAw%3D%3D">iParque Park, Coimbra, Portugal</a> ( 40º17'64.826''N; -8º46'86.631''W).<br>The dataset contains information on rosbags with different rostopics of various sensors, with a total distance covered of approximately 950 meters.<br>The dataset was broken into rosbags with a maximum storage of 10GB each.</p> <p>The recorded topics were as follows:</p> <ul> <li>Velarray M1600 LiDAR point clouds data (sensor_msgs/PointCloud2) @20Hz;</li> <li>Realsense Intel D435i compressed image data, compressed depth info, alongside camera info (sensor_msgs/CompressedImage + sensor_msgs/CompressedImage + sensor_msgs/CameraInfo) @30Hz;</li> <li>Mynt Eye S1030 left camera data alongside camera info (sensor_msgs/Image + sensor_msgs/CameraInfo) @20Hz;</li> <li>Mynt Eye S1030 right camera data alongside camera info (sensor_msgs/Image + sensor_msgs/CameraInfo) @20Hz;</li> <li>IMU sensor data, alongside magnetometer data (sensor_msgs/Imu + sensor_msgs/MagneticField) @100Hz;</li> <li>RTK GNSS information on the trajectory taken (sensor_msgs/NavSatFix) @1Hz.</li> </ul>
Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base
<p>Official Code and Dataset of Table Tennis Coaching System Based on a Multimodal Large Language Model with Knowledge Base</p>
Datasets for Evaluation of Multimodal Image Registration
<p><strong>Description</strong></p> <ul> <li><strong>Aerial data</strong></li> <li>The Aerial dataset is divided into 3 sub-groups by IDs: {7, 9, 20, 3, 15, 18}, {10, 1, 13, 4, 11, 6, 16}, {14, 8, 17, 5, 19, 12, 2}. Since the images vary in size, each image is subdivided into the maximal number of equal-sized non-overlapping regions such that each region can contain exactly one 300x300 px image patch. Then one 300x300 px image patch is extracted from the centre of each region. The particular 3-folded grouping followed by splitting leads to that each evaluation fold contains 72 test samples. <ul> <li> <p>Modality A: Near-Infrared (NIR)</p> </li> <li> <p>Modality B: three colour channels (in B-G-R order)</p> </li> </ul> </li> <li><strong>Cytological data</strong></li> <li>The Cytological data contains images from 3 different cell lines; all images from one cell line is treated as one fold in 3-folded cross-validation. Each image in the dataset is subdivided from 600x600 px into 2x2 patches of size 300x300 px, so that there are 420 test samples in each evaluation fold. <ul> <li> <p>Modality A: Fluorescence Images</p> </li> <li> <p>Modality B: Quantitative Phase Images (QPI)</p> </li> </ul> </li> <li><strong>Histological dataset</strong></li> <li>For the Histological data, to avoid too easy registration relying on the circular border of the TMA cores, the evaluation images are created by cutting 834x834 px patches from the centres of the original 134 TMA image pairs. <ul> <li> <p>Modality A: Second Harmonic Generation (SHG)</p> </li> <li> <p>Modality B: Bright-Field (BF)</p> </li> </ul> </li> </ul> <p>The evaluation set created from the above three publicly available 2D datasets consists of images undergone 4 levels of (rigid) transformations of increasing size of displacement. The level of transformations is determined by the size of the rotation angle θ and the displacement tx & ty, detailed in <a href="https://github.com/MIDA-group/MultiRegEval/tree/master/Datasets">this table</a>. Each image sample is transformed exactly once at each transformation level so that all levels have the same number of samples.</p> <ul> <li><strong>Radiological data</strong></li> <li>The Radiological dataset is divided into 3 sub-groups by patient IDs: {109, 106, 003, 006}, {108, 105, 007, 001}, {107, 102, 005, 009}. Since the Radiological dataset is non-isotropic (and also of varying resolution), it is resampled using B-spline interpolation to 1 mm<sup>3</sup> cubic voxels, taking explicit care to not resample twice; displaced volumes are transformed and resampled in one step. <ul> <li> <p>Modality A: T1-weighted MRI</p> </li> <li> <p>Modality B: T2-weighted MRI</p> </li> </ul> </li> </ul> <p>(Run <a href="https://github.com/MIDA-group/MultiRegEval/blob/master/utils/make_rire_patches.py"><code>make_rire_patches.py</code></a> to generate the sub-volumes.)</p> <p>Reference sub-volumes of size 210x210x70 voxels are cropped directly from centres of the (non-displaced) resampled volumes. Similarly as for the aforementioned 2D datasets, random (uniformly-distributed) transformations are composed of rotations θx, θy ∈ [-4, 4] degrees around the x- and y-axes, rotation θz ∈ [-20, 20] degrees around the z-axis, translations tx, ty ∈ [-19.6, 19.6] voxels in x and y directions and translation tz ∈ [-6.5, 6.5] voxels in z direction. 40 rigid transformations of increasing sizes of displacement are applied to each volume. Transformed sub-volumes, of size 210x210x70 voxels, are cropped from centres of the transformed and resampled volumes.</p> <p> </p> <p>In total, it contains 864 image pairs created from the aerial dataset, 5040 image pairs created from the cytological dataset, 536 image pairs created from the histological dataset, and metadata with scripts to create the 480 volume pairs from the radiological dataset. Each image pair consists of a reference patch <span class="math-tex">\(I^{\text{Ref}}\)</span> and its corresponding initial transformed patch <span class="math-tex">\(I^{\text{Init}}\)</span> in both modalities, along with the ground-truth transformation parameters to recover it.</p> <p>Scripts to calculate the registration performance and to plot the overall results can be found in <a href="https://github.com/MIDA-group/MultiRegEval">https://github.com/MIDA-group/MultiRegEval</a>, and instructions to generate more evaluation data with different settings can be found in <a href="https://github.com/MIDA-group/MultiRegEval/tree/master/Datasets#instructions-for-customising-evaluation-data">https://github.com/MIDA-group/MultiRegEval/tree/master/Datasets#instructions-for-customising-evaluation-data</a>.</p> <p> </p> <p><strong>Metadata</strong></p> <p>In the <code>*.zip</code> files, each row in <code>{Zurich,Balvan}_patches/fold[1-3]/patch_tlevel[1-4]/info_test.csv</code> or <code>Eliceiri_patches/patch_tlevel[1-4]/info_test.csv</code> provides the information of an image pair as follow:</p> <ul> <li> <p>Filename: identifier(ID) of the image pair</p> </li> <li> <p>X1_Ref: x-coordinate of the upper-left corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>Y1_Ref: y-coordinate of the upper-left corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>X2_Ref: x-coordinate of the lower-left corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>Y2_Ref: y-coordinate of the lower-left corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>X3_Ref: x-coordinate of the lower-right corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>Y3_Ref: y-coordinate of the lower-right corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>X4_Ref: x-coordinate of the upper-right corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>Y4_Ref: y-coordinate of the upper-right corner of reference patch I<sub>Ref</sub></p> </li> <li> <p>X1_Trans: x-coordinate of the upper-left corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>Y1_Trans: y-coordinate of the upper-left corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>X2_Trans: x-coordinate of the lower-left corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>Y2_Trans: y-coordinate of the lower-left corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>X3_Trans: x-coordinate of the lower-right corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>Y3_Trans: y-coordinate of the lower-right corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>X4_Trans: x-coordinate of the upper-right corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>Y4_Trans: y-coordinate of the upper-right corner of transformed patch I<sub>Init</sub></p> </li> <li> <p>Displacement: mean Euclidean distance between reference corner points and transformed corner points</p> </li> <li> <p>RelativeDisplacement: the ratio of displacement to the width/height of image patch</p> </li> <li> <p>Tx: randomly generated translation in the x-direction to synthesise the transformed patch I<sub>Init</sub></p> </li> <li> <p>Ty: randomly generated translation in the y-direction to synthesise the transformed patch I<sub>Init</sub></p> </li> <li> <p>AngleDegree: randomly generated rotation in degrees to synthesise the transformed patch I<sub>Init</sub></p> </li> <li> <p>AngleRad: randomly generated rotation in radian to synthesise the transformed patch I<sub>Init</sub></p> </li> </ul> <p>In addition, each row in <code>RIRE_patches/fold[1-3]/patch_tlevel[1-4]/info_test.csv</code> has following columns:</p> <ul> <li>Z1_Ref: z-coordinate of the upper-left corner of reference patch I<sub>Ref</sub></li> <li>Z2_Ref: z-coordinate of the lower-left corner of reference patch I<sub>Ref</sub></li> <li>Z3_Ref: z-coordinate of the lower-right corner of reference patch I<sub>Ref</sub></li> <li>Z4_Ref: z-coordinate of the upper-right corner of reference patch I<sub>Ref</sub></li> <li>Z1_Trans: z-coordinate of the upper-left corner of transformed patch I<sub>Init</sub></li> <li>Z2_Trans: z-coordinate of the lower-left corner of transformed patch I<sub>Init</sub></li> <li>Z3_Trans: z-coordinate of the lower-right corner of transformed patch I<sub>Init</sub></li> <li>Z4_Trans: z-coordinate of the upper-right corner of transformed patch I<sub>Init</sub></li> <li>(...and similarly, coordinates of the 5th-8th corners)</li> <li>Tz: randomly generated translation in z-direction to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleDegreeX: randomly generated rotation around X-axis in degrees to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleRadX: randomly generated rotation around X-axis in radian to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleDegreeY: randomly generated rotation around Y-axis in degrees to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleRadY: randomly generated rotation around Y-axis in radian to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleDegreeZ: randomly generated rotation around Z-axis in degrees to synthesise the transformed patch I<sub>Init</sub></li> <li>AngleRadZ: randomly generated rotation around Z-axis in radian to synthesise the transformed patch I<sub>Init</sub></li> </ul> <p> </p> <p><strong>Naming convention</strong></p> <ul> <li><strong>Aerial Data</strong> <ul> <li> <pre> zh{ID}_{iRow}_{iCol}_{ReferenceOrTransformed}.png</pre> </li> <li>Example: <code>zh5_03_02_R.png</code> indicates the <em>Reference </em>patch of the <em>3rd row</em> and <em>2nd column</em> cut from the image with ID <code>zh5</code>.</li> </ul> </li> <li><strong>Cytological data</strong> <ul> <li> <pre> {{cellline}_{treatment}_{fieldofview}_{iFrame}}_{iRow}_{iCol}_{ReferenceOrTransformed}.png</pre> </li> <li>Example: <code>PNT1A_do_1_f15_02_01_T.png</code> indicates the <em>Transformed </em>patch of the <em>2nd row</em> and <em>1st column</em> cut from the image with ID <code>PNT1A_do_1_f15</code>.</li> </ul> </li> <li><strong>Histological data</strong> <ul> <li> <pre> {ID}_{ReferenceOrTransformed}.tif</pre> </li> <li>Example: <code>1B_A4_T.tif</code> indicates the <em>Transformed </em>patch cut from the image with ID <code>1B_A4</code>.</li> </ul> </li> </ul> <ul> <li><strong>Radiological Data</strong> <ul> <li> <pre> patient_{ID}_{iTransform}_T.mhd</pre> </li> <li> <pre> patient_{ID}_R.mhd </pre> </li> <li>Example: <code>patient_003_8_T.mhd</code> indicates the sub-volume <em>Transformed </em>with the <em>8th random transformation</em> cut from the volume with patient ID <code>003</code>; <code>patient_003_R.mhd</code> indicates the <em>Reference </em>sub-volume the volume with patient ID <code>003</code>.</li> </ul> </li> </ul> <p> </p> <p>This dataset was originally produced by the authors of <em><a href="https://arxiv.org/abs/2103.16262">Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study</a></em>.</p>
Multimodal faces dataset for PracticalMEEG handson tutorials
<p>The dataset analyzed in PracticalMEEG tutorials is part of a dataset recorded by Rik Hanson and colleagues. During the hands-on sessions we will mainly work with the MEG data of a single representative subject, for the group statistics we will work with source-level processed data from all subjects.</p> <p>The “mmfaces” dataset contains EEG, MEG, functional MRI and structural MRI data from research participants that were recorded in multiple runs of a simple task performed on a large number of Famous, Unfamiliar and Scrambled faces. It is described in more detail the data descriptor publication <a href="https://www.nature.com/articles/sdata20151">doi:10.1038/sdata.2015.1</a> and analyzed in detail in <a href="http://journal.frontiersin.org/Journal/10.3389/fnhum.2011.00076/abstract">doi:10.3389/fnhum.2011.00076</a>.</p> <p>The original multimodal dataset included simultaneous MEG/EEG recordings on 19 healthy subjects. In the original study, three subjects (sub001, sub005, sub016) were excluded from further analysis.</p> <p>The dataset used to be available from the MRC-CBU FTP server, but is nowadays maintained on <a href="https://openneuro.org/datasets/ds000117">OpenNeuro</a>.</p> <p>Stimulation details</p> <ul> <li>The start of a trial was indicated with a fixation cross of random duration between 400 to 600 ms</li> <li>The face stimuli was superimposed on the fixation cross for a random duration of 800 to 1,000 ms</li> <li>Inter-stimulus interval of 1,700 ms comprised a central white circle</li> <li>Two types of stimulation patterns: <ul> <li>Immediate: The image was presented consecutively</li> <li>Long: The two images were presented with 5-15 intervening stimuli</li> </ul> </li> <li>For the purposes of our analysis, we treat these two stimulation patterns of stimuli together</li> <li>To maintain attention, subjects were asked to judge the symmetry of the image and respond with a keypress</li> </ul> <p>MEG/EEG acquisition details</p> <p>The MEG data consist of 102 magnetometers and 204 planar gradiometers from a Neuromag/Elekta/Megin VectorView system. The same system was used to simultaneously record EEG data from 70 electrodes (using a nose reference), which are stored in the same “FIF” format file. The above FTP site includes a raw .fif file for each run/subject, but also a second .fif file in which the MEG data have been “cleaned” using Signal-Space Separation as implemented in MaxFilter 2.1.</p> <p>A Polhemus was used to digitize three fiducial points and a large number of other points across the scalp, which can be used to co-register the M/EEG data with the structural MRI image. Six runs of approximately 10 minutes each were acquired for each subject, while they judged the left-right symmetry of each stimulus (face or scrambled), leading to nearly 300 trials in total for each of the 3 conditions.</p> <ul> <li>Sampling frequency: 1100 Hz</li> <li>Stimulation triggers: The trigger channel is STI101 with the following event codes:</li> <li>Famous faces: 5 (first), 6 (immediate), and 7 (long)</li> <li>Unfamiliar faces: 13 (first), 14 (immediate), and 15 (long)</li> <li>Scrambled faces: 17 (first), 18 (immediate), and 19 (long)</li> <li>Sensors <ul> <li>102 magnetometers</li> <li>204 planar gradiometers</li> <li>70 electrodes recorded with a nose reference (Easycap conforming to extended 10-20% system)</li> <li>Two sets of bipolar electrodes were used to measure vertical (left eye; EEG062) and horizontal Electro-oculograms (EEG061). Another set was used to measure ECG (EEG063)</li> </ul> </li> <li>A fixed 34 ms delay exists between the appearance of a trigger in the trigger channel STI101 and the appearance of the stimulus on the screen</li> </ul> <p>MRI acquisition details</p> <p>The MRI data were acquired on a 3T Siemens TIM Trio, and include a 1x1x1mm T1-weighted structural MRI (sMRI) as well as a large number of 3x3x4mm T2*-weighted functional MRI (fMRI) EPI volumes acquired during 9 runs of the same task (performed by same subjects with different set of stimuli on a separate visit). (The FTP site also contains DTI and ME-FLASH MRI images from the same subject, which could be used for improved head modeling for example, but these are not used here.) For full description of the data and paradigm, see README.txt on the FTP site or <a href="http://journal.frontiersin.org/Journal/10.3389/fnhum.2011.00076/abstract">Wakeman & Henson</a>.</p>
Simple Multimodal Algorithmic Reasoning Task Dataset (SMART-101)
<p><strong>Introduction</strong></p> <p>Recent times have witnessed an increasing number of applications of deep neural networks towards solving tasks that require superior cognitive abilities, e.g., playing Go, generating art, ChatGPT, etc. Such a dramatic progress raises the question: how generalizable are neural networks in solving problems that demand broad skills? To answer this question, we propose SMART: a Simple Multimodal Algorithmic Reasoning Task (and the associated SMART-101 dataset) for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed specifically for children of younger age (6--8). Our dataset consists of 101 unique puzzles; each puzzle comprises a picture and a question, and their solution needs a mix of several elementary skills, including pattern recognition, algebra, and spatial reasoning, among others. To train deep neural networks, we programmatically augment each puzzle to 2,000 new instances; each instance varied in appearance, associated natural language question, and its solution. To foster research and make progress in the quest for artificial general intelligence, we are publicly releasing our SMART-101 dataset, consisting of the full set of programmatically-generated instances of 101 puzzles and their solutions.</p> <p>The dataset was introduced in our paper <a href="https://arxiv.org/pdf/2212.09993.pdf">Are Deep Neural Networks SMARTer than Second Graders?</a> by Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Kevin A. Smith, and Joshua B. Tenenbaum, CVPR 2023</p> <p>Files in the unzipped folder:</p> <ol> <li>./README.md: This Markdown file</li> <li>./SMART101-Data: Folder containing all the puzzle data. See below for details.</li> <li>./puzzle_type_info.csv: Puzzle categorization (into 8 skill classes).</li> </ol> <p><strong>Dataset Organization</strong></p> <p>The dataset consists of `101` folders (numbered from 1-101); each folder corresponds to one distinct puzzle (root puzzle). There are 2000 puzzle instances programmatically created for each root puzzle, numbered from 1-2000. Every root puzzle index (in [1,101]) folder contains: (i) `img/` and (ii) `puzzle_<index>.csv`. The folder `img/` is the location where the puzzle instance images are stored, and `puzzle_<index>.csv` the non-image part of a puzzle. Specifically, a row of `puzzle_<index>.csv` is the following tuple: `<id, Question, image, A, B, C, D, E, Answer>`, where `id` is the puzzle instance id (in [1,2000]), `Question` is the puzzle question associated with the instance, `image` is the name of the image (in `img/` folder) corresponding to this instance `id`, `A, B, C, D, E` are the five answer candidates, and `Answer` is the answer to the question. </p> <p><strong>At a Glance</strong></p> <ul> <li>The size of the unzipped dataset is ~12GB. </li> <li>The dataset consists of `101` folders (numbered from 1-101); each folder corresponds to one distinct puzzle (root puzzle). </li> <li>There are 2000 puzzle instances programmatically created for each root puzzle, numbered from 1-2000. </li> <li>Every root puzzle index (in [1,101]) folder contains: (i) `img/` and (ii) `puzzle_<index>.csv`. </li> <li>The folder `img/` is the location where the puzzle instance images are stored, and `puzzle_<index>.csv` contains the non-image part of a puzzle. Specifically, a row of `puzzle_<index>.csv` is the following tuple: `<id, Question, image, A, B, C, D, E, Answer>`, where `id` is the puzzle instance id (in [1,2000]), `Question` is the puzzle question associated with the instance, `image` is the name of the image (in `img/` folder) corresponding to this instance `id`, `A, B, C, D, E` are the five answer candidates, and `Answer` is the correct answer to the question. </li> </ul> <p><strong>Other Details</strong><br> In our paper <a href="https://arxiv.org/pdf/2212.09993.pdf">Are Deep Neural Networks SMARTer than Second Graders?</a>, we provide four different dataset splits for evaluation: (i) Instance Split (IS), (ii) Answer Split (AS), (iii) Puzzle Split (PS), and (iv) Few-shot Split (FS). Below, we provide the details of each split to make fair comparisons to the results reported in our paper. </p> <p><em>Puzzle Split (PS)</em><br> We use the following root puzzle ids as the `Train` and `Test` sets. </p> <table> <thead> <tr> <th scope="col">Split</th> <th scope="col">Root Puzzle Id Sets</th> </tr> </thead> <tbody> <tr> <td>`Test`</td> <td>{ 94,95, 96, 97, 98, 99, 101, 61,62, 65, 66,67, 69, 70, 71,72,73,74,75,76,77}</td> </tr> <tr> <td>`Train`</td> <td>{1,2,...,101} \ Test</td> </tr> </tbody> </table> <p>Evaluation is done on all the `Test` puzzles and their accuracies averaged. For the 'Test' puzzles, we use the instance indices 1701-2000 in the evaluation.</p> <p><em>Few-shot Split (FS)</em></p> <p>We randomly select `k` number of instances from the `Test` sets (that are used in the PS split above) for training in FS split (e.g., `k=100`). These `k` few-shot samples are taken from instance indices 1-1600 of the respective puzzles and evaluation is conducted on all instance ids from 1701-2000.</p> <p><em>Instance Split (IS)</em></p> <p>We split the instances under every root puzzle as: Train = 1-1600, Val = 1601-1700, Test = 1701-2000. We train the neural network models using the `Train` split puzzle instances from all the root puzzles together and evaluate on the `Test` split of all puzzles.</p> <p><em>Answer Split (AS)</em></p> <p>We find the median answer value among all the 2000 instances for every root puzzle and only use this set of the respective instances (with the median answer value) as the `Test` set for evaluation (this set is excluded from the training of the neural networks).</p> <p><em>Puzzle Categorization</em></p> <p>Please see puzzle_type_info.csv for details on the categorization of the puzzles into eight classes, namely (i) counting, (ii) logic, (iii) measure, (iv) spatial, (v) arithmetic, (vi) algebra, (vii) pattern finding, and (viii) path tracing.</p> <p><strong>Other Resources</strong></p> <p>PyTorch code for using the dataset to train deep neural networks is available <a href="https://www.merl.com/publications/TR2023-014">here</a>.</p> <p><strong>Contact</strong><br> Anoop Cherian (cherian@merl.com), Kuan-Chuan Peng (kpeng@merl.com), or Suhas Lohit (slohit@merl.com)</p> <p><br> <strong>Citation</strong><br> If you use the SMART-101 dataset in your research, please cite our paper:</p> <pre><code>@article{cherian2022deep, title={Are Deep Neural Networks SMARTer than Second Graders?}, author={Cherian, Anoop and Peng, Kuan-Chuan and Lohit, Suhas and Smith, Kevin and Tenenbaum, Joshua B}, journal={arXiv preprint arXiv:2212.09993}, year={2022} }</code></pre> <p><strong>Copyright and Licenses</strong></p> <p>The SMART-101 dataset is released under `CC-BY-SA-4.0`.</p> <pre><code>Created by Mitsubishi Electric Research Laboratories (MERL), 2022-2023 SPDX-License-Identifier: CC-BY-SA-4.0 </code></pre> <p> </p>
MDCC: A Multimodal Dynamic Dataset for Donation-based Crowdfunding Campaigns
<p>This is the repository of MDCC: A Multimodal Dynamic Dataset for Donation-based<br> Crowdfunding Campaigns. For details, please refer: https://github.com/Jiayang-L1/mdcc</p> <p><br> V2: We added json and csv files for easier use and renamed it with standard name. Specifically, the "comment_cor_time" column in new files means the corresponding donation time of each comment.</p>
Plains zebra 2019 multimodal communication dataset
Open the record for dataset details and reuse information.
An fNIRS dataset for multimodal speech comprehension in normal hearing individuals and cochlear implant users
Open the record for dataset details and reuse information.
Multimodal Biomedical Dataset for Evaluating Registration Methods (patches from TMA Cores)
<p>The dataset consists of 206 aligned Bright-Field (BF) and Second-Harmonic Generation (SHG) images. All images are of the same size of 834x834 pixels. The training set contains 40 image pairs. A validation set 1 of 25 image pairs for tuning the hyperparameters for the training. A validation set 2 of 7 image pairs for tuning the registration method. The test set for evaluation: 134 image pairs (times two).</p> <p>For each image pair in the test set, a random rotation up to +/-30 degrees and random translations in x and y for up to 100px were applied. For each image pair that was transformed for the test set, we apply the transformation to both modalities and keep their corresponding reference images as well as transformed images, in order to allow for registration using each modality as a reference and the other as a floating image. The metadata.csv file includes the coordinates of the four corners of the reference image as well as the coordinates of the corners of the transformed image after displacement. The origin of this coordinate system is the upper left corner of the reference image, given by (0,0).</p> <p>Example of naming convention in the test set:</p> <ul> <li>R_1B_F8_BF.tif: reference image in Bright-Field</li> <li>R_1B_F8_SHG.tif: reference image in Second-Harmonic Generation</li> <li>T_1B_F8_BF.tif: transformed image in Bright-Field</li> <li>T_1B_F8_SHG.tif: transformed image in Second-Harmonic Generation.</li> </ul> <p>Where the transformations applied to T_1B_F8_BF.tif and T_1B_F8_SHG.tif are identical and hence the coordinates of the corners in metadata.csv are valid for both image pairs.</p> <p>The metadata.csv file contains the following columns:</p> <ul> <li>Filename: identifier for image pair</li> <li>X1_Ref: x-coordinate of upper left corner of reference image</li> <li>Y1_Ref: y-coordinate of upper left corner of reference image</li> <li>X2_Ref: x-coordinate of lower left corner of reference image</li> <li>Y2_Ref: y-coordinate of lower left corner of reference image</li> <li>X3_Ref: x-coordinate of upper right corner of reference image</li> <li>Y3_Ref: y-coordinate of upper right corner of reference image</li> <li>X4_Ref: x-coordinate of lower right corner of reference image</li> <li>Y4_Ref: y-coordinate of lower right corner of reference image</li> <li>X1_Trans: x-coordinate of upper left corner of transformed image</li> <li>Y1_Trans: y-coordinate of upper left corner of transformed image</li> <li>X2_Trans: x-coordinate of lower left corner of transformed image</li> <li>Y2_Trans: y-coordinate of lower left corner of transformed image</li> <li>X3_Trans: x-coordinate of upper right corner of transformed image</li> <li>Y3_Trans: y-coordinate of upper right corner of transformed image</li> <li>X4_Trans: x-coordinate of lower right corner of transformed image</li> <li>Y4_Trans: y-coordinate of lower right corner of transformed image</li> <li>Displacement: Mean Euclidean distance between reference corner points and transformed corner points</li> </ul> <p>The data set was originally produced by the authors of <em>Aligned Collagen Is a Prognostic Signature for Survival in Human Breast Carcinoma</em> (<a href="https://www.sciencedirect.com/science/article/pii/S0002944010002336">https://www.sciencedirect.com/science/article/pii/S0002944010002336</a>). The registered and non-registered sub-image pairs in this data set were created by Johan Öfverstedt and Elisabeth Wetzer.</p>
AUTH - Multimodal facial expressions dataset
<p>This is a dataset suitable for active and static single modality and multimodal facial expression recognition methods as well as other active perception tasks.. It contains (i) sequences of facial 3D models, (ii) Webots simulation environments and (ii) videos, along with synchronized audio (speech), captured from a grid of virtual cameras at 2 lighting levels. Each part of the dataset can be utilized in a standalone manner, i.e., it can be downloaded and utilized separately from the rest.</p>
SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation
<p>This dataset contains multimodal images captured in a real waste processing facility specialized in plastics, cartons, and cans. The data was collected with a true-to-life prototype of the conveyor belt installed on the waste separation line, closely mimicking the actual installation. The setup has two synchronized cameras: a line-scan RGB camera (Teledyne DALSA Linea) and line-scan hyperspectral sensor (Specim FX17) that captures 224 contiguous spectral bands in a range from 900 to 1700 nm.</p> <p>The images are annotated for semantic segmentation with categories based on the requirements of the facility. Each one represents elements that commonly cause operational problems in recycling lines and impact the efficiency of the sorting process. Among these problems, machinery jams pose significant issues because they can cause complete stoppages in the process until the obstructing object is removed. The categories include: <em>film</em> and <em>basket</em>, large objects that can clog the conveyor belts as they are not easily breakable; <em>video tape </em>and <em>filament</em>, representing long objects prone to entangling with mechanical parts and requiring manual intervention; <em>trash bag</em>, which encompasses closed bags containing waste that need to be mechanically opened for further processing; and <em>cardboard</em>, paper objects whose recovery adds value by sending them to another recycling process.</p> <p>Our project page can be found at <a href="https://sites.google.com/unizar.es/spectralwaste">https://sites.google.com/unizar.es/spectralwaste</a>.</p>
M-Arg: Multimodal Multimodal Argument Mining Dataset
<p>No description provided.</p>
Dataset - Low-hysteresis shape-memory ceramics designed by multimode modelling
<p>Accompanying Dataset for the article "Low-hysteresis shape-memory ceramics designed by multimode modelling". The dataset contains two .csv files, one each for tetragonal and monoclinic phase, containing lattice parameter data used to train the ML models. Lattice parameters are in angstroms, compositions are in mole percent (cation site), temperatures are in degrees Celsius.</p>
Augmented Brain Tumor Multimodal Dataset Based on Reconstruction Consistency Loss
<p>We solved the problem that the brain MRI data only has the T2 modality and the other three modalities are missing. The data synthesis of the missing modal I data was completed, and the missing modal data of Flair, T1, and T1ce were completed, and 10,685 pieces of data were generated for each group of modalities.</p>
Pipaset preview: A multimodal dataset for AMT and EA tasks dedicated to Chinese music instrument Pipa
<p>Yuancheng Wang, Yuyang Jing , Wei Wei, Dorian Cazau, Olivier Adam, Qiao Wang</p> <p>Accompanying <a href="http://github.com/yuanchengwang/TEAS">Website</a> here.</p> <p>If you make use of PipaSet for academic purposes, please cite the following publication:</p> <blockquote> <p>PipaSet and TEAS: A Multimodal Dataset and Annotation Platform for Automatic Music Transcription and Expressive Analysis dedicated to Chinese Traditional Plucked String Instrument Pipa. IEEE ACCESS 2022.</p> </blockquote> <p>This project was led by Yuancheng Wang at Information School of Information Science and Engineering, Southeast University, China, along with my supervisor Prof. Qiao Wang from same school and Dr. Yuyang Jing from Nanjing University of the Arts, Wei Wei form Xiaozhuang University, Dr. Dorian Cazau from Institute of Mines-Télécom Atlantique in Brest France, Prof. Olivier Adam from Sorbonne University.</p> <p>We present PipaSet, a dataset that provides multimodal pipa recordings alongside a high diversity of annotations for Automatic Music Transcription and Expressive Analysis tasks, including note, pitch contours, string and fret positions, and playing techniques. </p> <p>More information will be coming soon to cover more pieces of music played by pipa. </p>
Multimodal Fish Feeding Intensity Assessment dataset part 2
<p>Multimodal Fish Feeding Intensity Assessment in Aquaculture (The paper you can find on Arxiv: <a href="https://arxiv.org/pdf/2309.05058.pdf" rel="nofollow">https://arxiv.org/pdf/2309.05058.pdf</a>) </p>
Dataset and scripts for "Multimode characterization of an optical beam deflection setup"
<p>The main files are the following :</p> <p><strong>Multimode_characterization_of_an_optical_beam_deflection_setup.pdf </strong>is the preprint of the article.</p> <p><strong>flexion_analysis.m</strong> is a Matlab script which analyses the amplitudes and frequencies of the contrasts corresponding to deflexion modes of the cantilever and stored in flexion_data.mat to compute the OBD spot position and size, and plot Figs. 3a, 3b, and 5b of the article.<br><strong>torsion_analysis.m</strong> is a Matlab script which analyses the amplitudes and frequencies of the contrasts corresponding to torsion modes of the cantilever and stored in flexion_data.mat to compute the OBD spot position and size, and plot Figs. 3c, 3d, and 5c of the article.<br><strong>ModeNumberStudy.m</strong> is a Matlab script which creates Fig. 4 of the article.</p> <p>All other matlab scripts and mat files are dependecies, their use is commented in the main scripts.</p> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.