Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
13
datasets available to search
ShareScore release 0.9.0
Dataset results
13 results for “Sensor fusion”
Aircraft Marshaling Signals Dataset of FMCW Radar and Event-Based Camera for Sensor Fusion
<p><strong>Dataset Introduction</strong></p><p>The advent of neural networks capable of learning salient features from variance in the radar data has expanded the breadth of radar applications, often as an alternative sensor or a complementary modality to camera vision. Gesture recognition for command control is arguably the most commonly explored application. Nevertheless, more suitable benchmarking datasets than currently available are needed to assess and compare the merits of the different proposed solutions and explore a broader range of scenarios than simple hand-gesturing a few centimeters away from a radar transmitter/receiver. Most current publicly available radar datasets used in gesture recognition provide limited diversity, do not provide access to raw ADC data, and are not significantly challenging. To address these shortcomings, we created and make available a new dataset that combines FMCW radar and dynamic vision camera of 10 aircraft marshalling signals (whole body) at several distances and angles from the sensors, recorded from 13 people. The two modalities are hardware synchronized using the radar's PRI signal. Moreover, in the supporting publication we propose a sparse encoding of the time domain (ADC) signals that achieve a dramatic data rate reduction (>76%) while retaining the efficacy of the downstream FFT processing (<2% accuracy loss on recognition tasks), and can be used to create an sparse event-based representation of the radar data. In this way the dataset can be used as a two-modality neuromorphic dataset.</p><p><strong>Synchronization of the two modalities</strong></p><p>The PRI pulses from the radar have been hard-wired to the event stream of the DVS sensor, and timestamped using the DVS clock. Based on this signal the DVS event stream has been segmented such that groups of events (time-bins) of the DVS are mapped with individual radar pulses (chirps).</p><p><strong>Data storage</strong></p><p>DVS events (x,y coords and timestamps) are stored in structured arrays, and one such structured array object is associated with the data of a radar transmission (pulse/chirp). A radar transmission is a vector of 512 ADC levels that correspond to sampling points of chirping signal (FMCW radar) that lasts about ~1.3ms. Every 192 radar transmissions are stacked in a matrix called a radar frame (each transmission is a row in that matrix). A data capture (recording) consisting of some thousands of continuous radar transmissions is therefore segmented in a number of radar frames. Finally radar frames and the corresponding DVS structured arrays are stored in separate containers in a custom-made multi-container file format (extension .rad). We provide a (rad file) parser for extracting the data out of these files. There is one file per capture of continuous gesture recording of about 10s.</p><p>Note the number of 192 transmissions per radar frame is an ad-hoc segmentation that suits the purpose of obtaining sufficient signal resolution in a 2D FFT typical in radar signal processing, for the range resolution of the specific radar. It also served the purpose of fast streaming storing of the data during capture. For extracting individual data points for the dataset however, one can pool together (concat) all the radar frames from a single capture file and re-segment them according to liking. The data loader that we provide offers this, with a default of re-segmenting every 769 transmissions (about 1s of gesturing).</p><p><strong>Data captures directory organization (</strong><a href="https://zenodo.org/api/records/10359770/draft/files/radar8Ghz-DVS-marshaling_signals_20220901_publication_anonymized.7z/content">radar8Ghz-DVS-marshaling_signals_20220901_publication_anonymized.7z</a><strong>)</strong></p><p>The dataset captures (recordings) are organized in a common directory structure which encompasses additional metadata information about the captures.</p><p>dataset_dir/<stage>/<room>/<person>-<gesture>-<distance>/ofxRadar8Ghz_yyyy-mm-dd_HH-MM-SS.rad</p><p>Identifiers</p><ul><li>stage [train, test].</li><li>room: [conference_room, foyer, open_space].</li><li>subject: [0-9]. Note that 0 stands for no person, and 1 for an unlabeled, random person (only present in test).</li><li>gesture: ['none', 'emergency_stop', 'move_ahead', 'move_back_v1', 'move_back_v2', 'slow_down' 'start_engines', 'stop_engines', 'straight_ahead', 'turn_left', 'turn_right'].</li><li>distance: ['xxx', '100', '150', '200', '250', '300', '350', '400', '450'] (in cm). Note that xxx is used for none gestures when there is no person present in front of the radar (i.e. background samples), or when a person is walking in front of the radar with varying distances but performing no gesture.</li></ul><p>The test data captures contain both subjects that appear in the train data as well as previously <i>unseen</i> subjects. Similarly the test data contain captures from the spaces that train data were recorded at, as well as from a new <i>unseen</i> open space.</p><p><strong>Files List</strong></p><p><a href="https://zenodo.org/api/records/10359770/draft/files/radar8Ghz-DVS-marshaling_signals_20220901_publication_anonymized.7z/content">radar8Ghz-DVS-marshaling_signals_20220901_publication_anonymized.7z</a></p><p>This is the actual archive bundle with the data captures (recordings).</p><p><a href="https://zenodo.org/api/records/10359770/draft/files/rad_file_parser_2.py/content">rad_file_parser_2.py</a></p><p>Parser for individual .rad files, which contain capture data.</p><p><a href="https://zenodo.org/api/records/10359770/draft/files/loader.py/content">loader.py</a></p><p>A convenience PyTorch Dataset loader (partly Tonic compatible). You practically only need this to quick-start if you don't want to delve too much into code reading. When you init a DvsRadarAircraftMarshallingSignals class object it automatically downloads the dataset archive and the .rad file parser, unpacks the archive, and imports the .rad parser to load the data. One can then <i>request from it </i>a training set, a validation set and a test set as torch.Datasets to work with<i>.</i> </p><p><a href="https://zenodo.org/api/records/10359770/draft/files/aircraft_marshalling_signals_howto.ipynb/content">aircraft_marshalling_signals_howto.ipynb</a></p><p>Jupyter notebook for exemplary basic use of loader.py</p><p><strong>Contact</strong></p><p>For further information or questions try contacting first M. Sifalakis or F. Corradi.</p><p> </p>
Applying Sensor Fusion to Augment Hyperspectral Data with Depth Information
<p>The research data for the paper "Applying Sensor Fusion to Augment Hyperspectral Data with Depth Information"<br> <br> Data in the archive "hyperdepth.tar.gz" includes:</p> <p><br> <strong>calibration_images/</strong><br> includes preprocessed images for calibrating both cameras</p> <p><strong>pointclouds/</strong><br> Includes individual hyperspectral point clouds for each view (front, rightmost, right, leftmost, left with postfixes correspondingly: edesta, oikea, oikea2, vasen, vasen2)<br> <br> <strong>raw_images/</strong><br> Two directories "day5" and "day6" which include the raw hyperspectral images and kinect images<br> <br> Some extra images are included which were not used in the research paper.</p> <p> </p> <p><strong>2022-03-11_112336_stereocalibration.json</strong> includes calibration results (mainly the intrinsic camera matrix and extrinsic parameters) for the setup.</p>
[Data] Qualify-As-You-Go: Sensor Fusion of Optical and Acoustic Signatures with Contrastive Deep Learning for Multi-Material Composition Monitoring in Laser Powder Bed Fusion Process
<p><br>Growing demand for multi-material Laser Powder Bed Fusion (LPBF) faces process control and quality monitoring challenges, particularly in ensuring precise material composition. This study explores optical and acoustic emission signals during LPBF processes with multiple materials, addressing challenges in process control and ensuring accurate material composition. Experimental data from processing five powder compositions were collected using a custombuilt monitoring system in a commercial LPBF machine. The research categorised signals from LPBF processing various compositions, enhancing prediction accuracy by combining optical with acoustic data and training convolutional neural networks using contrastive learning. Latent spaces of trained models using two contrastive loss functions, clustered acoustic and optical<br>emissions based on similarities, aligning with five compositions. Contrastive learning and sensor fusion were found to be essential for monitoring LPBF processes involving multiple materials. This research advances the understanding of multi-material LPBF, highlighting sensor fusion strategies’ potential for improving quality control in additive manufacturing. Data set for this work is hosted here</p>
A Kalman Filter Approach to the Fusion of Acceleration, GNSS position and Rotation Sensor Data from Robot Motions
<p><strong>GNSS data:</strong></p> <ul> <li>Instrument: Javad antenna and Septentrio receiver</li> <li>sampling rate: 100 Hz</li> <li>Bandwidth of loop filter: auto adjust</li> <li>Relative positioning </li> <li>Baseline: ultra short with distance of 5 m</li> <li>files in Rinex format: Rover (moving antenna) and Base (stationary antenna), .20G (GLONASS Navigation data), .20N (GPS Navigation data), .20L (Galileo Navigation data), .20O (Observations)</li> </ul> <p><strong>Accelerometer data:</strong></p> <ul> <li>Instrument: EpiSensor and Centaur Digitizer</li> <li>Sampling rate: 250 Hz</li> <li>Unit: counts</li> <li>unfiltered</li> <li>file: XKUK_centaur-6_1233_20200908_114500.seed</li> </ul> <p><strong>Angular rate data:</strong></p> <ul> <li>Instrument: IMU KvH 1750 (includes accelerometer and rotational sensor)</li> <li>Sampling rate: 250 Hz</li> <li>Unit gyro: rad/s</li> <li>Unit accelerometer: g (gravitational acceleration)</li> <li>file: LOGGING_1750_IMU_1308K004_11_57_25_250.csv</li> </ul> <p><strong>Robot Feedback:</strong></p> <ul> <li>Instrument: KUKA model AGILUS KR 6 R900 sixx</li> <li>Sampling rate: 250 Hz</li> <li>Unit translation: m</li> <li>Unit rotation: degree</li> <li>files: kuka_motion_*.txt, 1-4 are consecutive in time.</li> </ul> <p><strong>Experiments:</strong></p> <ul> <li>T: translations, R: rotations, XL, L, S denote the relative amplitudes</li> <li>10 experiments: TLRXL, TLRXL, TLRL, TLRL, TLRS, unfinished TLRS, TLRS, TLRS, TSRS, TSRS (Robot feedback (1,2), angular rate, GNSS data)</li> <li>9 experiments: TLRXL, TLRXL, TLRL, TLRL, TLRS, unfinished TLRS, TLRS, TSRS, TSRS (Robot feedback (3,4), accelerometer data</li> </ul>
Data for OFDVDnet: A sensor fusion approach for video denoising in fluorescence guided surgery
<p>Many applications in machine vision and medical imaging require the capture of images from a scene with very low radiance, which may result in very noisy images and videos. An important example of such an application is the imaging of fluorescently-labeled tissue in fluorescence-guided surgery. Medical imaging systems, especially when intended to be used in surgery, are designed to operate in well-lit environments and use optical filters, time division, or other strategies that allow the simultaneous capture of low radiance fluorescence video and a well-lit visible light video of the scene. This work demonstrates video denoising can be dramatically improved by utilizing deep learning together with motion and textural cues from the noise-free video.</p>
GNSS Location Verification in Connected and Autonomous Vehicles Using in-Vehicle Multimodal Sensor Data Fusion (presentation recording)
<p>Video recording of the presentation for the publication N. Souli et al., "GNSS Location Verification in Connected and Autonomous Vehicles Using in-Vehicle Multimodal Sensor Data Fusion," 2020 22nd International Conference on Transparent Optical Networks (ICTON), Bari, Italy, 2020, pp. 1-4, doi: 10.1109/ICTON51198.2020.9203087.</p>
VIO-GNSS Dataset: Benchmarking Dataset for Sensor Fusion of Visual Inertial Odometry and GNSS Positioning
<p>This upload contains datasets for benchmarking and improving different Sensor Fusion implementations/algorithms. The documentation for these datasets can be found on <a href="https://github.com/AaltoVision/vio-gnss-dataset">GitHub</a>.</p> <p>The upload contains two datasets (version 1.0.0):</p> <ul> <li>urban_with_gnss_dead_zones (7.0 GB, ~16 minutes) <ul> <li>City streets</li> <li>A building is passed through on two occasions which makes the GNSS location signal unavailable at times.</li> <li>RTK Fix is acquired at times</li> </ul> </li> <li>suburban_nature (10.6 GB, ~19 minutes) <ul> <li>The route begins on a suburban street but quickly turns into a nature trail. Lots of vegetation</li> <li>The RTK solution is only Float or None most of the route.</li> </ul> </li> </ul> <p>Details on collecting the data:</p> <ul> <li>Software <ul> <li>The data was collected using <a href="https://github.com/AaltoVision/vio-gnss-recorder">this</a> open-source recorder. <ul> <li>Can be easily replayed using <a href="https://github.com/SpectacularAI/sdk-examples">SpectacularAI's SDK</a> (sdk-examples/python/oak/vio_replay.py)</li> </ul> </li> <li>Each dataset contains a map of the travelled route in Otaniemi, Espoo, Finland.</li> <li><strong>Necessary files to implement SLAM are included</strong> in the dataset.</li> <li>Use of NTRIP and the high precision GNSS antenna enables global positioning accuracy of only few centimeters.</li> </ul> </li> <li>Hardware <ul> <li>OAK-D stereo depth + color camera (Luxonis)</li> <li>C099-F9P GNSS module (u-blox)</li> <li>ANN-MB-00 high precision GNSS antenna (u-blox)</li> </ul> </li> </ul>
Data for OFDVDnet: A sensor fusion approach for video denoising in fluorescence guided surgery
Open the record for dataset details and reuse information.
EMG and Video Dataset for sensor fusion based hand gestures recognition
<p>This dataset contains data for hand gesture recognition recorded with 3 different sensors. </p> <p>sEMG: recorded via the Myo armband that is composed of 8 equally spaced non-invasive sEMG sensors that can be placed approximately around the middle of the forearm. The sampling frequency of Myo is 200 Hz. The output of the Myo is a.u </p> <p>DVS: Dynamic Video Sensor which is a very low power event-based camera with 128x128 resolution</p> <p>DAVIS: Dynamic Video Sensor which is a very low power event-based camera with 240x180 resolution that also acquires APS frames.</p> <p>The dataset contains recordings of 21 subjects. Each subject performed 3 sessions, where each of the 5 hand gesture was recorded 5 times, each lasting for 2s. Between the gestures a relaxing phase of 1s is present where the muscles could go to the rest position, removing any residual muscular activation.</p> <p> </p> <p>Note: All the information for the DVS sensor has been extracted and can be found in the *.npy files. In case the raw data (.aedat) was needed please contact</p> <p> </p> <p>enea.ceolini@ini.uzh.ch</p> <p>elisa@ini.uzh.ch</p> <p>==== README ====</p> <p> </p> <p>DATASET STRUCTURE:</p> <p>EMG, DVS and APS recordings</p> <p>21 subjects</p> <p>3 sessions for each subject</p> <p>5 gestures in each session ('pinky', 'elle', 'yo', 'index', 'thumb')</p> <p> </p> <p>SINGLE DATASETS:</p> <p>- relax21_raw_emg.zip: contains raw sEMG and annotations (ground truth of gestures) in the format `subjectXX_sessionYY_ZZZ` with `XX` subject ID (01 to 21), `YY` session ID (01-03) and `ZZZ` that can be ‘emg’ or ‘ann’.</p> <p> </p> <p>- relax21_raw_dvs.zip: contains the full-frame dvs events in an array with dimensions 0 -> addr_x, 1 -> addr_y, 2 -> timestamp, 3 -> polarity. The timestamps are in seconds and synchronized with the Myo. Each file is in the format `subjectXX_sessionYY_dvs` with `XX` subject ID (01 to 21), `YY` session ID (01-03).</p> <p> </p> <p>- relax21_cropped_aps.zip: contains the 40x40 pixel aps frames for all subjects and trials in the format `subjectXX_sessionYY_Z_W_K` with `XX` subject ID (01 to 21), `YY` session ID (01-03), Z gesture ('pinky', 'elle', 'yo', 'index', 'thumb’), W trial ID (1-5), `K` frame index.</p> <p> </p> <p>- relax21_cropped_dvs_emg_spikes.pkl: spiking dataset that can be used to reproduce the results in the paper. The dataset is a dictionary with the following keys:</p> <ul> <li><strong>- </strong><strong>y</strong>: array of size 1xN with the class (0->4).</li> <li><strong>- </strong><strong>sub</strong>: array of size 1xN with the subject id (1->10).</li> <li><strong>- </strong><strong>sess</strong>: array of size 1xN with the session id (1->3).</li> <li><strong>- </strong><strong>dvs</strong>: list of length N, each object in the list is a 2d array of size 4xT_n where T_n is the number of events in the trial and the 4 dimensions rappresent: 0 -> addr_x, 1 -> addr_y, 2 -> timestamp, 3 -> polarity .</li> <li><strong>- </strong><strong>emg</strong>: list of length N, each object in the list is a 2d array of size 3xT_n where T_n is the number of events in the trial and the 3 dimensions rappresent: 0 -> addr, 1 -> timestamp, 3 -> polarity.</li> </ul> <p> </p> <p> </p>
Data for "Quantifying nocturnal thrush migration using sensor data fusion between acoustics and vertical-looking radar"
<p>This repository contains the data used for the analyses conducted and described in the article "Quantifying nocturnal thrush migration using sensor data fusion between acoustics and vertical-looking radar", as well as the Random Forest classifier built from such data. </p> <p>Files:</p> <p><em>2021_AcousticDataset.xlsx:</em> excel dataset containing the daily number of calls for each of the three thrush species considered recorded in Helsinki (Finland) during the study period (year 2021). The dates refer to UTC time.</p> <p><em>2022_ AcousticDataset.xlsx: </em>excel dataset containing the daily number of calls for each of the three thrush species considered recorded in Helsinki (Finland) during the study period (year 2022) . The dates refer to UTC time.</p> <p><em>RandomForestClassifier.Rdata: </em>R object containing the Random Forest classifier built using the eight most important echo features. The purpose of the classifier is to categorize echoes into 'thrush' and 'non-thrush' classes. </p> <p><em>2021_RadarDataset.rds: </em>R object containing the radar data collected with AVLR BirdScan MR1 in Helsinki (Finland) during the study period (year 2021).</p> <p><em>2022_RadarDataset.rds:</em> R object containing the radar data collected with AVLR BirdScan MR1 in Helsinki (Finland) during the study period (year 2022).</p> <p><em>license.txt: </em>the license applying to the data.</p>
Robust perception fusion from deployed sensors
<p>This dataset contains the necessary data for the development of perception tools to fuse the information from the sensors on the crawler robot and the sensors on the aerial robots. It contains two Rosbag files that provide different measurements from an 3D Velodyne LIDAR, two visual sensors, an altimeter and an IMU, all mounted in an aerial vehicle. It is also given the position estimation of the crawler. The ground-truth localization of the robot is given by a RTK GPS with accuracy of < 2cm. The transformations between sensor frames are also given in the bags. The flight experiments were conducted in the Karting AEROARMS outdoors scenario near Seville, during September 2018.</p>
Deep learning empowered sensor fusion boosts infant movement classification
<p>This repository contains data set and code for the paper:</p> <p>Kulvicius, T., Zhang, D., Poustka, L., Bölte, S., Jahn, L., Flügge, S., Kraft, M., Zweckstetter, M., Nielsen-Saines, K., Wörgötter, F., and Marschik, P. B. (2024). Deep learning empowered sensor fusion boosts infant movement classification. Communications Medicine, 5(16). DOI:10.1038/s43856-024-00701-w.</p> <p>For more details please see README.md file.</p>
Testing the Fault-Tolerance of Multi-Sensor Fusion Perception in Autonomous Driving Systems
<p>Here we provide some example videos of safety violations of Apollo caused by sensor faults</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.