Millimeter-wave Multi-View Radar (MMVR) Dataset

<p><strong>Introduction</strong></p> <p>The&nbsp;Millimeter-wave Multi-View Radar (MMVR)&nbsp;dataset comprises&nbsp;345K&nbsp;multi-view radar frames collected from&nbsp;25&nbsp;human subjects over&nbsp;6&nbsp;different rooms. It includes&nbsp;446K&nbsp;annotated bounding boxes/segmentation instances, and&nbsp;7.59 million&nbsp;annotated keypoints to support three perception tasks: object detection, pose estimation, and instance segmentation in the image plane.</p> <p>MMVR consists of 35 data sessions, each stored in a&nbsp;data folder&nbsp;with the format&nbsp;d<em>x</em>sy, where&nbsp;<em>x</em>&nbsp;represents the day and&nbsp;y&nbsp;the session index of that day when the data were collected.</p> <p>Within each data folder, data frames are grouped into 395 non-overlapping one-minute&nbsp;data segments. Each data segment is stored in a folder named using a three-digit, zero-filled convention based on the chronological order. For example, the first one-minute data segment is saved in the folder&nbsp;000, while the second one-minute data segment is stored in the folder&nbsp;001.</p> <p>Within each data segment folder, each data frame consists of 5 NPZ files: meta, radar, bounding boxes (bbox), keypoints (pose) and segmentation masks (mask). Each data frame is named using a five-digit, zero-filled convention based on the chronological order within the one-minute data segment.</p> <pre><code>Root/ ├── d1s1/ │ └── ... ├── d1s2/ │ ├── 000/ │ │ ├── 00000_meta.npz ... Meta info │ │ ├── 00000_radar.npz ... Horizontal/Vertical heatmaps │ │ ├── 00000_bbox.npz ... 2D Bounding Boxes │ │ ├── 00000_pose.npz ... 2D keypoints │ │ ├── 00000_mask.npz ... 2D Segmentation masks │ │ ├── 00001_meta.npz │ │ ├── 00001_radar.npz │ │ ├── 00001_bbox.npz │ │ ├── 00001_pose.npz │ │ ├── 00001_mask.npz │ │ . │ │ ├── 00899_meta.npz │ │ ├── 00899_radar.npz │ │ ├── 00899_bbox.npz │ │ ├── 00899_pose.npz │ │ └── 00899_mask.npz │ └── 001/ │ ├── 00000_meta.npz │ ├── 00000_radar.npz │ ├── 00000_bbox.npz │ ├── 00000_pose.npz │ ├── 00000_mask.npz . └── ... ├── d9s5/ │ └── ... └── d9s6/ └── ... </code></pre> <p>&nbsp;</p> <p><strong>At a Glance</strong></p> <ul> <li> <ul> <li>The size of the unzipped dataset is ~81.8 GB</li> <li>The MMVR dataset consists of 345K data frames with each frame including 5 NPZ files:&nbsp;meta,&nbsp;radar,&nbsp;bounding boxes&nbsp;(bbox),&nbsp;keypoints&nbsp;(pose) and&nbsp;segmentation masks&nbsp;(mask).</li> <li>The MMVR dataset is divided into four smaller chunks to facilitate easier downloads: three chunks of approximately 20 GB each and one chunk of 12 GB. <ul> <li>P1.zip (24.2 GB): <ul> <li>README.md and its related figure folder (figs)</li> <li>load_sample.ipynb: a python snippet to load and visualize a data frame</li> <li>data_split.npz: an npz file containing the predefined data splits (S1 or S2) for training, validation, and test sets of data segments under P1 and P2.</li> <li>all data frames under P1 (d1s1, d1s2, d2s2, d3s1, d3s2, d4s1)</li> </ul> </li> <li>P2_00.zip (20.4 GB): all data frames in d5s1 &ndash; d5s6 and d6s1 &ndash; d6s6</li> <li>P2_01.zip (21.6 GB): all data frames in d7s1 &ndash; d7s5 and d8s1 &ndash; d8s6</li> <li>P2_02.zip (11.7 GB): all data frames in d9s1 &ndash; d9s6</li> </ul> </li> </ul> </li> </ul> <p><strong>Citation</strong></p> <p>If you use the MMVR dataset in your research, please cite our contribution:</p> <pre><code>@inproceedings{MMVR2024, title={MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception}, author={M. Mahbubur Rahman and Ryoma Yataka and Sorachi Kato and Pu Perry Wang and Peizhao Li and Adriano Cardace and Petros Boufounos}, booktitle={Proceedings of European Conference on Computer Vision (ECCV)}, pages={}, year={2024} }</code></pre> <p><strong>License</strong></p> <p>The MMVR dataset is released under&nbsp;<a href="https://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA-4.0 license</a>.</p> <p>All data:</p> <pre><code>Created by Mitsubishi Electric Research Laboratories (MERL), 2023 SPDX-License-Identifier: CC-BY-SA-4.0</code></pre>