Fusion of Underwater Camera and Multibeam Sonar for Diver Detection and Tracking

<div><strong>Context</strong></div> <div>&nbsp;</div> <div>This dataset is related to previously published public dataset "Sonar-to-RGB Image Translation for Diver Monitoring in Poor Visibility Environments".&nbsp;<a href="../records/7728089">https://zenodo.org/records/7728089</a></div> <div>It contains ZED-right camera and sonar images collected from Hemmoor Lake and DFKI Maritime Exploration Hall.</div> <div>&nbsp;</div> <div>Sensors: Low Frq (1.2MHz) Blueprint Oculus M1200d Sonar and ZED Right Camera</div> <div>&nbsp;</div> <div><strong>Content</strong></div> <div>&nbsp;</div> <div>The dataset is created for Diver Detection and Diver Tracking applications.</div> <div>&nbsp;</div> <div>For the Diver Detection part, the dataset is prepared to train, validate and test YOLOv7 model.</div> <div>7095 images are used for training data, and 3095 images are used for validation data. These sets are augmented from originally captured and sampled ZED camera images.&nbsp;Augmentation methods are not applied to the Test data, which contains 822 images. Train and validation contain images from both the DFKI pool and Hemmor Lake, while the test data is only collected from the lake.</div> <div>&nbsp;</div> <div>To distinguish between the original image and the augmented image, check the name coding.&nbsp;</div> <div>Naming of object detection images:</div> <div>original_image_name.jpg</div> <div>if augmented:</div> <div>original_image_name_&lt;augmentation_number_of_the_same_image&gt;.jpg</div> <div>&nbsp;</div> <div>Object Detection Label Format:&nbsp;</div> <div>YOLO [(class), ((x_min + (x_max - x_min)/2)&nbsp; / image_width), ((y_min + (y_max - y_min)/2)&nbsp; / image_height), ((x_max - x_min) / image_width), ((y_max - y_min) / image_height)]</div> <div>&nbsp;</div> <div>Class: "diver", represented by "0" in object detection labels.</div> <div>&nbsp;</div> <div>Resolution of Object Detection Camera Images: 640x640</div> <div>Resolution of Object Tracking Camera Images: 1280x720</div> <div>Resolution of Object Tracking Low Frequency Sonar: 932x514</div> <div>&nbsp;</div> <div>About the Object Tracking on Sonar, the sampled data is the part where diver moves around the table and the platform.&nbsp;</div> <div>There are 4 cases shared in the dataset, which contain a sonar stream, and corresponding ZED-right camera images.&nbsp;</div> <div>Totally, 1193 points represent the diver on sonar images for the diver tracking application.</div> <div>&nbsp;</div> <div>For the tracking, "tracking_sonar_coordinates_&lt;number&gt;.csv" contains x,y coordinates of a point where the diver is in the sonar image.&nbsp;</div> <div>And "image_sonar_&lt;number&gt;.csv" file contains the matching between sonar and camera images.</div> <div>&nbsp;</div> <div><strong>Acknowledgements</strong></div> <div>&nbsp;</div> <div>The data in this repository were collected as a joint effort between the German Center for Artificial Intelligence (DFKI), the German Federal Agency for technical Relief (THW), and Kraken Robotics GmbH. This work is part of the project DeeperSense that received funding from the European Commission. Program H2020-ICT-2020-2 ICT-47-2020 Project Number: 101016958.</div> <p>&nbsp;</p>