RoHuCAD: Robots and Humans Collaborative Anomaly Detection

<h1>RoHuCAD: Robots and Humans Collaborative Anomaly Detection</h1> <p>RoHuCAD is a dataset of human-robot collaboration in a robotic workshop (check <code>workshop_layout.png</code>). Two robots (collaborative manipulator - cobot, autonomous mobile robot - AMR) assist three human operators in assembly of electronic devices.</p> <p>There are two 8-min long recordings in the dataset. They mostly follow the same scenario, with slightly different anomalies. The data is in ROS Noetic rosbag format.</p> <h2>Included data&nbsp;</h2> <ul> <li>RGBD camera data (color + depth) <ul> <li>3 cameras: <a href="https://www.intelrealsense.com/depth-camera-d435i/">Intel Realsense D435i</a></li> <li>color and depth data at 6 frames per second</li> <li>Intrinsic calibration data</li> <li>Extrinsic calibration data (positions and orientations)</li> </ul> </li> <li>Information about positions of robots <ul> <li>AMR: <a href="https://www.ez-wheel.com/en/development-kit-for-agv-and-amr">Ez-Wheel SWD&reg; Starter Kit</a></li> <li>Cobot: <a href="https://www.universal-robots.com/products/ur10-robot/">Universal Robots UR10e</a></li> </ul> </li> </ul> <h2>Annotations</h2> <p>Annotations of specific anomalies are included (CSV file with columns: event_id, tstart, tend, event_type, person_id, camera_id)</p> <ul> <li>Gestures / poses <ul> <li>BENT</li> <li>T-POSE (hands horizontally to the sides)</li> <li>L+R-UP (both hands up)</li> <li>RH-UP (right hand up)</li> <li>LH-UP (left hand up)</li> <li>SQUAT</li> <li>HI-POSE (waving)</li> </ul> </li> <li>Unsafe behaviour <ul> <li>Human in robot working area</li> <li>Standing back to (moving) robot</li> <li>Looking at phone</li> <li>Human in the way of AMR</li> </ul> </li> <li>Normal activities <ul> <li>Assembling/Working</li> <li>Loading/unloading AMR</li> </ul> </li> </ul> <h2>ROS topics</h2> <ul> <li><code>/tf </code></li> <li><code>/tf_static</code></li> <li><code>/joint_states</code></li> <li>cam_ws2_box <ul> <li><code>/cam_ws2_box/color/camera_info</code></li> <li><code>/cam_ws2_box/color/image_raw/compressed</code></li> <li><code>/cam_ws2_box/depth_registered/camera_info</code></li> <li><code>/cam_ws2_box/depth_registered/image_rect_raw</code></li> </ul> </li> <li>cam_ta2_ws2 <ul> <li><code>/cam_ta2_ws2/color/camera_info</code></li> <li><code>/cam_ta2_ws2/color/image_raw/compressed</code></li> <li><code>/cam_ta2_ws2/depth_registered/camera_info</code></li> <li><code>/cam_ta2_ws2/depth_registered/image_rect_raw</code></li> </ul> </li> <li>cam_ta1_ws2 <ul> <li><code>/cam_ta1_ws2/color/camera_info</code></li> <li><code>/cam_ta1_ws2/color/image_raw/compressed</code></li> <li><code>/cam_ta1_ws2/aligned_depth_to_color/camera_info</code></li> <li><code>/cam_ta1_ws2/aligned_depth_to_color/image_raw</code></li> </ul> </li> </ul> <h2>Acknowledgement</h2> <p>The work leading to these results has received funding from the European Union&rsquo;s Horizon Europe research and innovation programme within the ULTIMATE project under the Grant Agreement no 101070162.</p>