Learning to Grasp Unknown Objects in Domestic Environments with GP-net+

<p>This record includes data for the paper "Learning to Grasp Unknown Objects in Domestic Environments", currently under review.<br><br><strong>Simulation environment with pre-trained GP-net+ model</strong><br><br>The paper presents a simulation environment for grasping objects in domestic environments. The presented objects and furniture units, as well as a pre-trained GP-net+ model can be found in the "gpnetplus_simulation_data.zip" file. After this zip file is downloaded, it can be unpacked it into the <a href="https://github.com/AuCoRoboticsMU/GP-netplus" target="_blank" rel="noopener">GP-net+ directory</a>. It includes all necessary data to use the simulation environment, for example, for testing GP-net+ or other grasping models in simulated domestic environments.</p> <p>&nbsp;</p> <p><strong>ROS model</strong></p> <p>The paper additionally presents an <a href="https://github.com/AuCoRoboticsMU/GP-netplus-ros" target="_blank" rel="noopener">ROS package</a> that can be deployed for grasping unknown objects in domestic environments with simulated or real robots. We make a ROS-compatbile model of GP-net+ available in the "ros_gpnet_plus.zip" file, which can be used with the ROS package.</p> <p>&nbsp;</p> <p><strong>Training dataset</strong></p> <p>We used the simulation environment in our paper to generate a training dataset and train GP-net+. This training dataset is included in this record and can be used to replicate our results or train modifications of GP-net+.</p> <p>To improve handling of the training dataset (total size 25GB+), we split the dataset into several .zip files, named val.zip (validation data) and train_[0-6].zip (training data). Download all files individually and extract them into a single folder. Combine all files train_[0-6].zip directory into a single directory called 'train', for example, by using the 'move_train_data.sh' script provided.<br><br>The final structure for the dataset should look similar to this:<br><br>gpnet_data</p> <p>|-- val</p> <p>&nbsp; &nbsp; &nbsp;|-- depth_image_0000000.npz</p> <p>&nbsp; &nbsp; &nbsp;|-- depth_image_0000001.npz</p> <p>&nbsp; &nbsp; &nbsp;...</p> <p>&nbsp; &nbsp; &nbsp;|--segmask_image_0052346.npz</p> <p>|-- train</p> <p>&nbsp; &nbsp; |-- depth_image_0000000.npz</p> <p>&nbsp; &nbsp; |-- depth_image_0000001.npz</p> <p>&nbsp; &nbsp; ...</p> <p>&nbsp; &nbsp; |-- segmask_image_0602506.npz</p> <p>&nbsp; &nbsp; |-- segmask_image_0602507.npz</p> <p><br><br><br>For generation of the training and simulation data, the following mesh databases have been used:<br><br>B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar,"Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set," IEEE Robotics and Automation Magazine, vol. 22, no. 3, pp. 36&ndash;52, 2015<br><br>A. Singh, J. Sha, K. S. Narayan, T. Achim, and P. Abbeel, "BigBIRD: A large-scale 3D database of object instances," 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 509&ndash;516, 2014.<br><br>A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, "ShapeNet: An Information-Rich 3D Model Repository," Tech. Rep. arXiv:1512.03012 [cs.GR], Stanford University &mdash; Princeton University &mdash; Toyota Technological Institute at Chicago, 2015.</p> <p>D. Morrison, P. Corke, and J. Leitner, "EGAD! An Evolved Grasping Analysis Dataset for Diversity and Reproducibility in Robotic Manipulation," IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4368&ndash;4375, 2020</p>