Skip to main content
ConductScienceScienceDex
zenodoopen

Mask R-CNN on NYUv2

<p><strong>Mask R-CNN on NYUv2</strong></p> <p>This repository mainly contains information from the execution of the <a href="https://github.com/facebookresearch/maskrcnn-benchmark">Mask R-CNN network </a>[1] on images from the <a href="https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html">NYUv2 dataset</a> [2] as well as additional metadata. It&nbsp;was created for analyzing the output of Mask R-CNN and post-processing it using contextual information for improving its performance. This work has been carried out by <a href="http://mapir.isa.uma.es/jotaraul">Dr. Jose-Raul Ruiz-Sarmiento</a> (MAPIR group, University of M&aacute;laga) and <a href="https://lishuda.wordpress.com/">Dr. Shuda Li</a> (AVG group, University of Oxford) in the scope of the European project<a href="http://www.movecare-project.eu/"> MoveCare: Multiple-actOrs Virtual Empathic CARgiver for the Elder</a> (Ref: 732158).</p> <p>Concretely, <strong>this repository includes</strong>:</p> <p>- metadata:<br> &nbsp;&nbsp; &nbsp;+ coco_nyu_mapping.txt: Mapping between the categories in COCO dataset and those in NYUv2.<br> &nbsp;&nbsp; &nbsp;+ coco_object_categories.txt: Object categories considered in COCO dataset.<br> &nbsp;&nbsp; &nbsp;+ nyu_object_categories.txt: Object categories used in NYUv2 dataset.<br> &nbsp;&nbsp; &nbsp;+ nyu_scene_categories.txt: Scene categories considered in NYUv2.<br> &nbsp;&nbsp; &nbsp;+ objects_and_categories_in_images.txt: For each image in NYUv2, the categories of the appearing objects.</p> <p>- nyu_content:<br> &nbsp;&nbsp; &nbsp;+ masks_in_X (Where X is the image index)<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;- Y.png: Where Y is the object index in the image, represents the binary mask of that object.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;- pixels_labelled.png: Binary mask indicating the labelled pixels in image X.<br> &nbsp;&nbsp; &nbsp;+ bboxesX.txt: Where X is the image index, includes the ground truth bounding boxes of the objects in it. Format is: min_x min_y max_x max_y.</p> <p>- preds:<br> &nbsp;&nbsp; &nbsp;+ X: Where X is the image index.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;- Y.png: Where Y is the object index in the image, as detected by Mask R-CNN. Binary image containing the mask of such detected object.<br> &nbsp;&nbsp; &nbsp;+ X.txt: Where X is the image index. File containing the objects detected by Mask R-CNN, including: idx class score min_x min_y max_x max_y masks_file, being min_x min_y max_x and max_y bounding box information, while masks_file refers to X/Y.png as described above.<br> &nbsp;&nbsp; &nbsp;+ result_X.png: Where X is the image index. Image showing the detections with a socre higher than 0.3.<br> &nbsp;&nbsp; &nbsp;+ gt_iou_X: Where X is the image index.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;- Y: Where Y is the index of the detected object.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;+ Z.png Where Z is the index of the object in the ground truth. Image showing the masks of both objects, Y and Z, for visually checking their overlapping.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;- Y.txt: Where Y is the index of the detected object. File containing:<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;+ The intersection ratio of the object mask Y with the labelled part of the image.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;+ The IoU value for the mask of object Y and those of ground truth objects.<br> &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp;<br> &nbsp;&nbsp; &nbsp;<br> <strong>References:</strong></p> <p>[1] He, Kaiming, Georgia Gkioxari, Piotr Doll&aacute;r, and Ross Girshick. &quot;Mask r-cnn.&quot; In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969. 2017.<br> [2] Silberman, Nathan, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. &quot;Indoor segmentation and support inference from rgbd images.&quot; In European Conference on Computer Vision, pp. 746-760. Springer, Berlin, Heidelberg, 2012.</p>

ShareScore

40/100

Overall dataset sharing score

Score breakdown

These five areas show where the dataset supports — or may limit — practical reuse.

Stewardship
8
Harmonization
4
Access
12
Reuse readiness
8
Engagement
8

Topics

Dataset details

Repository
zenodo
Published
June 13, 2019
License
cc-by-nc-sa-4.0
DOI
10.5281/zenodo.3246277

Scored with release 0.9.0

ScienceDex for faculty

Your academic CV and NIH biosketch

Make your CV in your school's format and your NIH biosketch from it, then download both as Word files.