Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
13
datasets available to search
ShareScore release 0.9.0
Dataset results
13 results for “objects and features”
Histograms of local features for the object dataset
<p>Pictures used in the experiments were acquired as frames from an interaction with an iCub robot, through an RGBD sensor (red, green, and blue camera coupled with a depth sensor). The acquisition was performed and is described in more details by Lyubova and Filliat. Each image contains one visual object and possibly the hand of the operator.</p> <p><br /> Images contains objects from a subset of 10 objects each appearing in more than a thousand frames.During the acquisition, the objects are moved and rotated. Hence they are presented from distinct points of view and they may be partially cluttered by the hand of the operator.</p> <p> </p> <p>For more information on the format please look at https://github.com/omangin/multimodal/blob/master/db/models/objects.py.</p> <p> </p>
Dataset supplementing Stoll, J., Thrun, M., Nuthmann, A., & Einhäuser, W. (2015). Overt attention in natural scenes: Objects dominate features. Vision Research, 107, 36-48. doi: 10.1016/j.visres.2014.11.006
<p>These data supplement the publication</p> <p>Stoll, J., Thrun, M., Nuthmann, A., & Einhäuser, W. (2015). Overt attention in natural scenes: Objects dominate features. Vision Research, 107, 36-48. doi: 10.1016/j.visres.2014.11.006</p> <p>and be used freely for scientific purposes provided the aforementioned paper is appropriately cited.</p> <p>Note that the image files cannot be provided on this site due to copyright restrictions.</p> <p>The dataset contains the following files:</p> <p>maps_01.mat - maps_72.mat:</p> <p>For each image the 6 maps used in the paper are contained, the maps of experiment 1 are labelled as in the paper (AWS, OOM, nOOM, PVL,UNI), AWS2 is the AWS map for the modified stimuli of experiments 2 and 3.</p> <p>exp?_fixations.mat contains all fixations of the respective experiment.</p> <p>For experiment 1, there are the variables xFix, yFix, durFix, which contain the x position, the y condition, and the fixation duration of each fixation. Dimensions are images x subjects x fixation number, where the first fixation is the 0th (initial) fixation. The variable condition (image x subject) contains the condition in which the respective image was shown to the subject. For the main analysis only the "0" condition was used, refer to the paper's appendix for the other conditions.</p> <p>For experiment 2 and 3, variables are called xFixByImage, yFixByImage, dFixByImage and the dimensions are subject x image x fixation number. In addition tFixByImage contains the start of the fixation relative to trial onset (negative for the 0th fixation).<br> In both cases, empty entries are filled with nans.</p> <p><br> computeROC.m is a helper function called by other functions.</p> <p><br> figure1.m through figure7.m reproduce the figures from the paper to exemplify data usage.</p> <p> </p>
VISIONE Feature Repository for VBS: Multi-Modal Features and Detected Objects from VBSLHE Dataset
<p>This repository contains a diverse set of features extracted from the VBSLHE dataset (laparoscopic gynecology) . These features will be utilized in the VISIONE system [Amato et al. 2023, Amato et al. 2022] in the next editions of the Video Browser Showdown (VBS) competition (<a href="https://www.videobrowsershowdown.org/">https://www.videobrowsershowdown.org/</a>). </p> <p>We used a snapshot of the dataset provided by the Medical University of Vienna and Toronto that can be downloaded using the instructions provided at <a href="https://download-dbis.dmi.unibas.ch/mvk/">https://download-dbis.dmi.unibas.ch/mvk/</a>. It comprises 75 video files. We divided each video into video shots with a maximum duration of 5 seconds.</p> <p>This repository is released under a Creative Commons Attribution license. If you use it in any form for your work, please cite the following paper:</p> <blockquote> <p>@inproceedings{amato2023visione, title={VISIONE at Video Browser Showdown 2023}, author={Amato, Giuseppe and Bolettieri, Paolo and Carrara, Fabio and Falchi, Fabrizio and Gennaro, Claudio and Messina, Nicola and Vadicamo, Lucia and Vairo, Claudio}, booktitle={International Conference on Multimedia Modeling}, pages={615--621}, year={2023}, organization={Springer} } </p> </blockquote> <p> </p> <p>This repository (v2) comprises the following files:</p> <ul> <li><em><strong>msb.tar.gz </strong></em> contains tab-separated files (.tsv) for each video. Each tsv file reports, for each video segment, the timestamp and frame number marking the start/end of the video segment, along with the timestamp of the extracted middle frame and the associated identifier ("id_visione").</li> <li><em><strong>extract-keyframes-from-msb.tar.gz</strong></em> contains a Python script designed to extract the middle frame of each video segment from the MSB files. To run the script successfully, please ensure that you have the original VBSLHE videos available.</li> <li><em><strong>features-aladin.tar.gz†</strong></em><strong> </strong>contains <a href="https://github.com/mesnico/ALADIN">ALADIN</a> [Messina N. et al. 2022] features extracted for all the segment's middle frames.</li> <li><em><strong>features-clip-laion.tar.gz†</strong></em> contains <a href="https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K">CLIP ViT-H/14 - LAION-2B </a>[Schuhmann et al. 2022] features extracted for all the segment's middle frames.</li> <li><em><strong>features-clip-openai.tar.gz† </strong></em>contains <a href="https://huggingface.co/openai/clip-vit-large-patch14">CLIP ViT-L/14</a> [Radford et al. 2021] features extracted for all the segment's middle frames.</li> <li><em><strong>features-clip2video.tar.gz† </strong></em>contains <a href="https://github.com/CryhanFang/CLIP2Video">CLIP2Video</a> [Fang H. et al. 2021] extracted for all the video segments. <strong> </strong></li> <li><em><strong>objects-frcnn-oiv4.tar.gz* </strong></em>contains the objects detected using <a href="http://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1">Faster R-CNN+Inception ResNet</a> (trained on the Open Images V4 [Kuznetsova et al. 2020]).</li> <li><em><strong>objects-mrcnn-lvis.tar.gz*</strong></em> contains the objects detected using Mask R-CNN [He et al. 2017] (trained on LVIS).</li> <li><em><strong>objects-vfnet64-coco.tar.gz*</strong></em> contains the objects detected using VfNet [Zhang et al. 2021] (trained on COCO dataset).</li> </ul> <p>*Please be sure to use the <strong>v2 version </strong>of this repository, since v1 feature files may contain inconsistencies that have now been corrected</p> <p><em><strong>*Note on the object annotations:</strong></em> Within an object archive, there is a jsonl file for each video, where each row contains a record of a video segment (the <em>"_id"</em> corresponds to the <em>"id_visione"</em> used in the msb.tar.gz) . Additionally, there are three arrays representing the objects detected, the corresponding scores, and the bounding boxes. The format of these arrays is as follows:</p> <ul> <li><em>"object_class_names"</em>: vector with the class name of each detected object.</li> <li><em>"object_scores"</em>: scores corresponding to each detected object.</li> <li><em>"object_boxes_yxyx"</em>: bounding boxes of the detected objects in the format <em>(ymin, xmin, ymax, xmax).</em></li> </ul> <p> </p> <p><em><strong>†Note on the cross-modal features: </strong></em>The extracted multi-modal features (ALADIN, CLIPs, CLIP2Video) enable internal searches within the VBSLHE dataset using the query-by-image approach (features can be compared with the dot product). However, to perform searches based on free text, the text needs to be transformed into the joint embedding space according to the specific network being used (see links above). Please be aware that t<strong>he service for transforming text into features is not provided within this repository and should be developed independently using the original feature repositories linked above.</strong></p> <p>We have plans to release the code in the future, allowing the reproduction of the VISIONE system, including the instantiation of all the services to transform text into cross-modal features. However, this work is still in progress, and the code is not currently available.</p> <p> </p> <p><strong>References:</strong></p> <p>[Amato et al. 2023] Amato, G.et al., 2023, January. VISIONE at Video Browser Showdown 2023. In International Conference on Multimedia Modeling (pp. 615-621). Cham: Springer International Publishing.</p> <p>[Amato et al. 2022] Amato, G. et al. (2022). VISIONE at Video Browser Showdown 2022. In: , et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. </p> <p>[Fang H. et al. 2021] Fang H. et al., 2021. Clip2video: Mastering video-text retrieval via image clip. arXiv preprint arXiv:2106.11097.</p> <p>[He et al. 2017] He, K., Gkioxari, G., Dollár, P. and Girshick, R., 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969).</p> <p>[Kuznetsova et al. 2020] Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A. and Duerig, T., 2020. The open images dataset v4. International Journal of Computer Vision, 128(7), pp.1956-1981.</p> <p>[Lin et al. 2014] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P. and Zitnick, C.L., 2014, September. Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.</p> <p>[Messina et al. 2022] Messina N. et al., 2022, September. Aladin: distilling fine-grained alignment scores for efficient image-text matching and retrieval. In Proceedings of the 19th International Conference on Content-based Multimedia Indexing (pp. 64-70).</p> <p>[Radford et al. 2021] Radford A. et al., 2021, July. Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PMLR.</p> <p>[Schuhmann et al. 2022] Schuhmann C. et al., 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35, pp.25278-25294.</p> <p>[Zhang et al. 2021] Zhang, H., Wang, Y., Dayoub, F. and Sunderhauf, N., 2021. Varifocalnet: An iou-aware dense object detector. In Proceedings of the IEEE/CV</p>
VISIONE Feature Repository for VBS: Multi-Modal Features and Detected Objects from MVK Dataset
<p>This repository contains a diverse set of features extracted from the marine video (underwater) dataset (MVK) . These features were utilized in the VISIONE system [Amato et al. 2023, Amato et al. 2022] during the latest editions of the Video Browser Showdown (VBS) competition (<a href="https://www.videobrowsershowdown.org/">https://www.videobrowsershowdown.org/</a>). </p> <p>We used a snapshot of the MVK dataset from 2023, that can be downloaded using the instructions provided at <a href="https://download-dbis.dmi.unibas.ch/mvk/">https://download-dbis.dmi.unibas.ch/mvk/</a>. It comprises 1,372 video files. We divided each video into 1 second segments. </p> <p>This repository is released under a Creative Commons Attribution license. If you use it in any form for your work, please cite the following paper:</p> <blockquote> <pre>@inproceedings{amato2023visione, title={VISIONE at Video Browser Showdown 2023}, author={Amato, Giuseppe and Bolettieri, Paolo and Carrara, Fabio and Falchi, Fabrizio and Gennaro, Claudio and Messina, Nicola and Vadicamo, Lucia and Vairo, Claudio}, booktitle={International Conference on Multimedia Modeling}, pages={615--621}, year={2023}, organization={Springer} }</pre> </blockquote> <p> </p> <p>This repository comprises the following files:</p> <ul> <li><strong><em>msb.tar.gz </em></strong> contains tab-separated files (.tsv) for each video. Each tsv file reports, for each video segment, the timestamp and frame number marking the start/end of the video segment, along with the timestamp of the extracted middle frame and the associated identifier ("id_visione"). </li> <li><em><strong>extract-keyframes-from-msb.tar.gz</strong></em> contains a Python script designed to extract the middle frame of each video segment from the MSB files. To run the script successfully, please ensure that you have the original MVK videos available.</li> <li><strong><em>features-aladin.tar.gz<sup>†</sup></em> </strong>contains <a href="https://github.com/mesnico/ALADIN">ALADIN</a> [Messina N. et al. 2022] features extracted for all the segment's middle frames. </li> <li><em><strong>features-clip-laion.tar.gz<sup>†</sup></strong></em> contains <a href="https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K">CLIP ViT-H/14 - LAION-2B </a>[Schuhmann et al. 2022] features extracted for all the segment's middle frames.</li> <li><em><strong>features-clip-openai.tar.gz<sup>†</sup> </strong></em>contains <a href="https://huggingface.co/openai/clip-vit-large-patch14">CLIP ViT-L/14</a> [Radford et al. 2021] features extracted for all the segment's middle frames. </li> <li><em><strong>features-clip2video.tar.gz<sup>†</sup> </strong></em>contains <a href="https://github.com/CryhanFang/CLIP2Video">CLIP2Video</a> [Fang H. et al. 2021] extracted for all the 1s video segments. <strong> </strong></li> <li><em><strong>objects-frcnn-oiv4.tar.gz<sup>*</sup> </strong></em>contains the objects detected using <a href="http://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1">Faster R-CNN+Inception ResNet</a> (trained on the Open Images V4 [Kuznetsova et al. 2020]). </li> <li><em><strong>objects-mrcnn-lvis.tar.gz<sup>*</sup></strong></em> contains the objects detected using Mask R-CNN [He et al. 2017] (trained on LVIS).</li> <li><em><strong>objects-vfnet64-coco.tar.gz<sup>*</sup></strong></em> contains the objects detected using VfNet [Zhang et al. 2021] (trained on COCO dataset).</li> </ul> <p>*Please be sure to use the <strong>v2 version </strong>of this repository, since v1 feature files may contain inconsistencies that have now been corrected</p> <p><em><strong>*Note on the object annotations:</strong></em> Within an object archive, there is a jsonl file for each video, where each row contains a record of a video segment (the <em>"_id"</em> corresponds to the <em>"id_visione"</em> used in the msb.tar.gz) . Additionally, there are three arrays representing the objects detected, the corresponding scores, and the bounding boxes. The format of these arrays is as follows:</p> <ul> <li><em>"object_class_names"</em>: vector with the class name of each detected object.</li> <li><em>"object_scores"</em>: scores corresponding to each detected object.</li> <li><em>"object_boxes_yxyx"</em>: bounding boxes of the detected objects in the format <em>(ymin, xmin, ymax, xmax).</em></li> </ul> <p> </p> <p><em><strong><sup>†</sup>Note on the cross-modal features: </strong></em>The extracted multi-modal features (ALADIN, CLIPs, CLIP2Video) enable internal searches within the MVK dataset using the query-by-image approach (features can be compared with the dot product). However, to perform searches based on free text, the text needs to be transformed into the joint embedding space according to the specific network being used (see links above). Please be aware that t<strong>he service for transforming text into features is not provided within this repository and should be developed independently using the original feature repositories linked above.</strong></p> <p>We have plans to release the code in the future, allowing the reproduction of the VISIONE system, including the instantiation of all the services to transform text into cross-modal features. However, this work is still in progress, and the code is not currently available.</p> <p> </p> <p><strong>References:</strong></p> <p>[Amato et al. 2023] Amato, G.et al., 2023, January. VISIONE at Video Browser Showdown 2023. In International Conference on Multimedia Modeling (pp. 615-621). Cham: Springer International Publishing.</p> <p>[Amato et al. 2022] Amato, G. et al. (2022). VISIONE at Video Browser Showdown 2022. In: , et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. </p> <p>[Fang H. et al. 2021] Fang H. et al., 2021. Clip2video: Mastering video-text retrieval via image clip. arXiv preprint arXiv:2106.11097.</p> <p>[He et al. 2017] He, K., Gkioxari, G., Dollár, P. and Girshick, R., 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969).</p> <p>[Kuznetsova et al. 2020] Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A. and Duerig, T., 2020. The open images dataset v4. International Journal of Computer Vision, 128(7), pp.1956-1981.</p> <p>[Lin et al. 2014] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P. and Zitnick, C.L., 2014, September. Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.</p> <p>[Messina et al. 2022] Messina N. et al., 2022, September. Aladin: distilling fine-grained alignment scores for efficient image-text matching and retrieval. In Proceedings of the 19th International Conference on Content-based Multimedia Indexing (pp. 64-70).</p> <p>[Radford et al. 2021] Radford A. et al., 2021, July. Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PMLR.</p> <p>[Schuhmann et al. 2022] Schuhmann C. et al., 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35, pp.25278-25294.</p> <p>[Zhang et al. 2021] Zhang, H., Wang, Y., Dayoub, F. and Sunderhauf, N., 2021. Varifocalnet: An iou-aware dense object detector. In Proceedings of the IEEE/CV</p>
Multiscale Spatial Patterns in Giant Dike Swarms Identified through Objective Feature Extraction Datasets
<p>S1 - Linked dike clusters for the Columbia River Flood Basalt group including the four identified subswarms: Chief Joseph, Monument, Ice Harbor, and Steens as compiled in Morriss et al., 2020. This dataset uses the a UTM Zone 11N projection (EPSG:26911).</p> <p>S2 - Linked dike clusters for the Deccan Traps including the four identified subswarms: Saurashtra, Narmada-Tapi, Central and Coastal. Due to their overlap Central and Coastal Swarms have been combined in this dataset into the Central Swarm. This dataset uses the a WGS 84 projection (EPSG:3857). </p> <p>S3 - Dike segment data for Spanish Peaks and Dike Mountain located in the Rio Grande Rift of Colorado. This dataset was digitized using QGIS based on the map by Johnson (1961). This dataset uses the a UTM Zone13N projection (EPSG:32613). The file includes the start, end points, and midpoints of the dikes; segment length; calculated $\rho$ and $\theta$ for the Hough Transform; the origin used for the Hough Transform which is different for each subswarm (xc,yc); dike rock type if known; and a unique identification calculated based on the start and endpoints. This dataset has been preprocessed to remove curving dikes and is the data set used to produce later products (Data set S4). </p> <p>S4 - Linked dike clusters for the Spanish Peaks and Dike Mountain. This dataset was produced using the Agglomerative Clustering algorithms using the parameters set in Table 1. This dataset uses the a UTM Zone 13N projection (EPSG:32613). </p> <p> </p> <p> </p> <p>These datasets were produced using the Agglomerative Clustering algorithms using the parameters set in Table 1. The datasets are in the format of a CSV file but can be read into GIS programs using Well Known Text (WKT) linestring. TThe file includes the start and end points of the average line in the cluster and it's mid points, cluster length and width (Xstart, Xend, Xmid, Ymid, in meters and UTM coordinates, Dike Cluster Width or R\_Width, Dike Cluster Length or R\_Length all in meters); calculated average $\rho$ and $\theta$ for the Hough Transform $\rho$ units measured in meters, $\theta$ units measured in degrees, unless otherwise stated); the origin used for the Hough Transform which is different for each subswarm ($xc$,$yc$, meters in UTM coordinates); average slope and intercept (AvgSlope, AvgIntercept meters); range and standard deviation for $\rho$ and $\theta$ for all objects in the cluster ($\rho$ units measured in meters, $\theta$ units measured in degrees); cluster size (Size); sum of segment lengths in a cluster (SegmentLSum, meters); whether the cluster crosses between negative and positive values (ClusterCrossesZero, boolean); overlap as calculated in the main text where the length of overlap is normalized by the sum of segment lengths in a cluster; maximum number of overlapping segments (nOverlapingSegments); twist angle which is the difference in angle betweeen the average cluster line and the average line formed by cluster midpoints (EnEchelonAngleDiff, degrees); the p-value for the midpoint line fit of the segments where $p<0.05$ is considered to be a significant fit (EEPValue); the maximum, median, and minimum segment nearest neighbors distances in the cluster which is calculated using the cartesian midpoints of each segment and normalized by the Cluster Length (MaxSegNNDist, MedianSegNNDist, MinSegNNDist); characterization of each cluster as filtered or not, filtered clusters are of size greater than $3$ and have a MaxSegNNDist of less than $0.5$ (TrustFilter, boolean); the date edited (Date\_Changed), and the clustering parameters used for each cluster (Rho\_Threshold in meters, Theta\_Threshold in degrees) and a unique identification calculated based on the start and endpoints (ClusterHash). </p>
Guerrero Iron Objects Feature No.1 (2019)
Site 2-04-8MO02343 Multiple Iron Concretions on Bedrock No.1 (2019). Along the eastern edge of the site, closest to where the reef meets deeper water, there are clusters of iron artifacts fused to the bedrock. This is one of two features of concreted iron objects documented in 2019. This particular feature was also modeled in 2108 (see https://skfb.ly/6AL9V), but this newer version shows a slightly broader view. Items observed within this group of artifacts are ship's fasteners, rigging elements, and many unidentified items. Black & white scale = 20 centimeters. Source: Objaverse 1.0 / Sketchfab
Guerreo Iron Objects Feature No.2 (2019)
Site 2-04-8MO02343 Multiple Iron Concretions on Bedrock No.2 (2019). Along the eastern edge of the site, closest to where the reef meets deeper water, there are clusters of iron artifacts fused to the bedrock. This is one of two features of concreted iron objects documented in 2019. Items observed within this feature include ship's fasteners, rigging elements, and many unidentified items. Black & white scale = 20 centimeters. Source: Objaverse 1.0 / Sketchfab
Objective identification of high-wind features within extratropical cyclones using a probabilistic random forest (RAMEFI). Part I: Method and illustrative case studies - Video Supplement
<p>These videos provide examples of application of RAMEFI (RAndom-forest based MEsoscale wind Feature Identification), a new objective identification of high-wind features within extratropical cyclones, for twelve selected case studies.</p>
Familiar object benefit more from transsaccadic feature predictions
<p>The transsaccadic feature prediction mechanism associates peripheral and foveal information<br> belonging to the same object to make predictions about how an object seen in the periphery<br> would appear in the fovea or vice versa. It is unclear if such transsaccadic predictions require<br> experience with the object such that only familiar objects benefit from this mechanism by virtue<br> of having peripheral-foveal associations. In two experiments, we tested whether familiar<br> objects have an advantage over novel objects in peripheral-foveal matching and transsaccadic<br> change detection tasks.<br> In both experiments, observers were unknowingly familiarized with a small set of stimuli by<br> completing a sham orientation change detection task. In the first experiment, observers<br> subsequently performed a peripheral-foveal matching task, where they needed to pick the<br> foveal test object that matched a briefly presented peripheral target. In the second experiment,<br> observers subsequently performed a transsaccadic object change detection task where a<br> peripheral target was exchanged or not exchanged with another target after the saccade, either<br> immediately or after a 300 ms blank period.<br> We found an advantage of familiar objects over novel objects in both experiments. While<br> foveal-peripheral associations explained the familiarity effect in the matching task of the first<br> experiment, the second experiment provided evidence for the advantage of peripheral-foveal<br> associations in transsaccadic object change detection. Introducing a postsaccadic blank<br> improved the change detection performance in general but more for familiar than for novel<br> objects. We conclude that familiar objects benefit from additional object-specific predictions.<br> Keywords: Saccades, Transsaccadic prediction, perceptual learning, object recognition,<br> visual stability.</p>
StreetScouting dataset: A Street-Level Image dataset for finetuning and applying custom object detectors for urban feature selection
<p>The dataset consists of two .zip files.</p> <p>The first .zip file named "annotated dataset" contains a folder named “annotated dataset" with annotated street images. It consists of the folder “images” that has 763 image files. The image format is PNG. 432 images have dimensions of 1080 x 2160 and 331 have dimensions of 866 x 2400. The filenames are random uuids. The “annotated dataset” folder also contains the annotations in the file “coco_annotations.json”. Annotations are provided in COCO format. Table 1 shows the total number of annotated objects per class.</p> <table align="center" summary="Total number of annotated objects per class"> <caption><em>Table 1. Total number of annotated objects per class</em></caption> <thead> <tr> <th scope="col"><strong><em>Class</em></strong></th> <th scope="col"><strong><em>Annotated Objects</em></strong></th> </tr> </thead> <tbody> <tr> <td><em>Tree</em></td> <td>1922</td> </tr> <tr> <td><em>Waste Bin</em></td> <td><em>223</em></td> </tr> <tr> <td><em>Recycling Bin</em></td> <td>181</td> </tr> <tr> <td><em>Lighting Pole</em></td> <td><em>716</em></td> </tr> <tr> <td><em>Shop Storefront</em></td> <td><em>628</em></td> </tr> </tbody> </table> <p>The second .zip file is named "routes" and contains a folder named “routes” with consecutive frames of four different driving routes in the city of Thessaloniki and their corresponding GPS signal. So the folder “routes” contains 4 folders in the following format “VID_<YYYYMMDD>_<HHmmSS>” where Y denotes digits for year, M denotes digits for month, D denotes digits for day, H denotes digits for hour, m denotes digits for minutes and S denotes digits for seconds. Not the filename represents the start of the collection sequence. All street data was collected in 2022. Each route folder has the “images” folder which contains the consecutive street image data. Image data in this folder is in JPEG format. Each filename in ‘images’ has the frame_<id>.jpg format where id denotes the order of the frame. Table 2 shows more details regarding the number of frames and frame dimension of the driving routes.</p> <table align="center" summary="Total number of annotated objects per class"> <caption><em>Table 2. Total number frames and frame dimensions for each of the routes</em></caption> <thead> <tr> <th scope="col"><strong><em>Route Name</em></strong></th> <th scope="col"><strong><em>Frames Number</em></strong></th> <th scope="col"><strong><em>Frame Dimension</em></strong></th> <th scope="col"><strong><em>Route duration</em></strong></th> </tr> </thead> <tbody> <tr> <td><em>VID_20220617_111456</em></td> <td><em>41,650</em></td> <td><em>1080 x 2160</em></td> <td>1h, 9m, 26s</td> </tr> <tr> <td><em>VID_20220210_112926</em></td> <td><em>23.035</em></td> <td><em>866 x 2400</em></td> <td>38m, 26s</td> </tr> <tr> <td><em>VID_20220209_114831</em></td> <td><em>18.000</em></td> <td><em>1080 x 2160</em></td> <td>30m, 3s</td> </tr> <tr> <td><em>VID_20220209_123323</em></td> <td><em>18.273</em></td> <td><em>1080 x 2160</em></td> <td>30m, 30s</td> </tr> </tbody> </table> <p>Each route folder contains a “gps.json” file which contains latitude and longitude information for each frame. This file is essentially a JSON list of objects that each object contains the “frame_name” attribute and the corresponding “coordinates” object which contains the “latitude” and "longitude" attributes.</p>
Objective assessment of the relationship between quantified neuron morphological features and convolutional neural network image analysis
<p>Relevant data for the publication titled: Objective assessment of the relationship between quantified neuron morphological features and convolutional neural network image analysis</p>
VISIONE Feature Repository for VBS: Multi-Modal Features and Detected Objects from V3C1+V3C2 Dataset
<p>This repository contains a diverse set of features extracted from the V3C1+V3C2 dataset, sourced from the Vimeo Creative Commons Collection. These features were utilized in the VISIONE system [Amato et al. 2023, Amato et al. 2022] during the latest editions of the Video Browser Showdown (VBS) competition (<a href="https://www.videobrowsershowdown.org/">https://www.videobrowsershowdown.org/</a>).</p> <p>The original V3C1+V3C2 dataset, provided by NIST, can be downloaded using the instructions provided at <a href="https://videobrowsershowdown.org/about-vbs/existing-data-and-tools/">https://videobrowsershowdown.org/about-vbs/existing-data-and-tools/</a>.</p> <p>It comprises 7,235 video files, amounting for 2,300h of video content and encompassing 2,508,113 predefined video segments.</p> <p>We subdivided the predefined video segments longer than 10 seconds into multiple segments, with each segment spanning no longer than 16 seconds. As a result, we obtained a total of 2,648,219 segments. For each segment, we extracted one frame, specifically the middle one, and computed several features, which are described in detail below.</p> <p>This repository is released under a Creative Commons Attribution license. If you use it in any form for your work, please cite the following paper:</p> <blockquote> <pre>@inproceedings{amato2023visione, title={VISIONE at Video Browser Showdown 2023}, author={Amato, Giuseppe and Bolettieri, Paolo and Carrara, Fabio and Falchi, Fabrizio and Gennaro, Claudio and Messina, Nicola and Vadicamo, Lucia and Vairo, Claudio}, booktitle={International Conference on Multimedia Modeling}, pages={615--621}, year={2023}, organization={Springer} }</pre> </blockquote> <p> </p> <p>This repository comprises the following files:</p> <ul> <li><strong><em>msb.tar.gz </em></strong> contains tab-separated files (.tsv) for each video. Each tsv file reports, for each video segment, the timestamp and frame number marking the start/end of the video segment, along with the timestamp of the extracted middle frame and the associated identifier ("id_visione"). </li> <li><em><strong>extract-keyframes-from-msb.tar.gz</strong></em> contains a Python script designed to extract the middle frame of each video segment from the MSB files. To run the script successfully, please ensure that you have the original V3C videos available.</li> <li><strong><em>features-aladin.tar.gz<sup>†</sup></em> </strong>contains <a href="https://github.com/mesnico/ALADIN">ALADIN</a> [Messina N. et al. 2022] features extracted for all the segment's middle frames. </li> <li><em><strong>features-clip-laion.tar.gz<sup>†</sup></strong></em> contains <a href="https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K">CLIP ViT-H/14 - LAION-2B </a>[Schuhmann et al. 2022] features extracted for all the segment's middle frames.</li> <li><em><strong>features-clip-openai.tar.gz<sup>†</sup> </strong></em>contains <a href="https://huggingface.co/openai/clip-vit-large-patch14">CLIP ViT-L/14</a> [Radford et al. 2021] features extracted for all the segment's middle frames. </li> <li><em><strong>features-clip2video.tar.gz<sup>†</sup> </strong></em>contains <a href="https://github.com/CryhanFang/CLIP2Video">CLIP2Video</a> [Fang H. et al. 2021] extracted for all the video segments. <strong> </strong>In particular 1) we concatenate consecutive short segments so to create segments at least 3 seconds long; 2) we downsample the obtained segments to 2.5 fps; 3) we feed the network with the first min(36, n) frames, where n is the number of frames of the segment. Notice that the minimum processed length consists of 7 frames, given that the segment is no shorter than 3s. </li> <li><em><strong>objects-frcnn-oiv4.tar.gz<sup>*</sup> </strong></em>contains the objects detected using <a href="http://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1">Faster R-CNN+Inception ResNet</a> (trained on the Open Images V4 [Kuznetsova et al. 2020]). </li> <li><em><strong>objects-mrcnn-lvis.tar.gz<sup>*</sup></strong></em> contains the objects detected using Mask R-CNN [He et al. 2017] (trained on LVIS).</li> <li><em><strong>objects-vfnet64-coco.tar.gz<sup>*</sup></strong></em> contains the objects detected using VfNet [Zhang et al. 2021] (trained on COCO dataset).</li> </ul> <p>*Please be sure to use the <strong>v2 version </strong>of this repository, since v1 feature files may contain inconsistencies that have now been corrected</p> <p><em><strong>*Note on the object annotations:</strong></em> Within an object archive, there is a jsonl file for each video, where each row contains a record of a video segment (the <em>"_id"</em> corresponds to the <em>"id_visione"</em> used in the msb.tar.gz) . Additionally, there are three arrays representing the objects detected, the corresponding scores, and the bounding boxes. The format of these arrays is as follows:</p> <ul> <li><em>"object_class_names"</em>: vector with the class name of each detected object.</li> <li><em>"object_scores"</em>: scores corresponding to each detected object.</li> <li><em>"object_boxes_yxyx"</em>: bounding boxes of the detected objects in the format <em>(ymin, xmin, ymax, xmax).</em></li> </ul> <p> </p> <p><em><strong><sup>†</sup>Note on the cross-modal features: </strong></em>The extracted multi-modal features (ALADIN, CLIPs, CLIP2Video) enable internal searches within the V3C1+V3C2 dataset using the query-by-image approach (features can be compared with the dot product). However, to perform searches based on free text, the text needs to be transformed into the joint embedding space according to the specific network being used. Please be aware that t<strong>he service for transforming text into features is not provided within this repository and should be developed independently using the original feature repositories linked above.</strong></p> <p>We have plans to release the code in the future, allowing the reproduction of the VISIONE system, including the instantiation of all the services to transform text into cross-modal features. However, this work is still in progress, and the code is not currently available.</p> <p> </p> <p><strong>References:</strong></p> <p>[Amato et al. 2023] Amato, G.et al., 2023, January. VISIONE at Video Browser Showdown 2023. In International Conference on Multimedia Modeling (pp. 615-621). Cham: Springer International Publishing.</p> <p>[Amato et al. 2022] Amato, G. et al. (2022). VISIONE at Video Browser Showdown 2022. In: , et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13142. Springer, Cham. </p> <p>[Fang H. et al. 2021] Fang H. et al., 2021. Clip2video: Mastering video-text retrieval via image clip. arXiv preprint arXiv:2106.11097.</p> <p>[He et al. 2017] He, K., Gkioxari, G., Dollár, P. and Girshick, R., 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969).</p> <p>[Kuznetsova et al. 2020] Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A. and Duerig, T., 2020. The open images dataset v4. International Journal of Computer Vision, 128(7), pp.1956-1981.</p> <p>[Lin et al. 2014] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P. and Zitnick, C.L., 2014, September. Microsoft coco: Common objects in context. In European conference on computer vision (pp. 740-755). Springer, Cham.</p> <p>[Messina et al. 2022] Messina N. et al., 2022, September. Aladin: distilling fine-grained alignment scores for efficient image-text matching and retrieval. In Proceedings of the 19th International Conference on Content-based Multimedia Indexing (pp. 64-70).</p> <p>[Radford et al. 2021] Radford A. et al., 2021, July. Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PMLR.</p> <p>[Schuhmann et al. 2022] Schuhmann C. et al., 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35, pp.25278-25294.</p> <p>[Zhang et al. 2021] Zhang, H., Wang, Y., Dayoub, F. and Sunderhauf, N., 2021. Varifocalnet: An iou-aware dense object detector. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8514-8523).</p>
Rotating Features for Object Discovery
<p>This repository contains the 4Shapes datasets from the paper "Rotating Features for Object Discovery" (<a href="https://arxiv.org/abs/2306.00600">link</a>) by Sindy Löwe, Phillip Lippe, Francesco Locatello and Max Welling.</p> <p>The <strong>4Shapes dataset</strong> comprises grayscale images of dimensions 32 x 32, each containing four distinct white shapes (square, up/downward facing triangle, circle) on a black background. The dataset consists of 50,000 images in the train set, and 10,000 images for the validation and test sets, respectively. All pixel values fall within the range [0,1].</p> <p>The <strong>4Shapes RGB(-D)</strong> datasets follow the same general setup, but randomly samples the color of each shape. To achieve this, we create sets of potential colors with varying sizes. Each set is generated by uniformly sampling an offset value within the range [0,1], and subsequently producing different colors by evenly dividing the hue space, starting from this offset value. The saturation and value are set to one for all colors, and the resulting HSV color representations are converted to RGB. To create the RGB-D variant, we incorporate a depth channel to each image and assign a unique depth value within the range [0,1] to every object, maintaining equal distances between them.</p> <p>For more details on how to use these datasets, see <a href="https://github.com/loeweX/RotatingFeatures/">our GitHub repository</a>.</p> <p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.