Find research datasets worth reusing

opencc-by-4.0Nov 2024View details →

Replication Package of "Exploiting Vision-Language Models in GUI Reuse"

This replication package is for the paper entitled "Exploiting Vision-Language Models in GUI Reuse". The authors remain anonymous for double-blind review purposes. The package contains six files. If the paper is accepted, then the authors will move the replication package to a public repository hosted by an institution.

opencc-by-4.0Nov 2011View details →

Figure 2. Catadioptric projection modelled by the unit sphere-Design and Implementation of a Fully Autonomous UAV's Navigator Based on Omni-directional Vision System

Since the beginning of UAV, the map building was one of the most addressed problems by researchers. Several researchers used Omni directional vision for robot navigation and map building. Because of the wide field of view in Omni directional sensors, the robot does not need to look around using moving parts (cameras or mirrors) or turning the moving parts. The global view offered by Omni directional vision is especially suitable for highly dynamic environments. The Omni directional vision system consists of a hyperbolic mirror, a USB color digital camera (Logitech C905) and a regulation device.

opencc-by-4.0Jun 2024View details →

Transformers Model Zoos and Soups: A Population of Language and Vision Models

Model Zoos submitted to the NeurIPS 2024 Dataset & Benchmark track: "Transformer Model Zoos and Soups: A Population of Language and Vision Models" We generate two model zoos, one for computer vision built on the ViT-S architecture, and one for language modeling based on the BERT architecture. For each, we train several backbone models with varying hyperparameters, and further fine-tune them using multiple hyperparameter combinations. We further annotate every model with performance metrics. These include test accuracy and F1-score, as well as the generalization gap. For the vision models, we also include the robust accuracy after a FGSM attack.

opencc-by-4.0Jun 2024View details →

Vision-Transformer, ViT, model validation dataset

The U.S. cotton industry is highly concerned with removing plastic contamination from cotton lint. A major source of this contamination is the plastic used to wrap cotton modules produced by John Deere round module harvesters. A machine-vision detection and removal system has been developed to address this problem, using low-cost color cameras to detect plastic in the cotton stream and remove it. However, the system requires a lot of calibration and is difficult for cotton gin workers to operate due to its reliance on custom machine-vision classifier running on low-cost ARM computers running Linux. This research aims to make the system more user-friendly by adding an auto-calibration feature that can track cotton colors and avoid plastic images, reducing the need for skilled personnel to operate the system and making it easier for the cotton ginning industry to adopt. This image dataset was created to validate several Vision-Transformer, ViT, AI models that in combination provides the key enabling technology for the auto-calibration code.

opencc-byJan 2022View details →

Styracosaurus model made with Beholder Vision

Generated using Beholder Vision (https://beholder.vision), a cloud-based photogrammetry solution. Blog post here: https://peterfalkingham.com/2022/01/18/photogrammetry-testing-beholder-vision-cloud-based/ Source: Objaverse 1.0 / Sketchfab

opencc-by-4.0Apr 2024View details →

ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Models

We introduce a new image-text dataset, providing high-quality natural language descriptions for global-scale satellite data. Specifically, we utilize Sentinel-2 data for its global coverage as the foundational image source, employing semantic segmentation labels from the European Space Agency's WorldCover project to enrich the descriptions of land covers. By conducting in-depth semantic analysis, we formulate detailed prompts to elicit rich descriptions from ChatGPT. We then include a manual verification process to enhance the dataset's quality further. This step involves manual inspection and correction to refine the dataset. Finally, we offer the community ChatEarthNet, a large-scale image-text dataset characterized by global coverage, high quality, wide-ranging diversity, and detailed descriptions. ChatEarthNet consists of 163,488 image-text pairs with captions generated by ChatGPT-3.5 and an additional 10,000 image-text pairs with captions generated by ChatGPT-4V(ision). This dataset has significant potential for both training and evaluating vision-language geo-foundation models for remote sensing.

opencc-by-4.0Apr 2024View details →

Dataset, Model Statistics, and 3D designs for "From Eyes to Cameras: Computer Vision for High-Throughput Liquid-Liquid Separation"

Dataset, model statistics, and 3D design of high throughput platform associated with HeinSight3.0.    Pre-print: https://chemrxiv.org/engage/chemrxiv/article-details/65e5481f9138d231619c1879   The code and model of HeinSight3.0 can be found at (https://doi.org/10.5281/zenodo.11053915)

opencc-by-4.0Aug 2024View details →

Langmark: annotations for scenes with semantic inconsistencies connecting distributional semantic models to vision science – data and code

Data (including object annotations) and code from the following manuscript: Langmark: annotations for scenes with semantic inconsistencies connecting distributional semantic models to vision science.

opencc-by-4.0Dec 2022View details →

Figures from the paper "The diversity of canonical and ubiquitous progress in computer vision: A dynamic topic modeling approach"(v2))

Figures from the paper  "The diversity of canonical and ubiquitous progress in computer vision: A dynamic topic modeling approach". The second version: Corrections to Figure 1. (fig1-> fig_v2).

opencc-by-4.0Jun 2023View details →

AccessFixer: Enhancing GUI Accessibility for Low Vision Users with R-GCN model

Here is the relevant dataset and open-source code for the article titled "AccessFixer: Enhancing GUI Accessibility for Low Vision Users with RGCN-based Method" Introduction In this work, we designed and implemented a tool, named AccessFixer, capable of fixing GUI accessibility issues in terms of small size, narrow interval, and low color contrast. It can ensure the consistency of GUI visual effects in the repair process, and will not produce inconsistent components. Functions <ol> <li>Convert the GUIs screenshots to GUI-graphs. AccessFixer converts the GUIs into their corresponding GUI-graphs by parsing the XML file (in Graph_Presentation()), and meanwhile, our tool could remove all edges that connected the problematic nodes detected by Google Accessibility Scanner.</li> <li>Pre-trained model based on Relational-Graph Convolutional Neural Network (R-GCN). Along with Capture_Signal(model), users can obtain the spectral signals of predicted links. Encoder and Decoder are responsible for generating new edges and scoring edges.</li> <li>AccessFixer builds a mapping relationship between signal values and attribute values of GUI components in Attribute_Mapp(). Based on this mapping, it is possible to input a GUI to be repaired and generate repair strategies for its various components.</li> </ol> Environment This tool can be run on an Android emulator based on Android 11.0 on a typical development machine, using Windows 11 with 2.4GHz core i7 CPU and 16 GB memory. The pre-trained RGCN model is run in PyCharm 4.5.4 with the packages of tensorflow, GraphConvolution, pandas, time, numpy, argparse, optimization, Parameter, and Module. Installation <ol> <li>Using pip install package-name to install the required packages.</li> <li>Configure GCN model training in the same directory.</li> <li>Click the run button in PyCharm or use the command run-train.sh [configuration]</li> </ol> Frequently Asked Questions <ol> <li>Unable to install packages using pip</li> </ol> Answer: It could be the version of the pip install command. Detailed solution could be found in https://stackoverflow.com/questions/17869101/unable-to-install-pygame-using-pip/74229901#74229901 <ol> <li>Some GUIs fail to build GUI-graphs</li> </ol> Answer: To create GII-graphs, the user is required to provide the GUI screenshot and the layout file parsed by UIAutomator. Contact Information If you have any questions about this tool, you can contact the author of this work at <a href="mailto:zmxalakay@126.com">zmxalakay@126.com</a> Copyright All copyright of the tool is owned by the author of the paper.

opencc-by-4.0Mar 2024View details →

$swint vision model training data

Training language models to see using strings that represent pixelized images.

opencdla-permissive-1.0Dec 2023View details →

WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models

############# <h1>WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models</h1> ############# Authors: Valentin Gabeff, Marc Russwurm, Devis Tuia & Alexander Mathis Affiliation: EPFL Date: January, 2024 Link to the article: <a href="https://link.springer.com/article/10.1007/s11263-024-02026-6">https://link.springer.com/article/10.1007/s11263-024-02026-6</a> -------------------------------- WildCLIP is a fine-tuned CLIP model that allows to retrieve camera-trap events with natural language from the Snapshot Serengeti dataset. This project intends to demonstrate how vision-language models may assist the annotation process of camera-trap datasets. Here we provide the processed Snapshot Serengeti data used to train and evaluate WildCLIP, along with two versions of WildCLIP (model weights). Details on how to run these models can be found in the project <a href="https://github.com/amathislab/wildclip">github repository</a>. <h2>Provided data (images and attribute annotations): </h2> The data consists of 380 x 380 image crops corresponding to the MegaDetector output of Snapshot Serengeti with a confidence threshold above 0.7. We considered only camera trap images containing single individuals. A description of the original data can be found on LILA <a href="https://lila.science/datasets/snapshot-serengeti">here</a>, released under the <a href="https://cdla.dev/permissive-1-0/" rel="nofollow">Community Data License Agreement (permissive variant)</a>. We warmly thank the authors of LILA for making the MegaDetector outputs publicly available, as well as for structuring the dataset and facilitating its access. <h2>Adapted CLIP model (model weights): </h2> WildCLIP models provided: <ul> <li>[New] WildCLIP_vitb16_t1.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Trained on both base and novel vocabulary (see paper for details).</li> <li>[New] WildCLIP_vitb16_t1_lwf.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1, and with the additional VR-LwF loss. Trained on both base and novel vocabulary (see paper for details).</li> <li>WildCLIP_vitb16_t1_base.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following template 1. Model used for evaluation and trained on base vocabulary only. (previously named WildCLIP_vitb16_t1.pth)</li> <li>WildCLIP_vitb16_t1t7_lwf_base.pth: CLIP model with the ViT-B/16 visual backbone trained on data with captions following templates 1 to 7, and with the additional VR-LwF loss. Model used for evaluation and trained on base vocabulary only. (previously named WildCLIP_vitb16_t1t7_lwf.pth)</li> </ul> We also provide the CSV files containing the train / val / test splits. The train / test splits follow camera split from LILA (https://lila.science/datasets/snapshot-serengeti). The validation split is custom, and also at the camera level. <ul> <li>train_dataset_crops_single_animal_template_captions_T1T7_ID.csv: Train set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1")</li> <li>val_dataset_crops_single_animal_template_captions_T1T7_ID.csv: Validation set with captions from templates 1 through 7 (column "all captions") or template 1 only (column "template 1")</li> <li>test_dataset_crops_single_animal_template_captions_T1T8T10.csv: Test set with captions from templates 1, 8, 9 and 10 (columns "all captions")</li> </ul> Details on how the models were trained can be found in the associated <a href="https://link.springer.com/article/10.1007/s11263-024-02026-6" target="_blank" rel="noopener">publication</a>. <h2>References: </h2> If you find our code, or weights, please cite: <pre>@article{gabeff2024wildclip, title={WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models}, author={Gabeff, Valentin and Ru{\ss}wurm, Marc and Tuia, Devis and Mathis, Alexander}, journal={International Journal of Computer Vision}, pages={1--17}, year={2024}, publisher={Springer} }</pre> If you use the adapted Snapshot Serengeti data please also cite their article: <pre>@article{swanson2015snapshot, title={Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna}, author={Swanson, Alexandra and Kosmala, Margaret and Lintott, Chris and Simpson, Robert and Smith, Arfon and Packer, Craig}, journal={Scientific data}, volume={2}, number={1}, pages={1--14}, year={2015}, publisher={Nature Publishing Group} }</pre>

opencc-by-4.0Jul 2021View details →

COMSOL models for Electronic "photoreceptors" enable prosthetic vision with acuity matching the natural resolution in rats

The in-vitro and in-vivo COMSOL models described in paper Electronic “photoreceptors” enable prosthetic vision with acuity matching the natural resolution in rats. <a href="https://zenodo.org/api/files/60e7380c-a31e-429c-a337-0a8b267d6bcc/Flat_pdish_final.mph">Flat_pdish_final.mph</a>: In-vitro model of the photovoltaic array tested in a Petri dish filled with 10x diluted saline water. <a href="https://zenodo.org/api/files/60e7380c-a31e-429c-a337-0a8b267d6bcc/Flat_RatEye_final.mph">Flat_RatEye_final.mph</a>: In-vivo model of the photovoltaic array implanted in the subretinal space of an RCS rat. The rat eye is anatomically realistic, while the head is replaced by a cylinder for computational tractability.

Figure 3 in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Figure 3. Example of Grad-CAM heatmaps obtained for Podarcis lusitanicus. The upper images show two common patterns observed in male dorsal images (also found, albeit with some differences, in females). The bottom images exhibit the patterns most frequently found in male and female head lateral images (here illustrated in two females).

Figure 2 in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Figure 2. Confusion matrix for male (upper) and female (lower) image classification for the two-class case based on a combination of predictions from six models. Abbreviations used: Pboc, P. bocagei; Plus, P. lusitanicus.

Highly variable (no clear pattern). All portions of the dorsal views were equally used. In head images the area around the eye, the top of the head, the snout and the throat were all used in similar proportions. P. carbonelli Variable for both views. Snout and middle of the dorsum used in dorsal view. Top of the head most frequently (but not strictly) used in lateral view. P. guadarramae Whole body used for dorsal view (but variable); either throat (most common) or ear region used in head lateral views. P. hispanicus Variable. Anterior portion of snout used more frequently than in other species for both dorsal and head lateral views. P. liolepis Highly variable. Whole body used in most dorsal images, area around the eye and throat used in head lateral views, but other patterns common. P. lusitanicus Highly variable. All parts of the dorsum used (but frequently the most posterior part); area around the ear frequently used in head lateral images. P. tunesiacus Highly variable. Dorsal area near the insertion of the posterior limbs used more frequently than in other species; different regions of the head used, often simultaneously. P. Ʋaucheri Highly variable. Different regions of dorsum (from head to the posterior region) used in dorsal images, all portions of the head, but most frequently the throat, used in lateral images. P. Ʋirescens Highly variable. All parts of both images used. Head and anterior part of the dorsum more used than in other species. in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Highly variable (no clear pattern). All portions of the dorsal views were equally used. In head images the area around the eye, the top of the head, the snout and the throat were all used in similar proportions. P. carbonelli Variable for both views. Snout and middle of the dorsum used in dorsal view. Top of the head most frequently (but not strictly) used in lateral view. P. guadarramae Whole body used for dorsal view (but variable); either throat (most common) or ear region used in head lateral views. P. hispanicus Variable. Anterior portion of snout used more frequently than in other species for both dorsal and head lateral views. P. liolepis Highly variable. Whole body used in most dorsal images, area around the eye and throat used in head lateral views, but other patterns common. P. lusitanicus Highly variable. All parts of the dorsum used (but frequently the most posterior part); area around the ear frequently used in head lateral images. P. tunesiacus Highly variable. Dorsal area near the insertion of the posterior limbs used more frequently than in other species; different regions of the head used, often simultaneously. P. Ʋaucheri Highly variable. Different regions of dorsum (from head to the posterior region) used in dorsal images, all portions of the head, but most frequently the throat, used in lateral images. P. Ʋirescens Highly variable. All parts of both images used. Head and anterior part of the dorsum more used than in other species.

Figure 1 in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Figure 1. The two image types analysed in this study (before pre-processing): above, a dorsal view; below, a head lateral image. Both images correspond to the same Podarcis Ʋaucheri s.l. male.

Highly variable. Mid-portion of the dorsum used frequently (although other areas as well). Tip of the snout used often, but area around the ear and throat are also relevant. P. carbonelli Variable. In the dorsal view, the tip of the snout is frequently used. In the head lateral view, the tip of the snout is also com- monly used, as well as the most posterior region of the head. P. guadarramae Variable. Mid portion of the dorsum and tip of the snout are the regions used more frequently in dorsal and head lateral views, respectively. P. hispanicus Variable. The head and most anterior part of the dorsum are frequently used in the dorsal view. Snout and/or top of posterior region of head used. P. liolepis Variable. Different parts of the dorsum are used, whereas the tip of the snout is used in most head lateral images. P. lusitanicus Anterior dorsum, in the dorsal view, and both snout and posterior side of the head (in head lateral views) frequently used. P. tunesiacus Variable. Tip of the snout and posterior part of the trunk more used than in other species; snout and top head region behind the eye used with some frequency. P. Ʋaucheri Highly variable. All parts of the dorsum used in dorsal images, various parts of the head (but frequently snout and throat combined) used in head lateral images. P. Ʋirescens Highly variable. All portions of the dorsum used in dorsal images, region around and behind the ear more used than in other species for head lateral images. in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Highly variable. Mid-portion of the dorsum used frequently (although other areas as well). Tip of the snout used often, but area around the ear and throat are also relevant. P. carbonelli Variable. In the dorsal view, the tip of the snout is frequently used. In the head lateral view, the tip of the snout is also com- monly used, as well as the most posterior region of the head. P. guadarramae Variable. Mid portion of the dorsum and tip of the snout are the regions used more frequently in dorsal and head lateral views, respectively. P. hispanicus Variable. The head and most anterior part of the dorsum are frequently used in the dorsal view. Snout and/or top of posterior region of head used. P. liolepis Variable. Different parts of the dorsum are used, whereas the tip of the snout is used in most head lateral images. P. lusitanicus Anterior dorsum, in the dorsal view, and both snout and posterior side of the head (in head lateral views) frequently used. P. tunesiacus Variable. Tip of the snout and posterior part of the trunk more used than in other species; snout and top head region behind the eye used with some frequency. P. Ʋaucheri Highly variable. All parts of the dorsum used in dorsal images, various parts of the head (but frequently snout and throat combined) used in head lateral images. P. Ʋirescens Highly variable. All portions of the dorsum used in dorsal images, region around and behind the ear more used than in other species for head lateral images.

Figure 4 in Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study

Figure 4. Confusion matrix for male (upper) and female (lower) image classification for the nine-class experiment based on a combination of predictions from six models. Abbreviations used: Pboc, P. bocagei; Pcar, P. carbonelli; Phis, P. hispanicus; Plio, P. liolepis; Plus, P. lusitanicus; Pvsl, P. Ʋaucheri s.l.; Pvss, P. Ʋaucheri s.s.; Pvir, P. Ʋirescens.