Robust-Minisets
<h3>Abstract</h3> <div>We introduce <strong>Robust-Minisets</strong>, a collection of robust benchmark classification datasets in the low resolution realm based on well-established image classification benchmarks, such as CIFAR, Tiny ImageNet, EuroSAT and the MedMNIST collection. We port existing robustness and generalization benchmarks (ImageNet-C, -R, -A and v2) to the small dataset domain introducing novel benchmarks to comprehensively evaluate the robustness and generalization capabilities of image classification models on low resolution datsets. This results in an extensive collection consisting of already existing test sets (e.g. CIFAR-10.1 and Tiny ImageNet-C) as well as the novel benchmarks EuroSAT-C, MedMNIST-C, and Tiny ImageNet-A, -R and -v2 introduced in our ICPR 2024 paper <a href="https://github.com/CeMOS-IS/GenFormer">"GenFormer - Generated Images are All You Need to Improve Robustness of Transformers on Small Datasets"</a>.</div> <div> </div> <h3>Installation and Requirements</h3> <div> <p>We recommend our official <a href="https://github.com/CeMOS-IS/Robust-Minisets">code</a> to download, parse and use the Robust-Minisets datasets:</p> <blockquote> <pre>% pip install robust-minisets<br>% python</pre> <div> <div>To use a standard version utilizing the downloaded files:</div> <br> <div>>>> from robust_minisets import TinyImageNetR</div> <div>>>> test_dataset = TinyImageNetR(split="test")</div> <br> <div>To enable automatic downloading by setting `download=True`:</div> <br> <div>>>> from robust_minisets import EuroSATC</div> <div>>>> test_dataset = EuroSATC(split="test", download=True)</div> <br> <div>Additionally, we include training and validation datasets that are not provided in common hubs (e.g. torchvision):</div> <br> <div>>>> from robust_minisets import EuroSAT</div> <div>>>> train_dataset = EuroSAT(split="train", download=True)</div> <div>>>> val_dataset = EuroSAT(split="val", download=True)</div> <div>>>> test_dataset = EuroSAT(split="test", download=True)</div> </div> </blockquote> </div> <p> </p> <h3>License</h3> <div>The code is under <a href="https://github.com/MedMNIST/MedMNIST/blob/main/LICENSE">Apache-2.0 License</a>.</div> <div> </div> <div>The publication licenses of the datasets can be found within the info dictionary of our official code via <code>robust_minisets.INFO[<dataset_flag>]</code> or <a href="https://github.com/CeMOS-IS/Robust-Minisets/blob/main/datasets.md">here</a>.</div> <p> </p> <h3><strong>Citation</strong></h3> <div>If you find this work useful, please consider citing us:</div> <blockquote> <div>Sven Oehri, Nikolas Ebert, Ahmed Abdullah, Didier Stricker, Oliver Wasenmüller. "GenFormer - Generated Images are All You Need to Improve Robustness on Small Datasets". International Conference on Pattern Recognition (ICPR), 2024.</div> </blockquote> <div>or using bibtex:</div> <blockquote>@inproceedings{oehri2024genformer,<br> title = {GenFormer – Generated Images are All You Need to Improve Robustness of Transformers on Small Datasets},<br> author = {Oehri, Sven and Ebert, Nikolas and Abdullah, Ahmed and Stricker, Didier and Wasenm{\"u}ller, Oliver},<br> booktitle = {International Conference on Pattern Recognition (ICPR)},<br> year = {2024},<br>}</blockquote> <p> </p> <div><strong>DISCLAIMER:</strong> Robust-Minisets is based on a wide range of existing datasets and benchmarks. Thus, please also cite source data paper(s) of the Robust-Minisets subset(s):</div> <div> <ul> <li><a href="https://arxiv.org/abs/1806.00451">CIFAR-10.1</a></li> <li><a href="https://arxiv.org/abs/1709.00029">EuroSAT</a></li> <li><a href="https://arxiv.org/abs/1907.07174">ImageNet-A</a></li> <li><a href="https://arxiv.org/abs/1903.12261">ImageNet-C</a></li> <li><a href="https://arxiv.org/abs/2006.16241">ImageNet-R</a></li> <li><a href="https://arxiv.org/abs/1902.10811">ImageNetv2</a></li> <li><a href="https://www.nature.com/articles/s41597-022-01721-8">MedMNIST</a>, the respective source datasets (described <a href="https://medmnist.com/">here)</a></li> </ul> </div> <div> </div> <h3>Release versions</h3> <div> <ul> <li>v1.0.0: Robust-Minisets v1 release</li> </ul> </div>
ShareScore
20/100
Overall dataset sharing score
Score breakdown
These five areas show where the dataset supports — or may limit — practical reuse.
- Stewardship
- 8
- Harmonization
- 4
- Access
- 8
- Reuse readiness
- 0
- Engagement
- 0
Topics
Dataset details
- Repository
- zenodo
- Published
- August 28, 2024
- License
- Not reported
- DOI
- 10.5281/zenodo.13382926
Scored with release 0.9.0
ScienceDex for faculty
Your academic CV and NIH biosketch
Make your CV in your school's format and your NIH biosketch from it, then download both as Word files.