Synthetic Airborne Intruder Dataset: A dataset based on High-Resolution Inpainting for Safety Critical Detect and Avoid

<p>Modern machine learning techniques have shown tremendous potential, especially for object detection on camera images. For this reason, they are also used to enable safety-critical automated processes such as autonomous drone flights. We present a study on object detection for Detect and Avoid, a safety critical function for drones that detects air traffic during automated flights for safety reasons. An ill-posed problem is the generation of good and especially large data sets, since detection itself is the corner case. Most models suffer from limited ground truth in raw data, e.g. recorded air traffic or frontal flight with a small aircraft. It often leads to poor and critical detection rates. We overcome this problem by using inpainting methods to bootstrap the dataset such that it explicitly contains the corner cases of the raw data. We provide an overview of inpainting methods and generative models and present an example pipeline given a small annotated dataset. We validate our method by generating a high-resolution dataset and present it to an independent object detector that was fully trained on real data.</p> <p>This dataset is represented in the following repository. The dataset is structured as follows:</p> <p># Synthetic Airborne Intruder Dataset</p> <p>This dataset was syntheticaly generated using an adapted [Pix2Pix](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) with different background images and object segementations. Each image contains one object instance.</p> <p>The annotations are in the COCO annotation format.</p> <p>## Data Structure<br> &nbsp; &nbsp; Synthetic Dataset Root:<br> &nbsp; &nbsp; &nbsp; &nbsp; --train<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--images<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--instances.json<br> &nbsp; &nbsp; &nbsp; &nbsp; --val<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--images<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--instances.json<br> &nbsp; &nbsp; &nbsp; &nbsp; --test<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--images<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--instances.json<br> &nbsp; &nbsp; &nbsp; &nbsp; --Background_Sources<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--sources_train.csv<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--sources_val.csv<br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; |--sources_test.cs<br> &nbsp; &nbsp; &nbsp; &nbsp; --README.md</p> <p>## Categories</p> <p>| Id | Name | Instances over all splits |<br> | ---| --- | --- |<br> | 0 | large airplane | 1695 |&nbsp;<br> | 1 | small airplane | 1255 |&nbsp;<br> | 2 | very small airplane | 46 |&nbsp;<br> | 3 | helicopter | 2201 |&nbsp;<br> | 4 | drone | 961 |<br> | 5 | hot air balloon | 315 |<br> | 6 | paraglider | 565 |<br> | 7 | airship | 42 |<br> | 8 | UFO | 0 |</p> <p>### Note:<br> UFO is a placeholder for future expansion of the dataset.</p> <p>## Splits<br> The dataset consists of 3 splits: train 5900 images, val 590 images, test 590 images.<br> The Number of instances per class and per split can be seen in the table below:</p> <p>Class | train | val | test<br> -------|-------|-----|--------<br> large airplane | 1416 | 142 | 137<br> small airplane | 1046 | 96 | 113<br> very small airplane | 38 | 2 | 6<br> helicopter | 1812 | 206 | 183<br> drone | 800 | 86 | 75<br> hot air balloon | 268 | 21 | 26<br> paragliders | 492 | 32 | 41<br> airship | 28 | 5 | 9<br> UFO | 0 | 0 | 0</p> <p>## Sources<br> The sources of the background images can be found in the files [here](./Background_Sources/).</p>