Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models

<h2>[Update] Looking for a larger unsafe image dataset? We publish a new dataset named UnsafeBench on Hugging Face. Take a look at&nbsp;<a href="https://huggingface.co/datasets/yiting/UnsafeBench">here</a>!</h2> <p>This dataset used in the paper&nbsp; <a href="https://arxiv.org/pdf/2305.13873.pdf">https://arxiv.org/pdf/2305.13873.pdf</a>&nbsp;contains four prompt sets and one image set.</p> <p>The four prompt sets were used to query Text-to-Image models and generate images for safety assessment. These sets include three&nbsp;harmful prompt sets&nbsp;and one harmless prompt&nbsp;set. The harmful prompts originate from different sources and contain&nbsp;various unsafe concepts, such as sexually explicit, violent, disturbing, hateful, and political content.</p> <p><strong>Prompt Sets</strong>:</p> <ul> <li>4chan Prompts: Harmful</li> <li>Lexica Prompts: Harmful</li> <li>Template Prompts: Harmful</li> <li>COCO Prompts: Harmless</li> </ul> <p><strong>Image Dataset</strong>:</p> <p>This dataset consists of 800 images, which were randomly selected from all the generated images from Text-to-Image models.</p> <ul> <li>Safe: 580 images</li> <li>Sexually Explicit: 48 images</li> <li>Violent: 45 images</li> <li>Disturbing: 68 images</li> <li>Hateful: 35 images</li> <li>Political: 50 images</li> </ul>