Domain-adaptive Data Synthesis for Large-scale Supermarket Product Recognition

<p><strong>Domain-Adaptive Data Synthesis for Large-Scale Supermarket Product Recognition</strong></p> <p>This repository contains the data synthesis pipeline and synthetic product recognition datasets proposed in [1].</p> <p><strong>Data Synthesis Pipeline:</strong></p> <p>We provide the Blender 3.1 project files and Python source code of our data synthesis pipeline <em>pipeline.zip,&nbsp;</em>accompanied by the<em>&nbsp;</em><a href="https://github.com/taesungp/contrastive-unpaired-translation">FastCUT</a> models used for synthetic-to-real domain translation<em>&nbsp;models.zip</em>. For the synthesis of new shelf images, a product assortment list and product images must be provided in the corresponding directories <em>products/assortment/</em> and <em>products/img/</em>. The pipeline expects product images to follow the naming convention <em>c</em>.png, with <em>c</em> corresponding to a GTIN or generic class label (e.g., 9120050882171.png). The assortment list, <em>assortment.csv</em>, is expected to use the sample format [<em>c, w, d, h</em>], with <em>c</em> being the class label and <em>w, d,</em> and <em>h</em> being the packaging dimensions of the given product in mm (e.g., [4004218143128, 140, 70, 160]). The assortment list to use and the number of images to generate can be specified in <em>generateImages.py </em>(see comments). The rendering process is initiated by either executing&nbsp;<em>load.py</em> from within Blender or within a command-line terminal as a background process.&nbsp;</p> <p><strong>Datasets:</strong></p> <ul> <li><strong>SG3k</strong> -&nbsp;Synthetic GroZi-3.2k (SG3k) dataset, consisting of 10,000 synthetic shelf images with 851,801 instances of 3,234&nbsp;GroZi-3.2k products.&nbsp;Instance-level bounding boxes and generic class labels are provided for all product instances.</li> <li><strong>SG3kt</strong>&nbsp;-&nbsp;Domain-translated version of&nbsp;SGI3k, utilizing GroZi-3.2k as the target domain.&nbsp;Instance-level bounding boxes and generic class labels are provided for all product instances.</li> <li><strong>SGI3k</strong> -&nbsp;Synthetic GroZi-3.2k (SG3k) dataset, consisting of 10,000 synthetic shelf images with 838,696&nbsp;instances of 1,063&nbsp;GroZi-3.2k products.&nbsp;Instance-level bounding boxes and&nbsp;generic class labels&nbsp;are provided for all product instances.</li> <li><strong>SGI3kt</strong>&nbsp;-&nbsp;Domain-translated version of&nbsp;SGI3k, utilizing GroZi-3.2k as the target domain.&nbsp;Instance-level bounding boxes and&nbsp;generic class labels are provided for all product instances.</li> <li><strong>SPS8k</strong> - Synthetic Product Shelves 8k (SPS8k) dataset, comprised&nbsp;of 16,224 synthetic shelf images with 1,981,967 instances of 8,112 supermarket products. Instance-level bounding boxes and GTIN class labels are provided for all product instances.</li> <li><strong>SPS8kt</strong>&nbsp;- Domain-translated version of&nbsp;SPS8k, utilizing&nbsp;SKU110k as the target domain.&nbsp;Instance-level bounding boxes and GTIN class labels for all product instances.</li> </ul> <p>Table 1: Dataset characteristics.&nbsp;</p> <table> <tbody> <tr> <td><strong>Dataset</strong></td> <td><strong>#images</strong></td> <td><strong>#products</strong></td> <td><strong>#instances</strong></td> <td>&nbsp;&nbsp;<strong>labels &nbsp; &nbsp; </strong></td> <td><strong>translation</strong></td> </tr> <tr> <td>SG3k</td> <td>10,000</td> <td>3,234</td> <td>851,801</td> <td>bounding box &amp; generic class&sup1;</td> <td>none</td> </tr> <tr> <td>SG3kt</td> <td>10,000</td> <td>3,234</td> <td>851,801</td> <td>bounding box &amp; generic class&sup1;</td> <td>GroZi-3.2k</td> </tr> <tr> <td>SGI3k</td> <td>10,000</td> <td>1,063</td> <td>838,696</td> <td>bounding box &amp; generic class&sup2;</td> <td>none</td> </tr> <tr> <td>SGI3kt</td> <td>10,000</td> <td>1,063</td> <td>838,696</td> <td>bounding box &amp; generic class&sup2;</td> <td>GroZi-3.2k</td> </tr> <tr> <td>SPS8k</td> <td>16,224</td> <td>8,112</td> <td>1,981,967</td> <td>bounding box &amp; GTIN</td> <td>none</td> </tr> <tr> <td>SPS8kt</td> <td>16,224</td> <td>8,112</td> <td>1,981,967</td> <td>bounding box &amp; GTIN</td> <td>SKU110k</td> </tr> </tbody> </table> <p>&nbsp;</p> <p><strong>Sample Format</strong></p> <p>A sample consists of an RGB image (i.png) and an accompanying label file (i.txt), which contains the labels for all product instances present in the image. Labels use the YOLO format&nbsp;[c, x, y, w, h].</p> <p>&sup1;SG3k and&nbsp;SG3kt&nbsp;use generic pseudo-GTIN&nbsp;class labels, created&nbsp;by combining the&nbsp;GroZi-3.2k food product category number <em>i</em> (1-27) with the product image index <em>j </em>(j.jpg)<em>, </em>following the convention<em>&nbsp;i0000j </em>(e.g., 13000097).</p> <p>&sup2;SGI3k and&nbsp;SGI3kt&nbsp;use the generic&nbsp;GroZi-3.2k class labels from&nbsp;<a href="https://arxiv.org/abs/2003.06800">https://arxiv.org/abs/2003.06800</a>.</p> <p><strong>Download and Use</strong><br>This data may be used for non-commercial research purposes only.&nbsp;If you publish material based on this data, we request that you include a reference to our paper [1].</p> <p>[1] Strohmayer, Julian, and Martin Kampel. "Domain-Adaptive Data Synthesis for Large-Scale Supermarket Product Recognition."&nbsp;<em>International Conference on Computer Analysis of Images and Patterns</em>. Cham: Springer Nature Switzerland, 2023.</p> <p>BibTeX&nbsp;citation:</p> <pre>@inproceedings{strohmayer2023domain, title={Domain-Adaptive Data Synthesis for Large-Scale Supermarket Product Recognition}, author={Strohmayer, Julian and Kampel, Martin}, booktitle={International Conference on Computer Analysis of Images and Patterns}, pages={239--250}, year={2023}, organization={Springer} }</pre>