Find research datasets worth reusing

Tesseract OCR of IIT-CDIP Dataset

<p>This is&nbsp;Tesseract generated&nbsp;<strong>transcriptions (no&nbsp;images)</strong>&nbsp;of (most of) the IIT-CDIP dataset. To download the images of the IIT-CDIP dataset go to&nbsp;<a href="https://data.nist.gov/od/id/mds2-2531">https://data.nist.gov/od/id/mds2-2531</a>&nbsp;</p> <p>The directory struture of this dataset is the same as the IIT-CDIP dataset (although has everything in one tar, with &quot;a.a&quot;, &quot;a.b&quot;, ... directories)&nbsp;and can thus be combine with the image&nbsp;IIT-CDIP dataset&nbsp;using rsync or similar tool. This dataset&nbsp;contains a &quot;X.layout.json&quot; for each &quot;X.png&quot; in the IIT-CDIP dataset (doesn&#39;t have sections &#39;a&#39;, &#39;w&#39;, &#39;x&#39;, &#39;y&#39;, and &#39;z&#39;).</p> <p>The jsons contain block/paragraph, line and word bounding boxes, with&nbsp;transcriptions for the words following the Tesseract format. The line and word annotations are directly taken from Tesseract. The block and paragraph output of Tesseract was discarded. The images were then run through both the&nbsp;Publaynet&nbsp;and PrimaNet&nbsp;models available on LayoutParser (<a href="https://layout-parser.github.io/">https://layout-parser.github.io/</a>). The combine output of these models became the block/paragraph annotations (we kept the Tesseract output format, but each block has 1 paragraph of exactly the same shape).</p> <p><strong>Important:</strong> There is also a &quot;rotation&quot; value in the json (0, 90, 180, or 270) indicating the json may be&nbsp;for a rotated version of the IIT-CDIP image by the given amount (attempted to rotated documents to upright position to get better OCR results).</p> <p>These are the annotations used to pre-train <strong>Dessurt</strong> (<a href="https://arxiv.org/abs/2203.16618">https://arxiv.org/abs/2203.16618</a>).</p> <p>These annotations will be worse than&nbsp;those that would be obtained&nbsp;using&nbsp;a commercial OCR system&nbsp;(like those used to pre-train LayoutLMv2/v3).</p> <p>The code used to produce these&nbsp;annotations is available here:&nbsp;<a href="https://github.com/herobd/ocr">https://github.com/herobd/ocr</a></p>