Datasets for "Reading Order Independent Metrics for Information Extraction in Handwritten Documents"

<p>This repository includes the five datasets used for our paper entitled <em>Reading Order Independent Metrics for Information Extraction in Handwritten Documents</em>, in which we compare various metrics to evaluate end-to-end information extraction from scanned documents.</p> <h2>Datasets</h2> <p>Five datasets are released following the BIO format:</p> <ul> <li>IAM</li> <li>Simara</li> <li>POPP</li> <li>Esposalles</li> <li>French Military Records</li> </ul> <p>For each dataset, we provide the following data (on test sets):</p> <ul> <li>Ground truth annotations (<code>gt/</code>)</li> <li>Automatic predictions (<code>dan/</code>)</li> <li>Automatic predictions with entities appearing in random order (<code>dan_shuffled/</code>)</li> </ul> <p>The data is organized as follows:</p> <p><code>├── Dataset name/</code><br><code>│ &nbsp; ├── gt/</code><br><code>│ &nbsp; ├── dan/</code><br><code>│ &nbsp; └── dan_shuffled/</code></p> <h2>Metrics</h2> <p>To install the <a href="https://pypi.org/project/ie-eval/"><code>ie-eval</code></a> package, run <code>pip install ie-eval</code>.</p> <p>To compute all metrics on a specific dataset, run:<br><br><code>ie-eval all --label-dir IAM_paragraph/gt/ --prediction-dir IAM_paragraph/dan/</code><br><br></p> <p>To learn more about the various options, use the <code>--help</code> argument or read the <a href="https://ie-eval-ner-metrics-050f40e80b04480e2310d39ad338de778f6bec80e18.pages.teklia.com/">documentation</a>.</p> <p>&nbsp;</p>