Find research datasets worth reusing

opencc-by-4.0Nov 2020View details →

Transkribus - Handwritten Text Recognition for Premodern Documents (SIMS 2020 Lightning Talk)

Transkribus is a platform for text recognition and can be used via the Transkribus Expert Software (available after registration: transkribus.eu). Through Transkribus different tools for document analysis and text recognition can be directly applied. The intro demonstrates very briefly how Transkribus can help with regards to premodern documents especially since a variety of pre-trained models are already available: for Latin (prints and handwriting), for early modern vernaculars in French, Dutch, English, and German. For more information go to transkribus.eu. Presented as a Schoenberg Symposium 2020 Lightning Talk

opencc-by-4.0May 2022View details →

Ground-Truthed Data Set of Zenon Papyri for Handwritten Text Recognition

Diplomatic transcription of papyri found in the Zenon archive [see <a href="https://en.wikipedia.org/wiki/Zenon_of_Kaunos">en.wikipedia.org/wiki/Zenon_of_Kaunos</a>] Manually prepared as PageXML with Transkribus within <a href="http://d-scribes.philhist.unibas.ch/">D-Scribes</a> project.

opencc-by-4.0May 2024View details →

A Brazilian Portuguese Dataset for Offline Handwritten Text Recognition (BRESSAY)

The BRESSAY dataset comprises images of handwritten essays in Brazilian Portuguese, which present a series of challenges to optical recognition models. These images were sourced from multiple online platforms, limiting our ability to standardize the capture process. Due to these varied sources and the lack of a uniform collection method, the dataset provides a realistic reflection of real-world conditions. Each essay is unique, contributed by different writers, and addresses a specific content topic. Furthermore, the constraints placed on the writers often lead to various handwriting scenarios, including hard-to-read words, connected words, noise, overwriting, and struck-through texts. <h3>Technical Details</h3> The BRESSAY dataset represents a comprehensive collection of handwritten essays in Brazilian Portuguese, offering detailed insights into various handwriting scenarios. It covers a total of 1,000 pages, each contributed by a unique writer, resulting in 1,000 distinct handwriting styles. This aspect of the dataset adds a layer of diversity, which is further emphasized by the total of 4,214 paragraphs, 30,090 lines, and 416,826 words. Regarding unique tokens, we have 41,318 unique words, and 107 unique characters. <h3>Data Structure</h3> The dataset is organized as follows: <ul> <li>data/: Main folder containing segmented essay images <ul> <li>lines/: Images of individual lines <ul> <li>  PNG files: Line images</li> <li>  TXT files: Transcriptions of lines</li> </ul> </li> <li>pages/: Full page essay images <ul> <li>  PNG files: Page images</li> <li>  TXT files: Transcriptions of pages</li> </ul> </li> <li>paragraphs/: Images of paragraphs <ul> <li>  PNG files: Paragraph images</li> <li>  TXT files: Transcriptions of paragraphs</li> </ul> </li> <li>words/: Images of individual words <ul> <li>  PNG files: Word images</li> <li>  TXT files: Transcriptions of words</li> </ul> </li> </ul> </li> <li>sets/: Contains partition files <ul> <li>test.txt: Names of images in the test set</li> <li>validation.txt: Names of images in the validation set</li> <li>training.txt: Names of images in the training set</li> </ul> </li> </ul> <h3>Dataset Usage and Annotations</h3> Each name in test.txt, validation.txt and training.txt represents the name of the page and all its content (words, lines, paragraphs) must be in the respective partition. Annotations used in the dataset: <ul> <li>  <code>##@@???@@##</code>: Superscript text that has become unidentifiable and unreadable.</li> <li>  <code>$$@@???@@$$</code>: Subscript text that has become unidentifiable and unreadable.</li> <li>  <code>@@???@@</code>: Text that cannot be read or identified due to its illegibility.</li> <li>  <code>##--xxx--##</code>: Text that has been added as a superscript and subsequently crossed out, rendering it illegible.</li> <li>  <code>$$--xxx--$$</code>: Text that has been added as a subscript and subsequently crossed out, rendering it illegible.</li> <li>  <code>--xxx--</code>: Text that has been crossed out in a way that makes it unreadable.</li> <li>  <code>##--text--##</code>: Text that has been added as a superscript and subsequently crossed out, but remains legible.</li> <li>  <code>$$--text--$$</code>: Text that has been added as a subscript and subsequently crossed out, but remains legible.</li> <li>  <code>##text##</code>: Text added as a superscript in the line, typically as a correction or additional note.</li> <li>  <code>$$text$$</code>: Text added as a subscript in the line, typically as a correction or additional note.</li> <li>  <code>--text--</code>: Text that has been crossed out but remains readable.</li> </ul>

opencc-by-4.0May 2024View details →

FloraNER: a Named Entity Recognition Dataset for Botanical French Text

FloraNER is a Named-Entity Recognition (NER) dataset for botanical french literature. The dataset covers plant species names and plant morphological terms for both plant organs/characteristics and their descriptors. the descriptors are annotated both in a coarse-grained manner as the named entity type "DESCRIPTOR" and in a fine-grained manner, categorized into the following named-entity types: Form, Measure, Surface, Color, Position, Disposition, Structure, and Development. FloraNER is distantly annotated using a specialized botanical corpus. Consequently, it's important to note that not all named entities within the text are captured and annotated for the coarse-grained and fine-grained datasets.

opencc-by-4.0Mar 2013View details →

Raw images (photographs) of urban text scenes for camera-based Thai text recognition

Raw image collection of city scenes in Thailand with text content. Text is photographed from diffeent angles. Also morning and evening photographs were taken in order to capture different lighting conditions. The material, 309 images, was photographed in 2013 by Bowornrat Sriman and volunteers. Example EXIF: JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=13, height=2448, manufacturer=SAMSUNG, model=GT-I9300, orientation=upper-right, xresolution=220, yresolution=228, resolutionunit=2, software=I9300XXEMA2, datetime=2013:03:14 18:17:49, GPS-Data, width=3264], baseline, precision 8, 3264x2448, frames 3 The images are not labeled. The orientation (landscape/portrait) is not corrected yet. This material was used in preparation of the publication: Sriman, B.  & Schomaker, L. (2015). Object Attention Patches for Text Detection and Recognition in Scene Images using SIFT, Proceedings of the International Conference on Pattern Recognition Applications and Methods: ICPRAM 2015.  De Marsico, M., Figueiredo, M.  & Fred, A. (Eds.).  Lisbon, Portugal: SciTePress, Vol.  1, p.  304-311 8 p. Please cite this publication when using these data.

opencc-by-4.0Sep 2018View details →

Dataset for ICFHR2018 Competition on Automated Text Recognition on a READ Dataset

The main idea of this dataset is to analyse the impact of training data. How many training data specific to the document, you are transcribing, is necessary?    general data: This is a collection of heterogeneous documents to train an initial system. For each text line there is an image file of that line, a file with the ground truth text and an information file containing an automatically generated surrounding polygon. specific data: The specific data contains documents related to the test data. For the specific systems only the images of the train list may be used. The file are of the same type as the general data. test data: The test data contains only the images and the information files. More Information, some published results and an evaluation procedure at https://scriptnet.iit.demokritos.gr/competitions/10/

opencc-by-4.0Feb 2023View details →

The e-NDP project : collaborative digital edition of the Chapter registers of Notre-Dame of Paris (1326-1504). Ground-truth for handwriting text recognition (HTR) on late medieval manuscripts.

The <a href="https://endp.hypotheses.org/">e-NDP project</a>, funded by the ANR, is led by the <a href="https://lamop.hypotheses.org/6870">LaMOP</a> (Julie Claustre and Darwin Smith). The project's partners are the Archives nationales, the Bibliothèque nationale de France (Department of Manuscripts, Bibliothèque de l'Arsenal), the École nationale des chartes and the Bibliothèque Mazarine. The e-NDP project aims at renewing our knowledge on Notre-Dame de Paris cathedral through the creation of a collaborative digital edition of the registers of its Chapter (1326-1504, AN LL 105-128), the community of 51 canons meeting three times a week on set days to take all administrative, financial and practical decisions pertaining to the cathedral, its estate and the society living in its cloister. This corpus has never been the object of a comprehensive study to understand the workings and history of this urban enclave and powerful community. The collaborative digital edition is based on a process of handwriting text recognition (HTR), tested and supervised by scholars, researchers and engineers combining expertise in Medieval history, paleography, philology and digital humanities. The edition shall allow a better insight into the Chapter’s administration, into its economical and political power within Paris, and the relationships it maintained with other institutions in the city.   Section 1 : The e-NDP ground-truth dataset for Handwriting text recognition. The full e-NDP corpus kept today in the French National Archives and was entirely digitized and described in its <a href="https://www.siv.archives-nationales.culture.gouv.fr/siv/rechercheconsultation/consultation/ir/consultationIR.action?formCaller=GENERALISTE&irId=FRAN_IR_059635">catalog</a> in 2022. The first major goal of the e-NDP projet is to propose a first automatic transcription of the 14k pages composing the 26 chapter registers. To achieve this goal representative samples from each one of the volumes were selected and transcribed in order to train a specialized HTR model able to propose a high quality automatic transcription. The collected ground-truth released on this repository currently has 512 pages from the 26 registers of the cathedral chapter preserved in the National Archives (LL105 - LL128, 1326-1504). The transcriptions were manually completed in two rounds by a group of 12 contributors, historians and paleographers, over the course of 2021-2022 using <a href="https://escriptorium.paris.inria.fr/">eScriptorium </a>as annotation environment.     Ground-truth features : Number of hands : according to our estimates no fewer than 18 main hands were involved in the writing of the registers during the medieval period.  Language : More than 98% of the content of the registers was written in Latin, the rest in French. The exact percentage is hard to estimate because the vernacular language is often used in formulae, notes and comments. It is rare to find entire pages or blocks written in French.  Script family : The registers were written using a Cursive script (ca. late XIIIe - XVIe). Documental typology : The volumes containing the chapter conclusions were conceived to serve as memorial records, but above all as documents for regular use and consultation in the daily practice of administration and management. In diplomatics the notion of "documentary manuscripts" is used to describe this kind of sources also by opposition to books and litterary or normative manuscripts. <table align="center"> <caption>Ground truth statistics</caption> <tbody> <tr> <th>Text units</th> <th>Count</th> </tr> <tr> <td>Pages</td> <td>512</td> </tr> <tr> <td>Annotated regions (see section 2)</td> <td>2448</td> </tr> <tr> <td>Lines of text</td> <td>34231</td> </tr> <tr> <td>Tokens</td> <td>205083</td> </tr> <tr> <td>Characters</td> <td>3320407</td> </tr> </tbody> </table>   Rules of transcription : <ul> <li>The abbreviations have been resolved, both those by suspension (<code>facimꝰ</code> ---> <code>facimus</code>) and by contraction (<code>dñi</code> --> <code>domini</code>). Likewise, those using conventional signs (<code>⁊</code> --> <code>et</code> ; <code>ꝓ</code> --> <code>pro</code>) have been resolved. </li> <li>The named entities (names of persons, places and institutions) have been <code>capitalized</code>. The beginning of a block of text as well as the original capitals used by the notary are also capitalized.</li> <li>The consonantal <code>i</code> and <code>u</code> characters have been transcribed as <code>j</code> and <code>v</code> in both French and Latin.</li> <li>The punctuation marks used in the text: <code>.</code> and <code>/</code> have been transcribed, but the transcription has not been standardized with modern punctuation.</li> <li>Corrections and words that appear cancelled in the manuscript have been transcribed surrounded by the sign <code>$</code> at the beginning and at the end.</li> <li>More specific transcription rules can be found into the file <code>transcription_guidelines.pdf</code></li> </ul>   Section 2. e-NDP Layout Segmentation. Layout segmentation is a compulsory step before HTR recognition in order to distinguish sections and regions inside a document. This process intend to separate interdependant page zones to produce a recognition in a section-sequence order and not in a line-sequence order which mix textual and peri-textual content. The regions of 364 pages (see <code>GT-layout_list</code>) of the e-NDP corpus were annotated using a 5 sections vocabulary (see <code>endp_layout_regions</code>) in order to describe the page distribution in all the 26 volumes : <ol> <li>Block : All the central text blocks, that normally corresponds to the main content called "conclusions" in registers.</li> <li>Liste : List of names of the canons who were present during the meeting. Normally located before the conclusions.</li> <li>Entrée : Marginal notes or entries to inform about the content of conclusions.</li> <li>Date : Paragraph contending the date. Normally at the head of a conclusion, but separate of the main body.</li> <li>Numérotation : Page numbers in roman or arabic. Usually appear in the top corners of the pages.</li> </ol> <table align="center"> <caption>Layout GT statistics</caption> <tbody> <tr> <th>Region</th> <th>Count</th> </tr> <tr> <td>block</td> <td>833</td> </tr> <tr> <td>liste</td> <td>431</td> </tr> <tr> <td>date</td> <td>448</td> </tr> <tr> <td>entrée</td> <td>205</td> </tr> <tr> <td>numérotation</td> <td>531</td> </tr> </tbody> </table>   Section 3. The e-NDP HTR modeling. The e-NDP project has progressively trained several HTR models adapted to work on late medieval cursive in order to accelerate the production of ground truth. Currently the best model delivers an average CER (Character error ratio) of 9.7% in handwriting recognition on the 26 registers (see <code>endp_learning_curve</code>) and can serve as generalist model for other manuscripts of the same period and similar script family. These models and their training implementation details can be found in the project's github <a href="https://github.com/chartes/e-NDP_HTR">repository</a>.  Additionally, the automatic HTR transcriptions of the 26 registers (14k pages, 4.5M tokens) enriched with lexical and semantical information has been the subject of a first <a href="https://nosketch-engine.lamop.fr/#dashboard?corpname=endp">online publication</a> using the NoSketch engine that allows advanced data mining based on the combination of data, metadata and NLP features.    Section 4. Dataset content. This zip dataset contains : - <code>HTR_ground_truth</code> : Two folders containing the jpg / jpeg images and their curated transcriptions in PAGE XML format. - <code>images_docs</code> : 4 files illustrating the different phases of the project (list of GT for layout segmentation, layout ontologie, transcription guideline and HTR evaluation curves)

opencc-by-4.0Jan 2017View details →

ICDAR 2015 Competition HTRtS: Handwritten Text Recognition on the tranScriptorium Dataset

This dataset comprises the dataset used for the ICDAR 2015 Competition on Handwritten Text Recognition on the tranScriptorium Dataset. The handwritten images for this contest were drawn from the English “Bentham collection” dataset used in the TRAN SCRIPTORIUM project. The selected data has been written by several hands and entails significant variabilities and difficulties regarding the quality of text images, writing styles and crossed-out text. This contest is clearly more difficult than the the first edition both for training and for testing. A portion of the training dataset and the full test dataset were provided in the form of carefully segmented line images, along with the corresponding transcripts. Another portion of the training dataset was provided as raw images and their corresponding transcripts at region level. ICDAR 2015 competition HTRtS: handwritten text recognition on the tranScriptorium dataset JA Sánchez, AH Toselli, V Romero, E Vidal. In International Conference on Document Analysis and Recognition (ICDAR), pp. 1166-1170, 2015.

opencc-by-4.0Apr 2017View details →

Train-B dataset for ICDAR2017 Competition on Handwritten Text Recognition on the READ Dataset (ICDAR2017 HTR). Batch 1 and Batch 2.

Train-B Dataset. Dataset of pages without any layout or text line information. The corresponding transcripts are provided at page level with line breaks. It has 10k pages, though for convenience it is divided into two 5k page batches. This information is provided in PAGE format. This dataset is complementary to this other dataset: https://zenodo.org/record/439807#.WOIBZ3WLSkA More information at: https://scriptnet.iit.demokritos.gr/competitions/~icdar2017htr/

opencc-by-4.0Apr 2017View details →

Train-A dataset for ICDAR2017 Competition on Handwritten Text Recognition on the READ Dataset (ICDAR2017 HTR)

Train-A Dataset of pages with manually revised baselines and the corresponding transcripts associated to them. This batch is small, 50 pages. Please, keep in mind that only the baselines have been manually corrected, The polygons associated to each line have not been manually reviewed. This dataset is complementary to this other dataset: https://zenodo.org/record/439811#.WOIF9HWLSkA More information at: https://scriptnet.iit.demokritos.gr/competitions/~icdar2017htr/

opencc-by-4.0Dec 2023View details →

NorHand v3 / Dataset for Handwritten Text Recognition in Norwegian

This dataset comprises Norwegian letter and diary documents from 19th and early 20th century. It can be used to train Handwritten Text Recognition (HTR) models.

opencc-by-nc-4.0Jul 2018View details →

Oficio de Hipotecas de Girona. A dataset of Spanish notarial deeds (18th Century) for Handwritten Text Recognition and Layout Analysis of historical documents.

This dataset is a subset of 596 documents from the Registre d'Hipoteques de Girona of 1769 collection, guarded by the <a href="http://xac.gencat.cat/ca/llista_arxius_comarcals/girones/">Arxiu Històric de Girona</a>. This collection, is composed by hundreds of thousands of notarial deeds from the XVIII-XIX century (1768-1862). Sales, redemption of censuses, inheritance and matrimonial chapters are among the most common documentary typologies in the collection. This dataset is composed of more than 23700 text lines written by a single hand, covering more that 50 different topics (documentary typologies) and a vocabulary of more than 2400 different words. The documents are transcribed using the so-called diplomatic criteria. Additionally, transcripts were tagged with  extra enriching/complementary information (e.g. expansion of the abbreviations, hyphen marks, etc.). Along with the transcripts  the layout of the document is detected and recorded. Pages have been labeled using six different layout regions. The images along with their respective ground-truth was compiled in PAGE compliant XML format by the <a href="http://www2.udg.edu/tabid/11296/Default.aspx">Centre de Recerca d'Història Rural</a> and the HTR group of the <a href="https://www.prhlt.upv.es">Pattern Recognition and Human Language Technologies Research Center</a>.

opencc-by-4.0Jun 2017View details →

Test-B1 ICDAR2017 Competition on Handwritten Text Recognition on the READ Dataset (ICDAR2017 HTR)

<ul> <li>Test-B1: a batch of page images annotated with the geometry of regions where to detect text line and recognize.</li> </ul>

opencc-by-4.0Jun 2017View details →

Test-B2 ICDAR2017 Competition on Handwritten Text Recognition on the READ Dataset (ICDAR2017 HTR)

Test-B2: a batch of page images annotated with the geometry of regions where to detect text line and recognize.

opencc-by-4.0Jul 2017View details →

Dataset for ICDAR2017 Competition on Handwritten Text Recognition on the READ Dataset (ICDAR2017 HTR)

Train-A: Dataset of pages with manually revised baselines and the corresponding transcripts associated to them. This batch is small, 50 pages. Please, keep in mind that only the baselines have been manually corrected, The polygons associated to each line have not been manually reviewed. Train-B: Dataset of pages without any layout or text line information. The corresponding transcripts are provided at page level with line breaks. It has 10k pages, though for convenience it is divided into two 5k page batches. This information is provided in PAGE format. Test A: Dataset of pages with manually revised baselines. This batch has 65 pages. The polygons associated to each line have not been manually reviewed. Test-B1: The same dataset of pages of the Test A, but annotated only with the geometry of regions. Text line information is not provided. Test-B2: Dataset of page images annotated with the geometry of regions where to detect text line and recognize. It has 57 pages. Baseline.tgz: Baseline system trained using the first 40 pages of Train-A. The system is based on the deep learning toolkit to transcribe handwritten text images called Laia. More information at: https://scriptnet.iit.demokritos.gr/competitions/~icdar2017htr/

opencc-by-4.0Nov 2023View details →

Appendix A - The Implications of Handwritten Text Recognition for Accessing the Past at Scale

List of works identified through a Grounded Theory Method (GTM) of the current and near future implications of Handwritten Text Recognition (HTR) on the historical method and wider information environment. The findings of this data collection are provided in 'The Implications of Handwritten Text Recognition for Accessing the Past at Scale'

opencc-by-4.0Jan 2022View details →

Appendix A - Understanding the application of Handwritten Text Recognition technology in heritage contexts

This appendix lists all the categorised works mentioning the HTR software Transkribus, used in 'Understanding the application of Handwritten Text Recognition technology in heritage contexts: a systematic review of Transkribus in published research'

opencc-by-4.0Jan 2018View details →

ICDAR 2015 Competition HTRtS: Handwritten Text Recognition on the tranScriptorium Dataset Rerelease

A new release of the dataset used in the ICDAR 2015 HTR competition in which all Page XML files are based on the same 2013-07-15 schema. It only contains page level images, Page XML files for train and test (including the ground truth transcripts for the test and train batch 1) and plain text files for train batch 2 that have the page level ground truth transcripts. The original version of this dataset can be found at http://doi.org/10.5281/zenodo.248733