OCR fulltexts of the Digital Collections of the Berlin State Library (DC-SBB)

<p>The digital collections of the SBB contain 153,942 digitized works from the time period of 1470 to 1945.</p> <p>At the time of publication, 28,909 works have been OCR-processed resulting in 4,988,099 full-text pages.<br> For each page with OCR text, the language has been determined by <em>langid </em>(Lui/Baldwin 2012).</p> <p>corpus-entropy.pkl &nbsp; &nbsp;&nbsp; entropy rate per document page</p> <p>corpus-language.pkl&nbsp;&nbsp; language per document page</p> <p>corpus.zip &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; fulltext corpus (extracts to .txt format)</p> <p>de_corpus.zip &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&nbsp; German sub-corpus (extracts to .txt format)</p> <p>selection_de.pkl&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Selection list of German documents</p> <p>xml2csv_alto.csv&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; fulltext corpus per document page (incl.OCR word confidences)</p> <p>&nbsp;</p> <p><em>Sources</em></p> <p>Marco Lui and Timothy Baldwin. 2012. Langid.py:</p> <p>An off-the-shelf language identification tool. In Proceedings of the ACL 2012 System Demonstrations,</p> <p>ACL &rsquo;12, pages 25&ndash;30, Stroudsburg, PA, USA. Association for Computational Linguistics</p>