unarXive: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network (open subset)

<h2><strong>Description</strong></h2><p>unarXive is a scholarly data set containing publications' structured full-text, annotated in-text citations, linked non-text content (mathematical notation, figure/table captions) and a citation network.</p><p>The data is generated from all LaTeX sources on <a href="https://arxiv.org/">arXiv</a> and therefore of higher quality than data generated from PDF files.</p><p>Typical uses are</p><ul><li>Training of ML models (citation recommendation, summarization, LLMs)</li><li>Citation context analysis</li><li>Bibliographic analyses</li></ul><h2><strong>Access</strong></h2><p>┏━━━━━━━━━━━━━━━━━━━━━━━━━━┓<br>┃ &nbsp;<a href="https://github.com/IllDepence/unarXive/raw/master/doc/unarXive_data_sample.tar.gz"><strong>D O W N L O A D &nbsp; S A M P L E</strong></a> &nbsp; ┃<br>┗━━━━━━━━━━━━━━━━━━━━━━━━━━┛</p><p>Regarding the full data set, please note the following:</p><blockquote><p><strong>Note</strong>: this Zenodo record is the "open subset" of unarXive, which contains all permissively licensed papers from arXiv.org. You can find the <a href="https://doi.org/10.5281/zenodo.7752754">full version here</a>.</p></blockquote><p>The code used for generating the data set is <a href="https://github.com/IllDepence/unarXive">publicly available</a>.</p>