Dataset for generating TL;DR

<p>This is the dataset for the TL;DR challenge containing posts&nbsp;from the Reddit corpus, suitable for abstractive summarization using deep learning. The format is a json file where each line is a JSON object representing a post. The schema of each post is shown below:</p> <ul> <li>author: string (nullable = true)</li> <li>body: string (nullable = true)</li> <li>normalizedBody: string (nullable = true)</li> <li>content: string (nullable = true)</li> <li>content_len: long (nullable = true)</li> <li>summary: string (nullable = true)</li> <li>summary_len: long (nullable = true)</li> <li>id: string (nullable = true)</li> <li>subreddit: string (nullable = true)</li> <li>subreddit_id: string (nullable = true)</li> <li>title: string (nullable = true)</li> </ul> <p>Specifically, the <strong>content</strong> and <strong>summary</strong> fields can be directly used as inputs to a deep learning model (e.g. Sequence to Sequence model ). The dataset consists of 3,084,410 posts with an average length of 211 words for content, and 25&nbsp;words for the summary.</p> <p><strong>Note :&nbsp;</strong>As this is the complete dataset for the challenge, it is up to the participants to split it into training and validation sets accordingly.</p>