Media Bias Aware Simulation Dataset

<p>We utilized the Hyperpartisan News Detection Dataset, released with the SemEval-2019 Task 4 Hyperpartisan detection task, due to its extensive bias labels. To ensure accurate bias labels, we used the Overlap-checking (1:1) model, retaining only articles where the distant supervision bias labels matched the model's predictions. This validation process resulted in 409,757 articles.</p> <p>These articles span from 1960 to 2018, with a sparse distribution in earlier years. We focused on articles from May 1, 2017, to December 31, 2017, resulting in a subset of 72,940 news articles, ensuring a consistent daily news flow. We processed this subset by removing HTML tags and special characters and generating news summaries using PEGASUS.&nbsp;We used Latent Dirichlet Allocation (LDA) to categorize the articles into 20 news themes, based on perplexity scores.</p> <p>This dataset was then fed into the simulation framework, with a cut-off date of June 24, 2017.</p> <ul> <li><strong>News Recommendation Dataset</strong>: Includes user-item interaction records from May 1 to June 24, providing users' reading histories and interacted news articles. <ul> <li><strong>Training Split</strong>: Data from May 1 to June 17, used to train news recommendation algorithms.</li> <li><strong>Evaluation Split</strong>: Data from June 17 to June 24, used to evaluate the trained recommendation algorithms.</li> </ul> </li> <li> <p><strong>Candidate News Dataset</strong>: News articles published from June 25 to December 31, presented to users during simulations.</p> </li> </ul> <p>For more information, please visit <a href="https://github.com/ruanqin0706/UserRecSimulation" target="_new" rel="noreferrer">https://github.com/ruanqin0706/UserRecSimulation</a>.</p>