Technical Debt Classification in Issue Trackers using Natural Language Processing based on Transformers

<p>In order to ensure transparency and reproducibility, we have&nbsp;made everything available publicly here, including the Code, Models, Datasets and more. All the files and their functionality used in this paper are explained clearly in the <strong>README.md</strong> file.</p> <p>Background: &nbsp;Technical Debt (TD) needs to be controlled and tracked during software development. Support to automatically track TD in issue trackers is limited.&nbsp;</p> <p>Aim: We explore the usage of a large dataset of developer-labeled TD issues in combination with cutting-edge Natural Language Processing (NLP) approaches to automatically classify TD in issue trackers.</p> <p>Method: &nbsp;We mine and analyze more than 160GB of textual data from GitHub projects, collecting over 55,600 TD issues and consolidating them into a large dataset (GTD dataset). We use such datasets to train and test Transformer ML models. Then we test the model&#39;s&nbsp;generalization ability by testing them on six unseen projects. Finally, we re-train the models including part of the TD issues from the target project to test their adaptability.&nbsp;</p> <p>Results and Conclusion: (i) We create and release the GTD dataset, a comprehensive dataset including TD issues from 6,401 public repositories with various contexts; (ii) By training Transformers using the GTD dataset, we achieve performance metrics that are promising; (iii) Our results are a significant step forward towards supporting the automatic classification of TD in issue trackers, especially when the models are adapted to the context of unseen projects after fine-tuning.</p>