On the feasibility of automated prediction of bug and non-bug issues
- PDF / 3,562,368 Bytes
- 37 Pages / 439.642 x 666.49 pts Page_size
- 80 Downloads / 206 Views
On the feasibility of automated prediction of bug and non-bug issues Steffen Herbold1
· Alexander Trautsch2 · Fabian Trautsch2
Published online: 14 September 2020 © The Author(s) 2020, corrected publication 2020
Abstract Context Issue tracking systems are used to track and describe tasks in the development process, e.g., requested feature improvements or reported bugs. However, past research has shown that the reported issue types often do not match the description of the issue. Objective We want to understand the overall maturity of the state of the art of issue type prediction with the goal to predict if issues are bugs and evaluate if we can improve existing models by incorporating manually specified knowledge about issues. Method We train different models for the title and description of the issue to account for the difference in structure between these fields, e.g., the length. Moreover, we manually detect issues whose description contains a null pointer exception, as these are strong indicators that issues are bugs. Results Our approach performs best overall, but not significantly different from an approach from the literature based on the fastText classifier from Facebook AI Research. The small improvements in prediction performance are due to structural information about the issues we used. We found that using information about the content of issues in form of null pointer exceptions is not useful. We demonstrate the usefulness of issue type prediction through the example of labelling bugfixing commits. Conclusions Issue type prediction can be a useful tool if the use case allows either for a certain amount of missed bug reports or the prediction of too many issues as bug is acceptable. Keywords Issue type prediction · Mislabeled issues · Issue tracking Communicated by: Burak Turhan Steffen Herbold
[email protected] Alexander Trautsch [email protected] Fabian Trautsch [email protected] 1
Institute AIFB, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
2
Institute of Computer Science, University of Goettingen, G¨ottingen, Germany
5334
Empirical Software Engineering (2020) 25:5333–5369
1 Introduction The tracking of tasks and issues is a common part of modern software engineering, e.g., through dedicated systems like Jira and Bugzilla, or integrated into other other systems like GitHub Issues. Developers and sometimes users of software file issues, e.g., to describe bugs, request improvements, organize work, or ask for feedback. This manifests in different types into which the issues are classified. However, past research has shown that the issue types are often not correct with respect to the content (Antoniol et al. 2008; Herzig et al. 2013; Herbold et al. 2020). Wrong types of issues can have different kinds of negative consequences, depending on the use of the issue tracking system. We distinguish between two important use cases, that are negatively affected by misclassifications. First, the types of issues are important for measuring the
Data Loading...
