Automated issue assignment: results and insights from an industrial case
- PDF / 1,503,744 Bytes
- 46 Pages / 439.642 x 666.49 pts Page_size
- 41 Downloads / 165 Views
Automated issue assignment: results and insights from an industrial case Ethem Utku Aktas1
· Cemal Yilmaz2
© Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract We automate the process of assigning issue reports to development teams by using data mining approaches and share our experience gained by deploying the resulting system, called IssueTAG, at Softtech. Being a subsidiary of the largest private bank in Turkey, Softtech on average receives 350 issue reports daily from the field, which need to be handled with utmost importance and urgency. IssueTAG has been making all the issue assignments at Softtech since its deployment on Jan 12, 2018. Deploying IssueTAG presented us not only with an unprecedented opportunity to observe the practical effects of automated issue assignment, but also with an opportunity to carry out user studies, both of which (to the best of our knowledge) have not been done before in this context. We first empirically determine the data mining approach to be used in IssueTAG. We then deploy IssueTAG and make a number of valuable observations. First, it is not just about deploying a system for automated issue assignment, but also about designing/changing the assignment process around the system. Second, the accuracy of the assignments does not have to be higher than that of manual assignments in order for the system to be useful. Third, deploying such a system requires the development of additional functionalities, such as creating human-readable explanations for the assignments and detecting deteriorations in assignment accuracies, for both of which we have developed and empirically evaluated different approaches. Last but not least, stakeholders do not necessarily resist change and gradual transition helps build confidence. Keywords Bug triaging · Issue report assignment · Text classification · Accountable machine learning · Change point detection
Communicated by: Per Runeson Ethem Utku Aktas
[email protected] Cemal Yilmaz [email protected] 1
Research and Development Center, Softtech Inc., Istanbul, 34947, Turkey
2
Faculty of Engineering and Natural Sciences, Sabanci University, Istanbul, 34956, Turkey
Empirical Software Engineering
1 Introduction When a software system produces an unexpected result or when an additional feature is requested from the system, a report is submitted to the vendor. These reports, which are often referred to as issue reports, bug reports, or problem reports, include all the information necessary for the vendor to resolve the reported issues. Once a report is received, the vendor carries out a number of tasks until the issue is resolved, including determining the validity of the reported issue and figuring out whether the same/similar issues have been reported in the past (Antoniol et al. 2008; Bettenburg et al. 2008a; Bettenburg et al. 2008b; Jalbert and Weimer 2008; Lamkanfi et al. 2010; Menzies and Marcus 2008; Pandey et al. 2017; Wang et al. 2008). An integral part of this process is to assign the issue repor
Data Loading...
