Challenges concerning test case specifications in automotive software testing: assessment of frequency and criticality
- PDF / 3,733,278 Bytes
- 62 Pages / 439.642 x 666.49 pts Page_size
- 8 Downloads / 182 Views
Challenges concerning test case specifications in automotive software testing: assessment of frequency and criticality Katharina Juhnke1,2
· Matthias Tichy2
· Frank Houdek1
© The Author(s) 2020
Abstract Automotive test case specifications document test cases to be performed for a specific test object at a defined test level. They are a fundamental part of a structured automotive testing process, as required by the ISO 26262. The aim of our research is to identify challenges from a practitioner’s point of view that lead to poor quality of test case specifications and thus negatively impact time, cost, and probability of defect detection. We designed an exploratory case study to systematically identify challenges focusing on (C) creation, (P) processing, and (Q) quality assurance related aspects of test case specifications. We conducted 17 semi-structured interviews covering a German OEM as well as three of its automotive suppliers and analyzed them qualitatively. We investigated causes and consequences arising from the challenges. Additionally, we conducted a descriptive survey to assess frequency and criticality. The identified challenges were summarized in a taxonomy consisting of nine main categories: (1) availability and (2) content-related problems with input artifacts, problems related to (3) a lack of knowledge, (4) the test case description, (5) the test case specification content, (6) processes, (7) communication, (8) quality assurance, and (9) tools. The challenges were assessed by 26 internal and 10 external employees. Hence, we identified differences between these groups in terms of access to documents, incomplete requirements, scope of model series, process, and tool-related aspects. Overall, the study results underline the necessity of quality assurance measures for test case specifications. Based on the assessments, our research indicates a broad range of test case description related challenges that are promising candidates for improving test case specification quality. Keywords Automotive software testing · Test case specification · Embedded systems · Challenges
Katharina Juhnke
[email protected]
Extended author information available on the last page of the article.
Software Quality Journal
1 Introduction Nowadays, innovation in vehicles is mainly realized by software and electronic systems. To verify that software works as expected, testing is an integral part of the development process in the automotive domain. Standards like ISO 26262 (2011) or Automotive SPICE (2016) must be implemented by original equipment manufacturers (OEMs). In addition to a systematic development process, standards require a mandatory documentation of the activities and relevant work products. In the context of test documentation, these work products are a verification plan and a verification specification. The latter is also called test case specification in the software testing standard ISO 29119 (2013) and this term will be used in this article. A test case specification contains a set of test case
Data Loading...
