Risk-Based Interoperability Testing Using Reinforcement Learning
Risk-based test strategies enable the tester to harmonize the number of specified test cases with imposed time and cost constraints. However, the risk assessment itself often requires a considerable effort of cost and time, since it is rarely automated. E
- PDF / 943,405 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 99 Downloads / 201 Views
Institute for Software and Systems Engineering, University of Augsburg, Augsburg, Germany {reichstaller,eberhardinger,knapp,reif}@isse.de 2 MaibornWolff GmbH, Munich, Germany [email protected]
Abstract. Risk-based test strategies enable the tester to harmonize the number of specified test cases with imposed time and cost constraints. However, the risk assessment itself often requires a considerable effort of cost and time, since it is rarely automated. Especially for complex tasks such as testing the interoperability of different components it is expensive to manually assess the criticality of possible faults. We present a method that operationalizes the risk assessment for interoperability testing. This method uses behavior models of the system under test and reinforcement learning techniques to break down the criticality of given failure situations to the relevance of single system actions for being tested. Based on this risk assessment, a desired number of test cases is generated which covers as much relevance as possible. Risk models and test cases have been generated for a mobile payment system within an industrial case study.
1
Introduction
Interoperability testing of a distributed system checks whether the components of the system are able to communicate with each other and thus render requested services correctly through interaction [5]. The typically high number of possible interaction scenarios (e.g., combination of messages) makes interoperability testing a complex task. Since it seems impossible to cover all scenarios, their relevance for being tested has to be prioritized somehow. A criticality-based test strategy should focus the test effort on revealing faults which are expected to lead to the most critical failures [2]. However, the existence of implementation faults and the ensuing reachability of failures in real operation is unknown. Still, extending behavior models of the communicating components of the System under Test (SuT ) with possible implementation faults, we can at least estimate the impact of a fault on the reachability of failures. The challenge is to find causal connections between the implementation faults and resulting failures. We tackle this challenge by combining model-based [16] and risk-based testing methods [1,2] with reinforcement learning [15]. Our approach builds on given c IFIP International Federation for Information Processing 2016 Published by Springer International Publishing AG 2016. All Rights Reserved F. Wotawa et al. (Eds.): ICTSS 2016, LNCS 9976, pp. 52–69, 2016. DOI: 10.1007/978-3-319-47443-4 4
Risk-Based Interoperability Testing Using Reinforcement Learning
53
behavior models of the interacting components of the SuT, common implementation faults, and a set of the most critical failure situations, each of them described by the combination of component states and a score of its deemed effect. A failure situation is reached if all of the combined component states are active at the same point in time. None of the failures is actually reachable in the
Data Loading...
