RETORCH: an approach for resource-aware orchestration of end-to-end test cases
- PDF / 1,467,632 Bytes
- 25 Pages / 439.37 x 666.142 pts Page_size
- 72 Downloads / 158 Views
RETORCH: an approach for resource-aware orchestration of end-to-end test cases Cristian Augusto 1 Javier Tuya 1
& Jesús Morán
1
& Antonia Bertolino
2
& Claudio de la Riva
1
&
# Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
Continuous integration practice mandates to continuously introduce incremental changes into code, but doing so may introduce new faults too. These faults could be detected automatically through regression testing, but this practice becomes prohibitive as the cost of executing the tests grows. This problem is preponderant in end-to-end testing where the whole system is requested for test execution. However, some of these test cases could be executed with fewer resources (e.g., memory, web services, computation, Cloud instances, among others), by deploying only the subsystems needed by each test. This paper is focused on the optimization of the resources employed in end-to-end testing by means of a resource-aware test orchestration technique in the context of continuous integration practices in the Cloud. The RETORCH approach proposes a novel way to identify the resources required by end-to-end test cases and to use this information to group together those tests requiring equivalent resources. Besides, the approach proposes to deploy the grouped tests in isolated and elastic environments, so that their execution can be scheduled in parallel on several machines. RETORCH is exemplified with a realworld application, and its performance evaluation shows promising savings in terms of resource usage and time. Keywords Software testing . Continuous integration . Continuous testing . Testing in the Cloud . End-to-end testing . Test orchestration
1 Introduction Continuous integration practices are based on incremental changes in the code to improve quality or add new functionalities (Meyer 2014). However, while introducing new features in the code, new faults can be introduced as well. Detection of these failures early in the code may reduce between 25 and 40% the amount of time and cost in fixing them compared to fix * Cristian Augusto [email protected] Extended author information available on the last page of the article
Software Quality Journal
them in production (Shull et al. 2002). To ensure that the modifications and the new code do not endanger the existing functionality, regression testing (Yoo and Harman 2012) is standard practice. In modern agile processes, though, in which new versions of software are continuously and frequently delivered within very short cycles, regression testing may face many challenges as the efficient execution of the test cases, the reliability improvement of the tested system, or the reduction of time between different releases. One emerging practice to shorten the validation of newly released versions is continuous testing (Fitzgerald and Stol 2017). Continuous testing consists of automating the test cases and re-executing them before any new release in the source code repository. However, a wellknown problem is that as
Data Loading...
