Two experiments for evaluating the impact of Hamcrest and AssertJ on assertion development
- PDF / 2,894,969 Bytes
- 33 Pages / 439.37 x 666.142 pts Page_size
- 8 Downloads / 174 Views
Two experiments for evaluating the impact of Hamcrest and AssertJ on assertion development Maurizio Leotta 1
1
1
& Maura Cerioli & Dario Olianas & Filippo Ricca
1
# Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
Test automation enables continuous testing, a cornerstone of agile methods, and DevOps. Assertions play a fundamental role in test automation, and recently competing assertion libraries for unit testing frameworks, such as, for example, JUnit or TestNG, emerged. Thus, it is imperative to gauge assertion libraries in terms of developer/tester productivity, allowing SQA managers and software testers to select the best. The goal of this work is comparing two assertion libraries having a different approach (matchers vs. fluent assertions) w.r.t. two dependent variables: correctness of developed assertions and time to develop them. We conducted two controlled experiments with Bachelor students in Computer Science and Master students in Computer Engineering. AssertJ (fluent assertions approach) is compared with Hamcrest (matchers), in a test development scenario with the Java language where 672 assertions were developed by 48 students overall. The results show that (a) adopting AssertJ improves the tester’s productivity significantly during the development of assertions only for Bachelor students, and (b) time of developing assertions is similar using AssertJ or Hamcrest in both the categories of participants. Testers and SQA managers selecting assertion libraries for their organizations should consider as first choice AssertJ in case of inexperienced developers/testers since our study shows that it increases the productivity of Bachelor students more than Hamcrest. Keywords Hamcrest matchers . AssertJ methods . Controlled experiment . Evidence-based Software Engineering . Tester productivity . Assertion development
1 Introduction The next frontier for development and quality assurance teams is to adopt continuous delivery in current practice, a process aiming at building, testing, and releasing good quality software This article belongs to the: Topical Collection on Quality Management for Information Systems Guest Editors: Mario Piattini, Ignacio García Rodríguez de Guzmán, Ricardo Pérez del Castillo
* Maurizio Leotta [email protected] Extended author information available on the last page of the article
Software Quality Journal
very quickly. That is particularly true for teams using agile methods or DevOps, where continuous integration and continuous testing constitute two critical cornerstones. Continuous testing is the process of testing an application continuously during the development cycle, after code integration, and before deployment. That is possible only by using test automation, and it has the significant advantage of giving the developers immediate feedback on their work. Continuous testing includes the validation of functional and nonfunctional requirements. For testing functional requirements, a “pyramid” of testing techniques is often used: unit tes
Data Loading...
