Specification testing of agent-based simulation using property-based testing
- PDF / 1,629,219 Bytes
- 22 Pages / 439.37 x 666.142 pts Page_size
- 21 Downloads / 164 Views
(2020) 34:47
Specification testing of agent‑based simulation using property‑based testing Jonathan Thaler1,2 · Peer‑Olaf Siebers1
© The Author(s) 2020
Abstract The importance of Agent-Based Simulation (ABS) as scientific method to generate data for scientific models in general and for informed policy decisions in particular has been widely recognised. However, the important technique of code testing of implementations like unit testing has not generated much research interested so far. As a possible solution, in previous work we have explored the conceptual use of property-based testing. In this code testing method, model specifications and invariants are expressed directly in code and tested through automated and randomised test data generation. This paper expands on our previous work and explores how to use property-based testing on a technical level to encode and test specifications of ABS. As use case the simple agent-based SIR model is used, where it is shown how to test agent behaviour, transition probabilities and model invariants. The outcome are specifications expressed directly in code, which relate whole classes of random input to expected classes of output. During test execution, random test data is generated automatically, potentially covering the equivalent of thousands of unit tests, run within seconds on modern hardware. This makes property-based testing in the context of ABS strictly more powerful than unit testing, as it is a much more natural fit due to its stochastic nature. Keywords Agent-based simulation testing · Code testing · Test driven development · Model specification
1 Introduction Since its inception in the early 1990s [17, 31, 36], Agent-Based Simulation (ABS) as a third way of doing science [3, 5] has matured substantially and has found its way into the mainstream of science [25]. Further, a number of ABS frameworks and tools like RePast, AnyLogic and NetLogo as well as open databases of ABS models [16] have been developed, allowing for quick and robust prototyping and development of models.
* Jonathan Thaler [email protected]; [email protected] 1
School Of Computer Science, University of Nottingham, 7301 Wollaton Rd, Nottingham, UK
2
Department of Computer Science, FH Vorarlberg, Hochschulstrasse 1, 6850 Dornbirn, Austria
13
Vol.:(0123456789)
47
Page 2 of 22
Autonomous Agents and Multi-Agent Systems
(2020) 34:47
However, despite the broad acceptance and adoption of ABS as methodology and generative way of doing science, there have been struggles as reported by Axelrod [4]. He discusses the vulnerability of ABS to misunderstanding: due to informal specifications of models and change requests amongst members of a research team, bugs are very likely to be introduced. Further, he reports how difficult it was to reproduce other work [2], which took the team four months, due to inconsistencies between the original code and the published paper. The consequence is that counter-intuitive simulation results can lead to weeks of checking whether the
Data Loading...
