Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO)
- PDF / 1,704,793 Bytes
- 32 Pages / 439.642 x 666.49 pts Page_size
- 82 Downloads / 149 Views
Data-driven software design with Constraint Oriented Multi-variate Bandit Optimization (COMBO) Rasmus Ros1
· Mikael Hammar2
Published online: 18 August 2020 © The Author(s) 2020
Abstract Context Software design in e-commerce can be improved with user data through controlled experiments (i.e. A/B tests) to better meet user needs. Machine learning-based algorithmic optimization techniques extends the approach to large number of variables to personalize software to different user needs. So far the optimization techniques has only been applied to optimize software of low complexity, such as colors and wordings of text. Objective In this paper, we introduce the COMBO toolkit with capability to model optimization variables and their relationship constraints specified through an embedded domain-specific language. The toolkit generates personalized software configurations for users as they arrive in the system, and the configurations improve over time in in relation to some given metric. COMBO has several implementations of machine learning algorithms and constraint solvers to optimize the model with user data by software developers without deep optimization knowledge. Method The toolkit was validated in a proof-of-concept by implementing two features that are relevant to Apptus, an e-commerce company that develops algorithms for web shops. The algorithmic performance was evaluated in simulations with realistic historic user data. Results The validation shows that the toolkit approach can model and improve relatively complex features with many types of variables and constraints, without causing noticeable delays for users. Conclusions We show that modeling software hierarchies in a formal model facilitates algorithmic optimization of more complex software. In this way, using COMBO, developers can make data-driven and personalized software products. Keywords Continuous experimentation · A/B testing · Machine learning · Multi-armed bandits · Combinatorial optimization
This article belongs to the Topical Collection: Software Engineering in the Age of Artificial Intelligence Communicated by: Tim Menzies, Chakkrit Tantithamthavorn and Burak Turhan Rasmus Ros
[email protected]
Extended author information available on the last page of the article.
3842
Empirical Software Engineering (2020) 25:3841–3872
1 Introduction Design of user-facing software involve many decisions that can be optimized with user data. The decision variables—called the search space—can include both product aspects that are directly or indirectly visible to the user. For example, what wordings to use in headings or how items should be ranked in recommender systems (Amatriain 2013) or search engines (Tang et al. 2010). Traditionally, randomized controlled experiments (i.e., A/B tests or split tests) are used to iteratively validate the design choices based on user data (Kohavi et al. 2008). Recently, data-driven optimization algorithms have been proposed to perform automated experimentation on software in larger scale on bigger search spaces simul
Data Loading...
