PROGRESS: Progressive Reinforcement-Learning-Based Surrogate Selection
In most engineering problems, experiments for evaluating the performance of different setups are time consuming, expensive, or even both. Therefore, sequential experimental designs have become an indispensable technique for optimizing the objective functi
- PDF / 385,845 Bytes
- 15 Pages / 439.37 x 666.142 pts Page_size
- 33 Downloads / 242 Views
2
Institute of Machining Technology (ISF), TU Dortmund University, Dortmund, Germany {hess,wagner}@isf.de Faculty of Statistics, TU Dortmund University, Dortmund, Germany [email protected]
Abstract. In most engineering problems, experiments for evaluating the performance of different setups are time consuming, expensive, or even both. Therefore, sequential experimental designs have become an indispensable technique for optimizing the objective functions of these problems. In this context, most of the problems can be considered as a black-box. Specifically, no function properties are known a priori to select the best suited surrogate model class. Therefore, we propose a new ensemble-based approach, which is capable of identifying the best surrogate model during the optimization process by using reinforcement learning techniques. The procedure is general and can be applied to arbitrary ensembles of surrogate models. Results are provided on 24 well-known black-box functions to show that the progressive procedure is capable of selecting suitable models from the ensemble and that it can compete with state-of-the-art methods for sequential optimization. Keywords: Model-based optimization · Sequential designs · Black-box optimization · Surrogate models · Kriging · Efficient global optimization · Reinforcement learning
1
Introduction
The optimization of real-world systems based on expensive experiments or timeconsuming simulations poses an important research area. Against the background of increasing flexibility and complexity of modern product portfolios, such kinds of problems have to be constantly solved. The use of surrogate (meta)models fˆ for approximating the expensive or time-consuming objective function f : x → y represents an established approach to this task. After determining the values of f for the points x of an initial design of experiments, the surrogate model fˆ is computed and then used for the further analysis and optimization. Here, we consider deterministic, i.e., noise-free minimization problems. In such a scenario, the former approach has a conceptual drawback. The location of G. Nicosia and P. Pardalos (Eds.): LION 7, LNCS 7997, pp. 110–124, 2013. c Springer-Verlag Berlin Heidelberg 2013 DOI: 10.1007/978-3-642-44973-4 13,
PROGRESS: Progressive Reinforcement-Learning-Based Surrogate Selection
111
the optimum can only roughly be determined based on the initial design. A high accuracy of the optimization on the model does not necessarily provide improved quality with respect to the original objective function. As a consequence, the resources expended for the usually uniform coverage of the experimental region for the approximation of the global response surface may be spent more efficiently in order to increase the accuracy of the surrogate in the regions of the actual optimum. A solution to this problem is provided by sequential techniques, called efficient global optimization (EGO) [16], sequential parameter optimization [1] and sequential designs [24] within the different disciplines.
Data Loading...
