Gaussian process optimization with failures: classification and convergence proof
- PDF / 628,440 Bytes
- 24 Pages / 439.37 x 666.142 pts Page_size
- 114 Downloads / 222 Views
Gaussian process optimization with failures: classification and convergence proof François Bachoc1 · Céline Helbert2 · Victor Picheny3 Received: 16 April 2019 / Accepted: 30 November 2019 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract We consider the optimization of a computer model where each simulation either fails or returns a valid output performance. We first propose a new joint Gaussian process model for classification of the inputs (computation failure or success) and for regression of the performance function. We provide results that allow for a computationally efficient maximum likelihood estimation of the covariance parameters, with a stochastic approximation of the likelihood gradient. We then extend the classical improvement criterion to our setting of joint classification and regression. We provide an efficient computation procedure for the extended criterion and its gradient. We prove the almost sure convergence of the global optimization algorithm following from this extended criterion. We also study the practical performances of this algorithm, both on simulated data and on a real computer model in the context of automotive fan design. Keywords Computation failures · Optimization · Gaussian processes · Classification · Convergence Mathematics Subject Classification 60G15 · 90C26
Electronic supplementary material The online version of this article (https://doi.org/10.1007/s10898-02000920-0) contains supplementary material, which is available to authorized users.
B
François Bachoc [email protected] Céline Helbert [email protected] Victor Picheny [email protected]
1
Institut de Mathématiques de Toulouse, Université Paul Sabatier, Toulouse, France
2
Ecole Centrale de Lyon, CNRS UMR 5208, Institut Camille Jordan, Univ. de Lyon, 36 av. G. de Collongue, 69134 Ecully Cedex, France
3
PROWLER.io, Cambridge, UK
123
Journal of Global Optimization
1 Introduction Bayesian optimization (BO) is now established as an efficient tool for solving optimization problems with non-linear objectives that are expensive to evaluate. A wide range of applications have been tackled, from hyperparameter tuning of machine learning algorithms [31] to wing shape design [16]. In the simplest BO setting, the aim is to find the maximum of a fixed unknown function f : D → R, where D is a box of dimension d. Under that configuration, the classical Efficient Global Optimization [EGO, [13]] and its underlying acquisition function Expected Improvement (EI) are still considered state-of-the-art. Several authors have adapted BO to the constrained optimization framework, i.e. when the acceptable design space A ⊂ D is defined by a set of non-linear, expensive-to-compute equations c: A = {x ∈ D s.t. c(x) ≤ 0}, either by adapting the EI function [6,11,25,29,30] or by proposing alternative acquisition functions [12,24]. We consider here the problem of crash constraints, where the objective f is typically evaluated using a computer code that fails to provide simulation results
Data Loading...
