Construction of Bayesian support vector regression in the feature space spanned by Bezier-Bernstein polynomial functions
- PDF / 130,348 Bytes
- 8 Pages / 595 x 842 pts (A4) Page_size
- 110 Downloads / 252 Views
CONSTRUCTION OF BAYESIAN SUPPORT VECTOR REGRESSION IN THE FEATURE SPACE SPANNED BY BEZIER–BERNSTEIN POLYNOMIAL FUNCTIONS
UDC 004.855:519.853
O. Yu. Mytn³k
Ill-posed inverse problems of recovering nonlinear dependencies in observational data are considered. An inductive method is developed for construction of a Bayesian model of support vector regression in Bernstein form. A new Bayesian evidence criterion is used to compare the adequacy of models. Keywords: support vector regression, Bayesian inference, polynomial in Bernstein form, feature space, evidence. INTRODUCTION The support vector method was first proposed by V. N. Vapnik for solution of image classification problems [1]. This approach was soon extended to ill-posed inverse problems of recovering a regression from observational data, which led to the emergence of support vector regression (SVR) [2]. A sufficiently complete historical review of support vector algorithms is presented by A. Smola in [3]. Since the performance of SVR algorithms depends on the values of regularization parameters (hyperparameters) that reflect noise characteristics in a training set, subsequent investigations were mainly devoted to the development of methods for determination of these hyperparameters. In particular, M. Law and J. Kwok [4] applied the Bayesian method for evidencing the adequacy an SVR model with hyperparameters that was proposed by D. Mackay [5] to the regularization of neural networks. The main obstacle in using this method is the nondifferentiability of an e-insensitive loss function. To provide the required smoothness level, as a rule, various approximations are proposed such as a soft insensitive loss function [6], but this violates the robustness properties of SVR. This paper proposes an approach that makes it possible to preserve robustness and to obtain a simple criterion of Bayesian evidence of adequacy of an SVR model. Another direction of investigations of SVR is the choice of a feature space. In [7], S. Gunn proposed the algorithm SUPANOVA in which a feature space is generated by component parts of decompositions of analysis of variance (ANOVA). Advantages of an ANOVA-decomposition such as a structured representation of dependences and their interpretation are used in SVR. The further development of decompositions based on ANOVA led to the occurrence of neurofuzzy models in Bernstein form [8] whose distinctive feature is the ability to interpret functional dependences in terms of fuzzy logic owing to the use of basic Bernstein polynomials as membership functions. The main objective of this work is the use of generalizing properties of a Bayesian SVR model with advantages of representation of neurofuzzy models in Bernstein form. 1. PROBLEM STATEMENT Let the nonlinear process being investigated be described by an unknown scalar function y( x ) , where x Î X = R n . We assume that y( x ) belong to some family of functions M and, in what follows, call it a model M from a given model space H. A model M( x, w, b ) parametrized by a vector of parameters w a
Data Loading...
