Robust Control Methods for On-Line Statistical Learning
- PDF / 194,950 Bytes
- 7 Pages / 600 x 792 pts Page_size
- 97 Downloads / 186 Views
obust Control Methods for On-Line Statistical Learning Enrico Capobianco CWI, Kruislaan 413, Amsterdam 1098-SJ, The Netherlands Email: [email protected] Received 8 January 2001 and in revised form 19 April 2001 The issue of controlling that data processing in an experiment results not affected by the presence of outliers is relevant for statistical control and learning studies. Learning schemes should thus be tested for their capacity of handling outliers in the observed training set so to achieve reliable estimates with respect to the crucial bias and variance aspects. We describe possible ways of endowing neural networks with statistically robust properties by defining feasible error criteria. It is convenient to cast neural nets in state space representations and apply both Kalman filter and stochastic approximation procedures in order to suggest statistically robustified solutions for on-line learning. Keywords and phrases: artificial learning, statistical control algorithms, robustness and efficiency of estimators, maximum likelihood inference.
1. INTRODUCTION This work represents, to our knowledge, one of the few attempts done to endow artificial neural networks learning schemes with statistical robustness properties. It is of great interest to realize why the statistical robustness field has not found a wide ground of applications outside statistics, given the fact that more and more applications of algorithms proposed by researchers in different fields are similar in the data sets they use and the goals of the analysis, usually prediction, one example being the very popular time series competitions. In many cases of empirical work dealing with a modelling experiment, one often finds out from the diagnostic checks run on the observed data that there is enough evidence for not relying on the convenient Gaussian probability distribution which characterize the disturbances driving the stochastic processes and thus their observed realizations. This fact may be due to the presence of outliers, occurring with small probability, or because of a particular nature of the data generating process underlying the data, like in financial time series. Simply ignoring the deviations from the Gaussian distribution is one way of conducting inference, which is feasible when these deviations are mild. But a stronger evidence for rejecting normality should bring the researcher to consider more robust statistical learning procedures, suitable to deliver more reliable parameter estimates and model predictions. In Section 2, we briefly present some well-known algorithms and their relations, starting from the stochastic approximation scheme. In Section 3, the likelihood-based standard inference and related quasi-likelihood ideas are presented, while in Section 4 the above framework is robustified with the definition of M-estimators. In Section 5,
some examples of useful M-estimation applications are introduced, and together with the loss functions the correspondent influence functions are given. Section 6 is for the conclusions and the appendix shows
Data Loading...
