Variable Selection by Perfect Sampling
- PDF / 733,094 Bytes
- 8 Pages / 600 x 792 pts Page_size
- 23 Downloads / 189 Views
ariable Selection by Perfect Sampling Yufei Huang Department of Electrical and Computer Engineering, State University of New York at Stony Brook, Stony Brook, NY 11794-2350, USA Email: [email protected]
Petar M. Djuri Department of Electrical and Computer Engineering, State University of New York at Stony Brook, Stony Brook, NY 11794-2350, USA Email: [email protected] Received 31 July 2001 and in revised form 30 September 2001 Variable selection is very important in many fields, and for its resolution many procedures have been proposed and investigated. Among them are Bayesian methods that use Markov chain Monte-Carlo (MCMC) sampling algorithms. A problem with MCMC sampling, however, is that it cannot guarantee that the samples are exactly from the target distributions. This drawback is overcome by related methods known as perfect sampling algorithms. In this paper, we propose the use of two perfect sampling algorithms to perform variable selection within the Bayesian framework. They are the sandwiched coupling from the past (CFTP) algorithm and the Gibbs coupler. We focus our attention to scenarios where the model coefficients and noise variance are known. We indicate the condition under which the sandwiched CFTP can be applied. Most importantly, we design a detailed scheme to adapt the Gibbs coupler algorithm to variable selection. In addition, we discuss the possibilities of applying perfect sampling when the model coefficients and noise variance are unknown. Test results that show the performance of the algorithms are provided. Keywords and phrases: variable selection, Markov chain Monte-Carlo, the Gibbs sampler, perfect sampling, coupling from the past, the Gibbs coupler.
1. INTRODUCTION The problem of variable selection is of great importance in many science and engineering areas. In time-varying system identification, variable selection is used to select a set of basis sequences for representing the time varying coefficients [1]. In radar or sonar, it is applied to detection of number of reflections and for determination of relevant parameters of signal models [2]. In visual recognition [3, 4], variable selection plays an important role in determining the identity of the desired object from the data. In neural networks [5], variable selection is a necessary procedure in building neural as well as other classifiers. Generally speaking, the basic goal of variable selection is to identify from a pool of available predictors the best subset with satisfactory predictive performance. For its resolution, many procedures have been proposed and investigated. A natural and direct approach is to exhaust all the possible subsets. However, this exhaustive search method become computationally burdensome if the available number of predictors is large. As a result, computationally efficient suboptimal ap-
proaches like genetic algorithms are used in practice [6]. In recent years, Bayesian signal processing methods that use Markov chain Monte-Carlo (MCMC) sampling algorithms [7, 8, 9, 10, 11] have drawn much attention. Th
Data Loading...
