Gaussian bandwidth selection for manifold learning and classification
- PDF / 2,595,871 Bytes
- 37 Pages / 439.37 x 666.142 pts Page_size
- 76 Downloads / 219 Views
Gaussian bandwidth selection for manifold learning and classification Ofir Lindenbaum1
· Moshe Salhov2 · Arie Yeredor1 · Amir Averbuch2
Received: 29 May 2018 / Accepted: 16 May 2020 © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2020
Abstract Kernel methods play a critical role in many machine learning algorithms. They are useful in manifold learning, classification, clustering and other data analysis tasks. Setting the kernel’s scale parameter, also referred to as the kernel’s bandwidth, highly affects the performance of the task in hand. We propose to set a scale parameter that is tailored to one of two types of tasks: classification and manifold learning. For manifold learning, we seek a scale which is best at capturing the manifold’s intrinsic dimension. For classification, we propose three methods for estimating the scale, which optimize the classification results in different senses. The proposed frameworks are simulated on artificial and on real datasets. The results show a high correlation between optimal classification rates and the estimated scales. Finally, we demonstrate the approach on a seismic event classification task. Keywords Dimensionality reduction · Kernel methods · Diffusion maps · Classification
1 Introduction Dimensionality reduction is an essential step in numerous machine learning tasks. Methods such as Principal Component Analysis (PCA) (Jolliffe 2002), Multidimensional Scaling (MDS) (Kruskal 1977), Isomap (Tenenbaum et al. 2000) and Local Linear Embedding (Roweis and Saul 2000) aim to extract essential information from high-dimensional data points based on their pairwise connectivities. Graph-based kernel methods such as Laplacian Eigenmaps (Belkin and Niyogi 2001) and Diffusion
Responsible editor: Jieping Ye.
B
Ofir Lindenbaum [email protected]
1
School of Electrical Engineering, Tel Aviv University, Tel Aviv, Israel
2
School of Computer Science, Tel Aviv University, Tel Aviv, Israel
123
O. Lindenbaum et al.
Maps (DM) (Coifman and Lafon 2006), construct a positive semi-definite kernel based on the multidimensional data points to recover the underlying structure of the data. Such methods have been proven effective for tasks such as clustering (Luo 2011), classification (Lindenbaum et al. 2015), manifold learning (Lin et al. 2006) and many more. Kernel methods rely on computing a distance function (usually Euclidean) between all pairs of data points x i , x j ∈ X ∈ R D×N and application of a data dependent kernel function. This kernel should encode the inherited relations between high-dimensional data points. An example for a kernel that encapsulates the Euclidean distance takes the form ||x i − x j ||2 = K i, j , (1.1) K(x i , x j ) K (where · denotes the Euclidean norm). As shown, for example, in Roweis and Saul (2000), Coifman and Lafon (2006), spectral analysis of such a kernel provides an efficient representation of the lower (d-) dimensional data (where d D) embedded in the ambient space. Devising
Data Loading...
