Multi-Modes Cascade SVMs: Fast Support Vector Machines in Distributed System
Machine learning is one field of Artificial Intelligence (AI) to help machines solve problems. Support Vector Machines (SVMs) are classic methods in machine learning field and are also used in many other AI fields. However, the model training is very time
- PDF / 1,289,710 Bytes
- 8 Pages / 439.37 x 666.14 pts Page_size
- 28 Downloads / 174 Views
)
1
Science and Technology on Parallel and Distributed Laboratory, National University of Defense Technology, Changsha, China [email protected], [email protected], [email protected], [email protected] 2 College of Computer Science and Electronic Engineering, Hunan University Changsha, Changsha 410073, Hunan, China [email protected]
Abstract. Machine learning is one field of Artificial Intelligence (AI) to help machines solve problems. Support Vector Machines (SVMs) are classic methods in machine learning field and are also used in many other AI fields. However, the model training is very time-consuming when meeting large scale data sets. Some efforts have been devoted to develop it for distributed memory clusters. Their bottleneck is the training phase, where the structure is immobile. In this paper, we propose Multi-Modes Cascade SVMs (MMCascadeSVMs) to adaptively reshape the structure. MMCascadeSVMs employs analytical hierarchy process to qualitatively analyse the similarity between adjacent hierarchies. Furthermore, MMCascadeSVMs leverages a two-stage algorithm: the first stage is to compute the similarity between two adjacent models, and the similarity is built for halting criterion. The second stage is to predict new samples based on multi models. MMCascadeSVMs can modify the structure of SVMs in distributed systems and reduce training time. Experiments show that our approach significantly reduces the total computation cost. Keywords: Cascade SVMs · Analytical hierarchy process
1
Introduction
In many real world problems such as text classification, image recognition, financial markets, Support Vector Machines (SVMs) is a popular classification tool [2, 4, 5]. SVMs establishes the separating hyperplane by discovering the support vectors (SVs). However, the training of SVMs requires to solve a quadratic programming (QP) problem with n inequality constraints and one equality constraint [8–10], where n is the training set size. The iterative process can’t be easily paralleled for its recurrence relation. To parallelize SVMs, some “cascade methods” [6, 7, 11] are proposed. They divide the data into several subproblems, then train local models on their own nodes. They pass the local models to the higher layer and combine them in a “tree” way. The process repeats until a single model remains. These works only differ from the way dividing data
© Springer Nature Singapore Pte Ltd. 2017 K. Kim and N. Joukov (eds.), Information Science and Applications 2017, Lecture Notes in Electrical Engineering 424, DOI 10.1007/978-981-10-4154-9_51
444
L. Cui et al.
sets and the concept passing to higher layers. The computational process is still slow when the data set is large, which narrows the applicable scope of cascade SVMs. We notice the fact that: (i) cascade SVMs are time-consuming in higher layers but identify few support vectors. We call those layers as ineffective structure; (ii) the model in the highest layer is not always the best one; (iii) the nodes utilization rate is low in higher layers. Motivated by this insight,
Data Loading...
