Multi-label optimal margin distribution machine
- PDF / 1,153,918 Bytes
- 20 Pages / 439.37 x 666.142 pts Page_size
- 58 Downloads / 214 Views
Multi-label optimal margin distribution machine Zhi-Hao Tan1
· Peng Tan1 · Yuan Jiang1 · Zhi-Hua Zhou1
Received: 6 May 2019 / Revised: 30 July 2019 / Accepted: 6 September 2019 © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2019
Abstract Multi-label support vector machine (Rank-SVM) is a classic and effective algorithm for multi-label classification. The pivotal idea is to maximize the minimum margin of label pairs, which is extended from SVM. However, recent studies disclosed that maximizing the minimum margin does not necessarily lead to better generalization performance, and instead, it is more crucial to optimize the margin distribution. Inspired by this idea, in this paper, we first introduce margin distribution to multi-label learning and propose multi-label Optimal margin Distribution Machine (mlODM), which optimizes the margin mean and variance of all label pairs efficiently. Extensive experiments in multiple multi-label evaluation metrics illustrate that mlODM outperforms SVM-style multi-label methods. Moreover, empirical study presents the best margin distribution and verifies the fast convergence of our method. Keywords Optimal margin distribution machine · Multi-label learning · Support vector machine · Margin theory
1 Introduction In contrast to traditional supervised learning, multi-label classification purports to build classification models for objects assigned with multiple labels simultaneously, which is a common learning paradigm in real-world tasks. In the past decades, it has attracted much attention (Zhang and Zhou 2014a). To name a few, in image classification, a scene image is usually annotated with several tags (Boutell et al. 2004); in text categorization, a docu-
Editors: Kee-Eung Kim and Jun Zhu.
B
Yuan Jiang [email protected] Zhi-Hao Tan [email protected] Peng Tan [email protected] Zhi-Hua Zhou [email protected]
1
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China
123
Machine Learning
ment may present multiple topics (McCallum 1999; Schapire and Singer 2000); in music information retrieval, a piece of music can convey various messages (Turnbull et al. 2008). To solve the multi-label tasks, a variety of methods have been proposed (Zhang and Zhou 2014a, Zhang et al. 2018), among which Rank-SVM (Elisseeff and Weston 2002) is one of the most eminent methods. It extended the idea of maximizing minimum margin in support vector machine (SVM) (Cortes and Vapnik 1995) to multi-label classification and achieved impressive performance. Specifically, the central idea of SVM is to search a large margin separator, i.e., maximizing the smallest distance from the instances to the classification boundary in a RKHS (reproducing kernel Hilbert space). Rank-SVM modified the definition of margin for label pairs and adapted maximizing margin strategy to deal with multi-label data, where a set of classifiers are optimized simultaneously. Benefiting from kernel tricks and co
Data Loading...
