Sequential Labeling with Structural SVM Under an Average Precision Loss
The average precision (AP) is an important and widely-adopted performance measure for information retrieval and classification systems. However, owing to its relatively complex formulation, very few approaches have been proposed to learn a classifier by m
- PDF / 692,966 Bytes
- 11 Pages / 439.37 x 666.142 pts Page_size
- 71 Downloads / 204 Views
Abstract. The average precision (AP) is an important and widelyadopted performance measure for information retrieval and classification systems. However, owing to its relatively complex formulation, very few approaches have been proposed to learn a classifier by maximising its average precision over a given training set. Moreover, most of the existing work is restricted to i.i.d. data and does not extend to sequential data. For this reason, we herewith propose a structural SVM learning algorithm for sequential labeling that maximises an average precision measure. A further contribution of this paper is an algorithm that computes the average precision of a sequential classifier at test time, making it possible to assess sequential labeling under this measure. Experimental results over challenging datasets which depict human actions in kitchen scenarios (i.e., TUM Kitchen and CMU Multimodal Activity) show that the proposed approach leads to an average precision improvement of up to 4.2 and 5.7 % points against the runner-up, respectively. Keywords: Sequential labeling Loss-augmented inference
1
· Structural SVM · Average precision ·
Introduction and Related Work
Choosing appropriate performance measures plays an important role in developing effective information retrieval and classification systems. Common figures include the false positive and false negative rates, the precision and recall, and the F-measure which can all assess the accuracy of a prediction by comparing the predicted labels with given ground-truth labels. However, in applications such as information retrieval, it is often important to assess not only the accuracy of the predicted labels, but also that of a complete ranking of the samples. In classification, too, it is often preferable to evaluate the prediction accuracy at various trade-offs of precision and recall, to ensure coverage of multiple operating points. For both these needs, the average precision (a discretised version of the area under the precision-recall curve) offers a very informative performance measure. Amongst the various flavours of classification, sequential labeling, or tagging, refers to the classification of each of the measurements in a sequence. It is a very c Springer International Publishing AG 2016 A. Robles-Kelly et al. (Eds.): S+SSPR 2016, LNCS 10029, pp. 344–354, 2016. DOI: 10.1007/978-3-319-49055-7 31
Sequential Labeling with Structural SVM Under an Average Precision Loss
345
important task in a variety of fields including video analysis, bioinformatics, financial time series and natural language processing [8]. Unlike the classification of independent samples, the typical sequential labeling algorithms such as Viterbi (including their n-best versions [7]) do not provide multiple predictions at varying trade-offs of precision and recall, and therefore the computation of their average precision is not trivial. In the literature, a number of papers have addressed the average precision as a performance measure in the case of independent samples. For instance, [5] has studied
Data Loading...
