Objective Evaluation Criteria for 2D-Shape Estimation Results of Moving Objects
- PDF / 970,303 Bytes
- 9 Pages / 612 x 792 pts (letter) Page_size
- 83 Downloads / 147 Views
Objective Evaluation Criteria for 2D-Shape Estimation Results of Moving Objects Roland Mech Institut f¨ur Theoretische Nachrichtentechnik und Informationsverarbeitung, Universit¨at Hannover, Appelstrasse 9A, 30167 Hannover, Germany Email: [email protected]
´ Ferran Marques Universitat Polit`ecnica de Catalunya, Campus Nord—M`odul D5, C/ Jordi Girona 1-3, Barcelona 08034, Spain Email: [email protected] Received 3 August 2001 and in revised form 15 January 2002 The objective evaluation of 2D-shape estimation results for moving objects in a video sequence is still an open problem. First approaches in the literature evaluate the spatial accuracy and the temporal coherency of the estimated 2D object shape. Thereby, it is not distinguished between several estimation errors located around the object contour and a few, but larger, estimation errors. Both cases would lead to similar evaluation results, although the 2D-shapes would be visually very different. To overcome this problem, in this paper, a new evaluation approach is proposed. In it, the evaluation of the spatial accuracy and the temporal coherency is based on the mean and the standard deviation of the 2D-shape estimation errors. Keywords and phrases: shape evaluation, objective evaluation, shape estimation, segmentation, video object, MPEG.
1.
INTRODUCTION
One major problem in the development of algorithms for 2D-shape estimation of moving objects, is to assess the quality of the estimation results. Up to now, mainly subjective evaluation, that is, tape viewing, has been used in order to decide upon the quality of a certain algorithm. Although this is very helpful and gives already some indication of the resulting quality, this procedure very much depends on the subjective conditions, that is, the attending people, the time of viewing, the used video equipment, and so forth. In the sequel, since we are only dealing with 2D-shape, the term “shape” will be used. In the literature, first approaches for objective evaluation of shape estimation results can be found [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. During the standardization work of ISO/MPEG-4 [13], within the core-experiment on automatic segmentation of moving objects it became necessary to compare the results of different proposed shape estimators, not only by subjective evaluation, but also by objective evaluation. The proposal for objective evaluation [9], which was agreed by the working group, uses an a priori known shape to evaluate the estimation result. This shape is denoted as reference shape, and has to be created once in an appropriate way, for example, by manual segmentation of each frame, by color-keying, or using synthetic image sequences, where
shapes are known. The shape of a moving object can be represented by a binary mask, where a pel has object label if it is inside the object and background label if it is outside the object. In [9], such a mask is called object mask. There are two objective evaluation criteria defined: (i) the first criterion evaluates the spatial accuracy of an
Data Loading...
