Fashion Landmark Detection in the Wild
Visual fashion analysis has attracted many attentions in the recent years. Previous work represented clothing regions by either bounding boxes or human joints. This work presents fashion landmark detection or fashion alignment, which is to predict the pos
- PDF / 4,584,625 Bytes
- 17 Pages / 439.37 x 666.142 pts Page_size
- 51 Downloads / 264 Views
Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong, China {lz013,siyan,pluo,xtang}@ie.cuhk.edu.hk, [email protected] 2 Shenzhen Key Laboratory for Computer Vision and Pattern Recognition, Shenzhen Institutes of Advanced Technology, CAS, Shenzhen, China
Abstract. Visual fashion analysis has attracted many attentions in the recent years. Previous work represented clothing regions by either bounding boxes or human joints. This work presents fashion landmark detection or fashion alignment, which is to predict the positions of functional key points defined on the fashion items, such as the corners of neckline, hemline, and cuff. To encourage future studies, we introduce a fashion landmark dataset (The dataset is available at http://mmlab.ie.cuhk.edu. hk/projects/DeepFashion/LandmarkDetection.html.) with over 120K images, where each image is labeled with eight landmarks. With this dataset, we study fashion alignment by cascading multiple convolutional neural networks in three stages. These stages gradually improve the accuracies of landmark predictions. Extensive experiments demonstrate the effectiveness of the proposed method, as well as its generalization ability to pose estimation. Fashion landmark is also compared to clothing bounding boxes and human joints in two applications, fashion attribute prediction and clothes retrieval, showing that fashion landmark is a more discriminative representation to understand fashion images. Keywords: Clothes landmark detection · Cascaded deep convolutional neural networks · Attribute prediction · Clothes retrieval
1
Introduction
Visual fashion analysis has drawn lots of attentions recently, due to its wide spectrum of applications such as clothes recognition [1–3], retrieval [3–5], and recommendation [6,7]. It is a challenging task because of the large variations presented in the clothing items, such as the changes of poses, scales, and appearances. To reduce these variations, existing works tackled the problem by looking for informative regions, i.e. detecting the clothes bounding boxes [1,2] or the human joints [8,9]. We go beyond the above by studying a more discriminative representation, fashion landmark, which is the key-point located at the functional region of clothes, for example the neckline and the cuff. The first two authors contribute equally and share first authorship. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part II, LNCS 9906, pp. 229–245, 2016. DOI: 10.1007/978-3-319-46475-6 15
230
Z. Liu et al. (b)
Fashion Landmarks
Human Joints
(c)
left shoulder : left collar
(a.1)
(a.2)
left elbow : left sleeve
right elbow : right sleeve
left elbow : left sleeve
left hip : left waistline
(a.3)
(a.4)
left ankle : left hem
right ankle : right hem
left ankle : left hem
Fig. 1. Comparisons of fashion landmarks and human joints: (a.1) sample annotations for human joints, (a.2) sample annotations for fashion landmarks (a.3–4) typical deformation and scale variations present in clothing item
Data Loading...
