Instance-Level Coupled Subspace Learning for Fine-Grained Sketch-Based Image Retrieval

Fine-grained sketch-based image retrieval (FG-SBIR) is a newly emerged topic in computer vision. The problem is challenging because in addition to bridging the sketch-photo domain gap, it also asks for instance-level discrimination within object categorie

  • PDF / 1,743,167 Bytes
  • 16 Pages / 439.37 x 666.142 pts Page_size
  • 49 Downloads / 168 Views

DOWNLOAD

REPORT


3

Pattern Recognition and Intelligent System Laboratory, Beijing University of Posts and Telecommunications, Beijing, China {peng.xu,qiyg,mazhanyu,guojun}@bupt.edu.cn 2 National Lab of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China {qyyin,wangliang}@nlpr.ia.ac.cn SketchX Lab, School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK [email protected]

Abstract. Fine-grained sketch-based image retrieval (FG-SBIR) is a newly emerged topic in computer vision. The problem is challenging because in addition to bridging the sketch-photo domain gap, it also asks for instance-level discrimination within object categories. Most prior approaches focused on feature engineering and fine-grained ranking, yet neglected an important and central problem: how to establish a finegrained cross-domain feature space to conduct retrieval. In this paper, for the first time we formulate a cross-domain framework specifically designed for the task of FG-SBIR that simultaneously conducts instancelevel retrieval and attribute prediction. Different to conventional phototext cross-domain frameworks that performs transfer on category-level data, our joint multi-view space uniquely learns from the instance-level pair-wise annotations of sketch and photo. More specifically, we propose a joint view selection and attribute subspace learning algorithm to learn domain projection matrices for photo and sketch, respectively. It follows that visual attributes can be extracted from such matrices through projection to build a coupled semantic space to conduct retrieval. Experimental results on two recently released fine-grained photo-sketch datasets show that the proposed method is able to perform at a level close to those of deep models, while removing the need for extensive manual annotations. Keywords: Fine-grained SBIR · Attribute supervision diction · Multi-view domain adaptation

1

· Attribute pre-

Introduction

Sketch-based image retrieval (SBIR) is traditionally casted into a classification problem, and most prior art evaluates retrieval performance at category-level. c Springer International Publishing Switzerland 2016  G. Hua and H. J´ egou (Eds.): ECCV 2016 Workshops, Part I, LNCS 9913, pp. 19–34, 2016. DOI: 10.1007/978-3-319-46604-0 2

20

P. Xu et al.

[1,2,4,8,13,14,16,17,19,24], i.e. given a query sketch, the goal is to discover photos with the same class label. However, it was recently argued [12,28] that SBIR is more reasonable to be conducted at a fine-grained level, where instead of conducting retrieval across object categories, it focuses on finding similar photos to the query sketch within specific categories. By specifically exploring the unique fine-grained visual characteristics captured in human sketches, finegrained SBIR is likely to transform the traditional landscape of image retrieval by introducing a new form of user interaction that underpins the ubiquitous commercial adoption of SBIR technology. Shared with conventional category-level SBIR, the co