Matching a composite sketch to a photographed face using fused HOG and deep feature models
- PDF / 1,396,547 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 53 Downloads / 174 Views
ORIGINAL ARTICLE
Matching a composite sketch to a photographed face using fused HOG and deep feature models Jiayi Xu1 · Xinying Xue1 · Yitiao Wu1 · Xiaoyang Mao2 Accepted: 5 September 2020 © Springer-Verlag GmbH Germany, part of Springer Nature 2020
Abstract In this paper, we focus on the research of matching a computer-generated composite face sketch to a photograph. This is of great importance in the field of criminal investigation. To blend the different facial representation modalities, we propose a robust feature model by combining pixel-level features extracted from multi-scale key face patches and high-level features learned from a pre-trained deep learning-based model. At first, texture features are captured by a two-level histogram of oriented gradient descriptor, considering both the overall structure and local details. The semantic-level facial characteristics are analyzed through the high-level features of the Visual Geometry Group-Face (VGG-Face) network. Next, feature similarities between each sketch/photograph pair are measured by feature distance. Then, adaptive weights are assigned to each feature similarity, and score level fused according to their visual saliency contribution. Finally, the fused feature similarity is evaluated for matching purposes. After experimenting on the Pattern Recognition and Image Processing-Viewed Software-Generated Composite (PRIP-VSGC) database and the expanded University of Malta Composite Face Sketch (UoM-SGFS) database, it is found that this framework could achieve more satisfying results compared to the existing methods. Keywords Composite sketch · HOG feature · VGG-face feature · Adaptive feature weight
1 Introduction With the development of facial recognition technology, law enforcement departments are able to identify the information regarding suspects with greater effectiveness. Sometimes, a clear picture of a suspect’s face cannot be obtained. In such cases, a facial sketch based on an eyewitness’s description of the suspect’s characteristics becomes the primary way to determine the suspect’s facial information. It would be different to match a photograph of a person’s face to a sketch of the face than to match a photograph to another photograph. This is due to the fact that a sketch has a higher abstraction
B
Xiaoyang Mao [email protected] Jiayi Xu [email protected] Xinying Xue [email protected] Yitiao Wu [email protected]
1
Hangzhou Dianzi University, Hangzhou, China
2
University of Yamanashi, Kofu, Japan
degree and fewer details than a photograph. Besides, since a sketch is influenced by the eyewitness’s memory and synthesis, only some of the facial features can be restored in the resulting sketch. In such cases, the direct application of photograph-to-photograph facial recognition algorithms cannot achieve high recognition rates. In the sketch–photograph recognition area, feature representation that could precisely capture invariant features under different modalities is the key to success. In order to directly match facial sketches to facial p
Data Loading...
