Convolutional Sketch Inversion
In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computer-generated face sketches with different styles and corre
- PDF / 4,380,217 Bytes
- 15 Pages / 439.37 x 666.142 pts Page_size
- 75 Downloads / 247 Views
ct. In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computergenerated face sketches with different styles and corresponding face images by expanding existing unconstrained face data sets. We then train models achieving state-of-the-art results on both computer-generated sketches and hand-drawn sketches by leveraging recent advances in deep learning such as batch normalization, deep residual learning, perceptual losses and stochastic optimization in combination with our new dataset. We finally demonstrate potential applications of our models in fine arts and forensic arts. In contrast to existing patch-based approaches, our deep-neural-network-based approach can be used for synthesizing photorealistic face images by inverting face sketches in the wild. Keywords: Deep neural network · Face synthesis · Face recognition Fine arts · Forensic arts · Sketch inversion · Sketch recognition
1
·
Introduction
Portrait and self-portrait sketches have an important role in art. From an art historical perspective, self-portraits serve as historical records of what the artists looked like. From the perspective of an artist, self-portraits can be seen as a way to practice and improve one’s skills without the need for a model to pose. Portraits of others further serve as memorabilia and a record of the person in the portrait. Artists most often are able to easily capture recognizable features of a person in their sketches. Therefore, hand-drawn sketches of people have further applications in law enforcement. Sketches of suspects drawn based on eye-witness accounts are used to identify suspects, either in person or from catalogues of mugshots (Fig. 1). However, a challenging task that remains is photorealistic face image synthesis from face sketches in uncontrolled conditions. That is, at present, there exist no sketch inversion models that are able to perform in realistic conditions. These Y. G¨ u¸cl¨ ut¨ urk and U. G¨ u¸cl¨ u contributed equally to this work. c Springer International Publishing Switzerland 2016 G. Hua and H. J´ egou (Eds.): ECCV 2016 Workshops, Part I, LNCS 9913, pp. 810–824, 2016. DOI: 10.1007/978-3-319-46604-0 56
Convolutional Sketch Inversion ground truth
sketch
deep neural network
811
inverse sketch
Fig. 1. Demonstration of our convolutional sketch inversion models. Our models invert face sketches to synthesize photorealistic face images. Each row shows the sketch inversion/image synthesis pipeline that transforms a different sketch of the same face to a different image of the same face via a different deep neural network. Each deep neural network layer is represented by the top three principal components of its feature maps.
conditions are characterized by changes in expression, pose, lighting condition and image quality, as well as the presence of varying amounts of background clutter and occlusions. Here, we use DNNs to tackle the problem of inverting face sketche
Data Loading...
