Learning Semantic Deformation Flows with 3D Convolutional Networks
Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self o
- PDF / 4,871,179 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 91 Downloads / 193 Views
Adobe Research, San Jose, CA, USA [email protected] University College London, London, UK [email protected]
Abstract. Shape deformation requires expert user manipulation even when the object under consideration is in a high fidelity format such as a 3D mesh. It becomes even more complicated if the data is represented as a point set or a depth scan with significant self occlusions. We introduce an end-to-end solution to this tedious process using a volumetric Convolutional Neural Network (CNN) that learns deformation flows in 3D. Our network architectures take the voxelized representation of the shape and a semantic deformation intention (e.g., make more sporty) as input and generate a deformation flow at the output. We show that such deformation flows can be trivially applied to the input shape, resulting in a novel deformed version of the input without losing detail information. Our experiments show that the CNN approach achieves comparable results with state of the art methods when applied to CAD models. When applied to single frame depth scans, and partial/noisy CAD models we achieve ∼60 % less error compared to the state-of-the-art.
1
Introduction
Shape deformation is a core component in 3D content synthesis. This problem has been well studied in graphics where low level, expert user manipulation is required [2,36]. It is acknowledged that this is an open and difficult problem, especially for deformations that follow semantic meaning, where very sparse high level information (e.g., make this shoe more durable) need to be extrapolated to a complex deformation. One way to solve this problem using traditional editing paradigms is through highly customized template matching [44], which does not scale. In this paper, we introduce a novel volumetric CNN, end-to-end trained for learning deformation flows on 3D data, which generalizes well to low fidelity models as well. CNNs have been shown to outperform hand-crafted features and domain knowledge engineered methods in many fields of computer vision. Promising applications to classification [23], dense segmentation [26], and more recently direct synthesis [8] and transformation [39,43] have been demonstrated. Electronic supplementary material The online version of this chapter (doi:10. 1007/978-3-319-46466-4 18) contains supplementary material, which is available to authorized users. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part VI, LNCS 9910, pp. 294–311, 2016. DOI: 10.1007/978-3-319-46466-4 18
Learning Semantic Deformation Flows with 3D Convolutional Networks
295
Fig. 1. Our 3D convolutional network (c) takes a volumetric representation (b) of an object (a) and high-level deformation intentions as input and predicts a deformation flow (d) at the output. Applying the predicted deformation flow to the original object yields a high quality novel deformed version (e) that displays the high-level transformation intentions (In this illustration, the car is deformed to be more compact).
Encouraged by these advances, we propose using the
Data Loading...
