Multi-view Inverse Rendering Under Arbitrary Illumination and Albedo
3D shape reconstruction with multi-view stereo (MVS) relies on a robust evaluation of photo consistencies across images. The robustness is ensured by isolating surface albedo and scene illumination from the shape recovery, i.e. shading and colour variatio
- PDF / 9,106,959 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 112 Downloads / 214 Views
Abstract. 3D shape reconstruction with multi-view stereo (MVS) relies on a robust evaluation of photo consistencies across images. The robustness is ensured by isolating surface albedo and scene illumination from the shape recovery, i.e. shading and colour variation are regarded as a nuisance in MVS. This yields a gap in the qualities between the recovered shape and the images used. We present a method to address it by jointly estimating detailed shape, illumination and albedo using the initial shape robustly recovered by MVS. This is achieved by solving the multi-view inverse rendering problem using the geometric and photometric smoothness terms and the normalized spherical harmonics illumination model. Our method allows spatially-varying albedo and per image illumination without any prerequisites such as training data or image segmentation. We demonstrate that our method can clearly improve the 3D shape and recover illumination and albedo on real world scenes. Keywords: Multi-view stereo · Shape from shading · Inverse rendering
1
Introduction
3D reconstruction from multiple images is becoming a basic tool in various applications, e.g. CAD model creation [1] and 3D mapping [2] thanks to softwares based on structure-from-motion (SfM) [3–5] followed by multi-view stereo (MVS) [6–8]. 3D reconstruction typically begins with SfM from imagery. MVS is then used to reconstruct surface while finding more correspondences via pixel/feature matching. Since this is a non-trivial task as real images are taken under a variety of conditions, photo consistencies among images are robustly computed [9,10] in order to suppress the appearance changes among images. Such matching schemes work well for recovering distinctive objects but often fail with weakly textured objects. In contrast to MVS, Shape from Shading (SfS) [11] recovers surface normals from a single image by solving the inverse problem of image rendering formed as a function of surface normal, albedo and shading. Since shading directly represents (high frequency) shape information under the smooth illumination, SfS can extract fine and detailed surface normal from shading variation. Although SfS c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part III, LNCS 9907, pp. 750–767, 2016. DOI: 10.1007/978-3-319-46487-9 46
Multi-view Inverse Rendering Under Arbitrary Illumination and Albedo
751
Fig. 1. Multi-view inverse rendering. We present a multi-view inverse rendering method for joint estimating of complex shape, spatially-varying albedo and per image illuminations. Table 1. Comparison with closely related methods. Our method can recover detailed shape from images taken with both spatially-varying albedo and per image (temporally-varying) illumination without any prerequisite. Method
Albedo
Illumination
Wu et al. [20]
Uniform
Constant among images
Shan et al. [21]
Spatially varying Cloudy images (→ temporally-varying)
Zollh¨ ofer et al. [22] Spatially varying Constant among images Proposed
Spatially varying Per image (temporally
Data Loading...
