A Continuous Optimization Approach for Efficient and Accurate Scene Flow
We propose a continuous optimization method for solving dense 3D scene flow problems from stereo imagery. As in recent work, we represent the dynamic 3D scene as a collection of rigidly moving planar segments. The scene flow problem then becomes the joint
- PDF / 4,341,810 Bytes
- 17 Pages / 439.37 x 666.142 pts Page_size
- 10 Downloads / 283 Views
Georgia Institute of Technology, Atlanta, USA {zlv30,cbeal3}@gatech.edu, [email protected] 2 Georgia Tech Research Institute, Atlanta, USA [email protected] 3 iRobot Corporation, London, UK [email protected] 4 Oregon State University, Corvallis, USA [email protected]
Abstract. We propose a continuous optimization method for solving dense 3D scene flow problems from stereo imagery. As in recent work, we represent the dynamic 3D scene as a collection of rigidly moving planar segments. The scene flow problem then becomes the joint estimation of pixel-to-segment assignment, 3D position, normal vector and rigid motion parameters for each segment, leading to a complex and expensive discrete-continuous optimization problem. In contrast, we propose a purely continuous formulation which can be solved more efficiently. Using a fine superpixel segmentation that is fixed a-priori, we propose a factor graph formulation that decomposes the problem into photometric, geometric, and smoothing constraints. We initialize the solution with a novel, high-quality initialization method, then independently refine the geometry and motion of the scene, and finally perform a global nonlinear refinement using Levenberg-Marquardt. We evaluate our method in the challenging KITTI Scene Flow benchmark, ranking in third position, while being 3 to 30 times faster than the top competitors (x37 [10] and x3.75 [24]). Keywords: Scene flow uous optimization
1
· Stereo · Optical flow · Factor graph · Contin-
Introduction
Understanding the geometry and motion within urban scenes, using either monocular or stereo imagery, is an important problem with increasingly relevant applications such as autonomous driving [15], urban scene understanding Electronic supplementary material The online version of this chapter (doi:10. 1007/978-3-319-46484-8 46) contains supplementary material, which is available to authorized users. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part VIII, LNCS 9912, pp. 757–773, 2016. DOI: 10.1007/978-3-319-46484-8 46
758
Z. Lv et al.
Fig. 1. An overview of our system: we estimate the 3D scene flow w.r.t. the reference image (the red bounding box), a stereo image pair and a temporal image pair as input. Image annotations show the results at each step. We assign a motion hypothesis to each superpixel as an initialization and optimize the factor graph for more accurate 3D motion. Finally, after global optimization, we show a projected 2D flow map in the reference frame and its 3D scene motion (static background are plotted in white). (Color figure online)
[13,15,26], video analysis [7], dynamic reconstruction [12,14], etc. In contrast to separately modeling 3D geometry (stereo) and characterizing the movement of 2D pixels in the image (optical flow), the scene flow problem is to characterize the 3D motion of points in the scene [20] (Fig. 1). Scene flow in the context of stereo sequences was first investigated by Huguet et al. [6]. Recent work [10,19,23] has shown that explicitly reasoning about th
Data Loading...
