Depth-Aware Motion Magnification
This paper adds depth to motion magnification. With the rise of cheap RGB+D cameras depth information is readily available. We make use of depth to make motion magnification robust to occlusion and large motions. Current approaches require a manual drawn
- PDF / 4,326,784 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 0 Downloads / 221 Views
2
Delft University of Technology, Delft, The Netherlands {J.F.P.Kooij,J.C.vanGemert}@tudelft.nl Leiden University Medical Center, Leiden, The Netherlands
Abstract. This paper adds depth to motion magnification. With the rise of cheap RGB+D cameras depth information is readily available. We make use of depth to make motion magnification robust to occlusion and large motions. Current approaches require a manual drawn pixel mask over all frames in the area of interest which is cumbersome and errorprone. By including depth, we avoid manual annotation and magnify motions at similar depth levels while ignoring occlusions at distant depth pixels. To achieve this, we propose an extension to the bilateral filter for non-Gaussian filters which allows us to treat pixels at very different depth layers as missing values. As our experiments will show, these missing values should be ignored, and not inferred with inpainting. We show results for a medical application (tremors) where we improve current baselines for motion magnification and motion measurements.
Keywords: Motion magnification
1
· Bilateral filter · RGB+D
Introduction
Magnifying tiny motions in video [3,4] opened up a wealth of applications. Examples include: reconstructing speech exclusively from small visual vibrations [5], detecting a heart-beat either from blood flow [4] or from tiny head motions [6], magnifying muscle tremors [7], segmenting blood vessels [8] or estimating material properties by the way it moves [9]. In this paper we propose to only magnify motion at selected depth ranges, which makes motion magnification robust to occlusions and large motions at other depths. Robustness is especially important to open up new applications in the medical domain such as tremor assessment [10–12], where the interaction between doctor and patient should not be disturbed, and prerequisites for video processing should not limit the poses and exercises dictated by the medical protocol. Currently though, magnifying tiny motions requires that there are no occlusions or large motions [1,3,4]. A recent solution proposes to manually indicate the large motions by drawing a binary pixel mask on the frames of interest [2]. Electronic supplementary material The online version of this chapter (doi:10. 1007/978-3-319-46484-8 28) contains supplementary material, which is available to authorized users. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part VIII, LNCS 9912, pp. 467–482, 2016. DOI: 10.1007/978-3-319-46484-8 28
468
J.F.P. Kooij and J.C. van Gemert
(a) Frame from sequence 1
(c) original
(d) magnified [1]
(b) Frame from sequence 2
(e) CVPR’15 [2]
(f) ours
Fig. 1. Comparison of our and baseline magnification approaches when magnifying small motions in the background (here, body) behind moving occluders (here, trembling hands). (a), (b) For two sequences, the input image, depth map, and depthdependent magnification matte of one frame (black/white is zero/full magnification). (c)–(f) Space-time slices for the red lines in input images. Our
Data Loading...
