Trainable TV- $$L^1$$ L 1 model as recurrent nets for low-level vision
- PDF / 1,540,548 Bytes
- 9 Pages / 595.276 x 790.866 pts Page_size
- 110 Downloads / 176 Views
(0123456789().,-volV)(0123456789(). ,- volV)
S.I. : DEEP LEARNING APPROACHES FOR REALTIME IMAGE SUPER RESOLUTION (DLRSR)
Trainable TV-L1 model as recurrent nets for low-level vision Yuqiang Fang1,2 • Zhihao Ma2 • Hao Zheng3
•
Wanting Ji4
Received: 11 December 2019 / Accepted: 17 June 2020 Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract TV-L1 is a classical diffusion–reaction model for low-level vision tasks, which can be solved by a duality-based iterative algorithm. Considering the recent success of end-to-end learned representations, we propose a TV-LSTM network to unfold the duality-based iterations of TV-L1 into long short-term memory (LSTM) cells. In particular, we formulate the iterations as customized layers of a LSTM neural network. Then, the proposed end-to-end trainable TV-LSTMs can be naturally connected with various task-specific networks, e.g., optical flow, image decomposition and event-based optical flow estimation. Extensive experiments on optical flow estimation and structure ? texture decomposition have demonstrated the effectiveness and efficiency of the proposed method. Keywords Total variation Optical flow Recurrent network Image decomposition
1 Introduction The total variation (TV) has been introduced in Computer Vision by Rudin, Osher, and Fatemi (ROF) [1] as a regularization term for solving inverse problems. In this paper, we study a version of the ROF model that uses the L1 -norm as a measure of the fidelity. Given a fidelity term f ðxÞ 2 L1 ðXÞ, the TV-based model minimizes the following energy function: inf
x2BVðXÞ
TVðxÞ þ kkf ðxÞkL1 ðXÞ ;
& Hao Zheng [email protected] 1
State Key Laboratory of Complex Electromagnetic Environment Effects on Electronics and Information System, Luoyang 471003, People’s Republic of China
2
Science and Technology on Complex Electronic System Simulation Laboratory, Space Engineering University, Beijing 101416, People’s Republic of China
3
Key Laboratory of Intelligent Information Processing, Nanjing Xiaozhuang University, Nanjing 211171, People’s Republic of China
4
School of Natural and Computational Sciences, Massey University, Auckland 0632, New Zealand
ð1Þ
where BVðÞ represents the bounded variation (BV) space, TVðxÞ is the total variation term, and kf ðxÞkL1 ðXÞ is the data fidelity term. A duality-based iterative algorithm can be used to solve problem in Eq. (1), due to its simplicity, promising performance and profound theoretical foundations. The progression of these iterations through time resembles activations passing through different layers (index by the iteration count k) of a deep neural network (DNN). A question is then naturally raised: can we train an appropriately structured DNN to obtain the minimum of the energy function in Eq. (1)? We will demonstrate in this paper that when appropriately unfold, the duality-based iterations for solving Eq. (1) can be formulated as variants of long short-term memory (LSTM) cells. The resulting network, namely TV-LSTMs, outperforms the existing duality-bas
Data Loading...
