Saliency-driven rate-distortion optimization for 360-degree image coding
- PDF / 952,327 Bytes
- 21 Pages / 439.37 x 666.142 pts Page_size
- 102 Downloads / 254 Views
Saliency-driven rate-distortion optimization for 360-degree image coding Jui-Chiu Chiang 1
1
2
& Cheng-Yu Yang & Bhishma Dedhia & Yi-Fan Char
1
Received: 19 November 2019 / Revised: 17 September 2020 / Accepted: 7 October 2020 # Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
360-degree images allow an immersive experience. They offer multiple views of a scene and the viewpoint can be selected by the user. However, the huge amount of data that is necessary for real-time transmission of 360-degree image and video requires efficient coding techniques, particularly for virtual reality (VR) and augmented reality (AR) applications. The viewer is only interested in a part of the scene so compressing the entire scene with equal quality is inefficient. This study initially constructs a saliency model of the 360-degree image and then a visual attention guided coding scheme is developed using a predicted saliency map. For saliency prediction, two methods of saliency prediction are used and the results are fused, to address the problem of geometry distortion in the ERP (Equirectangular Projection) format. A smoothing-based optimization is then realized in the spherical domain to improve the saliency map. Using the saliency map of the 360-degree image, the distortion of the rate-distortion optimization is modified to ensure a better visual experience. The experimental results show that the viewports of greatest interest are rendered with the highest quality and there is a maximum of 14.33% reduction in the bitrate when the quality measurement is performed in these regions. Keywords 360-degree image . ERP . Saliency prediction . Rate-distortion optimization
1 Introduction Advances in hardware and software technology bring more applications of VR and AR to our life, such as computer games, live events, and remote medicine. VR provides users with a new
* Jui-Chiu Chiang [email protected]
1
Department of Electrical Engineering, Advanced Institute of Manufacturing with High-tech Innovations, National Chung Cheng University, Chia-Yi, Taiwan
2
Department of Electrical Engineering, Indian Institute of Technology, Bombay, India
Multimedia Tools and Applications
media experience, offering the freedom to explore 360-degree content and giving users visual experience that is more realistic and immersive. This kind of service is available on popular image/video platforms, such as street exploration through Google map or 360-degree content sharing on social platforms. 360-degree images are usually described in terms of longitudes and latitudes or in terms of 3D coordinates on a spherical surface. They are defined as panoramic images, which include information in 360 degrees in the horizontal direction and 180 degrees in the vertical direction. 360-degree images are termed omnidirectional images and spherical images. For efficient storage and transmission, projections are used for 360degree images that convert each 3D coordinate to a location in the specific 2D plane. The viewport image is generated
Data Loading...
