Graphics and Media Lab

Approaches to multiview video compression

• independent views compression
• joint multiview video encoding in consideration of inter view information redundancy (e.g. H.264 MVC amendment)
• encoding of 2D and depth with further multiview generation

Initial 3D video compression scheme using 2D+depth representation

Proposed 3D video compression scheme

Depth maps contain uniform areas and though very good compression rates are achieved. Depth maps have very good correlation with 2D image so this information is used for 3D video compression.

Results

We used following codecs for testing:

• H.264 Reference software for Multiview Video Coding (JMVC 8.5) was used for single-view and multiview compression
• JPEG2000, x264, Xvid for depth key frames compression
• YUVsoft Depth Propagation for depth key frames propagation
• YUVsoft Depth Upscale for upscaling low resolution compressed depth key frames

Average quality for test data set. Results of 4 different compression rates assessment.

Problems and further work

• Depth Propagation algorithm is of high computational complexity. Using propagation of downscaled key frames and Depth Upscale can significantly improve speed of processing
• Construction of multiview video from depth maps. The final quality highly depends both from depth map generation way and from multiview from 2D+depth generation algorithm
• Depth key frames are set at constant intervals now. Selection of position of keyframes according to features of the video should Improve quality and compression rate
• Quality measurement.Traditional metrics like PSNR and SSIM do not cover all aspects of 3D Video quality

Projects

Keyframe-based Depth Compression (Update)

Keyframe-based Depth Compression with Depth Propagation for Multiview Video Coding

Team

• Dmitriy S. Vatolin, Lomonosov Moscow State University
• Sergey Matyunin,

Acknowledgements

This work is partially supported by the Intel/Cisco Video-Aware Wireless Network (VAWN) Program