Advanced three-dimensional multi-view video coding and evaluation techniques.

Abstract

3D video services constitute the next step in multimedia services, as they give the chance of more natural visualisation and provide a sense of "being there". A lot of research effort has been put towards to realising 3D video services, especially in the context of stereoscopic video. 3D multi-view video is a step beyond stereoscopic video, creating much wider scene navigation range and improved user interaction, despite the higher source data size. With the aid of the extracted scene geometry and depth information, any arbitrary viewpoint can be reconstructed. The level of research in multi-view video is not as mature as the level of research in stereoscopic video, although there is a lot of ongoing work towards the realisation of practical multi-view video based 3D video applications. This thesis addresses compression and quality assessment related aspects of 3D multi-view video, for reduced bandwidth usage and more reliable evaluation of perceived quality. In the first part of the thesis, efficient compression algorithms for multi-view video with depth information that take into account several constraints are studied. These include the ease of viewpoint scalability and fast viewpoint random access. To be standards conformant, the proposed methods are implemented on the multi-view codec standard. The second part of the thesis studies processing and block based coding approaches for depth map video sequences, taking into account their special characteristics and effects on the 3D scene reconstruction process. Some state-of-the-art compression approaches, like reduced resolution coding, are extended to exploit scene texture and geometry information for improved performance. The last part of the thesis is devoted to the quality assessment problem for synthesized camera viewpoints, a core element of multi-view based free-viewpoint video applications. Depth based rendering related aspects are exploited to quantify the objective quality of synthesized scenes in an improved way, by extending the state-of-the-art 2D video quality assessment tools