Perception-oriented video coding based on texture analysis and synthesis

ABSTRACT Perception-oriented video coding based on texture analysis and synthesis has gained importance over the past few years. Hence, the present paper overviews a selection of related approaches that have been proposed over the past decades. They are also referred to as content-based video coding (CBVC) methods in this paper. For better insight into CBVC, an overview on texture analysis and synthesis is also given. Furthermore, the principles common to a careful selection of CBVC methods are depicted and the requirements of each of the fundamental modules are extensively discussed in the context of the limitations of state-of-the-art hybrid video codecs like H.264/AVC

A New Fast Motion Estimation and Mode Decision algorithm for H.264 Depth Maps encoding in Free Viewpoint TV

In this paper, we consider a scenario where 3D scenes are modeled through a View+Depth representation. This representation is to be used at the rendering side to generate synthetic views for free viewpoint video. The encoding of both type of data (view and depth) is carried out using two H.264/AVC encoders. In this scenario we address the reduction of the encoding complexity of depth data. Firstly, an analysis of the Mode Decision and Motion Estimation processes has been conducted for both view and depth sequences, in order to capture the correlation between them. Taking advantage of this correlation, we propose a fast mode decision and motion estimation algorithm for the depth encoding. Results show that the proposed algorithm reduces the computational burden with a negligible loss in terms of quality of the rendered synthetic views. Quality measurements have been conducted using the Video Quality Metric

Towards a model of how designers mentally categorise design information

This study aims to explore how designers mentally categorise design information during the early sketching performed in the generative phase. An action research approach is particularly appropriate for identifying the various sorts of design information and the cognitive operations involved in this phase. Thus, we conducted a protocol study with eight product designers based on a descriptive model derived from cognitive psychological memory theories. Subsequent protocol analysis yielded a cognitive model depicting the mental categorisation of design information processing performed by designers. This cognitive model included a structure for design information (high, middle, and low levels) and linked cognitive operations (association and transformation). Finally, this paper concludes by discussing directions for future research on the development of new computational tools for designers

Confessions of a live coder

This paper describes the process involved when a live coder decides to learn a new musical programming language of another paradigm. The paper introduces the problems of running comparative experiments, or user studies, within the field of live coding. It suggests that an autoethnographic account of the process can be helpful for understanding the technological conditioning of contemporary musical tools. The author is conducting a larger research project on this theme: the part presented in this paper describes the adoption of a new musical programming environment, Impromptu, and how this affects the author’s musical practice

A content based method for perceptually driven joint color/depth compression

International audienceMulti-view Video plus Depth (MVD) data refer to a set of conventional color video sequences and an associated set of depth video sequences, all acquired at slightly different viewpoints. This huge amount of data necessitates a reliable compression method. However, there is no standardized compression method for MVD sequences. H.264/MVC compression method, which was standardized for Multi-View-Video representation (MVV), has been the subject of many adaptations to MVD. However, it has been shown that MVC is not well adapted to encode multi-view depth data. We propose a novel option as for compression of MVD data. Its main purpose is to preserve joint color/depth consistency. The originality of the proposed method relies on the use of the decoded color data as a prior for the associated depth compression. This is meant to ensure consistency in both types of data after decoding. Our strategy is motivated by previous studies of artifacts occurring in synthesized views: most annoying distortions are located around strong depth discontinuities and these distortions are due to misalignment of depth and color edges in decoded images. Thus the method is meant to preserve edges and to ensure consistent localization of color edges and depth edges. To ensure compatibility, colored sequences are encoded with H.264. Depth maps compression is based on a 2D still image codec, namely LAR (Locally adapted Resolution). It consists in a quad-tree representation of the images. The quad-tree representation contributes in the preservation of edges in both color and depth data. The adopted strategy is meant to be more perceptually driven than state-of-the-art methods. The proposed approach is compared to H.264 encoding of depth images. Objective metrics scores are similar with H.264 and with the proposed method, and visual quality of synthesized views is improved with the proposed approach