Gaze-Based Personalized Multi-View Experiences

This paper describes a solution for delivering andpresenting stereoscopic video content to users in aninnovative way. It adopts the multi-view paradigm of theH.264-MVC video coding standard and the emergentMPEG DASH specification to provide users inheterogeneous network environments multiple and varyingperspectives of stereoscopic video sequences. Unlike existing3D systems based on multi-view technology, which requirehigh transmission bandwidth and high processing power onthe terminal device to achieve the same objective, theproposed solution is able to make an efficient use of networkresources whilst being cost-effective. It offers users a higherquality of experience by seamlessly adapting the quality ofthe delivered video content according to the networkconditions, whilst providing a more realistic sense ofimmersion by offering stereoscopic views of the scene,dynamically switching the perspective to match the interestsof the user. A non-intrusive head-tracking system using anoff-the-shelf Web camera detects the focus of attention ofthe user, transmitting this information to the server thatselects the most appropriate view to send to the client.Additionally, the system is able to generate the multipleperspective stereoscopic scenes using 2D cameras

Gaze-Based Personalized Multi-View Experiences

This paper describes a solution for delivering andpresenting stereoscopic video content to users in aninnovative way. It adopts the multi-view paradigm of theH.264-MVC video coding standard and the emergentMPEG DASH specification to provide users inheterogeneous network environments multiple and varyingperspectives of stereoscopic video sequences. Unlike existing3D systems based on multi-view technology, which requirehigh transmission bandwidth and high processing power onthe terminal device to achieve the same objective, theproposed solution is able to make an efficient use of networkresources whilst being cost-effective. It offers users a higherquality of experience by seamlessly adapting the quality ofthe delivered video content according to the networkconditions, whilst providing a more realistic sense ofimmersion by offering stereoscopic views of the scene,dynamically switching the perspective to match the interestsof the user. A non-intrusive head-tracking system using anoff-the-shelf Web camera detects the focus of attention ofthe user, transmitting this information to the server thatselects the most appropriate view to send to the client.Additionally, the system is able to generate the multipleperspective stereoscopic scenes using 2D cameras

Light field image compression

Light field imaging based on a single-tier camera equipped with a micro-lens array has currently risen up as a practical and prospective approach for future visual applications and services. However, successfully deploying actual light field imaging applications and services will require identifying adequate coding solutions to efficiently handle the massive amount of data involved in these systems. In this context, this chapter presents some of the most recent light field image coding solutions that have been investigated. After a brief review of the current state of the art in image coding formats for light field photography, an experimental study of the rate-distortion performance for different coding formats and architectures is presented. Then, aiming at enabling faster deployment of light field applications and services in the consumer market, a scalable light field coding solution that provides backward compatibility with legacy display devices (e.g., 2D, 3D stereo, and 3D multiview) is also presented. Furthermore, a light field coding scheme based on a sparse set of microimages and the associated blockwise disparity is also presented. This coding scheme is scalable with three layers such that the rendering can be performed with the sparse micro-image set, the reconstructed light field image, and the decoded light field image.info:eu-repo/semantics/acceptedVersio

Codage de cartes de profondeur par deformation de courbes elastiques

In multiple-view video plus depth, depth maps can be represented by means of grayscale images and the corresponding temporal sequence can be thought as a standard grayscale video sequence. However depth maps have different properties from natural images: they present large areas of smooth surfaces separated by sharp edges. Arguably the most important information lies in object contours, as a consequence an interesting approach consists in performing a lossless coding of the contour map, possibly followed by a lossy coding of per-object depth values.In this context, we propose a new technique for the lossless coding of object contours, based on the elastic deformation of curves. A continuous evolution of elastic deformations between two reference contour curves can be modelled, and an elastically deformed version of the reference contours can be sent to the decoder with an extremely small coding cost and used as side information to improve the lossless coding of the actual contour. After the main discontinuities have been captured by the contour description, the depth field inside each region is rather smooth. We proposed and tested two different techniques for the coding of the depth field inside each region. The first technique performs the shape-adaptive wavelet transform followed by the shape-adaptive version of SPIHT. The second technique performs a prediction of the depth field from its subsampled version and the set of coded contours. It is generally recognized that a high quality view rendering at the receiver side is possible only by preserving the contour information, since distortions on edges during the encoding step would cause a sensible degradation on the synthesized view and on the 3D perception. We investigated this claim by conducting a subjective quality assessment test to compare an object-based technique and a hybrid block-based techniques for the coding of depth maps.Dans le format multiple-view video plus depth, les cartes de profondeur peuvent être représentées comme des images en niveaux de gris et la séquence temporelle correspondante peut être considérée comme une séquence vidéo standard en niveaux de gris. Cependant les cartes de profondeur ont des propriétés différentes des images naturelles: ils présentent de grandes surfaces lisses séparées par des arêtes vives. On peut dire que l'information la plus importante réside dans les contours de l'objet, en conséquence une approche intéressante consiste à effectuer un codage sans perte de la carte de contour, éventuellement suivie d'un codage lossy des valeurs de profondeur par-objet.Dans ce contexte, nous proposons une nouvelle technique pour le codage sans perte des contours de l'objet, basée sur la déformation élastique des courbes. Une évolution continue des déformations élastiques peut être modélisée entre deux courbes de référence, et une version du contour déformée élastiquement peut être envoyé au décodeur avec un coût de codage très faible et utilisé comme information latérale pour améliorer le codage sans perte du contour réel. Après que les principales discontinuités ont été capturés par la description du contour, la profondeur à l'intérieur de chaque région est assez lisse. Nous avons proposé et testé deux techniques différentes pour le codage du champ de profondeur à l'intérieur de chaque région. La première technique utilise la version adaptative à la forme de la transformation en ondelette, suivie par la version adaptative à la forme de SPIHT.La seconde technique effectue une prédiction du champ de profondeur à partir de sa version sous-échantillonnée et l'ensemble des contours codés. Il est généralement reconnu qu'un rendu de haute qualité au récepteur pour un nouveau point de vue est possible que avec la préservation de l'information de contour, car des distorsions sur les bords lors de l'étape de codage entraînerait une dégradation évidente sur la vue synthétisée et sur la perception 3D. Nous avons étudié cette affirmation en effectuant un test d'évaluation de la qualité perçue en comparant, pour le codage des cartes de profondeur, une technique basée sur la compression d'objects et une techniques de codage vidéo hybride à blocs