Modeling user perception of 3D video based on ambient illumination context for enhanced user centric media access and consumption

For enjoying 3D video to its full extent, it is imperative that access and consumption of it is user centric, which in turn ensures improved 3D video perception. Several important factors including video characteristics, users’ preferences, contexts prevailing in various usage environments, etc have influences on 3D video perception. Thus, to assist efficient provision of user centric media, user perception of 3D video should be modeled considering the factors affecting perception. Considering ambient illumination context to model 3D video perception is an interesting research topic, which has not been particularly investigated in literature. This context is taken into account while modeling video quality and depth perception of 3D video in this paper. For the video quality perception model: motion and structural feature characteristics of color texture sequences; and for the depth perception model: luminance contrast of color texture and depth intensity of depth map sequences of 3D video are used as primary content related factors in the paper. Results derived using the video quality and depth perception models demonstrate that these models can efficiently predict user perception of 3D video considering the ambient illumination context in user centric media access and consumption environments

Seamless video access for mobile devices by content-aware utility-based adaptation

Today's Internet multimedia services are characterized by heterogeneous networks, a wide range of terminals, diverse user preferences, and varying natural environment conditions. Heterogeneity of terminals, networks, and user preferences impose nontrivial challenges to the Internet multimedia services for providing seamless multimedia access particularly for mobile devices (e.g., laptops, tablet PCs, PDAs, mobile phones, etc.). Thus, it is essential that advanced multimedia technologies are developed to deal with these challenges. One of these technologies is video adaptation, which has gained significant importance with its main objective of enabling seamless access to video contents available over the Internet. Adaptation decision taking, which can be considered as the "brain" of video adaptation, assists video adaptation to achieve this objective. Scalable Video Coding (SVC) offers flexibility for video adaptation through providing a comprehensive set of scalability parameters (i.e., temporal, spatial, and quality) for producing scalable video streams. Deciding the best combination of scalability parameters to adapt a scalable video stream while satisfying a set of constraints (e.g., device specifics, network bandwidth, etc.) poses challenges for the existing adaptation services to enable seamless video access. To ease such challenges, an adaptation decision taking technique employing a utility-based approach to decide on the most adequate scalability parameters for adaptation operations is developed. A Utility Function (UF), which models the relationships among the scalability parameters and weights specifying the relative importance of these parameters considering video content characteristics (i.e., motion activity and structural feature), is proposed to assist the developed technique. In order to perform the developed adaptation decision taking technique, a video adaptation framework is also proposed in this paper. The adaptation experiments performed using the proposed framework prove the effectiveness of the framework to provide an important step towards enabling seamless video access for mobile devices to enhance viewing experience of users. © 2012 Springer Science+Business Media, LLC

3D video bit rate adaptation decision taking using ambient illumination context

3-Dimensional (3D) video adaptation decision taking is an open field in which not many researchers have carried out investigations yet compared to 3D video display, coding, etc. Moreover, utilizing ambient illumination as an environmental context for 3D video adaptation decision taking has particularly not been studied in literature to date. In this paper, a user perception model, which is based on determining perception characteristics of a user for a 3D video content viewed under a particular ambient illumination condition, is proposed. Using the proposed model, a 3D video bit rate adaptation decision taking technique is developed to determine the adapted bit rate for the 3D video content to maintain 3D video quality perception by considering the ambient illumination condition changes. Experimental results demonstrate that the proposed technique is capable of exploiting the changes in ambient illumination level to use network resources more efficiently without sacrificing the 3D video quality perception

3D video bit rate adaptation decision taking using ambient illumination context

3-Dimensional (3D) video adaptation decision taking is an open field in which not many researchers have carried out investigations yet compared to 3D video display, coding, etc. Moreover, utilizing ambient illumination as an environmental context for 3D video adaptation decision taking has particularly not been studied in literature to date. In this paper, a user perception model, which is based on determining perception characteristics of a user for a 3D video content viewed under a particular ambient illumination condition, is proposed. Using the proposed model, a 3D video bit rate adaptation decision taking technique is developed to determine the adapted bit rate for the 3D video content to maintain 3D video quality perception by considering the ambient illumination condition changes. Experimental results demonstrate that the proposed technique is capable of exploiting the changes in ambient illumination level to use network resources more efficiently without sacrificing the 3D video quality perception

A disocclusion replacement approach to subjective assessment for depth map quality evaluation

An inherent problem of Depth Image Based Rendering (DIBR) is the visual presence of disocclusions in the rendered views. This poses a significant challenge when the subjective assessment of these views is utilised for evaluating the quality of the depth maps used in the rendering process. Although various techniques are available to address this challenge, they result in concealing distortions, which are directly caused by the depth map imperfections. For the purposes of depth map quality evaluation, there is a need for an approach that deals with the presence of disocclusions without having further impact on other distortions. The aim of this approach is to enable the subjective assessments of rendered views to provide results, which are more representative of the quality of the depth map used in the rendering process

A disocclusion replacement approach to subjective assessment for depth map quality evaluation

An inherent problem of Depth Image Based Rendering (DIBR) is the visual presence of disocclusions in the rendered views. This poses a significant challenge when the subjective assessment of these views is utilised for evaluating the quality of the depth maps used in the rendering process. Although various techniques are available to address this challenge, they result in concealing distortions, which are directly caused by the depth map imperfections. For the purposes of depth map quality evaluation, there is a need for an approach that deals with the presence of disocclusions without having further impact on other distortions. The aim of this approach is to enable the subjective assessments of rendered views to provide results, which are more representative of the quality of the depth map used in the rendering process

Toward an impairment metric for stereoscopic video: a full-reference video quality metric to assess compressed stereoscopic video

The quality assessment of impaired stereoscopic video is a key element in designing and deploying advanced immersive media distribution platforms. A widely accepted quality metric to measure impairments of stereoscopic video is, however, still to be developed. As a step toward finding a solution to this problem, this paper proposes a full reference stereoscopic video quality metric to measure the perceptual quality of compressed stereoscopic video. A comprehensive set of subjective experiments is performed with 14 different stereoscopic video sequences, which are encoded using both the H.264 and high efficiency video coding compliant video codecs, to develop a subjective test results database of 116 test stimuli. The subjective results are analyzed using statistical techniques to uncover different patterns of subjective scoring for symmetrically and asymmetrically encoded stereoscopic video. The subjective result database is subsequently used for training and validating a simple but effective stereoscopic video quality metric considering heuristics of binocular vision. The proposed metric performs significantly better than state-of-the-art stereoscopic image and video quality metrics in predicting the subjective scores. The proposed metric and the subjective result database will be made publicly available, and it is expected that the proposed metric and the subjective assessments will have important uses in advanced 3D media delivery systems

Content-aware delivery of visual attention-based scalable multi-view video over P2P

3D media applications have become widespread thanks to the intense research being conducted on 3D enabling technologies, commercial products being released and service providers. There is also a huge potential for IP network to become a major means for delivering 3D video services, being highly flexible and allowing much custom and scalable applications to take up. Peer-to-Peer (P2P) video streaming, which offers high scalability in the presence of many media consuming peers, is suitable for multi-view video applications with significantly higher bandwidth requirements. To exploit flexible streaming and also serve a range of displays, P2P networking should be combined with scalable coded multi-view video that offers a useful bit-rate and maximum viewpoint adaptation range. However, since maintaining users' Quality of Experience (QoE) is a primary target, scalable coded video should ensure that perceptually sensible visual data is delivered with high accuracy at all times to users, even under severe network conditions. Hence, content-based visual attention models provide a suitable means for letting salient video data be delivered at all times to users. With the utilization of appropriate adaptation decision making process in users' equipment, the P2P protocol can adaptively stream the scalable 3D media. Work presented in this paper explains such a delivery framework over P2P networks

Psycho-physical limits of interocular blur suppression and its application to asymmetric stereoscopic video delivery

It is well known that when the two eyes are provided with two views of different resolutions the overall perception is dominated by the high resolution view. This property, known as binocular suppression, is effectively used to reduce the bit rate required for stereoscopic video delivery, where one view of the stereo pair is encoded at a much lower quality than the other. There have been significant amount of effort in the recent past to measure the just noticeable level of asymmetry between the two views, where asymmetry is achieved by encoding views at two quantization levels. However, encoding artifacts introduce both blurring and blocking artifacts in to the stereo views, which are perceived differently by the human visual system. Therefore, in this paper, we design a set of psycho-physical experiments to measure the just noticeable level of asymmetric blur at various spatial frequencies, luminance contrasts and orientations. The subjective results suggest that humans could tolerate a significant amount of asymmetry introduced by blur, and the level of tolerance is independent of the spatial frequency or luminance contrast. Furthermore, the results of this paper illustrate that when asymmetry is introduced by unequal quantization, the just noticeable level of asymmetry is driven by the blocking artifacts. In general, stereoscopic asymmetry introduced by way of asymmetric blurring is preferred over asymmetric compression. It is expected that the subjective results of this paper will have important use cases in objective measurement of stereoscopic video quality and asymmetric compression and processing of stereoscopic video