Impact of tone-mapping operators and viewing devices on visual quality of experience

© 2016 IEEE. The development of HDR imaging is seen as an important step towards improving the visual quality of experience (QoE) of the end user in many applications. In practice, Tone-mapping operators (TMOs) provide a useful means for converting a high dynamic range (HDR) image to a low dynamic range (LDR) image in order to achieve better visualization on standard displays. Although mobile devices are becoming popular, the techniques for displaying the content of HDR images on the screens of such devices are still in the early stages. While several studies have been conducted to evaluate TMOs on conventional displays, few studies have been carried out to date to evaluate TMOs on small screen displays, such as those used in mobile devices. In this paper we evaluate, using subjective and objective methods, the most popular Tone-mapping-operators in different mobile displays and resolutions under normal viewing conditions for the end-user. Preliminary results show that small screen displays (SSDs) have an impact on the performance of TMOs compared to computer displays. In general, the larger the mobile resolution, the better the subjective results. We also found clear differences between SSDs and LDRs performances. The best TMO for mobile displays is iCAM06 and for computer displays it is Photographic Reproduction

Towards prediction of Sense of Presence in immersive audiovisual communications

Tremendous progress has been made in audiovisual technologies in the last decades. Consequently, new technologies quality measures evolve and trend to be more user-centric. This is the reason why the Quality of Experience (QoE) assessment is presently meaningful and challenging, especially for users typical experiences during multimedia content consumption. Such an evaluation is the aim of this paper. More specifically, the Sense of Presence (SoP) was explored in place of QoE as it is a factor influencing the QoE. This paper presents the conducted subjective test investigating typical and practical user experiences. This latter consists of presenting one-minute video stimuli to twenty subjects, on three different devices (iPhone, iPad and UHD screen). Annotated subjective scores were collected and physiological signals (EEG, ECG, and Respiration) were recorded during the conducted subjective test. The resulting multimodal dataset, aiming an alternative evaluation of human experience while consuming multimedia, is publicly available

Assessment of Quality of Experience of High Dynamic Range Images Using the EEG and Applications in Healthcare

File embargoed until 30.09.2021 at author's request.Recent years have witnessed the widespread application of High Dynamic Range (HDR) imaging, which like the Human Visual System (HVS), has the ability to capture a wide range of luminance values. Areas of application include home-entertainment, security, scientific imaging, video processing, computer graphics, multimedia communications, and healthcare. However, in practice, HDR content cannot be displayed in full on standard or low dynamic range (LDR) displays, and this diminishes the benefits of HDR technology for many users. To address this problem, Tone-Mapping Operators (TMO) are used to convert HDR images so that they can be displayed on low-dynamic-range displays and preserve as far as possible the perception of HDR. However, this may affect the visual Quality of Experience (QoE) of the end-user. QoE is a vital issue in image and video applications. It is important to understand how humans perceive quality in response to visual stimuli as this can potentially be exploited to develop and optimise image and video processing algorithms. Image consumption using mobile devices has become increasingly popular, given the availability of smartphones capable of producing and consuming HDR images along with advances in high-speed wireless communication networks. One of the most critical issues associated with mobile HDR image delivery services concerns how to maximise the QoE of the delivered content for users. An open research question therefore addresses how HDR images with different types of content perform on mobile phones. Traditionally, evaluation of the perceived quality of multimedia content is conducted using subjective opinion tests (i.e., explicitly), such as Mean Opinion Scores (MOS). However, it is difficult for the user to link the quality they are experiencing to the quality scale. Moreover, MOS does not give an insight into how the user feels at a physiological level in response to satisfaction or dissatisfaction with the perceived quality. To address this issue, measures that can be taken directly (implicitly) from the participant have now begun to attract interest. The electroencephalogram (EEG) is a promising approach that can be used to assess quality related processes implicitly. However, implicit QoE approaches are still at an early stage and further research is necessary to fully understand the nature of the recorded neural signals and their associations with user-perceived quality. Nevertheless, the EEG is expected to provide additional and complementary information that will aid understanding of the human perception of content. Furthermore, it has the potential to facilitate real-time monitoring of QoE without the need for explicit rating activities. The main aim of this project was therefore to assess the QoE of HDR images employing a physiological method and to investigate its potential application in the field of healthcare. This resulted in the following five main contributions to the research literature: 1. A detailed understanding of the relationship between the subjective and objective evaluation of the most popular TMOs used for colour and greyscale HDR images. Different mobile displays and resolutions were therefore presented under normal viewing conditions for the end-user with an LDR display as a reference. Preliminary results show that, compared to computer displays, small screen devices (SSDs) such as those used in smartphones impact the performance of TMOs in that a higher resolution gave more favourable MOS results. 2. The development of a novel Electrophysiology-based QoE assessment of HDR image quality that can be used to predict perceived image quality. This was achieved by investigating the relationships between changes in EEG features and subjective quality test scores (i.e. MOS) for HDR images viewed with SSD. 3. The development of a novel QoE prediction model, based on the above findings. The model can predict user acceptability and satisfaction for various mobile HDR image scenarios based on delta-beta coupling. Subjective quality tests were conducted to develop and evaluate the model, where the HDR image quality was predicted in terms of MOS. 4. The development of a new method of detecting a colour vision deficiency (CVD) using EEG and HDR images. The results suggest that this method may provide an accurate way to detect CVD with high sensitivity and specificity (close to 100%). Potentially, the method may facilitate the development of a low-cost tool suitable for CVD diagnosis in younger people. 5. The development of an approach that enhances the quality of dental x-ray images. This uses the concepts of QoE in HDR images without re-exposing patients to ionising radiation, thus improving patient care. Potentially, the method provides the basis for an intelligent model that accurately predicts the quality of dental images. Such a model can be embedded into a tool to automatically enhance poor quality dental images.Ministry of Higher Education and Scientific Research (MoHESR

Encoding high dynamic range and wide color gamut imagery

In dieser Dissertation wird ein szenischer Bewegtbilddatensatz mit erweitertem Dynamikumfang (High Dynamic Range, HDR) und großem Farbumfang (Wide Color Gamut, WCG) eingeführt und es werden Modelle zur Kodierung von HDR und WCG Bildern vorgestellt. Die objektive und visuelle Evaluation neuer HDR und WCG Bildverarbeitungsalgorithmen, Kompressionsverfahren und Bildwiedergabegeräte erfordert einen Referenzdatensatz hoher Qualität. Daher wird ein neuer HDR- und WCG-Video-Datensatz mit einem Dynamikumfang von bis zu 18 fotografischen Blenden eingeführt. Er enthält inszenierte und dokumentarische Szenen. Die einzelnen Szenen sind konzipiert um eine Herausforderung für Tone Mapping Operatoren, Gamut Mapping Algorithmen, Kompressionscodecs und HDR und WCG Bildanzeigegeräte darzustellen. Die Szenen sind mit professionellem Licht, Maske und Filmausstattung aufgenommen. Um einen cinematischen Bildeindruck zu erhalten, werden digitale Filmkameras mit ‘Super-35 mm’ Sensorgröße verwendet. Der zusätzliche Informationsgehalt von HDR- und WCG-Videosignalen erfordert im Vergleich zu Signalen mit herkömmlichem Dynamikumfang eine neue und effizientere Signalkodierung. Ein Farbraum für HDR und WCG Video sollte nicht nur effizient quantisieren, sondern wegen der unterschiedlichen Monitoreigenschaften auf der Empfängerseite auch für die Dynamik- und Farbumfangsanpassung geeignet sein. Bisher wurden Methoden für die Quantisierung von HDR Luminanzsignalen vorgeschlagen. Es fehlt jedoch noch ein entsprechendes Modell für Farbdifferenzsignale. Es werden daher zwei neue Farbräume eingeführt, die sich sowohl für die effiziente Kodierung von HDR und WCG Signalen als auch für die Dynamik- und Farbumfangsanpassung eignen. Diese Farbräume werden mit existierenden HDR und WCG Farbsignalkodierungen des aktuellen Stands der Technik verglichen. Die vorgestellten Kodierungsschemata erlauben es, HDR- und WCG-Video mittels drei Farbkanälen mit 12 Bits tonaler Auflösung zu quantisieren, ohne dass Quantisierungsartefakte sichtbar werden. Während die Speicherung und Übertragung von HDR und WCG Video mit 12-Bit Farbtiefe pro Kanal angestrebt wird, unterstützen aktuell verbreitete Dateiformate, Videoschnittstellen und Kompressionscodecs oft nur niedrigere Bittiefen. Um diese existierende Infrastruktur für die HDR Videoübertragung und -speicherung nutzen zu können, wird ein neues bildinhaltsabhängiges Quantisierungsschema eingeführt. Diese Quantisierungsmethode nutzt Bildeigenschaften wie Rauschen und Textur um die benötigte tonale Auflösung für die visuell verlustlose Quantisierung zu schätzen. Die vorgestellte Methode erlaubt es HDR Video mit einer Bittiefe von 10 Bits ohne sichtbare Unterschiede zum Original zu quantisieren und kommt mit weniger Rechenkraft im Vergleich zu aktuellen HDR Bilddifferenzmetriken aus