STANDARDIZED TOOLCHAIN AND MODEL DEVELOPMENT FOR VIDEO QUALITY ASSESSMENT - THE MISSION OF THE JOINT EFFORT GROUP IN VQEG

International audienceSince 1997, the Video Quality Experts Group (VQEG) has been active in the field of subjective and objective video quality assessment. The group has validated competitive quality metrics throughout several projects. Each of these projects requires mandatory actions such as creating a testplan and obtaining databases consisting of degraded video sequences with corresponding subjective quality ratings. Recently, VQEG started a new open initiative, the Joint Effort Group (JEG), for encouraging joint collaboration on all mandatory actions needed to validate video quality metrics. Within the JEG, effort is made to advance the field of both subjective and objective video quality measurement by providing proper software tools and subjective databases to the community. One of the subprojects of the JEG is the joint development of a hybrid H.264/AVC objective quality metric. In this paper, we introduce the JEG and provide an overview of the different ongoing activities within this newly started group

Open Profiling of Quality: A Mixed Methods Research Approach for Audiovisual Quality Evaluations

Den Anforderungen der Konsumenten gerecht zu werden und ihnen eine immer besser werdende Quality of Experience zu bieten, ist eine der großen Herausforderungen jeder Neuentwicklung im Bereich der Multimediasysteme. Doch proportional zur technischen Komplexität neuer Systeme, in denen Komponenten unterschiedlicher Technologien zu neuen System wie zum Beispiel mobilem 3D-Fernsehen verschmolzen werden, steigt auch die Frage, wie eine optimierte Quality of Experience eigentlich zu erreichen ist. Daher werden seit langer Zeit Nutzertests zur subjektiven Qualitätsbewertung durchgeführt. Deren Ziel über den gesamten Entwcklungsprozesses ist es, die kritischen Komponenten des Systems mit so wenig wie möglich wahrnehmbarem Einfluss auf diewahrgenommene Qualität des Nutzers zu optimieren. Bereits seit den 1970er Jahren werden hierfür Leitfäden verschiedener Standardisierungsgremien zur Verfügung gestellt, in denen unterschiedliche Evaluationsmethoden definiert sind, um die wahrgenommene Gesamtqualität des Systems mit Hilfe von Skalen quantitativ evaluieren zu können. Aktuelle Ansätze erweitern diese klassische Methoden um Sichtweise, die über die klassische Evaluation hedonistischer Gesamtqualität hinausgehen, um das Wissen über individuell zugrundeliegende Qualitätsfaktoren zu erweitern.Die vorliegende Dissertation verfolgt dabei zwei Ziele. Zum einen soll eine audiovisuelle Evaluationsmethode entwickelt werden, die eine kombinierte Analyse quantitativer und qualitativer Daten ermöglicht, um eine Verknüpfung hedonistischer Qualität und zugrundeliegender Qualitätsfaktoren zu ermöglichen. Weiter soll diese Methode innerhalb des Gebiets der mobiler 3DTV-Systeme erprobt und validiert werden.Open Profiling of Quality (OPQ) als Evaluationsmethode kombiniert quantitative Evaluation wahrgenommener Gesamtqualität und deskriptive, sensorische Analyse zur Erhebung individueller Qualitätsfaktoren. Die Methode ist für Erhebungen mit naiven Probanden geeignet. OPQ wurde unter besonderer Beachtung von Validität und Reliabilität in einem konstruktivem Ansatz entwickelt und in einer Folge von Studien während der Entwicklung eines mobilem 3DTV-Systems mit über 300 Probanden angewendet. Die Ergebnisse dieser Studien unterstreichen die sich ergänzenden Ergebnisse quantitativer und sensorischer Analysen.Neben der Entwicklung von OPQ werden in der vorliegenden Arbeit weitere Ansätze sensorischer Analyse präsentiert und miteinander verglichen. Gerade dieser Vergleich ist ein wichtiger Bestandteil der Validierung der OPQ-Methode. Um die Stärken und Schwächen jeder Methode ganzheitlich erfassen und vergleichen zu können, wurde hierfür ein Methodenvergleichsmodell entwickelt und operationalisiert, das den methodischen Beitrag der Arbeit vervollständigtTo meet the requirements of consumers and to provide them with a greater quality of experience than existing systems do is a key issue for the success of modern multimedia systems. However, the question about an optimized quality of experience becomes more and more complex as technological systems are evolving and several systems are merged into new ones, e.g. systems for mobile 3D television and video. To be able to optimize critical components of a system under development with as little perceptual errors as possible, user studies are conducted throughout the whole process. A variety of research methods for different purposes have been provided by standardization bodies since the 1970s. These methods allow researchers to evaluate the hedonic excellence of a set of test stimuli. However, a broader view to quality has been taken recently to be able to evaluate quality beyond its hedonic excellence to obtain a greater knowledge about perceived quality and its subjective quality factors that impact on the user.The goal of this thesis is twofold. The primary goal is the development of a validated mixed methods research approach for audiovisual quality evaluations. The method shall allow collecting quantitative and descriptive data during the experiment to combine evaluation of hedonic excellence and the elicitation of underlying subjective quality factors. The second goal is the application of the developed method within a series of studies in the domain of mobile 3D video and television to show its applicability.Open Profiling of Quality (OPQ) is a mixed-methods research approach which combines a quantitative, psychoperceptual evaluation of hedonic excellence and a descriptive sensory analysis of underlying quality factors based on naive participants' individual vocabulary. This combination allows defining the excellence of overall quality, understanding the characteristics of quality perception, and, eventually, constructing a link between preferences and quality attributes. The method was developed under constructive research with respect to validity and reliability of test results. A series of quality evaluation studies with more than 300 test participants was conducted along different critical components of a system for optimized mobile 3DTV content delivery over DVB-H.The results complemented each other, and, even more importantly, quantitative quality preferences were explained by sensory descriptions in all studies. Beyond the development of OPQ, the thesis proposes further research approaches, e.g. a conventional profiling in which OPQ's individual vacobulary is substituted by a fixed set of Quality ofExperience components or Descriptive Sorted Napping which combines a sorting task and a short post-task interview. All approaches are compared to Open ProVling of Quality at the end of the thesis. To be able to holistically contrast strengths and weaknesses of each method, a comparison model for audiovisual evaluation methods was developed and a Vrst conceptual operationalization of the model was applied in the comparison

No-Reference Video Quality Assessment Model for Distortion Caused by Packet Loss in the Real-Time Mobile Video Services

Packet loss will make severe errors due to the corruption of related video data. For most video streams, because the predictive coding structures are employed, the transmission errors in one frame will not only cause decoding failure of itself at the receiver side, but also propagate to its subsequent frames along the motion prediction path, which will bring a significant degradation of end-to-end video quality. To quantify the effects of packet loss on video quality, a no-reference objective quality assessment model is presented in this paper. Considering the fact that the degradation of video quality significantly relies on the video content, the temporal complexity is estimated to reflect the varying characteristic of video content, using the macroblocks with different motion activities in each frame. Then, the quality of the frame affected by the reference frame loss, by error propagation, or by both of them is evaluated, respectively. Utilizing a two-level temporal pooling scheme, the video quality is finally obtained. Extensive experimental results show that the video quality estimated by the proposed method matches well with the subjective quality

Video Quality Prediction for Video over Wireless Access Networks (UMTS and WLAN)

Transmission of video content over wireless access networks (in particular, Wireless Local Area Networks (WLAN) and Third Generation Universal Mobile Telecommunication System (3G UMTS)) is growing exponentially and gaining popularity, and is predicted to expose new revenue streams for mobile network operators. However, the success of these video applications over wireless access networks very much depend on meeting the user’s Quality of Service (QoS) requirements. Thus, it is highly desirable to be able to predict and, if appropriate, to control video quality to meet user’s QoS requirements. Video quality is affected by distortions caused by the encoder and the wireless access network. The impact of these distortions is content dependent, but this feature has not been widely used in existing video quality prediction models. The main aim of the project is the development of novel and efficient models for video quality prediction in a non-intrusive way for low bitrate and resolution videos and to demonstrate their application in QoS-driven adaptation schemes for mobile video streaming applications. This led to five main contributions of the thesis as follows:(1) A thorough understanding of the relationships between video quality, wireless access network (UMTS and WLAN) parameters (e.g. packet/block loss, mean burst length and link bandwidth), encoder parameters (e.g. sender bitrate, frame rate) and content type is provided. An understanding of the relationships and interactions between them and their impact on video quality is important as it provides a basis for the development of non-intrusive video quality prediction models.(2) A new content classification method was proposed based on statistical tools as content type was found to be the most important parameter. (3) Efficient regression-based and artificial neural network-based learning models were developed for video quality prediction over WLAN and UMTS access networks. The models are light weight (can be implemented in real time monitoring), provide a measure for user perceived quality, without time consuming subjective tests. The models have potential applications in several other areas, including QoS control and optimization in network planning and content provisioning for network/service providers.(4) The applications of the proposed regression-based models were investigated in (i) optimization of content provisioning and network resource utilization and (ii) A new fuzzy sender bitrate adaptation scheme was presented at the sender side over WLAN and UMTS access networks. (5) Finally, Internet-based subjective tests that captured distortions caused by the encoder and the wireless access network for different types of contents were designed. The database of subjective results has been made available to research community as there is a lack of subjective video quality assessment databases.Partially sponsored by EU FP7 ADAMANTIUM Project (EU Contract 214751

The Impact of Multichannel Game Audio on the Quality of Player Experience and In-game Performance

Multichannel audio is a term used in reference to a collection of techniques designed to present sound to a listener from all directions. This can be done either over a collection of loudspeakers surrounding the listener, or over a pair of headphones by virtualising sound sources at specific positions. The most popular commercial example is surround-sound, a technique whereby sounds that make up an auditory scene are divided among a defined group of audio channels and played back over an array of loudspeakers. Interactive video games are well suited to this kind of audio presentation, due to the way in which in-game sounds react dynamically to player actions. Employing multichannel game audio gives the potential of immersive and enveloping soundscapes whilst also adding possible tactical advantages. However, it is unclear as to whether these factors actually impact a player’s overall experience. There is a general consensus in the wider gaming community that surround-sound audio is beneficial for gameplay but there is very little academic work to back this up. It is therefore important to investigate empirically how players react to multichannel game audio, and hence the main motivation for this thesis. The aim was to find if a surround-sound system can outperform other systems with fewer audio channels (like mono and stereo). This was done by performing listening tests that assessed the perceived spatial sound quality and preferences towards some commonly used multichannel systems for game audio playback over both loudspeakers and headphones. There was also a focus on how multichannel audio might influence the success of a player in a game, based on their in-game score and their navigation within a virtual world. Results suggest that surround-sound game audio is preferable over more regularly used two-channel stereo systems, because it is perceived to have higher spatial sound quality and there is an improvement in player performance. This illustrates the potential for multichannel game audio as a tool to positively influence player experiences, a core goal many game designers strive to achieve

Indicator Of Experince For Mobile Data Networks

Tez (Yüksek Lisans) -- İstanbul Teknik Üniversitesi, Fen Bilimleri Enstitüsü, 2013Thesis (M.Sc.) -- İstanbul Technical University, Institute of Science and Technology, 2013Bu çalışmada mobil veri şebekeleri için kullanıcıların yaşadıkları deneyim kalitesini ölçmek adına farklı bir metod önerilmektedir. Literatürde önerilen yöntemler ağırlıklı olarak kullanıcıların deneyimlerine ilişkin görüşlerinin anketler düzenlenerek alınması veya mobil cihazlara yerleştirilen akıllı yazılımlar ile deneyimlerin ölçülmesi şeklindedir. Bu sebeple yapılan çalışmalar örnek küme üzerinden deneyimin bütüne aitmiş gibi konumlandırılması esasına dayanmaktadır. Bu çalışmada bahsedilen yöntemlerden farklı olarak mobil şebekede bulunan sistemsel istatistiklerin kullanılması ve radyo şebekesinin deneyime etkisini hesaplayan bir modelin önerilmesiyle farklı bir hesaplama yöntemi önerilmektedir. Bu sayede sadece belli bir örnek küme değil, servisi deneyimleyen tüm kullanıcılar için bir sonuç elde etmek mümkün olmaktadır. Bu amaçla IoE kısaltması adı altında mobil veri servisleri kullanımlarında deneyimin başarısı ölçülebilir ve hesaplanabilir bir parametre ile ifade edilmiştir.In this study, introducing a new way of quality of experience calculation methodology is aimed. A new parameter, indicator of experience (IoE) has been introduced, which is a combination of quantised throughput and acceptability of the user for mobile internet service. Most QoE calculations use mobile client or user perception researches. Differently, network level statistics and radio network modeling is used to calculate IoE. This new approach is then compared with mobile client measurements in order to prove the usability of the approach.Yüksek LisansM.Sc