Computational Multimedia for Video Self Modeling

Video self modeling (VSM) is a behavioral intervention technique in which a learner models a target behavior by watching a video of oneself. This is the idea behind the psychological theory of self-efficacy - you can learn or model to perform certain tasks because you see yourself doing it, which provides the most ideal form of behavior modeling. The effectiveness of VSM has been demonstrated for many different types of disabilities and behavioral problems ranging from stuttering, inappropriate social behaviors, autism, selective mutism to sports training. However, there is an inherent difficulty associated with the production of VSM material. Prolonged and persistent video recording is required to capture the rare, if not existed at all, snippets that can be used to string together in forming novel video sequences of the target skill. To solve this problem, in this dissertation, we use computational multimedia techniques to facilitate the creation of synthetic visual content for self-modeling that can be used by a learner and his/her therapist with a minimum amount of training data. There are three major technical contributions in my research. First, I developed an Adaptive Video Re-sampling algorithm to synthesize realistic lip-synchronized video with minimal motion jitter. Second, to denoise and complete the depth map captured by structure-light sensing systems, I introduced a layer based probabilistic model to account for various types of uncertainties in the depth measurement. Third, I developed a simple and robust bundle-adjustment based framework for calibrating a network of multiple wide baseline RGB and depth cameras

Open Profiling of Quality: A Mixed Methods Research Approach for Audiovisual Quality Evaluations

Den Anforderungen der Konsumenten gerecht zu werden und ihnen eine immer besser werdende Quality of Experience zu bieten, ist eine der großen Herausforderungen jeder Neuentwicklung im Bereich der Multimediasysteme. Doch proportional zur technischen Komplexität neuer Systeme, in denen Komponenten unterschiedlicher Technologien zu neuen System wie zum Beispiel mobilem 3D-Fernsehen verschmolzen werden, steigt auch die Frage, wie eine optimierte Quality of Experience eigentlich zu erreichen ist. Daher werden seit langer Zeit Nutzertests zur subjektiven Qualitätsbewertung durchgeführt. Deren Ziel über den gesamten Entwcklungsprozesses ist es, die kritischen Komponenten des Systems mit so wenig wie möglich wahrnehmbarem Einfluss auf diewahrgenommene Qualität des Nutzers zu optimieren. Bereits seit den 1970er Jahren werden hierfür Leitfäden verschiedener Standardisierungsgremien zur Verfügung gestellt, in denen unterschiedliche Evaluationsmethoden definiert sind, um die wahrgenommene Gesamtqualität des Systems mit Hilfe von Skalen quantitativ evaluieren zu können. Aktuelle Ansätze erweitern diese klassische Methoden um Sichtweise, die über die klassische Evaluation hedonistischer Gesamtqualität hinausgehen, um das Wissen über individuell zugrundeliegende Qualitätsfaktoren zu erweitern.Die vorliegende Dissertation verfolgt dabei zwei Ziele. Zum einen soll eine audiovisuelle Evaluationsmethode entwickelt werden, die eine kombinierte Analyse quantitativer und qualitativer Daten ermöglicht, um eine Verknüpfung hedonistischer Qualität und zugrundeliegender Qualitätsfaktoren zu ermöglichen. Weiter soll diese Methode innerhalb des Gebiets der mobiler 3DTV-Systeme erprobt und validiert werden.Open Profiling of Quality (OPQ) als Evaluationsmethode kombiniert quantitative Evaluation wahrgenommener Gesamtqualität und deskriptive, sensorische Analyse zur Erhebung individueller Qualitätsfaktoren. Die Methode ist für Erhebungen mit naiven Probanden geeignet. OPQ wurde unter besonderer Beachtung von Validität und Reliabilität in einem konstruktivem Ansatz entwickelt und in einer Folge von Studien während der Entwicklung eines mobilem 3DTV-Systems mit über 300 Probanden angewendet. Die Ergebnisse dieser Studien unterstreichen die sich ergänzenden Ergebnisse quantitativer und sensorischer Analysen.Neben der Entwicklung von OPQ werden in der vorliegenden Arbeit weitere Ansätze sensorischer Analyse präsentiert und miteinander verglichen. Gerade dieser Vergleich ist ein wichtiger Bestandteil der Validierung der OPQ-Methode. Um die Stärken und Schwächen jeder Methode ganzheitlich erfassen und vergleichen zu können, wurde hierfür ein Methodenvergleichsmodell entwickelt und operationalisiert, das den methodischen Beitrag der Arbeit vervollständigtTo meet the requirements of consumers and to provide them with a greater quality of experience than existing systems do is a key issue for the success of modern multimedia systems. However, the question about an optimized quality of experience becomes more and more complex as technological systems are evolving and several systems are merged into new ones, e.g. systems for mobile 3D television and video. To be able to optimize critical components of a system under development with as little perceptual errors as possible, user studies are conducted throughout the whole process. A variety of research methods for different purposes have been provided by standardization bodies since the 1970s. These methods allow researchers to evaluate the hedonic excellence of a set of test stimuli. However, a broader view to quality has been taken recently to be able to evaluate quality beyond its hedonic excellence to obtain a greater knowledge about perceived quality and its subjective quality factors that impact on the user.The goal of this thesis is twofold. The primary goal is the development of a validated mixed methods research approach for audiovisual quality evaluations. The method shall allow collecting quantitative and descriptive data during the experiment to combine evaluation of hedonic excellence and the elicitation of underlying subjective quality factors. The second goal is the application of the developed method within a series of studies in the domain of mobile 3D video and television to show its applicability.Open Profiling of Quality (OPQ) is a mixed-methods research approach which combines a quantitative, psychoperceptual evaluation of hedonic excellence and a descriptive sensory analysis of underlying quality factors based on naive participants' individual vocabulary. This combination allows defining the excellence of overall quality, understanding the characteristics of quality perception, and, eventually, constructing a link between preferences and quality attributes. The method was developed under constructive research with respect to validity and reliability of test results. A series of quality evaluation studies with more than 300 test participants was conducted along different critical components of a system for optimized mobile 3DTV content delivery over DVB-H.The results complemented each other, and, even more importantly, quantitative quality preferences were explained by sensory descriptions in all studies. Beyond the development of OPQ, the thesis proposes further research approaches, e.g. a conventional profiling in which OPQ's individual vacobulary is substituted by a fixed set of Quality ofExperience components or Descriptive Sorted Napping which combines a sorting task and a short post-task interview. All approaches are compared to Open ProVling of Quality at the end of the thesis. To be able to holistically contrast strengths and weaknesses of each method, a comparison model for audiovisual evaluation methods was developed and a Vrst conceptual operationalization of the model was applied in the comparison

Holoscopic 3D image depth estimation and segmentation techniques

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonToday’s 3D imaging techniques offer significant benefits over conventional 2D imaging techniques. The presence of natural depth information in the scene affords the observer an overall improved sense of reality and naturalness. A variety of systems attempting to reach this goal have been designed by many independent research groups, such as stereoscopic and auto-stereoscopic systems. Though the images displayed by such systems tend to cause eye strain, fatigue and headaches after prolonged viewing as users are required to focus on the screen plane/accommodation to converge their eyes to a point in space in a different plane/convergence. Holoscopy is a 3D technology that targets overcoming the above limitations of current 3D technology and was recently developed at Brunel University. This work is part W4.1 of the 3D VIVANT project that is funded by the EU under the ICT program and coordinated by Dr. Aman Aggoun at Brunel University, West London, UK. The objective of the work described in this thesis is to develop estimation and segmentation techniques that are capable of estimating precise 3D depth, and are applicable for holoscopic 3D imaging system. Particular emphasis is given to the task of automatic techniques i.e. favours algorithms with broad generalisation abilities, as no constraints are placed on the setting. Algorithms that provide invariance to most appearance based variation of objects in the scene (e.g. viewpoint changes, deformable objects, presence of noise and changes in lighting). Moreover, have the ability to estimate depth information from both types of holoscopic 3D images i.e. Unidirectional and Omni-directional which gives horizontal parallax and full parallax (vertical and horizontal), respectively. The main aim of this research is to develop 3D depth estimation and 3D image segmentation techniques with great precision. In particular, emphasis on automation of thresholding techniques and cues identifications for development of robust algorithms. A method for depth-through-disparity feature analysis has been built based on the existing correlation between the pixels at a one micro-lens pitch which has been exploited to extract the viewpoint images (VPIs). The corresponding displacement among the VPIs has been exploited to estimate the depth information map via setting and extracting reliable sets of local features. ii Feature-based-point and feature-based-edge are two novel automatic thresholding techniques for detecting and extracting features that have been used in this approach. These techniques offer a solution to the problem of setting and extracting reliable features automatically to improve the performance of the depth estimation related to the generalizations, speed and quality. Due to the resolution limitation of the extracted VPIs, obtaining an accurate 3D depth map is challenging. Therefore, sub-pixel shift and integration is a novel interpolation technique that has been used in this approach to generate super-resolution VPIs. By shift and integration of a set of up-sampled low resolution VPIs, the new information contained in each viewpoint is exploited to obtain a super resolution VPI. This produces a high resolution perspective VPI with wide Field Of View (FOV). This means that the holoscopic 3D image system can be converted into a multi-view 3D image pixel format. Both depth accuracy and a fast execution time have been achieved that improved the 3D depth map. For a 3D object to be recognized the related foreground regions and depth information map needs to be identified. Two novel unsupervised segmentation methods that generate interactive depth maps from single viewpoint segmentation were developed. Both techniques offer new improvements over the existing methods due to their simple use and being fully automatic; therefore, producing the 3D depth interactive map without human interaction. The final contribution is a performance evaluation, to provide an equitable measurement for the extent of the success of the proposed techniques for foreground object segmentation, 3D depth interactive map creation and the generation of 2D super-resolution viewpoint techniques. The no-reference image quality assessment metrics and their correlation with the human perception of quality are used with the help of human participants in a subjective manner

Quality of Experience in Immersive Video Technologies

Over the last decades, several technological revolutions have impacted the television industry, such as the shifts from black & white to color and from standard to high-definition. Nevertheless, further considerable improvements can still be achieved to provide a better multimedia experience, for example with ultra-high-definition, high dynamic range & wide color gamut, or 3D. These so-called immersive technologies aim at providing better, more realistic, and emotionally stronger experiences. To measure quality of experience (QoE), subjective evaluation is the ultimate means since it relies on a pool of human subjects. However, reliable and meaningful results can only be obtained if experiments are properly designed and conducted following a strict methodology. In this thesis, we build a rigorous framework for subjective evaluation of new types of image and video content. We propose different procedures and analysis tools for measuring QoE in immersive technologies. As immersive technologies capture more information than conventional technologies, they have the ability to provide more details, enhanced depth perception, as well as better color, contrast, and brightness. To measure the impact of immersive technologies on the viewersâ QoE, we apply the proposed framework for designing experiments and analyzing collected subjectsâ ratings. We also analyze eye movements to study human visual attention during immersive content playback. Since immersive content carries more information than conventional content, efficient compression algorithms are needed for storage and transmission using existing infrastructures. To determine the required bandwidth for high-quality transmission of immersive content, we use the proposed framework to conduct meticulous evaluations of recent image and video codecs in the context of immersive technologies. Subjective evaluation is time consuming, expensive, and is not always feasible. Consequently, researchers have developed objective metrics to automatically predict quality. To measure the performance of objective metrics in assessing immersive content quality, we perform several in-depth benchmarks of state-of-the-art and commonly used objective metrics. For this aim, we use ground truth quality scores, which are collected under our subjective evaluation framework. To improve QoE, we propose different systems for stereoscopic and autostereoscopic 3D displays in particular. The proposed systems can help reducing the artifacts generated at the visualization stage, which impact picture quality, depth quality, and visual comfort. To demonstrate the effectiveness of these systems, we use the proposed framework to measure viewersâ preference between these systems and standard 2D & 3D modes. In summary, this thesis tackles the problems of measuring, predicting, and improving QoE in immersive technologies. To address these problems, we build a rigorous framework and we apply it through several in-depth investigations. We put essential concepts of multimedia QoE under this framework. These concepts not only are of fundamental nature, but also have shown their impact in very practical applications. In particular, the JPEG, MPEG, and VCEG standardization bodies have adopted these concepts to select technologies that were proposed for standardization and to validate the resulting standards in terms of compression efficiency

A 3D Digital Approach to the Stylistic and Typo-Technological Study of Small Figurines from Ayia Irini, Cyprus

The thesis aims to develop a 3D digital approach to the stylistic and typo-technological study of coroplastic, focusing on small figurines. The case study to test the method is a sample of terracotta statuettes from an assemblage of approximately 2000 statues and figurines found at the beginning of the 20th century in a rural open-air sanctuary at Ayia Irini (Cyprus) by the archaeologists of the Swedish Cyprus Expedition. The excavators identified continuity of worship at the sanctuary from the Late Cypriot III (circa 1200 BC) to the end of the Cypro-Archaic II period (ca. 475 BC). They attributed the small figurines to the Cypro-Archaic I-II. Although the excavation was one of the first performed through the newly established stratigraphic method, the archaeologists studied the site and its material following a traditional, merely qualitative approach. Theanalysis of the published results identified a classification of the material with no-clear-cut criteria, and their overlap between types highlights ambiguities in creating groups and classes. Similarly, stratigraphic arguments and different opinions among archaeologists highlight the need for revising. Moreover, pastlegislation allowed the excavators to export half of the excavated antiquities, creating a dispersion of the assemblage. Today, the assemblage is still partly exhibited at the Cyprus Museum in Nicosia and in four different museums in Sweden. Such a setting prevents to study, analyse and interpret the assemblageholistically. This research proposes a 3D chaîne opératoire methodology to study the collection’s small terracotta figurines, aiming to understand the context’s function and social role as reflected by the classification obtained with the 3D digital approach. The integration proposed in this research of traditional archaeological studies, and computer-assisted investigation based on quantitative criteria, identified and defined with 3D measurements and analytical investigations, is adopted as a solution to the biases of a solely qualitative approach. The 3D geometric analysis of the figurines focuses on the objects’ shape and components, mode of manufacture, level of expertise, specialisation or skills of the craftsman and production techniques. The analysis leads to the creation of classes of artefacts which allow archaeologists to formulate hypotheses on the production process, identify a common production (e.g., same hand, same workshop) and establish a relative chronological sequence. 3D reconstruction of the excavation’s area contributes to the virtual re-unification of the assemblage for its holistic study, the relative chronological dating of the figurines and the interpretation of their social and ritual purposes. The results obtained from the selected sample prove the efficacy of the proposed 3D approach and support the expansion of the analysis to the whole assemblage, and possibly initiate quantitative and systematic studies on Cypriot coroplastic production

Intelligence, Creativity and Fantasy

UID/HIS/04666/2019 This is the 2nd volume of PHI series, published by CRC Press, the 4th published by CRC Press and the 5th volume of PHI proceedings.The texts presented in Proportion Harmonies and Identities (PHI) - INTELLIGENCE, CREATIVITY AND FANTASY were compiled with the intent to establish a multidisciplinary platform for the presentation, interaction and dissemination of research. The aim is also to foster the awareness and discussion on the topics of Harmony and Proportion with a focus on different visions relevant to Architecture, Arts and Humanities, Design, Engineering, Social and Natural Sciences, and their importance and benefits for the sense of both individual and community identity. The idea of modernity has been a significant motor for development since the Western Early Modern Age. Its theoretical and practical foundations have become the working tools of scientists, philosophers, and artists, who seek strategies and policies to accelerate the development process in different contexts.authorsversionpublishe