Relating Objective and Subjective Performance Measures for AAM-based Visual Speech Synthesizers

We compare two approaches for synthesizing visual speech using Active Appearance Models (AAMs): one that utilizes acoustic features as input, and one that utilizes a phonetic transcription as input. Both synthesizers are trained using the same data and the performance is measured using both objective and subjective testing. We investigate the impact of likely sources of error in the synthesized visual speech by introducing typical errors into real visual speech sequences and subjectively measuring the perceived degradation. When only a small region (e.g. a single syllable) of ground-truth visual speech is incorrect we find that the subjective score for the entire sequence is subjectively lower than sequences generated by our synthesizers. This observation motivates further consideration of an often ignored issue, which is to what extent are subjective measures correlated with objective measures of performance? Significantly, we find that the most commonly used objective measures of performance are not necessarily the best indicator of viewer perception of quality. We empirically evaluate alternatives and show that the cost of a dynamic time warp of synthesized visual speech parameters to the respective ground-truth parameters is a better indicator of subjective quality

Smoothness perception : investigation of beat rate effect on frame rate perception

Despite the complexity of the Human Visual System (HVS), research over the last few decades has highlighted a number of its limitations. These limitations can be exploited in computer graphics to significantly reduce computational cost and thus required rendering time, without a viewer perceiving any difference in resultant image quality. Furthermore, cross-modal interaction between different modalities, such as the influence of audio on visual perception, has also been shown as significant both in psychology and computer graphics. In this paper we investigate the effect of beat rate on temporal visual perception, i.e. frame rate perception. For the visual quality and perception evaluation, a series of psychophysical experiments was conducted and the data analysed. The results indicate that beat rates in some cases do affect temporal visual perception and that certain beat rates can be used in order to reduce the amount of rendering required to achieve a perceptual high quality. This is another step towards a comprehensive understanding of auditory-visual cross-modal interaction and could be potentially used in high-fidelity interactive multi-sensory virtual environments

Assessment criteria for 2D shape transformations in animation

The assessment of 2D shape transformations (or morphing) for animation is a difficult task because it is a multi-dimensional problem. Existing morphing techniques pay most attention to shape information interactive control and mathematical simplicity. This paper shows that it is not enough to use shape information alone, and we should consider other factors such as structure, dynamics, timing, etc. The paper also shows that an overall objective assessment of morphing is impossible because factors such as timing are related to subjective judgement, yet local objective assessment criteria, e.g. based on shape, are available. We propose using “area preservation” as the shape criterion for the 2D case as an acceptable approximation to “volume preservation” in reality, and use it to establish cases in which a number of existing techniques give clearly incorrect results. The possibility of deriving objective assessment criteria for dynamics simulations and timing under certain conditions is discussed

The Big Picture on Small Screens Delivering Acceptable Video Quality in Mobile TV

Mobile TV viewers can change the viewing distance and (on some devices) scale the picture to their preferred viewing ratio, trading off size for angular resolution. We investigated optimal trade-offs between size and resolution through a series of studies. Participants selected their preferred size and rated the acceptability of the visual experience on a 200ppi device at a 4: 3 aspect ratio. They preferred viewing ratios similar to living room TV setups regardless of the much lower resolution: at a minimum 14 pixels per degree. While traveling on trains people required videos with a height larger than 35mm

Investigating facial animation production through artistic inquiry

Studies into dynamic facial expressions tend to make use of experimental methods based on objectively manipulated stimuli. New techniques for displaying increasingly realistic facial movement and methods of measuring observer responses are typical of computer animation and psychology facial expression research. However, few projects focus on the artistic nature of performance production. Instead, most concentrate on the naturalistic appearance of posed or acted expressions. In this paper, the authors discuss a method for exploring the creative process of emotional facial expression animation, and ask whether anything can be learned about authentic dynamic expressions through artistic inquiry

The sweet spot: How people trade off size and definition on mobile devices

Mobile TV can deliver up-to-date content to users on the move. But it is currently unclear how to best adapt higher resolution TV content. In this paper, we describe a laboratory study with 35 participants who watched short clips of different content and shot types on a 200ppi PDA display at a resolution of either 120x90 or 168x128. Participants selected their preferred size and rated the acceptability of the visual experience. The preferred viewing ratio depended on the resolution and had to be at least 9.8H. The minimal angular resolution people required and which limited the up-scaling factor was 14 pixels per degree. Extreme long shots were best when depicted actors were at least 0.7° high. A second study researched the ecological validity of previous lab results by comparing them to results from the field. Image size yielded more value for users in the field than was apparent from lab results. In conclusion, current prediction models based on preferred viewing distances for TV and large displays do not predict viewing preferences on mobile devices. Our results will help to further the understanding of multimedia perception and service designers to deliver both economically viable and enjoyable experiences

Can small be beautiful? assessing image resolution requirements for mobile TV

Mobile TV services are now being offered in several countries, but for cost reasons, most of these services offer material directly recoded for mobile consumption (i.e. without additional editing). The experiment reported in this paper, aims to assess the image resolution and bitrate requirements for displaying this type of material on mobile devices. The study, with 128 participants, examined responses to four different image resolutions, seven video encoding bitrates, two audio bitrates and four content types. The results show that acceptability is significantly lower for images smaller than 168×126, regardless of content type. The effect is more pronounced when bandwidth is abundant, and is due to important detail being lost in the smaller screens. In contrast to previous studies, participants are more likely to rate image quality as unacceptable when the audio quality is high