Gesture controlled interactive rendering in a panoramic scene

The demonstration described hereafter covers technical work carried out in the FascinatE project [1], related to the interactive retrieval and rendering of high-resolution panoramic scenes. The scenes have been captured by a special panoramic camera (the OMNICAM) [2] with is capturing high resolution video featuring a wide angle (180 degrees) field of view. Users can access the content by interacting based on a novel device-less and markerless gesture-based system that allows them to interact as naturally as possible, permitting the user to control the rendering of the scene by zooming, panning or framing through the panoramic scenePeer ReviewedPostprint (published version

Towards a format-agnostic approach for production, delivery and rendering of immersive media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalization (selection and control of content, availability on many devices). We investigate the feasibility of an end-to-end format-agnostic approach to support both these trends. In this paper, different aspects of a format- agnostic capture, production, delivery and rendering system are discussed. At the capture stage, the concept of layered scene representation is introduced, including panoramic video and 3D audio capture. At the analysis stage, a virtual director component is discussed that allows for automatic execution of cinematographic principles, using feature tracking and saliency detection. At the delivery stage, resolution-independent audiovisual transport mechanisms for both managed and unmanaged networks are treated. In the rendering stage, a rendering process that includes the manipulation of audiovisual content to match the connected display and loudspeaker properties is introduced. Different parts of the complete system are revisited demonstrating the requirements and the potential of this advanced concept

Towards a format-agnostic approach for production, delivery and rendering of immersive media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalization (selection and control of content, availability on many devices). We investigate the feasibility of an end-to-end format-agnostic approach to support both these trends. In this paper, different aspects of a format- agnostic capture, production, delivery and rendering system are discussed. At the capture stage, the concept of layered scene representation is introduced, including panoramic video and 3D audio capture. At the analysis stage, a virtual director component is discussed that allows for automatic execution of cinematographic principles, using feature tracking and saliency detection. At the delivery stage, resolution-independent audiovisual transport mechanisms for both managed and unmanaged networks are treated. In the rendering stage, a rendering process that includes the manipulation of audiovisual content to match the connected display and loudspeaker properties is introduced. Different parts of the complete system are revisited demonstrating the requirements and the potential of this advanced concept.Peer Reviewe

Improving television sound for people with hearing impairments

This thesis investigates how developments in audio for digital television can be utilised to improve the experience of hearing impaired people when watching television. The work has had significant impact on international digital TV broadcast standards; it led to the formation of the UK Clean Audio Forum whose recommendations based on the research have been included in ETSI international standards for digital television, adopted into ITU standards for IPTV and also into EBU and NorDig digital television receiver specifications. In this thesis listening tests are implemented to assess the impact of various processes with a phantom centre channel and with a centre loudspeaker. The impact of non-speech channel attenuation and dynamic range control on speech clarity, sound quality and enjoyment of audio-visual media are investigated for both hearing impaired and non-hearing impaired people. For the first time the impact of acoustical crosstalk in two channel stereo reproduction on intelligibility of speech is quantified using both subjective intelligibility assessments and acoustic measurement techniques with intelligibility benefits of 5.9% found by utilising a centre loudspeaker instead of a phantom centre. A novel implementation of principal component analysis as a pre- broadcast production tool for labelling AV media compatible with a clean audio mix is identified, and two research implementations of accessible audio are documented including an object based implementation of clean audio for live broadcast that has been developed and publicly demonstrated