Ambient Gestures

We present Ambient Gestures, a novel gesture-based system designed to support ubiquitous ‘in the environment’ interactions with everyday computing technology. Hand gestures and audio feedback allow users to control computer applications without reliance on a graphical user interface, and without having to switch from the context of a non-computer task to the context of the computer. The Ambient Gestures system is composed of a vision recognition software application, a set of gestures to be processed by a scripting application and a navigation and selection application that is controlled by the gestures. This system allows us to explore gestures as the primary means of interaction within a multimodal, multimedia environment. In this paper we describe the Ambient Gestures system, define the gestures and the interactions that can be achieved in this environment and present a formative study of the system. We conclude with a discussion of our findings and future applications of Ambient Gestures in ubiquitous computing

‘It’s a-me, Mario!’ Exploring dynamic changes and similarities in the composition of early Nintendo video game music

As with films, a thoughtfully composed video game soundtrack has the ability to dramatically enhance and elevate the experience for the audience or player. This article explores the potential issues and difficulties of composing for video game systems by studying the sound-producing hardware and music for two popular systems from one manufacturer. By comparing two of Nintendo’s Super Mario titles, which appeared on both 8-bit and 16-bit systems, through an analysis of the technology, audio, visual (audiovisual), music, and gameplay elements, it is shown that the musical composition was affected by the limitations of processing power. The discussion shows how the composer, Koji Kondo, overcame the issues of limited computing power by using layers of repetition while applying various functions of music for film to enhance player immersion. Kondo composed theme music that has become engrained in popular culture and is synonymous with one of Nintendo’s flagship franchises (Greening, 2014). By attempting to understand the method or approach behind the composition for earlier systems, it is possible to investigate and discuss the evolution of video game music while acknowledging and contributing to the study of music for games. A musical analysis of the Castle and Underwater themes on each system allows for a direct comparison of the compositional approach, while an audiovisual analysis reveals the presence of existing cinematic tropes and identifies potential influences on the creation of effective musical soundtracks for video games. Applying audiovisual theory to games will require the use of existing literature from Lissa (1965), Gorbman (1987), Chion (1994) and Tagg (2004), along with the work of Collins (2005; 2007a; 2007b; 2008a; 2008b), which adapts and applies audiovisual analysis to video games

General highlight detection in sport videos

Attention is a psychological measurement of human reflection against stimulus. We propose a general framework of highlight detection by comparing attention intensity during the watching of sports videos. Three steps are involved: adaptive selection on salient features, unified attention estimation and highlight identification. Adaptive selection computes feature correlation to decide an optimal set of salient features. Unified estimation combines these features by the technique of multi-resolution autoregressive (MAR) and thus creates a temporal curve of attention intensity. We rank the intensity of attention to discriminate boundaries of highlights. Such a framework alleviates semantic uncertainty around sport highlights and leads to an efficient and effective highlight detection. The advantages are as follows: (1) the capability of using data at coarse temporal resolutions; (2) the robustness against noise caused by modality asynchronism, perception uncertainty and feature mismatch; (3) the employment of Markovian constrains on content presentation, and (4) multi-resolution estimation on attention intensity, which enables the precise allocation of event boundaries