Human-display interaction technology: Emerging remote interfaces for pervasive display environments

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.We're living in a world where information processing isn't confined to desktop computers - it's being integrated into everyday objects and activities. Pervasive computation is human centered: it permeates our physical world, helping us achieve goals and fulfill our needs with minimum effort by exploiting natural interaction styles. Remote interaction with screen displays requires a sensor-based, multimodal, touchless approach. For example, by processing user hand gestures, this paradigm removes constraints requiring physical contact and permits natural interaction with tangible digital information. Such touchless interaction can be multimodal, exploiting the visual, auditory, and olfactory senses.Ministerio de Educación y Ciencia and Amper Sistemas, SA

Augmented reality meeting table: a novel multi-user interface for architectural design

Immersive virtual environments have received widespread attention as providing possible replacements for the media and systems that designers traditionally use, as well as, more generally, in providing support for collaborative work. Relatively little attention has been given to date however to the problem of how to merge immersive virtual environments into real world work settings, and so to add to the media at the disposal of the designer and the design team, rather than to replace it. In this paper we report on a research project in which optical see-through augmented reality displays have been developed together with prototype decision support software for architectural and urban design. We suggest that a critical characteristic of multi user augmented reality is its ability to generate visualisations from a first person perspective in which the scale of rendition of the design model follows many of the conventions that designers are used to. Different scales of model appear to allow designers to focus on different aspects of the design under consideration. Augmenting the scene with simulations of pedestrian movement appears to assist both in scale recognition, and in moving from a first person to a third person understanding of the design. This research project is funded by the European Commission IST program (IST-2000-28559)

Ambient Gestures

We present Ambient Gestures, a novel gesture-based system designed to support ubiquitous ‘in the environment’ interactions with everyday computing technology. Hand gestures and audio feedback allow users to control computer applications without reliance on a graphical user interface, and without having to switch from the context of a non-computer task to the context of the computer. The Ambient Gestures system is composed of a vision recognition software application, a set of gestures to be processed by a scripting application and a navigation and selection application that is controlled by the gestures. This system allows us to explore gestures as the primary means of interaction within a multimodal, multimedia environment. In this paper we describe the Ambient Gestures system, define the gestures and the interactions that can be achieved in this environment and present a formative study of the system. We conclude with a discussion of our findings and future applications of Ambient Gestures in ubiquitous computing

Developing a Multi-Touch Map Application for a Large Screen in a Nature Centre

The paper describes the development of a research prototype of a multi-touch map application for multi-use on a large multi-touch screen intended for a nature centre. The presented system and the development steps provide insight into what can be expected when similar systems are designed. A number of new considerations regarding multi-touch interaction, map browsing, and user needs for multi-use have been taken into account during the challenging ongoing development. These considerations include making a simple user interface used with intuitive, continuous and simultaneous gestures for map browsing, and taking different kinds of users and their needs to interact with each other into account. Since multi-user map applications in multi-touch environments are still rare, the given considerations may be helpful for the future development of similar map applications intended for public spaces

Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as Touchpads

In this paper, we present a multi-user interaction interface for a large immersive space that supports simultaneous screen interactions by combining (1) user input via personal smartphones and Bluetooth microphones, (2) spatial tracking via an overhead array of Kinect sensors, and (3) WebSocket interfaces to a webpage running on the large screen. Users are automatically, dynamically assigned personal and shared screen sub-spaces based on their tracked location with respect to the screen, and use a webpage on their personal smartphone for touchpad-type input. We report user experiments using our interaction framework that involve image selection and placement tasks, with the ultimate goal of realizing display-wall environments as viable, interactive workspaces with natural multimodal interfaces.Comment: 8 pages with reference

Space time pixels

This paper reports the design of a networked system, the aim of which is to provide an intermediate virtual space that will establish a connection and support interaction between multiple participants in two distant physical spaces. The intention of the project is to explore the potential of the digital space to generate original social relationships between people that their current (spatial or social) position can difficultly allow the establishment of innovative connections. Furthermore, to explore if digital space can sustain, in time, low-level connections like these, by balancing between the two contradicting needs of communication and anonymity. The generated intermediate digital space is a dynamic reactive environment where time and space information of two physical places is superimposed to create a complex common ground where interaction can take place. It is a system that provides awareness of activity in a distant space through an abstract mutable virtual environment, which can be perceived in several different ways – varying from a simple dynamic background image to a common public space in the junction of two private spaces or to a fully opened window to the other space – according to the participants will. The thesis is that the creation of an intermediary environment that operates as an activity abstraction filter between several users, and selectively communicates information, could give significance to the ambient data that people unconsciously transmit to others when co-existing. It can therefore generate a new layer of connections and original interactivity patterns; in contrary to a straight-forward direct real video and sound system, that although it is functionally more feasible, it preserves the existing social constraints that limit interaction into predefined patterns