Ambient Gestures

We present Ambient Gestures, a novel gesture-based system designed to support ubiquitous ‘in the environment’ interactions with everyday computing technology. Hand gestures and audio feedback allow users to control computer applications without reliance on a graphical user interface, and without having to switch from the context of a non-computer task to the context of the computer. The Ambient Gestures system is composed of a vision recognition software application, a set of gestures to be processed by a scripting application and a navigation and selection application that is controlled by the gestures. This system allows us to explore gestures as the primary means of interaction within a multimodal, multimedia environment. In this paper we describe the Ambient Gestures system, define the gestures and the interactions that can be achieved in this environment and present a formative study of the system. We conclude with a discussion of our findings and future applications of Ambient Gestures in ubiquitous computing

Human Robot Interface for Assistive Grasping

This work describes a new human-in-the-loop (HitL) assistive grasping system for individuals with varying levels of physical capabilities. We investigated the feasibility of using four potential input devices with our assistive grasping system interface, using able-bodied individuals to define a set of quantitative metrics that could be used to assess an assistive grasping system. We then took these measurements and created a generalized benchmark for evaluating the effectiveness of any arbitrary input device into a HitL grasping system. The four input devices were a mouse, a speech recognition device, an assistive switch, and a novel sEMG device developed by our group that was connected either to the forearm or behind the ear of the subject. These preliminary results provide insight into how different interface devices perform for generalized assistive grasping tasks and also highlight the potential of sEMG based control for severely disabled individuals.Comment: 8 pages, 21 figure

Applications of high and low fidelity prototypes in researching intuitive interaction

This paper addresses some of the issues involved in incorporating use of prototypes into a research program. Definitions, merits and uses of both low and high-fidelity prototypes are discussed and then the applications of prototypes in our research program into intuitive interaction are explored. It has previously been established that intuitive interaction is based on past experience, and can be encouraged by designing interfaces that contain familiar features (Blackler, 2006; Blackler, Popovic, & Mahar, 2007b). Two aspects of the research program which are relevant to prototyping are: researching the issues of how intuitive use happens and how it can be better facilitated; and developing ways to help designers include investigations about users and their existing knowledge into their design processes in order to make interfaces more intuitive. The current and future planned applications of high and low-fidelity prototypes in each of these areas are explored. Then experiences with using high-fidelity touchscreen prototypes for experimental research into intuitive interaction are discussed, including problems with the prototypes, how they were addressed and what we have learned from the process. Next the potential for low-fidelity prototypes to elicit users’ tacit knowledge during the design process is explored. This has exciting possibilities due to the link between intuitive interaction and tacit knowledge. Finally, the challenges of developing prototype-based design tools for use by older people are discussed and future directions for using prototypes in our research program are considered. Keywords: Prototypes; intuitive interaction; experimental methodology; implicit or tacit knowledge</p

Direct Manipulation-like Tools for Designing Intelligent Virtual Agents

If intelligent virtual agents are to become widely adopted it is vital that they can be designed using the user friendly graphical tools that are used in other areas of graphics. However, extending this sort of tool to autonomous, interactive behaviour, an area with more in common with artificial intelligence, is not trivial. This paper discusses the issues involved in creating user-friendly design tools for IVAs and proposes an extension of the direct manipulation methodology to IVAs. It also presents an initial implementation of this methodology