Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

A major challenge for the realization of intelligent robots is to supply them with cognitive abilities in order to allow ordinary users to program them easily and intuitively. One way of such programming is teaching work tasks by interactive demonstration. To make this effective and convenient for the user, the machine must be capable to establish a common focus of attention and be able to use and integrate spoken instructions, visual perceptions, and non-verbal clues like gestural commands. We report progress in building a hybrid architecture that combines statistical methods, neural networks, and finite state machines into an integrated system for instructing grasping tasks by man-machine interaction. The system combines the GRAVIS-robot for visual attention and gestural instruction with an intelligent interface for speech recognition and linguistic interpretation, and an modality fusion module to allow multi-modal task-oriented man-machine communication with respect to dextrous robot manipulation of objects.Comment: 7 pages, 8 figure

The telematic dress: Evolving garments and distributed proprioception in streaming media and fashion performance

Centered around several short films from streaming performances created in 2005, this paper explores new ideas for movement technologies and garment design in an arts and digital research context. The "telematic dress" project, developed at the DAP Lab in Nottingham, involves transdisciplinary intersections between fashion and live performance, interactive system architecture, electronic textiles, wearable technologies, choreography, and anthropology. The concept on an evolving garment design that is materialized (moved) in live performance originates from DAP Lab's experimentation with telematics and distributed media (http://art.ntu.ac. uk/performance_research/birringer/dap.htm] addressing "connective tissues" through a study of perception/proprioception in the wearer (tactile sensory processing) and the dancer/designer/viewer relationship. This study is conducted as cross-cultural communication with online performance partners in Europe, the US, Brazil and Japan. The inter-active space is predicated on transcultural questions: how does the movement with an evolving design and wearable interactive sensors travel, how does movement - and capturing of movement - allow the design to emerge toward a garment statement, and how are bodies-in-relation-to sensory fabrics affected by the multidimensional kinesthetics of a media-rich, responsive environment

Embedding Robotic Agents in the Social Environment

This paper discusses the interactive vision approach, which advocates using knowledge from the human sciences on the structure and dynamics of human-human interaction in the development of machine vision systems and interactive robots. While this approach is discussed generally, the particular case of the system being developed for the Aurora project (which aims to produce a robot to be used as a tool in the therapy of children with autism) is especially considered, with description of the design of the machine vision system being employed and discussion of ideas from the human sciences with particular reference to the Aurora system. An example architecture for a simple interactive agent, which will likely form the basis for the first implementation of this system, is briefly described and a description of hardware used for the Aurora system is given.Peer reviewe

PIWeCS: enhancing human/machine agency in an interactive composition system

This paper focuses on the infrastructure and aesthetic approach used in PIWeCS: a Public Space Interactive Web-based Composition System. The concern was to increase the sense of dialogue between human and machine agency in an interactive work by adapting Paine's (2002) notion of a conversational model of interaction as a ‘complex system’. The machine implementation of PIWeCS is achieved through integrating intelligent agent programming with MAX/MSP. Human input is through a web infrastructure. The conversation is initiated and continued by participants through arrangements and composition based on short performed samples of traditional New Zealand Maori instruments. The system allows the extension of a composition through the electroacoustic manipulation of the source material

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion