Collaboration with agents in VR environments

Virtual reality is gaining on importance in many fields – scientific simulation, training, therapy and also more and more in entertainment. All these applications require the human user to interact with virtual worlds inhabited by intelligent characters and to solve simulated or real problems. This thesis will present an integrated approach to simulated problem solving in virtual reality environments, with the emphasis on teamwork and the ability to control the simulations. A simulation framework satisfying these goals will be presented. A unified approach to the representation of semantic information in virtual environments based on predicate calculus will be introduced, including the representation of the world state, action semantics and basic axioms holding in the simulated world. Afterwards, the focus will be on the collaboration model based on task delegation and facilitator-centric architecture. A simple but efficient facilitator design will be presented. The issues of the collaborative problem solving will be examined. A new technique using propositional (STRIPS-like) planning with delegated actions and object-specific planning will be described. A control technique for virtual characters/objects will be detailed, enabling run-time exchange of control and control sharing over a virtual entity between multiple autonomous agents and/or human users. Finally, a set of case studies will be shown, illustrating the possible applications of the techniques developed and described in this dissertation

Customising games for non-formal rehabilitation

The field of rehabilitation has increasingly adopted commercially available games using perceptual interfaces as a means for physically training patients. The adaptability of such systems to match each person’s need and rehabilitation goal remains problematic. This paper presents a rapid prototyping approach for customising gaming technology using various affordable commercial devices and open source software. We first demonstrate how a freely available game is adapted for training disabled people through different sensors and control modes. We then show how an open online virtual world such as Second Life® offers sufficient conditions for quickly building custom content for testing with interactive devices. When presented with these prototyping possibilities, people from the target groups (healthcare professionals, patients, people with disabilities, older people, families) related such systems to their needs and further elaborated on the use of such systems. Our research indicates how availability of simple prototyping platforms expands upon the possibilities for developers and practitioners

Evaluation of Gaze Tracking Technology for Social Interaction in Virtual Environments

This paper presents a study of the application of eye tracking technology in the context of social interaction in a virtual environment. We evaluate the reliability and precision of gaze tracking in two different virtual reality applications. In spite of the known drawbacks, the technology still has a potential for interaction with virtual humans

Planning with Smart Objects

This paper presents a novel method of employing "smart objects" for problem solving in virtual environments. Smart objects were primarily used for behavioral animation in the past. The paper demonstrates how to use them for AI and planning purposes as well. We formally define which operations can be performed on a smart object in terms of their requirements and their effects. A planner uses this information to determine the correct sequence of actions needed to achieve a goal. This approach enables intelligent agents to solve problems requiring a collaboration of several agents and complex interactions with objects

Abstract Flying Cities: building a 3D world from vocal input

The Flying Cities artistic installation brings to life imaginary cities made from the speech input of visitors. In this article we describe the original interactive process generating real time 3D graphics from spectators ' vocal inputs. This example of cross-modal interaction has the nice property of providing a tangible correspondence between the two spaces. This interaction mean has proved to suit the artistic expression well but it also aims at providing anyone with a pleasant and stimulating feedback from speech activity, a new medium for creativity and a way to visually perceive a vocal performance. As the feedback we have received when presenting Flying Cities was very positive, our objective now is to cross the bridge between art and the potential applications to the rehabilitation of people with reduced mobility or for the treatment of language impairments