ANGELICA : choice of output modality in an embodied agent

The ANGELICA project addresses the problem of modality choice in information presentation by embodied, humanlike agents. The output modalities available to such agents include both language and various nonverbal signals such as pointing and gesturing. For each piece of information to be presented by the agent it must be decided whether it should be expressed using language, a nonverbal signal, or both. In the ANGELICA project a model of the different factors influencing this choice will be developed and integrated in a natural language generation system. The application domain is the presentation of route descriptions by an embodied agent in a 3D environment. Evaluation and testing form an integral part of the project. In particular, we will investigate the effect of different modality choices on the effectiveness and naturalness of the generated presentations and on the user's perception of the agent's personality

VirtualHome: Simulating Household Activities via Programs

In this paper, we are interested in modeling complex activities that occur in a typical household. We propose to use programs, i.e., sequences of atomic actions and interactions, as a high level representation of complex tasks. Programs are interesting because they provide a non-ambiguous representation of a task, and allow agents to execute them. However, nowadays, there is no database providing this type of information. Towards this goal, we first crowd-source programs for a variety of activities that happen in people's homes, via a game-like interface used for teaching kids how to code. Using the collected dataset, we show how we can learn to extract programs directly from natural language descriptions or from videos. We then implement the most common atomic (inter)actions in the Unity3D game engine, and use our programs to "drive" an artificial agent to execute tasks in a simulated household environment. Our VirtualHome simulator allows us to create a large activity video dataset with rich ground-truth, enabling training and testing of video understanding models. We further showcase examples of our agent performing tasks in our VirtualHome based on language descriptions.Comment: CVPR 2018 (Oral

Generating a 3D Simulation of a Car Accident from a Formal Description: the CarSim System

The problem of generating a 3D simulation of a car accident from a written description can be divided into two subtasks: the linguistic analysis and the virtual scene generation. As a means of communication between these two system parts, we designed a template formalism to represent a written accident report. The CarSim system processes template formal descriptions and creates corresponding 3D simulations. A planning component models the trajectories and temporal values of every vehicle that is involved in the accident

A Case Study of Using Online Communities and Virtual Environment in Massively Multiplayer Role Playing Games (MMORPGs) as a Learning and Teaching Tool for Second Language Learners

Massively Multiplayer Online Role Playing Games (MMORPGs) create large virtual communities. Online gaming shows potential not just for entertaining, but also in education. This research investigates the use of commercial MMORPGs to support second language teaching. MMORPGs offer virtual safe spaces in which students can communicate by using their target second language with global players. Using a mix of ethnography and action research, this study explores the students’ experiences of language learning and performing while playing MMORPGs. The results show that the use of MMORPGs can facilitate language development by offering fun, informal, individualised and secure virtual spaces for students to practise their language with native and other second language speakers

The Virtual Storyteller: story generation by simulation

The Virtual Storyteller is a multi-agent framework that generates stories based on a concept called emergent narrative. In this paper, we describe the motivation and approach of the Virtual Storyteller, and give an overview of the computational processes involved in the story generation process. We also discuss some of the challenges posed by our chosen approach

A 'likely benefit' from aligning Web 2.0 technologies with an institutions learning and teaching agenda

This paper demonstrates a ‘likely benefit’, and a practical view of expected challenges, when incorporating Web 2.0 technologies in a contemporary higher education context. After first exploring which factors potentially influence a shift in thinking about learning and teaching in a Web 2.0 context this paper then addresses the important role, or the affordance, of an integrated Learning Management System (LMS) and the pedagogical applications of Web 2.0 technologies. It then uses a series of case study from the University of Southern Queensland, a large distance education provider in Australia, to support these propositions. Overall, this paper suggests that the goals and ideals of Web 2.0/ Pedagogy 2.0 can be achieved, or at least stimulated, within an institutional LMS environment, as long as the LMS environment is aligned with these ideals