Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Generating multimedia briefings: coordinating language and illustration

AbstractCommunication can be more effective when several media (such as text, speech, or graphics) are integrated and coordinated to present information. This changes the nature of media-specific generation (e.g., language or graphics generation), which must take into account the multimedia context in which it occurs. This paper presents work on coordinating and integrating speech, text, static and animated three-dimensional graphics, and stored images, as part of several systems we have developed at Columbia University. A particular focus of our work has been on the generation of presentations that brief a user on information of interes

Image-Enabled Discourse: Investigating the Creation of Visual Information as Communicative Practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making as an interactive tool for conveying information. Images are increasingly ubiquitous in daily communication, in large part due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems and visually enabled collaborative work tools. Human abilities to use images to communicate are however far more sophisticated and nuanced than these technologies currently support. In order to learn more about the practice of image making as a specialized form of information and communication behavior, this study examined face-to-face conversations involving the creation of ad hoc visualizations (i.e., napkin drawings ). A model of image-enabled discourse is introduced, which positions image making as a specialized form of communicative practice. Multimodal analysis of video-recorded conversations focused on identifying image-enabled communicative activities in terms of interactional sociolinguistic concepts of conversational involvement and coordination, specifically framing, footing and stance. The study shows that when drawing occurs in the context of an ongoing dialogue, the activity of visual representation performs key communicative tasks. Visualization is a form of social interaction that contributes to the maintenance of conversational involvement in ways that are not often evident in the image artifact. For example, drawing enables us to coordinate with each other, to introduce alternative perspectives into a conversation and even to temporarily suspend the primary thread of a discussion in order to explore a tangential thought. The study compares attributes of the image artifact with those of the activity of image making, described as a series of contrasting affordances. Visual information in complex systems is generally represented and managed based on the affordances of the artifact, neglecting to account for all that is communicated through the situated action of creating. These finding have heuristic and best-practice implications for a range of areas related to the design and evaluation of virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools

The Rhetoric Of The Regional Image Interpreting The Visual Products Of Regional Plannning

The Rhetoric of the Regional Image: Interpreting the Visual Products of Regional Planning investigates the manner in which visual conventions and visual contexts of regional visioning scenarios affect their interpretation by urban and regional planners, who use visual communication to meet the technical and rhetorical demands of their professional practice. The research assesses Central Florida‘s ―How Shall We Grow?‖ regional land use scenario using focus groups and interviews with planning professionals, a corresponding survey of community values, and rhetorical analysis to explore the ―How Shall We Grow?‖ scenario as persuasive communication. The Rhetoric of the Regional Image proposes specific recommendations for technologybased visual communication and scenario development in urban and regional planning practice, while contributing to literature in technical communication and rhetoric by examining planners‘ professional communication within their discourse communit

Using an ontology for guiding natural language interaction with knowledge based systems

Des dels anys 80, els sistemes basats en el coneixement, programes que utilitzen una gran quantitat de informació per modelar situacions i resoldre problemes, han adquirit gran importància en el camp industrial, financer i científic. La complexitat d'aquests sistemes fa que el seu ús presenti més dificultats que altres aplicacions informàtiques. La comunicació entre els sistemes basats en el coneixement i l'usuari presenta, doncs, nous reptes. Tot i que el llenguate natural es especialment apropiat per comunicar-se amb aquests sistemes, són pocs els que incorporen interfícies en llenguatge natural. Els motius principals són els problemes d'eficiència que presenta el processament del llenguatge natural i l'elevat cost de desenvolupar les bases de coneixement (conceptual i lingüístic) necessàries per a cada aplicació. L'objectiu d'aquesta tesi és millorar la comunicació en llenguatge natural amb els sistemes basats en el coneixement. Aquesta recerca s'ha centrat en el disseny d'una representació reutilitzable dels diferents tipus de coneixement involucrats en aquesta comunicació, que permetir de generar de forma automàtica la interfície més adequada per a cada aplicació. S'ha desenvolupat un sistema, GISE (Generador de Interfaces a Sistemas Expertos), que genera interfícies en llenguatge natural per diferents tipus d'aplicacions. Aquest sistema adapta automàticament les bases de coneixement lingüístic generals als requeriments d'una aplicació concreta, obtenint la gramàtica més apropiada. El disseny del sistema està basat en una representació reutilitzable i modular dels diferents tipus de coneixement necessaris en la comunicació en llenguatge natural. Aquesta informació consisteix en els conceptes de l'aplicació, les tasques de comunicació, el coneixement lingüístic i les relacions generals entre el coneixement conceptual i la seva realització lingüística. Tres bases de coneixement s'han dissenyat per representar aquesta informació: la ontologia conceptual, la ontologia lingüística i un conjunt de relges de producció. El coneixement conceptual s'ha representat en la ontologia conceptual. Aquest coneixement inclou aspectes sobre el domini i la funcionalitat. Tota la informació necessària per modelar l'aplicació i tots els possibles actes de comunicació estan representats en la ontologia conceptual. La complexitat dels sistemes basats en el coneixement fa necessària una representació formal i explícita de la seva funcionalitat i domini.El coneixement lingüístic general necessari per expressar en llenguatge natural les possibles tasques del sistema es representen en la ontologia lingüística.La informació que permet relacionar el coneixement lingüístic general a una aplicació concreta per tal d'obtenir la gramàtica més adequada es representada mitjançant un conjunt de regles de producció.L'organització modular dels diferents tipus de coneixement que intervenen en la comunicació facilita l'adaptació del sistema a diferents tipus d'aplicacions i usuaris.Les gramàtiques generades pel sistema GISE utilitzen un llenguatge alhora ric i precís, adaptat a l'aplicació. La interfície del sistema incorpora un sistema de finestres que guia a l'usuari a introduir les opcions en llenguatge natural que el sistema reconeix.GISE s'ha aplicat a diferents sistemes: a SIREDOJ, un sistema expert en lleis i a un sistema que dóna informació sobre trens.Since the 1980's, knowledge based systems (KBSs), programs that use knowledge to model situations and solve problems, have spread throughout industry, finance and science. Human communication with these systems deals with complex concepts and relationships that are not present in other software applications. Allthough the natural language (NL) is especially appropriate for expressing these concepts, there are not many KBSs incorporating NL interfaces. The main reasons for this are problems of efficiency in NLI performance, lack of adequacy to the communication needs of the applications and the high cost of developing and maintaining them.The aim of this thesis is to study how the communication process and engineering features can be improved in NL interaction with KBSs. This study has been focused on the efficient and reusable representation of the knowledge involved in NL communication with KBSs. GISE (Generador de Interfaces a Sistemas Expertos), a system supporting NL communication with KBSs has been developed. This system adapts the general linguistic resources to application requirements in order to automatically obtain application-restricted grammars. The main issue of the system design is a separate and reusable representation of all types of knowledge involved in communication with KBSs. This knowledge consists of the application knowledge appearing in the communication, the tasks of communication, the linguistic knowledge supporting their expression and the general relationships between conceptual knowledge and its linguistic realization. Three general bases were designed to represent all this knowledge : the Conceptual Ontology (CO), the Linguistic Ontology (LO) and a set of control rules.Conceptual knowledge is represented in the CO. This conceptual knowledge includes domain and functionality issues. All knowledge required to model the applications as well as the description of all possible communication acts is provided in the CO. The CO is the skeleton for anchoring the domain and the functionality of the applications. The complexity of KBS performance makes a formal and explicit representation of their domain and functionality necessary. The general linguistic knowledge needed to cover the expression in NL of the tasks the system performs is represented by means of the LO and a set containing all possible realizations of the application terms. The LO is domain and application independent. The control information to relate the general linguistic knowledge to conceptual application knowledge in order to generate the application-restricted grammars is represented by a set of production rules. The modular organization of the relevant knowledge into separate data structures provides great flexibility for adapting the system to different types of applications and users.The grammars generated by GISE use expressive and precise language tuned to the application and adapted to the evolution of the communicative process. A menu-system to guide the user in introducing the NL is integrated into the GISE interface. GISE has been applied to a couple of applications: SIREDOJ, an ES in law and a railway communication system