Agent-based autonomous systems and abstraction engines: Theory meets practice

We report on experiences in the development of hybrid autonomous systems where high-level decisions are made by a rational agent. This rational agent interacts with other sub-systems via an abstraction engine. We describe three systems we have developed using the EASS BDI agent programming language and framework which supports this architecture. As a result of these experiences we recommend changes to the theoretical operational semantics that underpins the EASS framework and present a fourth implementation using the new semantics

Integration of psychological models in the design of artificial creatures

Artificial creatures form an increasingly important component of interactive computer games. Examples of such creatures exist which can interact with each other and the game player and learn from their experiences. However, we argue, the design of the underlying architecture and algorithms has to a large extent overlooked knowledge from psychology and cognitive sciences. We explore the integration of observations from studies of motivational systems and emotional behaviour into the design of artificial creatures. An initial implementation of our ideas using the “sim agent” toolkit illustrates that physiological models can be used as the basis for creatures with animal like behaviour attributes. The current aim of this research is to increase the “realism” of artificial creatures in interactive game-play, but it may have wider implications for the development of AI

Towards Learning ‘Self’ and Emotional Knowledge in Social and Cultural Human-Agent Interactions

Original article can be found at: http://www.igi-global.com/articles/details.asp?ID=35052 Copyright IGI. Posted by permission of the publisher.This article presents research towards the development of a virtual learning environment (VLE) inhabited by intelligent virtual agents (IVAs) and modeling a scenario of inter-cultural interactions. The ultimate aim of this VLE is to allow users to reflect upon and learn about intercultural communication and collaboration. Rather than predefining the interactions among the virtual agents and scripting the possible interactions afforded by this environment, we pursue a bottomup approach whereby inter-cultural communication emerges from interactions with and among autonomous agents and the user(s). The intelligent virtual agents that are inhabiting this environment are expected to be able to broaden their knowledge about the world and other agents, which may be of different cultural backgrounds, through interactions. This work is part of a collaborative effort within a European research project called eCIRCUS. Specifically, this article focuses on our continuing research concerned with emotional knowledge learning in autobiographic social agents.Peer reviewe

Developing social action capabilities in a humanoid robot using an interaction history architecture

“This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." “Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.” DOI: 10.1109/ICHR.2008.4756013We present experimental results for the humanoid robot Kaspar2 engaging in a simple “peekaboo” interaction game with a human partner. The robot develops the capability to engage in the game by using its history of interactions coupled with audio and visual feedback from the interaction partner to continually generate increasingly appropriate behaviour. The robot also uses facial expressions to feedback its level of reward to the partner. The results support the hypothesis that reinforcement of time-extended experiences through interaction allows a robot to act appropriately in an interaction

Interaction and Experience in Enactive Intelligence and Humanoid Robotics

We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a “form-of-life”’ in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful framework for the construction of complex adaptive systems, based on interaction, habit, and experience. Enactive cognitive architectures (following insights of Varela, Thompson and Rosch) that we have developed support social learning and robot ontogeny by harnessing information-theoretic methods and raw uninterpreted sensorimotor experience to scaffold the acquisition of behaviours. The success criterion here is validation by the robot engaging in ongoing human-robot interaction with naive participants who, over the course of iterated interactions, shape the robot’s behavioural and linguistic development. Engagement in such interaction exhibiting aspects of purposeful, habitual recurring structure evidences the developed capability of the humanoid to enact language and interaction games as a successful participant

Learning Representations in Model-Free Hierarchical Reinforcement Learning

Common approaches to Reinforcement Learning (RL) are seriously challenged by large-scale applications involving huge state spaces and sparse delayed reward feedback. Hierarchical Reinforcement Learning (HRL) methods attempt to address this scalability issue by learning action selection policies at multiple levels of temporal abstraction. Abstraction can be had by identifying a relatively small set of states that are likely to be useful as subgoals, in concert with the learning of corresponding skill policies to achieve those subgoals. Many approaches to subgoal discovery in HRL depend on the analysis of a model of the environment, but the need to learn such a model introduces its own problems of scale. Once subgoals are identified, skills may be learned through intrinsic motivation, introducing an internal reward signal marking subgoal attainment. In this paper, we present a novel model-free method for subgoal discovery using incremental unsupervised learning over a small memory of the most recent experiences (trajectories) of the agent. When combined with an intrinsic motivation learning mechanism, this method learns both subgoals and skills, based on experiences in the environment. Thus, we offer an original approach to HRL that does not require the acquisition of a model of the environment, suitable for large-scale applications. We demonstrate the efficiency of our method on two RL problems with sparse delayed feedback: a variant of the rooms environment and the first screen of the ATARI 2600 Montezuma's Revenge game

Issues in integrating existing multi-agent systems for power engineering applications

Multi-agent systems (MAS) have proven to be an effective platform for diagnostic and condition monitoring applications in the power industry. For example, a multi-agent system architecture, entitled condition monitoring multi-agent system (COMMAS) (McArthur et al., 2004), has been applied to the ultra high frequency (UHF) monitoring of partial discharge activity inside transformers. Additionally, a multi-agent system, entitled protection engineering diagnostic agents (PEDA) (Hossack et al., 2003), has demonstrated the use of MAS technology for automated and enhanced post-fault analysis of power systems disturbances based on SCADA and digital fault recorder (DFR) data. In this paper, the authors propose the integration of COMMAS and PEDA as a means of offering enhanced decision support to engineers tasked with managing transformer assets. By providing automatically interpreted data related to condition monitoring and power system disturbances, the proposed integrated system offer engineers a more comprehensive picture of the health of a given transformer. Defects and deterioration in performance can be correlated with the operating conditions it experiences. The integration of COMMAS and PEDA has highlighted the issues inherent to the inter-operation of existing multi-agent systems and, in particular, the issues surrounding the use of differing ontologies. The authors believe that these issues need to be addressed if there is to be widespread deployment of MAS technology within the power industry. This paper presents research undertaken to integrate the two MAS and to deal with ontology issues