Decision-Theoretic Golog with Qualitative Preferences

Personalization is becoming increasingly important in agent programming, particularly as it relates to the Web. We propose to develop underspecified, task-specific agent programs, and to automatically personalize them to the preferences of individual users. To this end, we propose a framework for agent programming that integrates rich, non-Markovian, qualitative user preferences expressed in a linear temporal logic with quantitative Markovian reward functions. We begin with DTGOLOG, a first-order, decisiontheoretic agent programming language in the situation calculus. We present an algorithm that compiles qualitative preferences into GOLOG programs and prove it sound and complete with respect to the space of solutions. To integrate these preferences into DTGOLOG we introduce the notion of multiprogram synchronization and restate the semantics of the language as a transition semantics. We demonstrate the utility of this framework with an application to personalized travel planning over the Web. To the best of our knowledge this is the first work to combine qualitative and quantitative preferences for agent programming. Further, while the focus of this paper is on the integration of qualitative and quantitative preferences, a side effect of this work is realization of the simpler task of integrating qualitative preferences alone into agent programming as well as the generation of GOLOG programs from LTL formulae.

ARBEITSBEREICH WISSENSBASIERTE SYSTEME TEAM PROGRAMMING IN GOLOG UNDER PARTIAL OBSERVABILITY

Abstract. We present and explore the agent programming language TEAMGOLOG, which is a novel approach to programming a team of cooperative agents under partial observability. Every agent is associated with a partial control program in Golog, which is completed by the TEAMGOLOG interpreter in an optimal way by assuming a decision-theoretic semantics. The approach is based on the key concepts of a synchronization state and a communication state, which allow the agents to passively resp. actively coordinate their behavior, while keeping their belief states, observations, and activities invisible to the other agents. We show the practical usefulness of the TEAMGOLOG approach in a rescue simulated domain. We describe the algorithms behind the TEAMGOLOG interpreter and provide a prototype implementation. We also show through experimental results that the TEAMGOLOG approach outperforms a standard greedy one in the rescue simulated domain

A belief-desire-intention architechture with a logic-based planner for agents in stochastic domains

This dissertation investigates high-level decision making for agents that are both goal and utility driven. We develop a partially observable Markov decision process (POMDP) planner which is an extension of an agent programming language called DTGolog, itself an extension of the Golog language. Golog is based on a logic for reasoning about action—the situation calculus. A POMDP planner on its own cannot cope well with dynamically changing environments and complicated goals. This is exactly a strength of the belief-desire-intention (BDI) model: BDI theory has been developed to design agents that can select goals intelligently, dynamically abandon and adopt new goals, and yet commit to intentions for achieving goals. The contribution of this research is twofold: (1) developing a relational POMDP planner for cognitive robotics, (2) specifying a preliminary BDI architecture that can deal with stochasticity in action and perception, by employing the planner.ComputingM. Sc. (Computer Science

Logic-Based Specification Languages for Intelligent Software Agents

The research field of Agent-Oriented Software Engineering (AOSE) aims to find abstractions, languages, methodologies and toolkits for modeling, verifying, validating and prototyping complex applications conceptualized as Multiagent Systems (MASs). A very lively research sub-field studies how formal methods can be used for AOSE. This paper presents a detailed survey of six logic-based executable agent specification languages that have been chosen for their potential to be integrated in our ARPEGGIO project, an open framework for specifying and prototyping a MAS. The six languages are ConGoLog, Agent-0, the IMPACT agent programming language, DyLog, Concurrent METATEM and Ehhf. For each executable language, the logic foundations are described and an example of use is shown. A comparison of the six languages and a survey of similar approaches complete the paper, together with considerations of the advantages of using logic-based languages in MAS modeling and prototyping.Comment: 67 pages, 1 table, 1 figure. Accepted for publication by the Journal "Theory and Practice of Logic Programming", volume 4, Maurice Bruynooghe Editor-in-Chie

Learning how to combine sensory-motor functions into a robust behavior

AbstractThis article describes a system, called Robel, for defining a robot controller that learns from experience very robust ways of performing a high-level task such as “navigate to”. The designer specifies a collection of skills, represented as hierarchical tasks networks, whose primitives are sensory-motor functions. The skills provide different ways of combining these sensory-motor functions to achieve the desired task. The specified skills are assumed to be complementary and to cover different situations. The relationship between control states, defined through a set of task-dependent features, and the appropriate skills for pursuing the task is learned as a finite observable Markov decision process (MDP). This MDP provides a general policy for the task; it is independent of the environment and characterizes the abilities of the robot for the task

Risk-minimizing program execution in robotic domains

Thesis (Sc. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 153-161).In this thesis, we argue that autonomous robots operating in hostile and uncertain environments can improve robustness by computing and reasoning explicitly about risk. Autonomous robots with a keen sensitivity to risk can be trusted with critical missions, such as exploring deep space and assisting on the battlefield. We introduce a novel, risk-minimizing approach to program execution that utilizes program flexibility and estimation of risk in order to make runtime decisions that minimize the probability of program failure. Our risk-minimizing executive, called Murphy, utilizes two forms of program flexibility, 1) flexible scheduling of activity timing, and 2) redundant choice between subprocedures, in order to minimize two forms of program risk, 1) exceptions arising from activity failures, and 2) exceptions arising from timing constraint violations in a program. Murphy takes two inputs, a program written in a nondeterministic variant of the Reactive Model-based Programming Language (RMPL) and a set of stochastic activity failure models, one for each activity in a program, and computes two outputs, a risk-minimizing decision policy and value function. The decision policy informs Murphy which decisions to make at runtime in order to minimize risk, while the value function quantifies risk. In order to execute with low latency, Murphy computes the decision policy and value function offline, as a compilation step prior to program execution. In this thesis, we develop three approaches to RMPL program execution. First, we develop an approach that is guaranteed to minimize risk. For this approach, we reason probabilistically about risk by framing program execution as a Markov Decision Process (MDP). Next, we develop an approach that avoids risk altogether. For this approach, we frame program execution as a novel form of constraint-based temporal reasoning. Finally, we develop an execution approach that trades optimality in risk avoidance for tractability. For this approach, we leverage prior work in hierarchical decomposition of MDPs in order to mitigate complexity. We benchmark the tractability of each approach on a set of representative RMPL programs, and we demonstrate the applicability of the approach on a humanoid robot simulator.by Robert T. Effinger, IV.Sc.D

Cognitive system framework for brain-training exercise based on human-robot interactif

“This is a post-peer-review, pre-copyedit version of an article published inCognitive computation. The final authenticated version is available online at: http://dx.doi.org/10.1007/s12559-019-09696-2Every 3 seconds, someone develops dementia worldwide. Brain-training exercises, preferably involving also physical activity, have shown their potential to monitor and improve the brain function of people affected by Alzheimer disease (AD) or mild cognitive impairment (MCI). This paper presents a cognitive robotic system designed to assist mild dementia patients during brain-training sessions of sorting tokens, an exercise inspired by the Syndrom KurzTest neuropsychological test (SKT). The system is able to perceive, learn and adapt to the user’s behaviour and is composed of two main modules. The adaptive module based on representing the human-robot interaction as a planning problem, that can adapt to the user performance offering different encouragement and recommendation actions using both verbal and gesture communication in order to minimize the time spent to solve the exercise. As safety is a very important issue, the cognitive system is enriched with a safety module that monitors the possibility of physical contact and reacts accordingly. The cognitive system is presented as well as its embodiment in a real robot. Simulated experiments are performed to (i) evaluate the adaptability of the system to different patient use-cases and (ii) validate the coherence of the proposed safety module. A real experiment in the lab, with able users, is used as preliminary evaluation to validate the overall approach. Results in laboratory conditions show that the two presented modules effectively provide additional and essential functionalities to the system, although further work is necessary to guarantee robustness and timely response of the robot before testing it with patientsPeer ReviewedPostprint (author's final draft

LTL Verification of Online Executions with Sensing in Bounded Situation Calculus

Abstract. We look at agents reasoning about actions from a firstperson perspective. The agent has a representation of world as situation calculus action theory. It can perform sensing actions to acquire information. The agent acts “online”, i.e., it performs an action only if it is certain that the action can be executed, and collects sensing results from the actual world. When the agent reasons about its future actions, it indeed considers that it is acting online; however only possible sensing values are available. The kind of reasoning about actions we consider for the agent is verifying a first-order (FO) variant (without quantification across situations) of linear time temporal logic (LTL). We mainly focus on bounded action theories, where the number of facts that are true in any situation is bounded. The main results of this paper are: (i) possible sensing values can be based on consistency if the initial situation description is FO; (ii) for bounded action theories, progression over histories that include sensing results is always FO; (iii) for bounded theories, verifying our FO LTL against online executions with sensing is decidable.

Agent programming in the cognitive era

It is claimed that, in the nascent ‘Cognitive Era’, intelligent systems will be trained using machine learning techniques rather than programmed by software developers. A contrary point of view argues that machine learning has limitations, and, taken in isolation, cannot form the basis of autonomous systems capable of intelligent behaviour in complex environments. In this paper, we explore the contributions that agent-oriented programming can make to the development of future intelligent systems. We briefly review the state of the art in agent programming, focussing particularly on BDI-based agent programming languages, and discuss previous work on integrating AI techniques (including machine learning) in agent-oriented programming. We argue that the unique strengths of BDI agent languages provide an ideal framework for integrating the wide range of AI capabilities necessary for progress towards the next-generation of intelligent systems. We identify a range of possible approaches to integrating AI into a BDI agent architecture. Some of these approaches, e.g., ‘AI as a service’, exploit immediate synergies between rapidly maturing AI techniques and agent programming, while others, e.g., ‘AI embedded into agents’ raise more fundamental research questions, and we sketch a programme of research directed towards identifying the most appropriate ways of integrating AI capabilities into agent programs

A Stochastic Belief Change Framework with an Observation Stream and Defaults as Expired Observations

Abstract. A framework for an agent to change its probabilistic beliefs after a stream of noisy observations is received is proposed. Observations which are no longer relevant, become default assumptions until overridden by newer, more prevalent observations. A distinction is made between background and foreground beliefs. Agent actions and environment events are distinguishable and form part of the agent model. It is left up to the agent designer to provide an environment model; a submodel of the agent model. An example of an environment model is provided in the paper, and an example scenario is based on it. Given the particular form of the agent model, several 'patterns of cognition' can be identified. An argument is made for four particular patterns