Decision theoretic agent design for personal rapid transit systems

This paper details a learning decision-theoretic intelligent agent designed to solve the problem of guiding vehicles in the context of Personal Rapid Transit (PRT). The intelligent agents are designed using Bayesian Decision Networks. The agents are designed to utilize the known methods of machine learning with Bayesian Networks (BN): parameter learning and structure learning. In addition, a new method of machine learning with BNs, termed utility learning in this paper, is introduced. BN software for Matlab is used to realize the proposed agent. Additional software is written to simulate the PRT problem using various intelligent agents that utilize one or more learning methods

Online Structure, Parameter, and Utility Updating of Bayesian Decision Networks for Cooperative Decision-Theoretic Agents

Multi-agent systems, systems consisting of more than one acting and decision making entities, are of great interest to researchers because they have advantages for some specific tasks where it would be more effective to use multiple small and simple robots rather than a large and complex one. One of the major problems with multi-agent systems is developing a means to organize or control the overall behavior of the system. Typically, multi-agent control involves one of two structures. In some designs, there is a hierarchy with some robots being leaders and other followers. Other designs involve robot specialization towards one particular task or individual robots which loosely or strongly cooperate in some manner to yield the desired behavior. This thesis studies using bayesian decision networks (BDNs) as a method to control individual robots to achieve some group or cooperative behavior. BDNs are powerful tools enabling designers of intelligent agents to model the agent\u27s environment and the behavior of other agents without expert knowledge about a system. The probabilistic nature of these networks allows agents to learn about themselves and their environment by updating their bayesian network (BN) with new observations. While two methods of learning and responding to change in the environment with BNs, parameter learning and structure learning, have been studied by many researchers as a means to control a single robot or teams of robots, a third method, utility updating, has seen little study. This work is thus a novel study of BN control since it incorporates all three methods to develop a decision theoretic agent (DTA). The agent is applied to a modified version of a personal rapid transit (PRT) problem (or personal automated transport (PAT)) that is simulated in Matlab. PRT is a proposed public transport method which offers automated on-demand transportation between any two nodes of the transportation network. The PRT problem of interest is that of autonomous control. This can be likened to one of multi-agent control of many identical agents. Several agents are developed to solve the problem, a rule based agent and BN-agents which use various subsets of the three network updating methods. The experimental results show that the DTA that uses parameter, structure, and utility updating could be a superior solution to agents based only on some subset of those methods

Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems as a Translation of a Psychological Model

Assistive systems for persons with cognitive disabilities (e.g. dementia) are difficult to build due to the wide range of different approaches people can take to accomplishing the same task, and the significant uncertainties that arise from both the unpredictability of client's behaviours and from noise in sensor readings. Partially observable Markov decision process (POMDP) models have been used successfully as the reasoning engine behind such assistive systems for small multi-step tasks such as hand washing. POMDP models are a powerful, yet flexible framework for modelling assistance that can deal with uncertainty and utility. Unfortunately, POMDPs usually require a very labour intensive, manual procedure for their definition and construction. Our previous work has described a knowledge driven method for automatically generating POMDP activity recognition and context sensitive prompting systems for complex tasks. We call the resulting POMDP a SNAP (SyNdetic Assistance Process). The spreadsheet-like result of the analysis does not correspond to the POMDP model directly and the translation to a formal POMDP representation is required. To date, this translation had to be performed manually by a trained POMDP expert. In this paper, we formalise and automate this translation process using a probabilistic relational model (PRM) encoded in a relational database. We demonstrate the method by eliciting three assistance tasks from non-experts. We validate the resulting POMDP models using case-based simulations to show that they are reasonable for the domains. We also show a complete case study of a designer specifying one database, including an evaluation in a real-life experiment with a human actor