Behavioral Modeling Based on Probabilistic Finite Automata: An Empirical Study

Imagine an agent that performs tasks according to different strategies. The goal of Behavioral Recognition (BR) is to identify which of the available strategies is the one being used by the agent, by simply observing the agent?s actions and the environmental conditions during a certain period of time. The goal of Behavioral Cloning (BC) is more ambitious. In this last case, the learner must be able to build a model of the behavior of the agent. In both settings, the only assumption is that the learner has access to a training set that contains instances of observed behavioral traces for each available strategy. This paper studies a machine learning approach based on Probabilistic Finite Automata (PFAs), capable of achieving both the recognition and cloning tasks. We evaluate the performance of PFAs in the context of a simulated learning environment (in this case, a virtual Roomba vacuum cleaner robot), and compare it with a collection of other machine learning approaches.This work was partially supported by project PAC::LFO (MTM2014-55262-P) of Programa Estatal de Fomento de la Investigación Científica y Técnica de Excelencia, Ministerio de Ciencia e Innovación (MICINN), Spain, and by the National Science Foundation (NSF) project SCH-1521943, USA

Learning Internal State Memory Representations from Observation

Learning from Observation (LfO) is a machine learning paradigm that mimics how people learn in daily life: learning how to do something simply by watching someone else do it. LfO has been used in various applications, from video game agent creation to driving a car, but it has always been limited by the inability of an observer to know what a performing entity chooses to remember as they act in an environment. Various methods have either ignored the effects of memory or otherwise made simplistic assumptions about its structure. In this dissertation, we propose a new method, Memory Composition Learning, that captures the influence of a performer\u27s memory in an observed behavior through the creation of an auxiliary memory feature set that explicitly models the aspects of the environment with significance for future decisions, and which can be used with a machine learning technique to provide salient information from memory. It advances the state of the art by automatically learning the internal structure of memory instead of ignoring or predefining it. This research is difficult in that memory modeling is an unsupervised learning problem that we elect to solve solely from unobtrusive observation. This research is significant for LfO in that it will allow learning techniques that otherwise could not use information from memory to use a tailored set of learned memory features that capture salient influences from memory and enable decision-making based on these influences for more effective learning performance. To validate our hypothesis, we implemented a prototype for modeling observed memory influences with our approach and applied it to simulated vacuum cleaner and lawn mower domains. Our investigation revealed that MCL was able to automatically learn memory features that describe the influences on an observed actor\u27s internal state, and which improved learning performance of observed behaviors

Discovering Contexts from Observed Human Performance

This paper describes an investigation to determine the technical feasibility of discovering and identifying the various contexts experienced by a human performer (called an actor) solely from a trace of time-stamped values of variables. More specifically, the goal of this research was to discover the contexts that a human actor experienced, while performing a tactical task in a simulated environment, the sequence of these contexts and their temporal duration. We refer to this process as the contextualization of the performance trace. In the process of doing this, we devised a context discovery algorithm called context partitioning and clustering (COPAC). The relevant variables that were observed in the trace were selected a priori by a human. The output of the COPAC algorithm was qualitatively compared with manual (human) contextualization of the same traces. One possible use of such automated context discovery is to help build autonomous tactical agents capable of performing the same tasks as the human actor. As such, we also quantitatively compared the results of using the COPAC-derived contexts with those obtained with human-derived contextualization in building autonomous tactical agents. Test results are described and discussed

Discovering Contexts From Observed Human Performance

This paper describes an investigation to determine the technical feasibility of discovering and identifying the various contexts experienced by a human performer (called an actor) solely from a trace of time-stamped values of variables. More specifically, the goal of this research was to discover the contexts that a human actor experienced, while performing a tactical task in a simulated environment, the sequence of these contexts and their temporal duration.We refer to this process as the contextualization of the performance trace. In the process of doing this, we devised a context discovery algorithm called context partitioning and clustering (COPAC). The relevant variables that were observed in the trace were selected a priori by a human. The output of the COPAC algorithm was qualitatively compared with manual (human) contextualization of the same traces. One possible use of such automated context discovery is to help build autonomous tactical agents capable of performing the same tasks as the human actor. As such, we also quantitatively compared the results of using the COPAC-derived contexts with those obtained with human-derived contextualization in building autonomous tactical agents. Test results are described and discussed. © 2013 IEEE