Factored Reasoning for Monitoring Dynamic Team and Goal Formation

Engineering and Applied Science

Factored Online Planning in Many-Agent POMDPs

In centralized multi-agent systems, often modeled as multi-agent partially observable Markov decision processes (MPOMDPs), the action and observation spaces grow exponentially with the number of agents, making the value and belief estimation of single-agent online planning ineffective. Prior work partially tackles value estimation by exploiting the inherent structure of multi-agent settings via so-called coordination graphs. Additionally, belief estimation methods have been improved by incorporating the likelihood of observations into the approximation. However, the challenges of value estimation and belief estimation have only been tackled individually, which prevents existing methods from scaling to settings with many agents. Therefore, we address these challenges simultaneously. First, we introduce weighted particle filtering to a sample-based online planner for MPOMDPs. Second, we present a scalable approximation of the belief. Third, we bring an approach that exploits the typical locality of agent interactions to novel online planning algorithms for MPOMDPs operating on a so-called sparse particle filter tree. Our experimental evaluation against several state-of-the-art baselines shows that our methods (1) are competitive in settings with only a few agents and (2) improve over the baselines in the presence of many agents.Comment: Extended version (includes the Appendix) of the paper accepted at AAAI-2

Influence of State-Variable Constraints on Partially Observable Monte Carlo Planning

Online planning methods for partially observable Markov decision processes (POMDPs) have re- cently gained much interest. In this paper, we pro- pose the introduction of prior knowledge in the form of (probabilistic) relationships among dis- crete state-variables, for online planning based on the well-known POMCP algorithm. In particu- lar, we propose the use of hard constraint net- works and probabilistic Markov random fields to formalize state-variable constraints and we extend the POMCP algorithm to take advantage of these constraints. Results on a case study based on Rock- sample show that the usage of this knowledge pro- vides significant improvements to the performance of the algorithm. The extent of this improvement depends on the amount of knowledge encoded in the constraints and reaches the 50% of the average discounted return in the most favorable cases that we analyzed

Standardization of a Volumetric Displacement Measurement for Two-Body Abrasion Scratch Test Data Analysis

A limitation has been identified in the existing test standards used for making controlled, two-body abrasion scratch measurements based solely on the width of the resultant score on the surface of the material. A new, more robust method is proposed for analyzing a surface scratch that takes into account the full three-dimensional profile of the displaced material. To accomplish this, a set of four volume displacement metrics are systematically defined by normalizing the overall surface profile to statistically denote the area of relevance, termed the Zone of Interaction (ZOI). From this baseline, depth of the trough and height of the ploughed material are factored into the overall deformation assessment. Proof of concept data were collected and analyzed to demonstrate the performance of this proposed methodology. This technique takes advantage of advanced imaging capabilities that now allow resolution of the scratched surface to be quantified in greater detail than was previously achievable. A quantified understanding of fundamental particle-material interaction is critical to anticipating how well components can withstand prolonged use in highly abrasive environments, specifically for our intended applications on the surface of the Moon and other planets or asteroids, as well as in similarly demanding, harsh terrestrial setting

A framework for distributed managing uncertain data in RFID traceability networks

The ability to track and trace individual items, especially through large-scale and distributed networks, is the key to realizing many important business applications such as supply chain management, asset tracking, and counterfeit detection. Networked RFID (radio frequency identification), which uses the Internet to connect otherwise isolated RFID systems and software, is an emerging technology to support traceability applications. Despite its promising benefits, there remains many challenges to be overcome before these benefits can be realized. One significant challenge centers around dealing with uncertainty of raw RFID data. In this paper, we propose a novel framework to effectively manage the uncertainty of RFID data in large scale traceability networks. The framework consists of a global object tracking model and a local RFID data cleaning model. In particular, we propose a Markov-based model for tracking objects globally and a particle filter based approach for processing noisy, low-level RFID data locally. Our implementation validates the proposed approach and the experimental results show its effectiveness.Jiangang Ma, Quan Z. Sheng, Damith Ranasinghe, Jen Min Chuah and Yanbo W

Towards Intelligent Dynamic Deployment of Mobile Sensors in Complex Resource-Bounded Environments

Decision-making in the face of uncertainty requires an understanding of the probabilistic mechanisms that govern the complex behavior of these systems. This issue applies to many domains: financial investments, disease control, military planning and homeland security. In each of these areas, there is a practical need for efficient resource-bounded reasoning capabilities to support optimal decision-making. Specifically, given a highly complex system, with numerous random variables and their dynamic interactions, how do we monitor such a system and detect crucial events that might impact our decision making process? More importantly, how do we perform this reasoning efficiently--to an acceptable degree of accuracy in real time--when there are only limited computational power and sensory capabilities? These questions encapsulate nontrivial key issues faced by many high-profile Laboratory missions: the problem of efficient inference and dynamic sensor deployment for risk/uncertainty reduction. By leveraging solid ideas such as system decomposition into loosely coupled subsystems and smart resource allocation among these subsystems, we can parallelize inference and data acquisition for faster and improved computational performance. In this report, we propose technical approaches for developing algorithmic tools to enable future scientific and engineering endeavors to better achieve the optimal use of limited resources for maximal return of information on a complex system. The result of the proposed research effort will be an efficient reasoning framework that would enable mobile sensors to work collaboratively as teams of adaptive and responsive agents, whose joint goal is to gather useful information that would assist in the inference process

Global/Local Dynamic Models

Many dynamic systems involve a number of entities that are largely independent of each other but interact with each other via a subset of state variables. We present global/local dynamic models (GLDMs) to capture these kinds of systems. In a GLDM, the state of an entity is decomposed into a globally influenced state that depends on other entities, and a locally influenced state that depends only on the entity itself. We present an inference algorithm for GLDMs called global/local particle filtering, that introduces the principle of reasoning globally about global dynamics and locally about local dynamics. We have applied GLDMs to an asymmetric urban warfare environment, in which enemy units form teams to attack important targets, and the task is to detect such teams as they form. Experimental results for this application show that global/local particle filtering outperforms ordinary particle filtering and factored particle filtering