Active Learning in Persistent Surveillance UAV Missions

The performance of many complex UAV decision-making problems can be extremely sensitive to small errors in the model parameters. One way of mitigating this sensitivity is by designing algorithms that more effectively learn the model throughout the course of a mission. This paper addresses this important problem by considering model uncertainty in a multi-agent Markov Decision Process (MDP) and using an active learning approach to quickly learn transition model parameters. We build on previous research that allowed UAVs to passively update model parameter estimates by incorporating new state transition observations. In this work, however, the UAVs choose to actively reduce the uncertainty in their model parameters by taking exploratory and informative actions. These actions result in a faster adaptation and, by explicitly accounting for UAV fuel dynamics, also mitigates the risk of the exploration. This paper compares the nominal, passive learning approach against two methods for incorporating active learning into the MDP framework: (1) All state transitions are rewarded equally, and (2) State transition rewards are weighted according to the expected resulting reduction in the variance of the model parameter. In both cases, agent behaviors emerge that enable faster convergence of the uncertain model parameters to their true values

Operator Choice Modeling for UAV Visual Search Tasks

Unmanned aerial vehicles (UAVs) provide unprecedented access to imagery of possible ground targets of interest in real time. The availability of this imagery is expected to increase with envisaged future missions of one operator controlling multiple UAVs. This research investigates decision models that can be used to develop assistive decision support for UAV operators involved in these complex search missions. Previous human-in-the-loop experiments have shown that operator detection probabilities may decay with increased search time. Providing the operators with the ability to requeue difficult images with the option of relooking at targets later was hypothesized to help operators improve their search accuracy. However, it was not well understood how mission performance could be impacted by operators performing requeues with multiple UAVs. This work extends a queuing model of the human operator by developing a retrial queue model (ReQM) that mathematically describes the use of relooks. We use ReQM to generate performance predictions through discrete event simulation. We validate these predictions through a human-in-the-loop experiment that evaluates the impact of requeuing on a simulated multiple-UAV mission. Our results suggest that, while requeuing can improve detection accuracy and decrease mean search times, operators may need additional decision support to use relooks effectively

Glycine receptors in the Striatum, Globus pallidus, and substantia nigra of the human brain: an immunohisotochemical study

http://www.aiaa.org/agenda.cfm?lumeetingid=1998Information consensus in sensor networks has received much attention due to its numerous applications in distributed decision making. This paper discusses the problem of a distributed group of agents coming to agreement on a probability vector over a network, such as would be required in a decentralized estimation of state transition probabilities or agreement on a probabilistic search map. Unique from other recent consensus literature, however, the agents in this problem must reach agreement while accounting for the uncertainties in their respective probabilities, which are formulated according to generally non-Gaussian distributions. The first part of this paper considers the problem in which the agents seek agreement to the centralized Bayesian estimate of the probabilities, which is accomplished using consensus on hyperparameters. The second part shows that the new hyperparameter consensus methodology can ensure convergence to the centralized estimate even while measurements of a static process are occurring concurrently with the consensus algorithm. A machine repair example is used to illustrate the advantages of hyperparameter consensus over conventional consensus approaches.United States. Air Force Office of Scientific Research (Grant FA9550-08-1-0086

Correctness of Service Components and Service Component Ensembles

Nowadays, cyber-physical systems consist of a large and possibly unbounded number of nodes operating in a partially unknown environment to which they need to adapt. They also have strong requirements in terms of performances, resource usage, reliability, or security. To face this inherent complexity it is crucial to develop adequate tools and underlying models to analyze these properties at design time. Proposed models must be able to capture essential aspects of the behavior (e.g. interactions between the components, adaptive behavior, uncertain or changing environments), and the corresponding analysis techniques can only succeed if they exploit as much as possible the specific structure of the considered systems (e.g. large replication of the same component, hierarchical compositions). We consider qualitative analyses targeting boolean properties stating that the system behaves without any flaw, as well as quantitative analyses that evaluate expected performances according to predefined metrics (energy/memory consumption, average/maximum time to accomplish a task, probability to fulfil a goal, etc.). We also address security specific issues such as control policies and information flow