Optimal control approaches for persistent monitoring problems.

Thesis (Ph. D.)--Boston UniversityPersistent monitoring tasks arise when agents must monitor a dynamically changing environment which cannot be fully covered by a stationary team of available agents. It differs from traditional coverage tasks due to the perpetual need to cover a changing environment, i.e., all areas of the mission space must be visited infinitely often. This dissertation presents an optimal control framework for persistent monitoring problems where the objective is to control the movement of multiple cooperating agents to minimize an uncertainty metric in a given mission space. In an one-dimensional mission space, it is shown that the optimal solution is for each agent to move at maximal speed from one switching point to the next, possibly waiting some time at each point before reversing its direction. Thus, the solution is reduced to a simpler parametric optimization problem: determining a sequence of switching locations and associated waiting times at these switching points for each agent. This amounts to a hybrid system which is analyzed using Infinitesimal Perturbation Analysis (IPA) , to obtain a complete on-line solution through a gradient-based algorithm. IPA is a method used to provide unbiased gradient estimates of performance metrics with respect to various controllable parameters in Discrete Event Systems (DES) as well as in Hybrid Systems (HS). It is also shown that the solution is robust with respect to the uncertainty model used, i.e., IPA provides an unbiased estimate of the gradient without any detailed knowledge of how uncertainty affects the mission space. In a two-dimensional mission space, such simple solutions can no longer be derived. An alternative is to optimally assign each agent a linear trajectory, motivated by the one dimensional analysis. It is proved, however, that elliptical trajectories outperform linear ones. With this motivation, the dissertation formulates a parametric optimization problem to determine such trajectories. It is again shown that the problem can be solved using IPA to obtain performance gradients on line and obtain a complete and scalable solution. Since the solutions obtained are generally locally optimal, a stochastic comparison algorithm is incorporated for deriving globally optimal elliptical trajectories. The dissertation also approaches the problem by representing an agent trajectory in terms of general function families characterized by a set of parameters to be optimized. The approach is applied to the family of Lissajous functions as well as a Fourier series representation of an agent trajectory. Numerical examples indicate that this scalable approach provides solutions that are near optimal relative to those obtained through a computationally intensive two point boundary value problem (TPBVP) solver. In the end, the problem is tackled using centralized and decentralized Receding Horizon Control (RHC) algorithms, which dynamically determine the control for agents by solving a sequence of optimization problems over a planning horizon and executing them over a shorter action horizon

Formal methods paradigms for estimation and machine learning in dynamical systems

Formal methods are widely used in engineering to determine whether a system exhibits a certain property (verification) or to design controllers that are guaranteed to drive the system to achieve a certain property (synthesis). Most existing techniques require a large amount of accurate information about the system in order to be successful. The methods presented in this work can operate with significantly less prior information. In the domain of formal synthesis for robotics, the assumptions of perfect sensing and perfect knowledge of system dynamics are unrealistic. To address this issue, we present control algorithms that use active estimation and reinforcement learning to mitigate the effects of uncertainty. In the domain of cyber-physical system analysis, we relax the assumption that the system model is known and identify system properties automatically from execution data. First, we address the problem of planning the path of a robot under temporal logic constraints (e.g. "avoid obstacles and periodically visit a recharging station") while simultaneously minimizing the uncertainty about the state of an unknown feature of the environment (e.g. locations of fires after a natural disaster). We present synthesis algorithms and evaluate them via simulation and experiments with aerial robots. Second, we develop a new specification language for tasks that require gathering information about and interacting with a partially observable environment, e.g. "Maintain localization error below a certain level while also avoiding obstacles.'' Third, we consider learning temporal logic properties of a dynamical system from a finite set of system outputs. For example, given maritime surveillance data we wish to find the specification that corresponds only to those vessels that are deemed law-abiding. Algorithms for performing off-line supervised and unsupervised learning and on-line supervised learning are presented. Finally, we consider the case in which we want to steer a system with unknown dynamics to satisfy a given temporal logic specification. We present a novel reinforcement learning paradigm to solve this problem. Our procedure gives "partial credit'' for executions that almost satisfy the specification, which can lead to faster convergence rates and produce better solutions when the specification is not satisfiable