Quadrotor control for persistent surveillance of dynamic environments

Thesis (M.S.)--Boston UniversityThe last decade has witnessed many advances in the field of small scale unmanned aerial vehicles (UAVs). In particular, the quadrotor has attracted significant attention. Due to its ability to perform vertical takeoff and landing, and to operate in cluttered spaces, the quadrotor is utilized in numerous practical applications, such as reconnaissance and information gathering in unsafe or otherwise unreachable environments. This work considers the application of aerial surveillance over a city-like environment. The thesis presents a framework for automatic deployment of quadrotors to monitor and react to dynamically changing events. The framework has a hierarchical structure. At the top level, the UAVs perform complex behaviors that satisfy high- level mission specifications. At the bottom level, low-level controllers drive actuators on vehicles to perform the desired maneuvers. In parallel with the development of controllers, this work covers the implementation of the system into an experimental testbed. The testbed emulates a city using physical objects to represent static features and projectors to display dynamic events occurring on the ground as seen by an aerial vehicle. The experimental platform features a motion capture system that provides position data for UAVs and physical features of the environment, allowing for precise, closed-loop control of the vehicles. Experimental runs in the testbed are used to validate the effectiveness of the developed control strategies

Probabilistic Hybrid Action Models for Predicting Concurrent Percept-driven Robot Behavior

This article develops Probabilistic Hybrid Action Models (PHAMs), a realistic causal model for predicting the behavior generated by modern percept-driven robot plans. PHAMs represent aspects of robot behavior that cannot be represented by most action models used in AI planning: the temporal structure of continuous control processes, their non-deterministic effects, several modes of their interferences, and the achievement of triggering conditions in closed-loop robot plans. The main contributions of this article are: (1) PHAMs, a model of concurrent percept-driven behavior, its formalization, and proofs that the model generates probably, qualitatively accurate predictions; and (2) a resource-efficient inference method for PHAMs based on sampling projections from probabilistic action models and state descriptions. We show how PHAMs can be applied to planning the course of action of an autonomous robot office courier based on analytical and experimental results

CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environments

In this paper we study a new reinforcement learning setting where the environment is non-rewarding, contains several possibly related objects of various controllability, and where an apt agent Bob acts independently, with non-observable intentions. We argue that this setting defines a realistic scenario and we present a generic discrete-state discrete-action model of such environments. To learn in this environment, we propose an unsupervised reinforcement learning agent called CLIC for Curriculum Learning and Imitation for Control. CLIC learns to control individual objects in its environment, and imitates Bob's interactions with these objects. It selects objects to focus on when training and imitating by maximizing its learning progress. We show that CLIC is an effective baseline in our new setting. It can effectively observe Bob to gain control of objects faster, even if Bob is not explicitly teaching. It can also follow Bob when he acts as a mentor and provides ordered demonstrations. Finally, when Bob controls objects that the agent cannot, or in presence of a hierarchy between objects in the environment, we show that CLIC ignores non-reproducible and already mastered interactions with objects, resulting in a greater benefit from imitation