Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the ``single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domain-expert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.Comment: Portions of this paper were published in the Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper consists of 50 pages with 11 figures and 4 table

Creating an Agent Based Framework to Maximize Information Utility

With increased reliance on communications to conduct military operations, information centric network management becomes vital. A Defense department study of information management for net-centric operations lists the need for tools for information triage (based on relevance, priority, and quality) to counter information overload, semi-automated mechanisms for assessment of quality and relevance of information, and advances to enhance cognition and information understanding in the context of missions [30]. Maximizing information utility to match mission objectives is a complex problem that requires a comprehensive solution in information classification, in scheduling, in resource allocation, and in QoS support. Of these research areas, the resource allocation mechanism provides a framework to build the entire solution. Through an agent based mindset, the lessons of robot control architecture are applied to the network domain. The task of managing information flows is achieved with a hybrid reactive architecture. By demonstration, the reactive agent responds to the observed state of the network through the Unified Behavior Framework (UBF). As information flows relay through the network, agents in the network nodes limit resource contention to improve average utility and create a network with smarter bandwidth utilization. While this is an important result for information maximization, the agent based framework may have broader applications for managing communication networks

Attention and Sensor Planning in Autonomous Robotic Visual Search

This thesis is concerned with the incorporation of saliency in visual search and the development of sensor planning strategies for visual search. The saliency model is a mixture of two schemes that extracts visual clues regarding the structure of the environment and object specific features. The sensor planning methods, namely Greedy Search with Constraint (GSC), Extended Greedy Search (EGS) and Dynamic Look Ahead Search (DLAS) are approximations to the optimal solution for the problem of object search, as extensions to the work of Yiming Ye. Experiments were conducted to evaluate the proposed methods. They show that by using saliency in search a performance improvement up to 75% is attainable in terms of number of actions taken to complete the search. As for the planning strategies, the GSC algorithm achieved the highest detection rate and the best efficiency in terms of cost it incurs to explore every percentage of an environment

Jointly Optimizing Placement and Inference for Beacon-based Localization

The ability of robots to estimate their location is crucial for a wide variety of autonomous operations. In settings where GPS is unavailable, measurements of transmissions from fixed beacons provide an effective means of estimating a robot's location as it navigates. The accuracy of such a beacon-based localization system depends both on how beacons are distributed in the environment, and how the robot's location is inferred based on noisy and potentially ambiguous measurements. We propose an approach for making these design decisions automatically and without expert supervision, by explicitly searching for the placement and inference strategies that, together, are optimal for a given environment. Since this search is computationally expensive, our approach encodes beacon placement as a differential neural layer that interfaces with a neural network for inference. This formulation allows us to employ standard techniques for training neural networks to carry out the joint optimization. We evaluate this approach on a variety of environments and settings, and find that it is able to discover designs that enable high localization accuracy.Comment: Appeared at 2017 International Conference on Intelligent Robots and Systems (IROS

Efficient Simultaneous Task and Motion Planning for Multiple Mobile Robots Using Task Reachability Graphs

In this thesis, we consider the problem of efficient navigation by robots in initially unknown environments while performing tasks at certain locations. In initially unknown environments, the path plans might change dynamically as the robot discovers obstacles along its route. Because robots have limited energy, adaptations to the task schedule of the robot in conjunction with updates to its path plan are required so that the robot can perform its tasks while reducing time and energy expended. However, most existing techniques consider robot path planning and task planning as separate problems. This thesis plans to bridge this gap by developing a unified approach for navigating multiple robots in uncertain environments. We first formalize this as a problem called task ordering with path uncertainty (TOP-U) where robots are provided with a set of task locations to visit in a bounded environment, but the length of the path between a pair of task locations is initially known only coarsely by the robots. The robots must find the order of tasks that reduces the path length to visit the task locations. We then propose an abstraction called a task reachability graph (TRG) that integrates the robots task ordering and path planning. The TRG is updated dynamically based on inter-task path costs calculated by the path planner. A Hidden Markov Model-based technique calculates the belief in the current path costs based on the environment perceived by the robot’s sensors. We then describe a Markov Decision Process-based algorithm used by each robot in a distributed manner to reason about the path lengths between tasks and select the paths that reduce the overall path length to visit the task locations. We have evaluated our algorithm in simulated and hardware robots. Our results show that the TRG-based approach performs up to 60% better in planning and locomotion times with 44% fewer replans, while traveling almost-similar distances as compared to a greedy, nearest task-first selection algorithm