Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the ``single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domain-expert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.Comment: Portions of this paper were published in the Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) in 2016 and in the Proceedings of Robotics: Science and Systems (RSS) in 2016. The paper consists of 50 pages with 11 figures and 4 table

Energy and Route Optimization of Moving Devices

This thesis highlights our efforts in energy and route optimization of moving devices. We have focused on three categories of such devices; industrial robots in a multi-robot environment, generic vehicles in a vehicle routing problem (VRP) context, automatedguided vehicles (AGVs) in a large-scale flexible manufacturing system (FMS). In the first category, the aim is to develop a non-intrusive energy optimization technique, based on a given set of paths and sequences of operations, such that the original cycle time is not exceeded. We develop an optimization procedure based on a mathematical programming model that aims to minimize the energy consumption and peak power. Our technique has several advantages. It is non-intrusive, i.e. it requires limited changes in the robot program and can be implemented easily. Moreover,it is model-free, in the sense that no particular, and perhaps secret, parameter or dynamic model is required. Furthermore, the optimization can be done offline, within seconds using a generic solver. Through careful experiments, we have shown that it is possible to reduce energy and peak-power up to about 30% and 50% respectively. The second category of moving devices comprises of generic vehicles in a VRP context. We have developed a hybrid optimization approach that integrates a distributed algorithm based on a gossip protocol with a column generation (CG) algorithm, which manages to solve the tested problems faster than the CG algorithm alone. The algorithm is developed for a VRP variation including time windows (VRPTW), which is meant to model the task of scheduling and routing of caregivers in the context of home healthcare routing and scheduling problems (HHRSPs). Moreover,the developed algorithm can easily be parallelized to further increase its efficiency. The last category deals with AGVs. The choice of AGVs was not arbitrary; by design, we decided to transfer our knowledge of energy optimization and routing algorithms to a class of moving devices in which both techniques are of interest. Initially, we improve an existing method of conflict-free AGV scheduling and routing, such that the new algorithm can manage larger problems. A heuristic version of the algorithm manages to solve the problem instances in a reasonable amount of time. Later, we develop strategies to reduce the energy consumption. The study is carried out using an AGV system installed at Volvo Cars. The results are promising; (1)the algorithm reduces performance measures such as makespan up to 50%, while reducing the total travelled distance of the vehicles about 14%, leading to an energy saving of roughly 14%, compared to the results obtained from the original traffic controller. (2) It is possible to reduce the cruise velocities such that more energy is saved, up to 20%, while the new makespan remains better than the original one

Stochastic Programming Models For Electric Vehicles’ Operation: Network Design And Routing Strategies

Logistic and transportation (L&T) activities become a significant contributor to social and economic advances throughout the modern world Road L&T activities are responsible for a large percentage of CO2 emissions, with more than 24% of the total emission, which mostly caused by fossil fuel vehicles. Researchers, governments, and automotive companies put extensive effort to incorporate new solutions and innovations into the L&T system. As a result, Electric Vehicles (EVs) are introduced and universally accepted as one of the solutions to environmental issues. Subsequently, L&T companies are encouraged to adopt fleets of EVs. Integrating the EVs into the logistic and transportation systems introduces new challenges from strategic, planning, and operational perspectives. At the strategical level, one of the main challenges to be addressed to expand the EV charging infrastructures is the location of charging stations. Due to the longer charging time in EVs compared to the conventional vehicles, the parking locations can be considered as the candidate locations for installing charging stations. Another essential factor that should be considered in designing the Electric Vehicle Charging Station (EVCS) network is the size or capacity of charging stations. EV drivers\u27 arrival times in a community vary depending on various factors such as the purpose of the trip, time of the day, and day of the week. So, the capacity of stations and the number of chargers significantly affect the accessibility and utilization of charging stations. Also, the EVCSs can be equipped by distinct types of chargers, which are different in terms of installation cost, charging time, and charging price. City planners and EVCS owners can make low-risk and high-utilization investment decisions by considering EV users charging pattern and their willingness to pay for different charger types. At the operational level, managing a fleet of electric vehicles can offer several incentives to the L&T companies. EVs can be equipped with autonomous driving technologies to facilitate online decision making, on-board computation, and connectivity. Energy-efficient routing decisions for a fleet of autonomous electric vehicles (AEV) can significantly improve the asset utilization and vehicles’ battery life. However, employing AEVs also comes with new challenges. Two of the main operational challenges for AEVs in transport applications is their limited range and the availability of charging stations. Effective routing strategies for an AEV fleet require solving the vehicle routing problem (VRP) while considering additional constraints related to the limited range and number of charging stations. In this dissertation, we develop models and algorithms to address the challenges in integrating the EVs into the logistic and transportation systems

Scalable Approach to Uncertainty Quantification and Robust Design of Interconnected Dynamical Systems

Development of robust dynamical systems and networks such as autonomous aircraft systems capable of accomplishing complex missions faces challenges due to the dynamically evolving uncertainties coming from model uncertainties, necessity to operate in a hostile cluttered urban environment, and the distributed and dynamic nature of the communication and computation resources. Model-based robust design is difficult because of the complexity of the hybrid dynamic models including continuous vehicle dynamics, the discrete models of computations and communications, and the size of the problem. We will overview recent advances in methodology and tools to model, analyze, and design robust autonomous aerospace systems operating in uncertain environment, with stress on efficient uncertainty quantification and robust design using the case studies of the mission including model-based target tracking and search, and trajectory planning in uncertain urban environment. To show that the methodology is generally applicable to uncertain dynamical systems, we will also show examples of application of the new methods to efficient uncertainty quantification of energy usage in buildings, and stability assessment of interconnected power networks

Control for Localization and Visibility Maintenance of an Independent Agent using Robotic Teams

Given a non-cooperative agent, we seek to formulate a control strategy to enable a team of robots to localize and track the agent in a complex but known environment while maintaining a continuously optimized line-of-sight communication chain to a fixed base station. We focus on two aspects of the problem. First, we investigate the estimation of the agent\u27s location by using nonlinear sensing modalities, in particular that of range-only sensing, and formulate a control strategy based on improving this estimation using one or more robots working to independently gather information. Second, we develop methods to plan and sequence robot deployments that will establish and maintain line-of-sight chains for communication between the independent agent and the fixed base station using a minimum number of robots. These methods will lead to feedback control laws that can realize this plan and ensure proper navigation and collision avoidance

Sampling-Based Coverage Path Planning for Inspection of Complex Structures

We present several new contributions in sampling-based coverage path planning, the task of finding feasible paths that give 100% sensor coverage of complex structures in obstacle-filled and visually occluded environments. First, we establish a framework for analyzing the probabilistic completeness of a sampling-based coverage algorithm, and derive results on the completeness and convergence of existing algorithms. Second, we introduce a new algorithm for the iterative improvement of a feasible coverage path; this relies on a sampling-based subroutine that makes asymptotically optimal local improvements to a feasible coverage path based on a strong generalization of the RRT algorithm. We then apply the algorithm to the real-world task of autonomous in-water ship hull inspection. We use our improvement algorithm in conjunction with redundant roadmap coverage planning algorithm to produce paths that cover complex 3D environments with unprecedented efficiency.United States. Office of Naval Research (ONR Grant N0014-06-10043

Reactive evolutionary path planning for autonomous surface vehicles in lake environments.

Autonomous Surface Vehicles (ASVs) have found a lot of promising applications in aquatic environments, i.e., sea, lakes, rivers, etc. They can be used for applications of paramount importance, such as environmental monitoring of water resources, and for bathymetry to study the characteristics of the basing of a lake/sea or for surveillance in patrol missions, among others. These vehicles can be built with smaller dimensions when compared to regular ships since they do not need an on-board crew for operation. However, they do require at least a telemetry control as well as certain intelligence for making decisions and responding to changing scenarios. Water resources are very important in Paraguay since they provide fresh water for its inhabitants and they are crucial for the main economic activities such as agriculture and cattle raising. Furthermore, they are natural borders with the surrounding countries, and consequently the main transportation route for importing/exporting products. In fact, Paraguay is the third country in the world with the largest fleet of barges after USA and China. Thus, maintaining and monitoring the environmental conditions of these resources is key in the development of the country. This work is focused on the maintenance and monitoring of the greatest lake of the country called Ypacarai Lake. In recent years, the quality of its water has been seriously degraded due to the pollution caused by the low control of the dumping of waste thrown into the Lake. Since it is also a national icon, the government of Paraguay has put a lot of effort in recovering water quality of the Lake. As a result, it is monitored periodically but using manual procedures. Therefore, the primary objective of this work is to develop these monitoring tasks autonomously by means of an ASV with a suitable path planning strategy. Path planning is an active research area in robotics. A particular case is the Coverage Path Planning (CPP) problem, where an algorithm should find a path that achieves the best coverage of the target region to be monitored. This work mainly studies the global CPP, which returns a suitable path considering the initial conditions of the environment where the vehicle moves. The first contribution of this thesis is the modeling of the CPP using Hamiltonian Circuits (HCs) and Eulerian Circuits (ECs). Therefore, a graph adapted to the Ypacarai Lake is created by using a network of wireless beacons located at the shore of the lake, so that they can be used as data exchange points between a control center and the ASV, and also as waypoints. Regarding the proposed modeling, HCs and ECs are paths that begin and end at the same point. Therefore, the ASV travels across a given graph that is defined by a set of wireless beacons. The main difference between HC and EC is that a HC is a tour that visits each vertex only once while EC visits each edge only once. Finding optimal HCs or ECs that minimize the total distance traveled by the ASV are very complex problems known as NP-complete. To solve such problems, a meta-heuristic algorithm can be a suitable approach since they provide quasi-optimal solutions in a reasonable time. In this work, a GA (Genetic Algorithm) approach is proposed and tested. First, an evaluation of the performance of the algorithm with different values of its hyper-parameters has been carried out. Second, the proposed approach has been compared to other approaches such as randomized and greedy algorithms. Third, a thorough comparison between the performance of HC and EC based approaches is presented. The simulation results show that EC-based approach outperforms the HC-based approach almost 2% which in terms of the Lake size is about 1.4 km2 or 140 ha (hectares). Therefore, it has been demonstrated that the modeling of the problem as an Eulerian graph provides better results. Furthermore, it has been investigated the relationship between the number of beacons to be visited and the distance traveled by the ASV in the EC-based approach. Findings indicate that there is a quasi-lineal relationship between the number of beacons and the distance traveled. The second contribution of this work is the development of an on-line learning strategy using the same model but considering dynamic contamination events in the Lake. Dynamic events mean the appearance and evolution of an algae bloom, which is a strong indicator of the degradation of the lake. The strategy is divided into two-phases, the initial exploration phase to discover the presence of the algae bloom and next the intensification phase to focus on the region where the contamination event is detected. This intensification effect is achieved by modifying the beacon-based graph, reducing the number of vertices and selecting those that are closer to the region of interest. The simulation results reveal that the proposed strategy detects two events and monitors them, keeping a high level of coverage while minimizing the distance traveled by the ASV. The proposed scheme is a reactive path planning that adapts to the environmental conditions. This scheme makes decisions in an autonomous way and it switches from the exploratory phase to the intensification phase depending on the external conditions, leading to a variable granularity in the monitoring task. Therefore, there is a balance between coverage and the energy consumed by the ASV. The main benefits obtained from the second contribution includes a better monitoring in the quality of water and control of waste dumping, and the possibility to predict the appearance of algae-bloom from the collected environmental data

Early aspects: aspect-oriented requirements engineering and architecture design

This paper reports on the third Early Aspects: Aspect-Oriented Requirements Engineering and Architecture Design Workshop, which has been held in Lancaster, UK, on March 21, 2004. The workshop included a presentation session and working sessions in which the particular topics on early aspects were discussed. The primary goal of the workshop was to focus on challenges to defining methodical software development processes for aspects from early on in the software life cycle and explore the potential of proposed methods and techniques to scale up to industrial applications