Optimal Event-Driven Multi-Agent Persistent Monitoring of a Finite Set of Targets

We consider the problem of controlling the movement of multiple cooperating agents so as to minimize an uncertainty metric associated with a finite number of targets. In a one-dimensional mission space, we adopt an optimal control framework and show that the solution is reduced to a simpler parametric optimization problem: determining a sequence of locations where each agent may dwell for a finite amount of time and then switch direction. This amounts to a hybrid system which we analyze using Infinitesimal Perturbation Analysis (IPA) to obtain a complete on-line solution through an event-driven gradient-based algorithm which is also robust with respect to the uncertainty model used. The resulting controller depends on observing the events required to excite the gradient-based algorithm, which cannot be guaranteed. We solve this problem by proposing a new metric for the objective function which creates a potential field guaranteeing that gradient values are non-zero. This approach is compared to an alternative graph-based task scheduling algorithm for determining an optimal sequence of target visits. Simulation examples are included to demonstrate the proposed methods.Comment: 12 pages full version, IEEE Conference on Decision and Control, 201

Topics in perturbation analysis for stochastic hybrid systems

Control and optimization of Stochastic Hybrid Systems (SHS) constitute increasingly active fields of research. However, the size and complexity of SHS frequently render the use of exhaustive verification techniques prohibitive. In this context, Perturbation Analysis techniques, and in particular Infinitesimal Perturbation Analysis (IPA), have proven to be particularly useful for this class of systems. This work focuses on applying IPA to two different problems: Traffic Light Control (TLC) and control of cancer progression, both of which are viewed as dynamic optimization problems in an SHS environment. The first part of this thesis addresses the TLC problem for a single intersection modeled as a SHS. A quasi-dynamic control policy is proposed based on partial state information defined by detecting whether vehicle backlogs are above or below certain controllable threshold values. At first, the threshold parameters are controlled while assuming fixed cycle lengths and online gradient estimates of a cost metric with respect to these controllable parameters are derived using IPA techniques. These estimators are subsequently used to iteratively adjust the threshold values so as to improve overall system performance. This quasi-dynamic analysis of the TLC\ problem is subsequently extended to parameterize the control policy by green and red cycle lengths as well as queue content thresholds. IPA estimators necessary to simultaneously control the light cycles and thresholds are rederived and thereafter incorporated into a standard gradient based scheme in order to further ameliorate system performance. In the second part of this thesis, the problem of controlling cancer progression is formulated within a Stochastic Hybrid Automaton (SHA) framework. Leveraging the fact that cell-biologic changes necessary for cancer development may be schematized as a series of discrete steps, an integrative closed-loop framework is proposed for describing the progressive development of cancer and determining optimal personalized therapies. First, the problem of cancer heterogeneity is addressed through a novel Mixed Integer Linear Programming (MILP) formulation that integrates somatic mutation and gene expression data to infer the temporal sequence of events from cross-sectional data. This formulation is tested using both simulated data and real breast cancer data with matched somatic mutation and gene expression measurements from The Cancer Genome Atlas (TCGA). Second, the use of basic IPA techniques for optimal personalized cancer therapy design is introduced and a methodology applicable to stochastic models of cancer progression is developed. A case study of optimal therapy design for advanced prostate cancer is performed. Given the importance of accurate modeling in conjunction with optimal therapy design, an ensuing analysis is performed in which sensitivity estimates with respect to several model parameters are evaluated and critical parameters are identified. Finally, the tradeoff between system optimality and robustness (or, equivalently, fragility) is explored so as to generate valuable insights on modeling and control of cancer progression

Stochastic Approximation Methods for Systems Over an Infinite Horizon

The paper develops efficient and general stochastic approximation (SA) methods for improving the operation of parametrized systems of either the continuous- or discrete-event dynamical systems types and which are of interest over a long time period. For example, one might wish to optimize or improve the stationary (or average cost per unit time) performance by adjusting the systems parameters. The number of applications and the associated literature are increasing at a rapid rate. This is partly due to the increasing activity in computing pathwise derivatives and adapting them to the average-cost problem. Although the original motivation and the examples come from an interest in the infinite-horizon problem, the techniques and results are of general applicability in SA. We present an updating and review of powerful ordinary differential equation-type methods, in a fairly general context, and based on weak convergence ideas. The results and proof techniques are applicable to a wide variety of applications. Exploiting the full potential of these ideas can greatly simplify and extend much current work. Their breadth as well as the relative ease of using the basic ideas are illustrated in detail via typical examples drawn from discrete-event dynamical systems, piecewise deterministic dynamical systems, and a stochastic differential equations model. In these particular illustrations, we use either infinitesimal perturbation analysis-type estimators, mean square derivative-type estimators, or finite-difference type estimators. Markov and non-Markov models are discussed. The algorithms for distributed/asynchronous updating as well as the fully synchronous schemes are developed

Optimal Variance Swaps Portfolios and Estimating Greeks for Variance-Gamma

In this dissertation, we investigate two problems: constructing optimal variance swaps portfolios and estimating Greeks for options with underlying assets following a Variance Gamma process. By modeling the dependent non-Gaussian residual in a linear regression model through a L'evy Mixture (LM) model and a Variance Gamma Correlated (VGC) model, and running some optimizations, we construct an optimal variance swap portfolio. By implementing gradient estimation techniques, we estimate the Greeks for a series of basket options called Mountain Range options. Constructing an optimal variance swap portfolio consists of two steps: evaluations and optimization. Each variance swap has two legs: a fixed leg (also called the variance strike) and a floating leg (also called the realized variance). The value of a variance swap is the discounted difference between the realized variance and the variance strike. For the latter, one can use an option surface calibration to evaluate. For the former, the procedure is complicated due to the non-negligible residuals from a linear regression model. Through LM and VGC, we can estimate the realized variance on different sample paths and obtain the payoff of a variance swap numerically. Based on these numerical results, we can apply the optimization method to construct an optimal portfolio. In the second part of this dissertation, we consider gradient estimation for Mountain Range options including Everest options, Atlas options, Altiplano/Annapurna options and Himalayan options. Assuming the underlying assets follow a Variance-Gamma (VG) process, we derive estimators for sensitivities such as Greeks through Monte Carlo simulation. We implement and compare using numerical experiments several gradient estimation approaches: finite difference methods (forward difference), infinitesimal perturbation analysis (IPA), and likelihood ratio (LR) method using either the density function or the characteristic function

High volume conveyor sortation system analysis

The design and operation of a high volume conveyor sortation system are important due to its high cost, large footprint and critical role in the system. In this thesis, we study the characteristics of the conveyor sortation system from performance evaluation and design perspectives employing continuous modeling approaches. We present two continuous conveyor models (Delay and Stock Model and Batch on Conveyor Model) with different representation accuracy in a unified mathematical framework. Based on the Batch on Conveyor Model, we develop a fast fluid simulation methodology. We address the feasibility of implementing fluid simulation from modeling capabilities, algorithm design and simulation performance in terms of accuracy and simulation time. From a design perspective, we focus on rates determination and accumulation design in the accumulation and merge subsystem. The optimization problem is to find a minimum cost design that satisfies some predefined performance requirements under stochastic conditions. We first transform this stochastic programming problem into a deterministic nonlinear programming problem through sample path based optimization method. A gradient based method is adopted to solve the deterministic problem. Since there is no closed form for performance metric even for a deterministic input stream, we adopt continuous modeling to develop deterministic performance evaluation models and conduct sensitivity analysis on these models. We explore the prospects of using the two continuous conveyor models we presented.Ph.D.Committee Chair: Chen Zhou; Committee Member: Gunter Sharp; Committee Member: Leon F. McGinnis; Committee Member: Spiridon Reveliotis; Committee Member: Yorai Ward

Seepage criteria based optimal design of water retaining structures with reliability quantification utilizing surrogate model linked simulation-optimization approach

The safety of hydraulic water retaining structures (HWRS) is an important issue as many instances of HWRS failure have been reported. Failure of HWRS may lead to catastrophic events, especially those associated with seepage failures. Therefore, seepage safety factors recommended for HWRS design are generally very conservative. These safety factors have been developed based on approximation calculations, unreliable assumptions, and ideal experimental conditions, which are rarely replicated in real field situations. However, with the development of the numerical methods, and high speed processors, more accurate seepage analysis has become possible, even for complex flow domains, different scenarios of boundary conditions, and varied hydraulic conductivity. On the other hand, because construction of HWRS requires a significant amount of construction material and engineering effort, the construction cost efficiency of HWRS is an issue that must be considered in design of HWRS. This study aims to determine the minimum cost design of HWRS constructed on permeable soils, incorporating numerical solutions of a seepage system related to HWRS, utilizing linked a simulation–optimization (S-O) model. Due to the complexity and inefficacy of directly linking a simulation model to the optimization model, the numerical simulation model was replaced by trained surrogate models. These surrogate models can be trained based on numerically simulated data sets. Therefore, trained surrogate models expeditiously and accurately provide predicted responses relating to seepage characteristics pertaining to HWRS. The optimization model based on the linked S-O technique incorporated different safety factors and hydraulic structure design requirements as constraints. The majority of these constraints and objective function(s) were affected by the responses of predicted seepage characteristics based on the developed surrogate models. To improve the safety of HWRS design, the effect of non-homogenous and anisotropic hydraulic conductivity were incorporated in the S-O model. Obtained solution results demonstrated that considering stratification of the flow domain due to different hydraulic conductivity values or anisotropic ratios can significantly change the optimum design of HWRS. Low hydraulic conductivity and anisotropic ratios resulted in more critical seepage characteristics. Consequently, the minimum construction cost increased due to an increase of dimensions of involved seepage protection design variables. Furthermore, uncertainty in estimating hydraulic conductivity is incorporated in the S-O model. The reliability based optimal design (RBOD) framework based on the multi-realization optimization technique was implemented using the S-O model. The uncertainty in seepage quantities due to uncertainty of hydraulic conductivity was represented using many stochastic ensemble surrogate models. Each ensemble model included many surrogate models trained in utilizing input– output data sets simulated with different scenarios of hydraulic conductivity drawn from diverse random fields based on different log-normal distributions. Obtained results of this approach demonstrated substantial consequences of considering uncertainty in hydraulic conductivity. Also, the deterministic safety factors, especially for those pertaining to the exit gradient, were insufficient to provide prescribed safety in the long term. Although surrogate models are utilized in S-O approaches, each run of the S-O model takes a long time as developed S-O models are applied to complex and large scale problems. Hence, efficiency of the S-O model was a key factor to successfully implement the methodology. Three main techniques were utilized to increase the efficiency of the S-O technique: using parallel computing, utilizing nested function technique, and using a vectorised formulation system. These strategies substantially boosted efficiency of implementing the S-O model. The S-O models were implemented for many hypothetical scenarios for different purposes. In general, results demonstrated that optimum design of the seepage protection system relating to HWRS design must include two end cut-offs with an apron between them. The dimensions of these components were augmented with an increase of upstream water head, and reduction of anisotropic ratios or hydraulic conductivity value. The main role of the downstream cut-off was to decrease the actual exit gradient value. This impact is more pronounced if the inclination angle of the cut-off is toward the downstream side (>90 degrees). The role of the upstream cut-off was to decrease uplift pressure values on the HWRS base. Consequently, this partially contributed to decreasing the exit gradient value. The effect of the upstream cut-off in reducing the uplift pressure was more when the inclination angle was toward the upstream side (<90 degrees). Moreover, the apron (floor) width helped to increase the stability of HWRS. This variable provided the required weight to improve HWRS resistance to external hydraulic forces and to uplift pressure. Incorporating the weight of water (hydrostatic pressure) at the upstream side in counterbalancing momentum and hydraulic forces showed improvement in the safety of the HWRS. Also, all conditions and safety factors pertaining to HWRS design were satisfied. The exit gradient safety factor was the most important critical factor affecting optimum design as obtained optimum solutions satisfied the minimum permissible values of the exit gradient safety factor, i.e., at the minimum permissible value. Also, the eccentric load condition played a crucial role in resulting optimum solutions. Finally, applying the S-O model to obtain reliable and safe design of HWRS at minimum cost was successfully implemented for performance evaluation purposes. This technique may be extended to incorporate more complex scenarios in HWRS design where the impact of dynamic and seismic load could be incorporated. The effect of unsteady state seepage system could be another interesting direction for future studies. Further, incorporating more sources of the uncertainty associated with design parameters could achieve a more accurate estimation of actual safety for the HWRS design