Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such controllers need to be robust with the respect to a series of exogenous sources of uncertainty. In this paper, we develop an open-source callback-based framework for promoting the flexible evaluation of different deep RL configurations under a traffic simulation environment. With this framework, we investigate how deep RL-based adaptive traffic controllers perform under different scenarios, namely under demand surges caused by special events, capacity reductions from incidents and sensor failures. We extract several key insights for the development of robust deep RL algorithms for traffic control and propose concrete designs to mitigate the impact of the considered exogenous uncertainties.Comment: 8 page

Reliable design of interdependent service facility systems under correlated disruption risks

Facility location decisions lie at the center of planning many infrastructure systems. In many practice, public agencies (e.g., governments) and private companies (e.g., retailers) need to locate facilities to serve spatially distributed demands. For example, governments locate public facilities, e.g., hospitals, schools, fire stations, to provide public services; retail companies determine the locations of their warehouses and stores to provide business. The design of such facility systems involves considerations of investment of facility construction and transportation cost of serving demands, so as to maximize the system operational efficiency and profit. Recently, devastating infrastructure damages observed in real world show that infrastructure facilities may be subject to disruptions that compromise individual facility functionality as well as overall system performance. This emphasizes the necessity of taking facility disruptions into consideration during planning to balance between system efficiency and reliability. Furthermore, facility systems often exhibit complex interdependence when: (1) facilities are spatially correlated due to physical connections/interrelations, and (2) facilities provide combinatorial service under cooperation, competition and/or restrictions. These further complicate the facility location design. Therefore, facility location models need to be extended to tackle all these challenges and design a reliable interdependent facility system. This dissertation aims at investigating several important and challenging topics in the reliable facility location context, including facility correlations, facility combinations, and facility districting. The main work of this PhD research consist of: (1) establishing a new systematic methodological framework based on supporting stations and quasi-probabilities to describe and decompose facility correlations into succinct mathematical representations, which allows compact mathematical formulations to be developed for planning facility locations under correlated facility disruptions; (2) expanding the modeling framework to allow facilities to provide combinatorial service; e.g., in the context of sensor deployment problems, where sensors work in combinations to provide positioning/surveillance service via trilateration procedure; and (3) incorporating the concepts of spatial districting into the reliable facility location context, with the criteria of spatial contiguity, compactness, and demand balance being ensured. First, in many real-world facility systems, facility disruptions exhibit spatial correlations, which have strong impacts on the system performance, but are difficult to be described with succinct mathematical models. We first investigate facility systems with correlations caused by facilities’ share of network access points (e.g., bridges, railway crossings), which are required to be passed through by customers to visit facilities. We incorporate these network access points and their probabilistic failures into a joint optimization framework. A layer of supporting stations are added to represent the network access points, and are connected to facilities to indicate their real-world relationships. We then develop a compact mixed-integer mathematical model to optimize the facility location and customer assignment decisions. Lagrangian relaxation based algorithms are designed to effectively solve the model. Multiple case studies are constructed to test the model and algorithm, and to demonstrate their performance and applicability. Next, when there exists no real access points, facilities could also be correlated if they are exposed to shared hazards. We develop a virtual station structure framework to decompose these types of facility correlations. First, we define three probabilistic representations of correlated facility disruptions (i.e., with scenario, marginal, and conditional probabilities), derive pairwise transformations between them, and theoretically prove their equivalence. We then provide detailed formulas to transform these probabilistic representations into an equivalent virtual station structure, which enables the decomposition of any correlated facility disruptions into a compact network structure with only independent failures, and helps avoid enumerating an exponential number of disruption scenarios. Based on the augmented system, we propose a compact mixed-integer optimization program, and design several customized solution approaches based on Lagrangian relaxation to efficiently solve the model. We demonstrate our methodology on a series of numerical examples involving different correlation patterns and varying network and parameter settings. We then apply the reliable location modeling framework to sensor deployment problems, where multiple sensors work in combinations to provide combinatorial coverage service to customers via trilateration procedure. Since various sensor combinations may share common sensors, one combination is typically interrelated with some other combinations, which leads to internal correlations among the functionality of sensors and sensor combinations. We address the problem of where to deploy sensors, which sensor combinations are selected to use, and in what sequence and probability to use these combinations in case of disruptions. A compact mixed-integer mathematical model is developed to formulate the problem, by combining and extending the ideas of assigning back-up sensors and correlation decomposition via supporting stations. A customized solution algorithm based on Lagrangian relaxation and branch-and-bound is developed, together with several embedded approximation subroutines for solving subproblems. A series of numerical examples are investigated to illustrate the performance of the proposed methodology and to draw managerial insights. Finally, we develop an innovative reliable network districting framework to incorporate districting concepts into the reliable facility location context. Districting criteria including spatial contiguity, compactness, and demand balance are enforced for location design and extended in considerations of facility disruptions. The problem is modeled into a reliable network districting problem, in the form of a location-assignment based model. We develop customized solution approaches, including heuristics (i.e., constructive heuristic and neighborhood search) and set-cover based algorithms (e.g., district generation, lower bound estimation) to provide near-optimum solution with optimality gap. A series of hypothetical cases and an empirical full-scale application are presented to demonstrate the performance of our methodology for different network and parameter settings

Joint inventory-location problem under the risk of probabilistic facility disruptions

This paper studies a reliable joint inventory-location problem that optimizes facility locations, customer allocations, and inventory management decisions when facilities are subject to disruption risks (e.g., due to natural or man-made hazards). When a facility fails, its customers may be reassigned to other operational facilities in order to avoid the high penalty costs associated with losing service. We propose an integer programming model that minimizes the sum of facility construction costs, expected inventory holding costs and expected customer costs under normal and failure scenarios. We develop a Lagrangian relaxation solution framework for this problem, including a polynomial-time exact algorithm for the relaxed nonlinear subproblems. Numerical experiment results show that this proposed model is capable of providing a near-optimum solution within a short computation time. Managerial insights on the optimal facility deployment, inventory control strategies, and the corresponding cost constitutions are drawn. Highlights ► A reliable inventory-location model is proposed for optimal facility location and inventory management under facility disruptions. ► The model allows customer re-assignments and minimizes the expected total system cost across all facility disruption scenarios. ► A customized Lagrangian relaxation approach is developed for the mixed-integer nonlinear formulation. ► Numerical experiments are conducted to demonstrate the model performance and draw managerial insights

A Survey on Wireless Sensor Network Security

Wireless sensor networks (WSNs) have recently attracted a lot of interest in the research community due their wide range of applications. Due to distributed nature of these networks and their deployment in remote areas, these networks are vulnerable to numerous security threats that can adversely affect their proper functioning. This problem is more critical if the network is deployed for some mission-critical applications such as in a tactical battlefield. Random failure of nodes is also very likely in real-life deployment scenarios. Due to resource constraints in the sensor nodes, traditional security mechanisms with large overhead of computation and communication are infeasible in WSNs. Security in sensor networks is, therefore, a particularly challenging task. This paper discusses the current state of the art in security mechanisms for WSNs. Various types of attacks are discussed and their countermeasures presented. A brief discussion on the future direction of research in WSN security is also included.Comment: 24 pages, 4 figures, 2 table

Doctor of Philosophy

dissertationA safe and secure transportation system is critical to providing protection to those who employ it. Safety is being increasingly demanded within the transportation system and transportation facilities and services will need to adapt to change to provide it. This dissertation provides innovate methodologies to identify current shortcomings and provide theoretic frameworks for enhancing the safety and security of the transportation network. This dissertation is designed to provide multilevel enhanced safety and security within the transportation network by providing methodologies to identify, monitor, and control major hazards associated within the transportation network. The risks specifically addressed are: (1) enhancing nuclear materials sensor networks to better deter and interdict smugglers, (2) use game theory as an interdiction model to design better sensor networks and forensically track smugglers, (3) incorporate safety into regional transportation planning to provide decision-makers a basis for choosing safety design alternatives, and (4) use a simplified car-following model that can incorporate errors to predict situational-dependent safety effects of distracted driving in an ITS infrastructure to deploy live-saving countermeasures

OPTIMIZATION MODELS AND METHODOLOGIES TO SUPPORT EMERGENCY PREPAREDNESS AND POST-DISASTER RESPONSE

This dissertation addresses three important optimization problems arising during the phases of pre-disaster emergency preparedness and post-disaster response in time-dependent, stochastic and dynamic environments. The first problem studied is the building evacuation problem with shared information (BEPSI), which seeks a set of evacuation routes and the assignment of evacuees to these routes with the minimum total evacuation time. The BEPSI incorporates the constraints of shared information in providing on-line instructions to evacuees and ensures that evacuees departing from an intermediate or source location at a mutual point in time receive common instructions. A mixed-integer linear program is formulated for the BEPSI and an exact technique based on Benders decomposition is proposed for its solution. Numerical experiments conducted on a mid-sized real-world example demonstrate the effectiveness of the proposed algorithm. The second problem addressed is the network resilience problem (NRP), involving an indicator of network resilience proposed to quantify the ability of a network to recover from randomly arising disruptions resulting from a disaster event. A stochastic, mixed integer program is proposed for quantifying network resilience and identifying the optimal post-event course of action to take. A solution technique based on concepts of Benders decomposition, column generation and Monte Carlo simulation is proposed. Experiments were conducted to illustrate the resilience concept and procedure for its measurement, and to assess the role of network topology in its magnitude. The last problem addressed is the urban search and rescue team deployment problem (USAR-TDP). The USAR-TDP seeks an optimal deployment of USAR teams to disaster sites, including the order of site visits, with the ultimate goal of maximizing the expected number of saved lives over the search and rescue period. A multistage stochastic program is proposed to capture problem uncertainty and dynamics. The solution technique involves the solution of a sequence of interrelated two-stage stochastic programs with recourse. A column generation-based technique is proposed for the solution of each problem instance arising as the start of each decision epoch over a time horizon. Numerical experiments conducted on an example of the 2010 Haiti earthquake are presented to illustrate the effectiveness of the proposed approach

Journey of Artificial Intelligence Frontier: A Comprehensive Overview

The field of Artificial Intelligence AI is a transformational force with limitless promise in the age of fast technological growth This paper sets out on a thorough tour through the frontiers of AI providing a detailed understanding of its complex environment Starting with a historical context followed by the development of AI seeing its beginnings and growth On this journey fundamental ideas are explored looking at things like Machine Learning Neural Networks and Natural Language Processing Taking center stage are ethical issues and societal repercussions emphasising the significance of responsible AI application This voyage comes to a close by looking ahead to AI s potential for human-AI collaboration ground-breaking discoveries and the difficult obstacles that lie ahead This provides with a well-informed view on AI s past present and the unexplored regions it promises to explore by thoroughly navigating this terrai