Adaptive Railway Traffic Control using Approximate Dynamic Programming

Railway networks around the world have become challenging to operate in recent decades, with a mixture of track layouts running several different classes of trains with varying operational speeds. This complexity has come about as a result of the sustained increase in passenger numbers where in many countries railways are now more popular than ever before as means of commuting to cities. To address operational challenges, governments and railway undertakings are encouraging development of intelligent and digital transport systems to regulate and optimise train operations in real-time to increase capacity and customer satisfaction by improved usage of existing railway infrastructure. Accordingly, this thesis presents an adaptive railway traffic control system for realtime operations based on a data-based approximate dynamic programming (ADP) approach with integrated reinforcement learning (RL). By assessing requirements and opportunities, the controller aims to reduce delays resulting from trains that entered a control area behind schedule by re-scheduling control plans in real-time at critical locations in a timely manner. The present data-based approach depends on an approximation to the value function of dynamic programming after optimisation from a specified state, which is estimated dynamically from operational experience using RL techniques. By using this approximation, ADP avoids extensive explicit evaluation of performance and so reduces the computational burden substantially. In this thesis, formulations of the approximation function and variants of the RL learning techniques used to estimate it are explored. Evaluation of this controller shows considerable improvements in delays by comparison with current industry practices

IEEE Access Special Section Editorial: Big Data Technology and Applications in Intelligent Transportation

During the last few years, information technology and transportation industries, along with automotive manufacturers and academia, are focusing on leveraging intelligent transportation systems (ITS) to improve services related to driver experience, connected cars, Internet data plans for vehicles, traffic infrastructure, urban transportation systems, traffic collaborative management, road traffic accidents analysis, road traffic flow prediction, public transportation service plan, personal travel route plans, and the development of an effective ecosystem for vehicles, drivers, traffic controllers, city planners, and transportation applications. Moreover, the emerging technologies of the Internet of Things (IoT) and cloud computing have provided unprecedented opportunities for the development and realization of innovative intelligent transportation systems where sensors and mobile devices can gather information and cloud computing, allowing knowledge discovery, information sharing, and supported decision making. However, the development of such data-driven ITS requires the integration, processing, and analysis of plentiful information obtained from millions of vehicles, traffic infrastructures, smartphones, and other collaborative systems like weather stations and road safety and early warning systems. The huge amount of data generated by ITS devices is only of value if utilized in data analytics for decision-making such as accident prevention and detection, controlling road risks, reducing traffic carbon emissions, and other applications which bring big data analytics into the picture

Intelligent Transportation Related Complex Systems and Sensors

Building around innovative services related to different modes of transport and traffic management, intelligent transport systems (ITS) are being widely adopted worldwide to improve the efficiency and safety of the transportation system. They enable users to be better informed and make safer, more coordinated, and smarter decisions on the use of transport networks. Current ITSs are complex systems, made up of several components/sub-systems characterized by time-dependent interactions among themselves. Some examples of these transportation-related complex systems include: road traffic sensors, autonomous/automated cars, smart cities, smart sensors, virtual sensors, traffic control systems, smart roads, logistics systems, smart mobility systems, and many others that are emerging from niche areas. The efficient operation of these complex systems requires: i) efficient solutions to the issues of sensors/actuators used to capture and control the physical parameters of these systems, as well as the quality of data collected from these systems; ii) tackling complexities using simulations and analytical modelling techniques; and iii) applying optimization techniques to improve the performance of these systems. It includes twenty-four papers, which cover scientific concepts, frameworks, architectures and various other ideas on analytics, trends and applications of transportation-related data

QoE-Based Low-Delay Live Streaming Using Throughput Predictions

Recently, HTTP-based adaptive streaming has become the de facto standard for video streaming over the Internet. It allows clients to dynamically adapt media characteristics to network conditions in order to ensure a high quality of experience, that is, minimize playback interruptions, while maximizing video quality at a reasonable level of quality changes. In the case of live streaming, this task becomes particularly challenging due to the latency constraints. The challenge further increases if a client uses a wireless network, where the throughput is subject to considerable fluctuations. Consequently, live streams often exhibit latencies of up to 30 seconds. In the present work, we introduce an adaptation algorithm for HTTP-based live streaming called LOLYPOP (Low-Latency Prediction-Based Adaptation) that is designed to operate with a transport latency of few seconds. To reach this goal, LOLYPOP leverages TCP throughput predictions on multiple time scales, from 1 to 10 seconds, along with an estimate of the prediction error distribution. In addition to satisfying the latency constraint, the algorithm heuristically maximizes the quality of experience by maximizing the average video quality as a function of the number of skipped segments and quality transitions. In order to select an efficient prediction method, we studied the performance of several time series prediction methods in IEEE 802.11 wireless access networks. We evaluated LOLYPOP under a large set of experimental conditions limiting the transport latency to 3 seconds, against a state-of-the-art adaptation algorithm from the literature, called FESTIVE. We observed that the average video quality is by up to a factor of 3 higher than with FESTIVE. We also observed that LOLYPOP is able to reach a broader region in the quality of experience space, and thus it is better adjustable to the user profile or service provider requirements.Comment: Technical Report TKN-16-001, Telecommunication Networks Group, Technische Universitaet Berlin. This TR updated TR TKN-15-00

Surrogate model for real time signal control: theories and applications

Traffic signal controls play a vital role in urban road traffic networks. Compared with fixed-time signal control, which is solely based on historical data, real time signal control is flexible and responsive to varying traffic conditions, and hence promises better performance and robustness in managing traffic congestion. Real time signal control can be divided into model-based and model-free approaches. The former requires a traffic model (analytical or simulation-based) in the generation, optimisation and evaluation of signal control plans, which means that its efficacy in real-world deployment depends on the validity and accuracy of the underlying traffic model. Model-free real time signal control, on the other hand, is constructed based on expert experience and empirical observations. Most of the existing model-free real time signal controls, however, focus on learning-based and rule-based approaches, and either lack interpretability or are non-optimised. This thesis proposes a surrogate-based real time signal control and optimisation framework, that can determine signal decisions in a centralised manner without the use of any traffic model. Surrogate models offer analytical and efficient approximations of complex models or black-box processes by fitting their input-output structures with appropriate mathematical tools. Current research on surrogate-based optimisation is limited to strategic and off-line optimisation, which only approximates the relationship between decisions and outputs under highly specific conditions based on certain traffic simulation models and is still to be attempted for real time optimisation. This thesis proposes a framework for surrogate-based real time signal control, by constructing a response surface that encompasses, (1) traffic states, (2) control parameters, and (3) network performance indicators at the same time. A series of comprehensive evaluations are conducted to assess the effectiveness, robustness and computational efficiency of the surrogate-based real time signal control. In the numerical test, the Kriging model is selected to approximate the traffic dynamics of the test network. The results show that this Kriging-based real time signal control can increase the total throughput by 5.3% and reduce the average delay by 8.1% compared with the fixed-time baseline signal plan. In addition, the optimisation time can be reduced by more than 99% if the simulation model is replaced by a Kriging model. The proposed signal controller is further investigated via multi-scenario analyses involving different levels of information availability, network saturation and traffic uncertainty, which shows the robustness and reliability of the controller. Moreover, the influence of the baseline signal on the Kriging-based signal control can be eliminated by a series of off-line updates. By virtue of the model-free nature and the adaptive learning capability of the surrogate model, the Kriging-based real time signal control can adapt to systematic network changes (such as seasonal variations in traffic demand). The adaptive Kriging-based real time signal control can update the response surface according to the feedback from the actual traffic environment. The test results show that the adaptive Kriging-based real time signal control maintains the signal control performance better in response to systematic network changes than either fixed-time signal control or non-adaptive Kriging-based signal control.Open Acces

DATA-DRIVEN PREDICTION, DESIGN, AND CONTROL OF SYSTEM BEHAVIOR USING STATISTICAL LEARNING

The goal in this dissertation is to develop new data-driven techniques for prediction, design, and control of the behavior of a variety of engineering systems. The data used can be obtained from a variety of sources, including from offline, high-fidelity system’s simulation, physical experiments, and online, sparse measurements from sensors. Three inter-related research directions are followed in this dissertation. Following the first direction, the author presents a multi-step-ahead prediction technique for evaluating a single-response (or single-output of the) system’s behavior through an integration of the data obtained offline from the system’s high-fidelity simulation, and online from single sensor measurements. With regard to the first research direction, the key contribution includes a reasonably fast and accurate prediction strategy that can be used, among others, for online, multi-step ahead forecasting of the system’s operational behavior. Building on the work from the first direction, the author follows a second research direction to present a multi-step ahead prediction technique, this time for a multi-response system’s behavior, that can be used for evaluating various system’s designs and corresponding operations. Data in this case is obtained from the offline, high-fidelity system’s simulations, and online sparse measurements from multiple sensors (or limited number of physical experiments). The main contribution for this second direction is in construction of a new data-driven, multi-response prediction framework that has a robust predictive capability. Along the third research direction, a data-driven technique is used for prediction and co-optimization of a system’s design and control. The data in this case is obtained from sensor measurements or a simulator. The main contribution achieved through the third direction is a new data-driven reinforcement learning-based prediction and co-optimization approach. The methods from this dissertation have numerous applications, including those demonstrated here: (i) assessment of safe aircraft flight conditions (Chapters 2 and 3), (ii) evaluation of design and operation of a robotic appendage (Chapter 3), and (iii) design and control of a traffic system (Chapter 4)

Intelligent Transportation Systems, Hybrid Electric Vehicles, Powertrain Control, Cooperative Adaptive Cruise Control, Model Predictive Control

Information obtainable from Intelligent Transportation Systems (ITS) provides the possibility of improving the safety and efficiency of vehicles at different levels. In particular, such information has the potential to be utilized for prediction of driving conditions and traffic flow, which allows us to improve the performance of the control systems in different vehicular applications, such as Hybrid Electric Vehicles (HEVs) powertrain control and Cooperative Adaptive Cruise Control (CACC). In the first part of this work, we study the design of an MPC controller for a Cooperative Adaptive Cruise Control (CACC) system, which is an automated application that provides the drivers with extra benefits, such as traffic throughput maximization and collision avoidance. CACC systems must be designed in a way that are sufficiently robust against all special maneuvers such as interfering vehicles cutting-into the CACC platoons or hard braking by leading cars. To address this problem, we first propose a Neural- Network (NN)-based cut-in detection and trajectory prediction scheme. Then, the predicted trajectory of each vehicle in the adjacent lanes is used to estimate the probability of that vehicle cutting-into the CACC platoon. To consider the calculated probability in control system decisions, a Stochastic Model Predictive Controller (SMPC) needs to be designed which incorporates this cut-in probability, and enhances the reaction against the detected dangerous cut-in maneuver. However, in this work, we propose an alternative way of solving this problem. We convert the SMPC problem into modeling the CACC as a Stochastic Hybrid System (SHS) while we still use a deterministic MPC controller running in the only state of the SHS model. Finally, we find the conditions under which the designed deterministic controller is stable and feasible for the proposed SHS model of the CACC platoon. In the second part of this work, we propose to improve the performance of one of the most promising realtime powertrain control strategies, called Adaptive Equivalent Consumption Minimization Strategy (AECMS), using predicted driving conditions. In this part, two different real-time powertrain control strategies are proposed for HEVs. The first proposed method, including three different variations, introduces an adjustment factor for the cost of using electrical energy (equivalent factor) in AECMS. The factor is proportional to the predicted energy requirements of the vehicle, regenerative braking energy, and the cost of battery charging and discharging in a finite time window. Simulation results using detailed vehicle powertrain models illustrate that the proposed control strategies improve the performance of AECMS in terms of fuel economy by 4\%. Finally, we integrate the recent development in reinforcement learning to design a novel multi-level power distribution control. The proposed controller reacts in two levels, namely high-level and low-level control. The high-level control decision estimates the most probable driving profile matched to the current (and near future) state of the vehicle. Then, the corresponding low-level controller of the selected profile is utilized to distribute the requested power between Electric Motor (EM) and Internal Combustion Engine (ICE). This is important because there is no other prior work addressing this problem using a controller which can adjust its decision to the driving pattern. We proposed to use two reinforcement learning agents in two levels of abstraction. The first agent, selects the most optimal low-level controller (second agent) based on the overall pattern of the drive cycle in the near past and future, i.e., urban, highway and harsh. Then, the selected agent by the high-level controller (first agent) decides how to distribute the demanded power between the EM and ICE. We found that by carefully designing a training scheme, it is possible to effectively improve the performance of this data-driven controller. Simulation results show up to 6\% improvement in fuel economy compared to the AECMS

Advances in Automated Driving Systems

Electrification, automation of vehicle control, digitalization and new mobility are the mega-trends in automotive engineering, and they are strongly connected. While many demonstrations for highly automated vehicles have been made worldwide, many challenges remain in bringing automated vehicles to the market for private and commercial use. The main challenges are as follows: reliable machine perception; accepted standards for vehicle-type approval and homologation; verification and validation of the functional safety, especially at SAE level 3+ systems; legal and ethical implications; acceptance of vehicle automation by occupants and society; interaction between automated and human-controlled vehicles in mixed traffic; human–machine interaction and usability; manipulation, misuse and cyber-security; the system costs of hard- and software and development efforts. This Special Issue was prepared in the years 2021 and 2022 and includes 15 papers with original research related to recent advances in the aforementioned challenges. The topics of this Special Issue cover: Machine perception for SAE L3+ driving automation; Trajectory planning and decision-making in complex traffic situations; X-by-Wire system components; Verification and validation of SAE L3+ systems; Misuse, manipulation and cybersecurity; Human–machine interactions, driver monitoring and driver-intention recognition; Road infrastructure measures for the introduction of SAE L3+ systems; Solutions for interactions between human- and machine-controlled vehicles in mixed traffic