Store-and-forward based methods for the signal control problem in large-scale congested urban road networks

The problem of designing network-wide traffic signal control strategies for large-scale congested urban road networks is considered. One known and two novel methodologies, all based on the store-and-forward modeling paradigm, are presented and compared. The known methodology is a linear multivariable feedback regulator derived through the formulation of a linear-quadratic optimal control problem. An alternative, novel methodology consists of an open-loop constrained quadratic optimal control problem, whose numerical solution is achieved via quadratic programming. Yet a different formulation leads to an open-loop constrained nonlinear optimal control problem, whose numerical solution is achieved by use of a feasible-direction algorithm. A preliminary simulation-based investigation of the signal control problem for a large-scale urban road network using these methodologies demonstrates the comparative efficiency and real-time feasibility of the developed signal control methods

Robust Optimal Sliding-Mode Tracking Control for a Class of Uncertain Nonlinear MIMO Systems

This paper addresses the problem of tracking a reference trajectory asymptotically given by a linear time-varying exosystem for a class of uncertain nonlinear MIMO systems based on the robust optimal sliding-mode control. The nonlinear MIMO system is transformed into a linear one by the input-output linearization technique, and at the same time the input-output decoupling is realized. Thus, the tracking error equation is established in a linear form, and the original nonlinear tracking problem is transformed into an optimal linear quadratic regulator (LQR) tracking problem. A LQR tracking controller (LQRTC) is designed for the corresponding nominal system, and the integral sliding-mode strategy is used to robustify the LQRTC. As a result, the original system exhibits global robustness to the uncertainties, and the tracking dynamics is the same as that of LQRTC for the nominal system. So a robust optimal sliding-mode tracking controller (ROSMTC) is realized. The proposed controller is applied to a two-link robot system, and simulation results show its effectiveness and superiority

Control and optimization methods for traffic signal control in large-scale congested urban road networks

The problem of designing real-time traffic signal control strategies for large-scale congested urban road networks via suitable application of control and optimization methods is considered. Three alternative methodologies are proposed, all based on the store-and-forward modeling (SFM) paradigm. The first methodology results in a linear multivariable feedback regulator derived through the formulation of the problem as a linear-quadratic (LQ) optimal control problem. The second methodology leads to an open-loop constrained quadratic optimal control problem whose numerical solution is achieved via quadratic-programming (QP). Finally, the third methodology leads to an open-loop constrained nonlinear optimal control problem whose numerical solution is effectuated by use of a feasible-direction algorithm. A simulation-based investigation of the signal control problem for a large-scale urban network using these methodologies is presented. Results demonstrate the efficiency and real-time feasibility of the developed generic control methods

Limitations of Nonlinear Controls For a Curved and Offset Beam and Ball

The problem of stabilizing the position of a ball on a straight beam is a common element of laboratory courses in control. Replacing the straight beam with a circular beam and moving the center of rotation off of the beam produces a harder control problem. This experiment was developed here at UMCP by Sheng, Renner, and Levine [1]. They designed and implemented a stabilizing controller based on a linearized model of the plant and an LQR optimal regulator. Here, the set of initial states that is stabilizable is explored. Also, nonlinear controls based on ideas from feedback linearization and backstepping are developed

Initial Value Problem Enhanced Sampling for Closed-Loop Optimal Control Design with Deep Neural Networks

Closed-loop optimal control design for high-dimensional nonlinear systems has been a long-standing challenge. Traditional methods, such as solving the associated Hamilton-Jacobi-Bellman equation, suffer from the curse of dimensionality. Recent literature proposed a new promising approach based on supervised learning, by leveraging powerful open-loop optimal control solvers to generate training data and neural networks as efficient high-dimensional function approximators to fit the closed-loop optimal control. This approach successfully handles certain high-dimensional optimal control problems but still performs poorly on more challenging problems. One of the crucial reasons for the failure is the so-called distribution mismatch phenomenon brought by the controlled dynamics. In this paper, we investigate this phenomenon and propose the initial value problem enhanced sampling method to mitigate this problem. We theoretically prove that this sampling strategy improves over the vanilla strategy on the classical linear-quadratic regulator by a factor proportional to the total time duration. We further numerically demonstrate that the proposed sampling strategy significantly improves the performance on tested control problems, including the optimal landing problem of a quadrotor and the optimal reaching problem of a 7 DoF manipulator

Model-free two-step design for improving transient learning performance in nonlinear optimal regulator problems

Reinforcement learning (RL) provides a model-free approach to designing an optimal controller for nonlinear dynamical systems. However, the learning process requires a considerable number of trial-and-error experiments using the poorly controlled system, and accumulates wear and tear on the plant. Thus, it is desirable to maintain some degree of control performance during the learning process. In this paper, we propose a model-free two-step design approach to improve the transient learning performance of RL in an optimal regulator design problem for unknown nonlinear systems. Specifically, a linear control law pre-designed in a model-free manner is used in parallel with online RL to ensure a certain level of performance at the early stage of learning. Numerical simulations show that the proposed method improves the transient learning performance and efficiency in hyperparameter tuning of RL

Data-driven nonlinear MPC using dynamic response surface methodology

For many complex processes, it is desirable to use a nonlinear model in the MPC design, and the recently proposed Dynamic Response Surface Methodology (DRSM) is capable of accurately modeling nonlinear continuous processes over semi-infinite time horizons. We exploit the DRSM to identify nonlinear data-driven dynamic models that are used in an NMPC. We demonstrate the ability and effectiveness of the DRSM data-driven model to be used as the prediction model for a nonlinear MPC regulator. This DRSM model is efficiently used to solve a non-equally-spaced finite-horizon optimal control problem so that the number of decision variables is reduced. The proposed DRSM-based NMPC is tested on a representative nonlinear process, an isothermal CSTR in which a second-order irreversible reaction is taking place. It is shown that the obtained quadratic data-driven model accurately represents the open-loop process dynamics and that DRSM-based NMPC is an effective data-driven implementation of nonlinear MPC