Parameter estimation for stochastic hybrid model applied to urban traffic flow estimation

This study proposes a novel data-based approach for estimating the parameters of a stochastic hybrid model describing the traffic flow in an urban traffic network with signalized intersections. The model represents the evolution of the traffic flow rate, measuring the number of vehicles passing a given location per time unit. This traffic flow rate is described using a mode-dependent first-order autoregressive (AR) stochastic process. The parameters of the AR process take different values depending on the mode of traffic operation – free flowing, congested or faulty – making this a hybrid stochastic process. Mode switching occurs according to a first-order Markov chain. This study proposes an expectation-maximization (EM) technique for estimating the transition matrix of this Markovian mode process and the parameters of the AR models for each mode. The technique is applied to actual traffic flow data from the city of Jakarta, Indonesia. The model thus obtained is validated by using the smoothed inference algorithms and an online particle filter. The authors also develop an EM parameter estimation that, in combination with a time-window shift technique, can be useful and practical for periodically updating the parameters of hybrid model leading to an adaptive traffic flow state estimator

Applied Harmonic Analysis and Sparse Approximation

Efficiently analyzing functions, in particular multivariate functions, is a key problem in applied mathematics. The area of applied harmonic analysis has a significant impact on this problem by providing methodologies both for theoretical questions and for a wide range of applications in technology and science, such as image processing. Approximation theory, in particular the branch of the theory of sparse approximations, is closely intertwined with this area with a lot of recent exciting developments in the intersection of both. Research topics typically also involve related areas such as convex optimization, probability theory, and Banach space geometry. The workshop was the continuation of a first event in 2012 and intended to bring together world leading experts in these areas, to report on recent developments, and to foster new developments and collaborations

Computation Approaches for Continuous Reinforcement Learning Problems

Optimisation theory is at the heart of any control process, where we seek to control the behaviour of a system through a set of actions. Linear control problems have been extensively studied, and optimal control laws have been identified. But the world around us is highly non-linear and unpredictable. For these dynamic systems, which don’t possess the nice mathematical properties of the linear counterpart, the classic control theory breaks and other methods have to be employed. But nature thrives by optimising non-linear and over-complicated systems. Evolutionary Computing (EC) methods exploit nature’s way by imitating the evolution process and avoid to solve the control problem analytically. Reinforcement Learning (RL) from the other side regards the optimal control problem as a sequential one. In every discrete time step an action is applied. The transition of the system to a new state is accompanied by a sole numerical value, the “reward” that designate the quality of the control action. Even though the amount of feedback information is limited into a sole real number, the introduction of the Temporal Difference method made possible to have accurate predictions of the value-functions. This paved the way to optimise complex structures, like the Neural Networks, which are used to approximate the value functions. In this thesis we investigate the solution of continuous Reinforcement Learning control problems by EC methodologies. The accumulated reward of such problems throughout an episode suffices as information to formulate the required measure, fitness, in order to optimise a population of candidate solutions. Especially, we explore the limits of applicability of a specific branch of EC, that of Genetic Programming (GP). The evolving population in the GP case is comprised from individuals, which are immediately translated to mathematical functions, which can serve as a control law. The major contribution of this thesis is the proposed unification of these disparate Artificial Intelligence paradigms. The provided information from the systems are exploited by a step by step basis from the RL part of the proposed scheme and by an episodic basis from GP. This makes possible to augment the function set of the GP scheme with adaptable Neural Networks. In the quest to achieve stable behaviour of the RL part of the system a modification of the Actor-Critic algorithm has been implemented. Finally we successfully apply the GP method in multi-action control problems extending the spectrum of the problems that this method has been proved to solve. Also we investigated the capability of GP in relation to problems from the food industry. These type of problems exhibit also non-linearity and there is no definite model describing its behaviour

Nonlinear Control and Estimation with General Performance Criteria

This dissertation is concerned with nonlinear systems control and estimation with general performance criteria. The purpose of this work is to propose general design methods to provide systematic and effective design frameworks for nonlinear system control and estimation problems. First, novel State Dependent Linear Matrix Inequality control approach is proposed, which is optimally robust for model uncertainties and resilient against control feedback gain perturbations in achieving general performance criteria to secure quadratic optimality with inherent asymptotic stability property together with quadratic dissipative type of disturbance reduction. By solving a state dependent linear matrix inequality at each time step, the sufficient condition for the control solution can be found which satisfies the general performance criteria. The results of this dissertation unify existing results on nonlinear quadratic regulator, Hinfinity and positive real control. Secondly, an H2-Hinfinity State Dependent Riccati Equation controller is proposed in this dissertation. By solving the generalized State Dependent Riccati Equation, the optimal control solution not only achieves the optimal quadratic regulation performance, but also has the capability of external disturbance reduction. Numerically efficient algorithms are developed to facilitate effective computation. Thirdly, a robust multi-criteria optimal fuzzy control of nonlinear systems is proposed. To improve the optimality and robustness, optimal fuzzy control is proposed for nonlinear systems with general performance criteria. The Takagi-Sugeno fuzzy model is used as an effective tool to control nonlinear systems through fuzzy rule models. General performance criteria have been used to design the controller and the relative weighting matrices of these criteria can be achieved by choosing different coefficient matrices. The optimal control can be achieved by solving the LMI at each time step. Lastly, since any type of controller and observer is subject to actuator failures and sensors failures respectively, novel robust and resilient controllers and estimators are also proposed for nonlinear stochastic systems to address these failure problems. The effectiveness of the proposed control and estimation techniques are demonstrated by simulations of nonlinear systems: the inverted pendulum on a cart and the Lorenz chaotic system, respectively

Investigations into implementation of an iterative feedback tuning algorithm into microcontroller

Includes abstract.Includes bibliographical references (leaves 73-75).Implementation of an Iterative Feedback Tuning (IFT) and Myopic Unfalsified Control (MUC) Algorithm into microcontroller is investigated in this dissertation. Motivation in carrying out this research emanates from successful results obtained in application of IFT algorithm to various physical systems since the method was originated in 1995 by Hjalmarsson [4]. The Motorola DSP56F807C microcontroller is selected for use in the investigations due to its matching characteristics with the requirements of IFT algorithm. Speed of program execution, large memory, in-built ADC & DAC and C compiler type are the key parameters qualifying for its usage. The Analog Devices ARM7024 microcontroller was chosen as an alternative to the DSP56F807C where it is not available. Myopic Unfalsified Control (MUC) is noted to be similar to IFT since it also employs ‘myopic’ gradient based steepest descent approach to parameter optimization. It is easier to implement in that its algorithm is not as complex as the IFT one, meaning that successful implementation of IFT algorithm in a microcontroller would obviously permit the implementation of MUC into microcontroller as well

34th Midwest Symposium on Circuits and Systems-Final Program

Organized by the Naval Postgraduate School Monterey California. Cosponsored by the IEEE Circuits and Systems Society. Symposium Organizing Committee: General Chairman-Sherif Michael, Technical Program-Roberto Cristi, Publications-Michael Soderstrand, Special Sessions- Charles W. Therrien, Publicity: Jeffrey Burl, Finance: Ralph Hippenstiel, and Local Arrangements: Barbara Cristi

Intelligent Control Agent for Autonomous UAS

A self reconfiguring autopilot system is presented, which is based on a rational agent framework that integrates decision making with abstractions of sensing and actions for next generation unmanned aerial vehicles. The objective of the new intelligent control system is to provide advanced capabilities of self-tuning control for a new UAS airframe or adaptation for an old UAS in the presence of failures in adverse flight conditions. High-level system performance is achieved through on-board dynamical monitoring and estimation associated with controller switching and tuning by the agent. The agent can handle an untuned autopilot or retune the autopilot when dynamical changes occur due to aerodynamic and on-board system changes. The system integrates dynamical modelling, hybrid adaptive control, model validation, flight condition diagnosis, control performance evaluation through software agent development. An important feature of the agent is its abstractions from real-time measurements and also its abstractions from model based on-board simulation. The agent, while tuning and supervising the autopilot, also performs real-time evaluations on the effects of its actions