On the almost sure central limit theorem for ARX processes in adaptive tracking

The goal of this paper is to highlight the almost sure central limit theorem for martingales to the control community and to show the usefulness of this result for the system identification of controllable ARX(p,q) process in adaptive tracking. We also provide strongly consistent estimators of the even moments of the driven noise of a controllable ARX(p,q) process as well as quadratic strong laws for the average costs and estimation errors sequences. Our theoretical results are illustrated by numerical experiments

Differential observation and integral action in LTI state-space controllers and the PID special case

This paper makes the case that practical differentiation of measured state variables may be seen as an observation or estimation scheme for linear time invariant state space controllers. It is shown that, although not having the separation property, the estimation error of this scheme converges to zero if the resulting closed loop system is strictly stable. On the basis of this concept, it is shown that PID controllers may be interpreted as a special case of state space controllers endowed with differential observation. The interesting consequences of this interpretation are discussed.Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/202

Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processes

Information-theoretic principles for learning and acting have been proposed to solve particular classes of Markov Decision Problems. Mathematically, such approaches are governed by a variational free energy principle and allow solving MDP planning problems with information-processing constraints expressed in terms of a Kullback-Leibler divergence with respect to a reference distribution. Here we consider a generalization of such MDP planners by taking model uncertainty into account. As model uncertainty can also be formalized as an information-processing constraint, we can derive a unified solution from a single generalized variational principle. We provide a generalized value iteration scheme together with a convergence proof. As limit cases, this generalized scheme includes standard value iteration with a known model, Bayesian MDP planning, and robust planning. We demonstrate the benefits of this approach in a grid world simulation.Comment: 16 pages, 3 figure

A novel coupling control with decision-maker and PID controller for minimizing heating energy consumption and ensuring indoor environmental quality

Due to climate change, global energy crisis, and high-quality life requirement for people, decreasing building energy consumption and enhancing indoor environment quality through control of heating, ventilation, and air conditioning systems tend to be increasingly important. Therefore, favorable control methods for heating and ventilation systems are urgently necessary. In this work, a new coupling control with decision-maker was proposed, developed, and investigated; meanwhile, several demand controlled ventilation strategies combined with heating control method was compared considering heating energy consumption, thermal comfort, and indoor air quality. In order to properly model the service systems, the air change rates and thermal time constants have been first measured in a reference office installed with commonly applied bottom-hinged tilted windows in our low-energy building supplied by geothermal district heating. Then, simulations have been carried out across two typical winter days in the reference office. The results illustrate that the proposed combination of suitable heating and demand controlled ventilation coupling control methods with decision-maker and proportional-integral-derivative (PID) controller could greatly reduce heating consumption in the reference room during the office time: around 52.4% (4.4 kW h energy saving) per day in winter in comparison to a commonly suggested method of intensive and brief airing. At the same time, it could ensure indoor CO2 concentration to keep within the pre-set ranges (Pettenkofer limit: 1000 ppm) as well as low variations of indoor temperature (standard deviation (SD): 0.1°C)

Trade-Offs and Constraints in Allosteric Sensing

Sensing extracellular changes initiates signal transduction and is the first stage of cellular decision-making. Yet relatively little is known about why one form of sensing biochemistry has been selected over another. To gain insight into this question, we studied the sensing characteristics of one of the biochemically simplest of sensors: the allosteric transcription factor. Such proteins, common in microbes, directly transduce the detection of a sensed molecule to changes in gene regulation. Using the Monod-Wyman-Changeux model, we determined six sensing characteristics – the dynamic range, the Hill number, the intrinsic noise, the information transfer capacity, the static gain, and the mean response time – as a function of the biochemical parameters of individual sensors and of the number of sensors. We found that specifying one characteristic strongly constrains others. For example, a high dynamic range implies a high Hill number and a high capacity, and vice versa. Perhaps surprisingly, these constraints are so strong that most of the space of characteristics is inaccessible given biophysically plausible ranges of parameter values. Within our approximations, we can calculate the probability distribution of the numbers of input molecules that maximizes information transfer and show that a population of one hundred allosteric transcription factors can in principle distinguish between more than four bands of input concentrations. Our results imply that allosteric sensors are unlikely to have been selected for high performance in one sensing characteristic but for a compromise in the performance of many

Potassium Starvation in Yeast: Mechanisms of Homeostasis Revealed by Mathematical Modeling

The intrinsic ability of cells to adapt to a wide range of environmental conditions is a fundamental process required for survival. Potassium is the most abundant cation in living cells and is required for essential cellular processes, including the regulation of cell volume, pH and protein synthesis. Yeast cells can grow from low micromolar to molar potassium concentrations and utilize sophisticated control mechanisms to keep the internal potassium concentration in a viable range. We developed a mathematical model for Saccharomyces cerevisiae to explore the complex interplay between biophysical forces and molecular regulation facilitating potassium homeostasis. By using a novel inference method (“the reverse tracking algorithm”) we predicted and then verified experimentally that the main regulators under conditions of potassium starvation are proton fluxes responding to changes of potassium concentrations. In contrast to the prevailing view, we show that regulation of the main potassium transport systems (Trk1,2 and Nha1) in the plasma membrane is not sufficient to achieve homeostasis