Interference-based dynamic pricing for WCDMA networks using neurodynamic programming

Copyright © 2007 IEEEWe study the problem of optimal integrated dynamic pricing and radio resource management, in terms of resource allocation and call admission control, in a WCDMA network. In such interference-limited network, one's resource usage also degrades the utility of others. A new parameter noise rise factor, which indicates the amount of interference generated by a call, is suggested as a basis for setting price to make users accountable for the congestion externality of their usage. The methods of dynamic programming (DP) are unsuitable for problems with large state spaces due to the associated ldquocurse of dimensionality.rdquo To overcome this, we solve the problem using a simulation-based neurodynamic programming (NDP) method with an action-dependent approximation architecture. Our results show that the proposed optimal policy provides significant average reward and congestion improvement over conventional policies that charge users based on their load factor.Siew-Lee Hew and Langford B. Whit

Event-triggered near optimal adaptive control of interconnected systems

Increased interest in complex interconnected systems like smart-grid, cyber manufacturing have attracted researchers to develop optimal adaptive control schemes to elicit a desired performance when the complex system dynamics are uncertain. In this dissertation, motivated by the fact that aperiodic event sampling saves network resources while ensuring system stability, a suite of novel event-sampled distributed near-optimal adaptive control schemes are introduced for uncertain linear and affine nonlinear interconnected systems in a forward-in-time and online manner. First, a novel stochastic hybrid Q-learning scheme is proposed to generate optimal adaptive control law and to accelerate the learning process in the presence of random delays and packet losses resulting from the communication network for an uncertain linear interconnected system. Subsequently, a novel online reinforcement learning (RL) approach is proposed to solve the Hamilton-Jacobi-Bellman (HJB) equation by using neural networks (NNs) for generating distributed optimal control of nonlinear interconnected systems using state and output feedback. To relax the state vector measurements, distributed observers are introduced. Next, using RL, an improved NN learning rule is derived to solve the HJB equation for uncertain nonlinear interconnected systems with event-triggered feedback. Distributed NN identifiers are introduced both for approximating the uncertain nonlinear dynamics and to serve as a model for online exploration. Next, the control policy and the event-sampling errors are considered as non-cooperative players and a min-max optimization problem is formulated for linear and affine nonlinear systems by using zero-sum game approach for simultaneous optimization of both the control policy and the event based sampling instants. The net result is the development of optimal adaptive event-triggered control of uncertain dynamic systems --Abstract, page iv

Online optimal and adaptive integral tracking control for varying discrete‐time systems using reinforcement learning

Conventional closed‐form solution to the optimal control problem using optimal control theory is only available under the assumption that there are known system dynamics/models described as differential equations. Without such models, reinforcement learning (RL) as a candidate technique has been successfully applied to iteratively solve the optimal control problem for unknown or varying systems. For the optimal tracking control problem, existing RL techniques in the literature assume either the use of a predetermined feedforward input for the tracking control, restrictive assumptions on the reference model dynamics, or discounted tracking costs. Furthermore, by using discounted tracking costs, zero steady‐state error cannot be guaranteed by the existing RL methods. This article therefore presents an optimal online RL tracking control framework for discrete‐time (DT) systems, which does not impose any restrictive assumptions of the existing methods and equally guarantees zero steady‐state tracking error. This is achieved by augmenting the original system dynamics with the integral of the error between the reference inputs and the tracked outputs for use in the online RL framework. It is further shown that the resulting value function for the DT linear quadratic tracker using the augmented formulation with integral control is also quadratic. This enables the development of Bellman equations, which use only the system measurements to solve the corresponding DT algebraic Riccati equation and obtain the optimal tracking control inputs online. Two RL strategies are thereafter proposed based on both the value function approximation and the Q‐learning along with bounds on excitation for the convergence of the parameter estimates. Simulation case studies show the effectiveness of the proposed approach

Suboptimal Safety-Critical Control for Continuous Systems Using Prediction-Correction Online Optimization

This paper investigates the control barrier function (CBF) based safety-critical control for continuous nonlinear control affine systems using more efficient online algorithms by the time-varying optimization method. The idea of the algorithms is that when quadratic programming (QP) or other convex optimization algorithms needed in the CBF-based method is not computation affordable, the alternative suboptimal feasible solutions can be obtained more economically. By using the barrier-based interior point method, the constrained CBF-QP problems are transformed into unconstrained ones with suboptimal solutions tracked by two continuous descent-based algorithms. Considering the lag effect of tracking and exploiting the system information, the prediction method is added to the algorithms, which achieves exponential convergence to the time-varying suboptimal solutions. The convergence and robustness of the designed methods as well as the safety criteria of the algorithms are studied theoretically. The effectiveness is illustrated by simulations on the anti-swing and obstacle avoidance tasks

Review of distributed control and optimization in energy internet: From traditional methods to artificial intelligence-based methods

Abstract: Energy internet (EI) can alleviate the arduous challenges brought about by the energy crisis and global warming and has aroused the concern of many scholars. In the research of EI control systems, the access of distributed energy causes the power system to exhibit complex nonlinearity, high uncertainty and strong coupling. Traditional control and optimization methods often have limited effectiveness in solving these problems. With the widespread application of distributed control technology and the maturity of artificial intelligence (AI) technology, the combination of distributed control and AI has become an effective method to break through current research bottlenecks. This study reviews the research progress of EI distributed control technologies based on AI in recent years. It can be found that AI‐based distributed control methods have many advantages in maintaining EI stability and achieving optimal energy management. This combination of AI and distributed control makes EI control systems more intelligent, safe and efficient, which will be an important direction for future research. The purpose of this study is to provide a reference as well as useful research ideas for the study of EI control systems