Distributive Power Control Algorithm for Multicarrier Interference Network over Time-Varying Fading Channels - Tracking Performance Analysis and Optimization

Distributed power control over interference limited network has received an increasing intensity of interest over the past few years. Distributed solutions (like the iterative water-filling, gradient projection, etc.) have been intensively investigated under \emph{quasi-static} channels. However, as such distributed solutions involve iterative updating and explicit message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. Unfortunately, the behavior of those distributed solutions under \emph{time-varying} channels is in general unknown. In this paper, we shall investigate the distributed scaled gradient projection algorithm (DSGPA) in a $K$ pairs multicarrier interference network under a finite-state Markov channel (FSMC) model. We shall analyze the \emph{convergence property} as well as \emph{tracking performance} of the proposed DSGPA. Our analysis shows that the proposed DSGPA converges to a limit region rather than a single point under the FSMC model. We also show that the order of growth of the tracking errors is given by \mathcal{O}\(1 \big/ \bar{N}\), where $\bar{N}$ is the \emph{average sojourn time} of the FSMC. Based on the analysis, we shall derive the \emph{tracking error optimal scaling matrices} via Markov decision process modeling. We shall show that the tracking error optimal scaling matrices can be implemented distributively at each transmitter. The numerical results show the superior performance of the proposed DSGPA over three baseline schemes, such as the gradient projection algorithm with a constant stepsize.Comment: To Appear on the IEEE Transaction on Signal Processin

Distributive Network Utility Maximization (NUM) over Time-Varying Fading Channels

Distributed network utility maximization (NUM) has received an increasing intensity of interest over the past few years. Distributed solutions (e.g., the primal-dual gradient method) have been intensively investigated under fading channels. As such distributed solutions involve iterative updating and explicit message passing, it is unrealistic to assume that the wireless channel remains unchanged during the iterations. Unfortunately, the behavior of those distributed solutions under time-varying channels is in general unknown. In this paper, we shall investigate the convergence behavior and tracking errors of the iterative primal-dual scaled gradient algorithm (PDSGA) with dynamic scaling matrices (DSC) for solving distributive NUM problems under time-varying fading channels. We shall also study a specific application example, namely the multi-commodity flow control and multi-carrier power allocation problem in multi-hop ad hoc networks. Our analysis shows that the PDSGA converges to a limit region rather than a single point under the finite state Markov chain (FSMC) fading channels. We also show that the order of growth of the tracking errors is given by O(T/N), where T and N are the update interval and the average sojourn time of the FSMC, respectively. Based on this analysis, we derive a low complexity distributive adaptation algorithm for determining the adaptive scaling matrices, which can be implemented distributively at each transmitter. The numerical results show the superior performance of the proposed dynamic scaling matrix algorithm over several baseline schemes, such as the regular primal-dual gradient algorithm

Convergence Analysis of Mixed Timescale Cross-Layer Stochastic Optimization

This paper considers a cross-layer optimization problem driven by multi-timescale stochastic exogenous processes in wireless communication networks. Due to the hierarchical information structure in a wireless network, a mixed timescale stochastic iterative algorithm is proposed to track the time-varying optimal solution of the cross-layer optimization problem, where the variables are partitioned into short-term controls updated in a faster timescale, and long-term controls updated in a slower timescale. We focus on establishing a convergence analysis framework for such multi-timescale algorithms, which is difficult due to the timescale separation of the algorithm and the time-varying nature of the exogenous processes. To cope with this challenge, we model the algorithm dynamics using stochastic differential equations (SDEs) and show that the study of the algorithm convergence is equivalent to the study of the stochastic stability of a virtual stochastic dynamic system (VSDS). Leveraging the techniques of Lyapunov stability, we derive a sufficient condition for the algorithm stability and a tracking error bound in terms of the parameters of the multi-timescale exogenous processes. Based on these results, an adaptive compensation algorithm is proposed to enhance the tracking performance. Finally, we illustrate the framework by an application example in wireless heterogeneous network

Error Bounds and Applications for Stochastic Approximation with Non-Decaying Gain

This work analyzes the stochastic approximation algorithm with non-decaying gains as applied in time-varying problems. The setting is to minimize a sequence of scalar-valued loss functions $f_k(\cdot)$ at sampling times $\tau_k$ or to locate the root of a sequence of vector-valued functions $g_k(\cdot)$ at $\tau_k$ with respect to a parameter $\theta\in R^p$. The available information is the noise-corrupted observation(s) of either $f_k(\cdot)$ or $g_k(\cdot)$ evaluated at one or two design points only. Given the time-varying stochastic approximation setup, we apply stochastic approximation algorithms with non-decaying gains, so that the recursive estimate denoted as $\hat{\theta}_k$ can maintain its momentum in tracking the time-varying optimum denoted as $\theta_k^*$. Chapter 3 provides a bound for the root-mean-squared error $\sqrt{E(\|\hat{\theta}_k-\theta_k^*\|^2})$. Overall, the bounds are applicable under a mild assumption on the time-varying drift and a modest restriction on the observation noise and the bias term. After establishing the tracking capability in Chapter 3, we also discuss the concentration behavior of $\hat{\theta}_k$ in Chapter 4. The weak convergence limit of the continuous interpolation of $\hat{\theta}_k$ is shown to follow the trajectory of a non-autonomous ordinary differential equation. Both Chapter 3 and Chapter 4 are probabilistic arguments and may not provide much guidance on the gain-tuning strategies useful for one single experiment run. Therefore, Chapter 5 discusses a data-dependent gain-tuning strategy based on estimating the Hessian information and the noise level. Overall, this work answers the questions "what is the estimate for the dynamical system $\theta_k^*$" and "how much we can trust $\hat{\theta}_k$ as an estimate for $\theta_k^*$."Comment: arXiv admin note: text overlap with arXiv:1906.0953

Mathematical theory of the Goddard trajectory determination system

Basic mathematical formulations depict coordinate and time systems, perturbation models, orbital estimation techniques, observation models, and numerical integration methods

Flight controller synthesis via deep reinforcement learning

Traditional control methods are inadequate in many deployment settings involving autonomous control of Cyber-Physical Systems (CPS). In such settings, CPS controllers must operate and respond to unpredictable interactions, conditions, or failure modes. Dealing with such unpredictability requires the use of executive and cognitive control functions that allow for planning and reasoning. Motivated by the sport of drone racing, this dissertation addresses these concerns for state-of-the-art flight control by investigating the use of deep artificial neural networks to bring essential elements of higher-level cognition to bear on the design, implementation, deployment, and evaluation of low level (attitude) flight controllers. First, this thesis presents a feasibility analyses and results which confirm that neural networks, trained via reinforcement learning, are more accurate than traditional control methods used by commercial uncrewed aerial vehicles (UAVs) for attitude control. Second, armed with these results, this thesis reports on the development and release of an open source, full solution stack for building neuro-flight controllers. This stack consists of a tuning framework for implementing training environments (GymFC) and firmware for the world’s first neural network supported flight controller (Neuroflight). GymFC’s novel approach fuses together the digital twinning paradigm with flight control training to provide seamless transfer to hardware. Third, to transfer models synthesized by GymFC to hardware, this thesis reports on the toolchain that has been released for compiling neural networks into Neuroflight, which can be flashed to off-the-shelf microcontrollers. This toolchain includes detailed procedures for constructing a multicopter digital twin to allow the research and development community to synthesize flight controllers unique to their own aircraft. Finally, this thesis examines alternative reward system functions as well as changes to the software environment to bridge the gap between simulation and real world deployment environments. The design, evaluation, and experimental work summarized in this thesis demonstrates that deep reinforcement learning is able to be leveraged for the design and implementation of neural network controllers capable not only of maintaining stable flight, but also precision aerobatic maneuvers in real world settings. As such, this work provides a foundation for developing the next generation of flight control systems