Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems

In many Cyber-Physical Systems, we encounter the problem of remote state estimation of geographically distributed and remote physical processes. This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes. Information from the different sensors have to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored. For effective estimation at the gateway, the sensors need to be scheduled appropriately, i.e., at each time instant one needs to decide which sensors have network access and which ones do not. To address this scheduling problem, we formulate an associated Markov decision process (MDP). This MDP is then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free. We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waiting-time, among others. Our algorithm is shown to significantly outperform these algorithms for many example scenarios

Performance Analysis of a 5G Transceiver Implementation for Remote Areas Scenarios

The fifth generation of mobile communication networks will support a large set of new services and applications. One important use case is the remote area coverage for broadband Internet access. This use case ha significant social and economic impact, since a considerable percentage of the global population living in low populated area does not have Internet access and the communication infrastructure in rural areas can be used to improve agribusiness productivity. The aim of this paper is to analyze the performance of a 5G for Remote Areas transceiver, implemented on field programmable gate array based hardware for real-time processing. This transceiver employs the latest digital communication techniques, such as generalized frequency division multiplexing waveform combined with 2 by 2 multiple-input multiple-output diversity scheme and polar channel coding. The performance of the prototype is evaluated regarding its out-of-band emissions and bit error rate under AWGN channel.Comment: Presented in 2018 European Conference on Networks and Communications (EuCNC),18-21 June, 2018, Ljubljana, Sloveni

Remote State Estimation with Smart Sensors over Markov Fading Channels

We consider a fundamental remote state estimation problem of discrete-time linear time-invariant (LTI) systems. A smart sensor forwards its local state estimate to a remote estimator over a time-correlated $M$-state Markov fading channel, where the packet drop probability is time-varying and depends on the current fading channel state. We establish a necessary and sufficient condition for mean-square stability of the remote estimation error covariance as $\rho^2(\mathbf{A})\rho(\mathbf{DM})<1$, where $\rho(\cdot)$ denotes the spectral radius, $\mathbf{A}$ is the state transition matrix of the LTI system, $\mathbf{D}$ is a diagonal matrix containing the packet drop probabilities in different channel states, and $\mathbf{M}$ is the transition probability matrix of the Markov channel states. To derive this result, we propose a novel estimation-cycle based approach, and provide new element-wise bounds of matrix powers. The stability condition is verified by numerical results, and is shown more effective than existing sufficient conditions in the literature. We observe that the stability region in terms of the packet drop probabilities in different channel states can either be convex or concave depending on the transition probability matrix $\mathbf{M}$. Our numerical results suggest that the stability conditions for remote estimation may coincide for setups with a smart sensor and with a conventional one (which sends raw measurements to the remote estimator), though the smart sensor setup achieves a better estimation performance.Comment: The paper has been accepted by IEEE Transactions on Automatic Control. Copyright may be transferred without notice, after which this version may no longer be accessibl