Minimization of Sensor Activation in Discrete-Event Systems with Control Delays and Observation Delays

In discrete-event systems, to save sensor resources, the agent continuously adjusts sensor activation decisions according to a sensor activation policy based on the changing observations. However, new challenges arise for sensor activations in networked discrete-event systems, where observation delays and control delays exist between the sensor systems and the agent. In this paper, a new framework for activating sensors in networked discrete-event systems is established. In this framework, we construct a communication automaton that explicitly expresses the interaction process between the agent and the sensor systems over the observation channel and the control channel. Based on the communication automaton, we can define dynamic observations of a communicated string. To guarantee that a sensor activation policy is physically implementable and insensitive to random control delays and observation delays, we further introduce the definition of delay feasibility. We show that a delay feasible sensor activation policy can be used to dynamically activate sensors even if control delays and observation delays exist. A set of algorithms are developed to minimize sensor activations in a transition-based domain while ensuring a given specification condition is satisfied. A practical example is provided to show the application of the developed sensor activation methods. Finally, we briefly discuss how to extend the proposed framework to a decentralized sensing architecture

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Fast Discrete Consensus Based on Gossip for Makespan Minimization in Networked Systems

In this paper we propose a novel algorithm to solve the discrete consensus problem, i.e., the problem of distributing evenly a set of tokens of arbitrary weight among the nodes of a networked system. Tokens are tasks to be executed by the nodes and the proposed distributed algorithm minimizes monotonically the makespan of the assigned tasks. The algorithm is based on gossip-like asynchronous local interactions between the nodes. The convergence time of the proposed algorithm is superior with respect to the state of the art of discrete and quantized consensus by at least a factor O(n) in both theoretical and empirical comparisons

Property Enforcement for Partially-Observed Discrete-Event Systems

Engineering systems that involve physical elements, such as automobiles, aircraft, or electric power pants, that are controlled by a computational infrastructure that consists of several computers that communicate through a communication network, are called Cyber-Physical Systems. Ever-increasing demands for safety, security, performance, and certi cation of these critical systems put stringent constraints on their design and necessitate the use of formal model-based approaches to synthesize provably-correct feedback controllers. This dissertation aims to tackle these challenges by developing a novel methodology for synthesis of control and sensing strategies for Discrete Event Systems (DES), an important class of cyber-physical systems. First, we develop a uniform approach for synthesizing property enforcing supervisors for a wide class of properties called information-state-based (IS-based) properties. We then consider the enforcement of non-blockingness in addition to IS-based properties. We develop a nite structure called the All Enforcement Structure (AES) that embeds all valid supervisors. Furthermore, we propose novel and general approaches to solve the sensor activation problem for partially-observed DES. We extend our results for the sensor activation problem from the centralized case to the decentralized case. The methodology in the dissertation has the following novel features: (i) it explicitly considers and handles imperfect state information, due to sensor noise, and limited controllability, due to unexpected environmental disturbances; (ii) it is a uniform information-state-based approach that can be applied to a variety of user-speci ed requirements; (iii) it is a formal model-based approach, which results in provably correct solutions; and (iv) the methodology and associated theoretical foundations developed are generic and applicable to many types of networked cyber-physical systems with safety-critical requirements, in particular networked systems such as aircraft electric power systems and intelligent transportation systems.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/137097/1/xiangyin_1.pd

Deep Reinforcement Learning for Event-Triggered Control

Event-triggered control (ETC) methods can achieve high-performance control with a significantly lower number of samples compared to usual, time-triggered methods. These frameworks are often based on a mathematical model of the system and specific designs of controller and event trigger. In this paper, we show how deep reinforcement learning (DRL) algorithms can be leveraged to simultaneously learn control and communication behavior from scratch, and present a DRL approach that is particularly suitable for ETC. To our knowledge, this is the first work to apply DRL to ETC. We validate the approach on multiple control tasks and compare it to model-based event-triggering frameworks. In particular, we demonstrate that it can, other than many model-based ETC designs, be straightforwardly applied to nonlinear systems

From Packet to Power Switching: Digital Direct Load Scheduling

At present, the power grid has tight control over its dispatchable generation capacity but a very coarse control on the demand. Energy consumers are shielded from making price-aware decisions, which degrades the efficiency of the market. This state of affairs tends to favor fossil fuel generation over renewable sources. Because of the technological difficulties of storing electric energy, the quest for mechanisms that would make the demand for electricity controllable on a day-to-day basis is gaining prominence. The goal of this paper is to provide one such mechanisms, which we call Digital Direct Load Scheduling (DDLS). DDLS is a direct load control mechanism in which we unbundle individual requests for energy and digitize them so that they can be automatically scheduled in a cellular architecture. Specifically, rather than storing energy or interrupting the job of appliances, we choose to hold requests for energy in queues and optimize the service time of individual appliances belonging to a broad class which we refer to as "deferrable loads". The function of each neighborhood scheduler is to optimize the time at which these appliances start to function. This process is intended to shape the aggregate load profile of the neighborhood so as to optimize an objective function which incorporates the spot price of energy, and also allows distributed energy resources to supply part of the generation dynamically.Comment: Accepted by the IEEE journal of Selected Areas in Communications (JSAC): Smart Grid Communications series, to appea