Optimality of Event-Based Policies for Decentralized Estimation over Shared Networks

Cyber-physical systems often consist of multiple non-collocated components that sense, exchange information and act as a team through a network. Although this new paradigm provides convenience, flexibility and robustness to modern systems, design methods to achieve optimal performance are elusive as they must account for certain detrimental characteristics of the underlying network. These include constrained connectivity among agents, rate-limited communication links, physical noise at the antennas, packet drops and interference. We propose a new class of problems in optimal networked estimation where multiple sensors operating as a team communicate their measurements to a fusion center over an interference prone network modeled by a collision channel. Using a team decision theoretic approach, we characterize jointly optimal communication policies for one-shot problems under different performance criteria. First we study the problem of estimating two independent continuous random variables observed by two different sensors communicating with a fusion center over a collision channel. For a minimum mean squared estimation error criterion, we show that there exist team-optimal strategies where each sensor uses a threshold policy. This result is independent of the distribution of the observations and, can be extended to vector observations and to any number of sensors. Consequently, the existence of team-optimal threshold policies is a result of practical significance, because it can be applied to a wide class of systems without requiring collision avoidance protocols. Next we study the problem of estimating independent discrete random variables over a collision channel. Using two different criteria involving the probability of estimation error, we show the existence of team-optimal strategies where the sensors either transmit all but the most likely observation; transmit only the second most likely observation; or remain always silent. These results are also independent of the distributions and are valid for any number of sensors. In our analysis, the proof of the structural result involves the minimization of a concave functional, which is an evidence of the inherent complexity of team decision problems with nonclassical information structure. In the last part of the dissertation, the assumption on the cooperation among sensors is relaxed, and we show that similar structural results can also be obtained for systems with one or more selfish sensors. Finally the assumption of the independence is lifted by introducing the observation of a common random variable in addition to the private observations of each sensor. The structural result obtained provides valuable insights on the characterization of team-optimal policies for a general correlation structure between the observed random variables

An Optimal Medium Access Control with Partial Observations for Sensor Networks

We consider medium access control (MAC) in multihop sensor networks, where only partial information about the shared medium is available to the transmitter. We model our setting as a queuing problem in which the service rate of a queue is a function of a partially observed Markov chain representing the available bandwidth, and in which the arrivals are controlled based on the partial observations so as to keep the system in a desirable mildly unstable regime. The optimal controller for this problem satisfies a separation property: we first compute a probability measure on the state space of the chain, namely the information state, then use this measure as the new state on which the control decisions are based. We give a formal description of the system considered and of its dynamics, we formalize and solve an optimal control problem, and we show numerical simulations to illustrate with concrete examples properties of the optimal control law. We show how the ergodic behavior of our queuing model is characterized by an invariant measure over all possible information states, and we construct that measure. Our results can be specifically applied for designing efficient and stable algorithms for medium access control in multiple-accessed systems, in particular for sensor networks

Solving Common-Payoff Games with Approximate Policy Iteration

For artificially intelligent learning systems to have widespread applicability in real-world settings, it is important that they be able to operate decentrally. Unfortunately, decentralized control is difficult -- computing even an epsilon-optimal joint policy is a NEXP complete problem. Nevertheless, a recently rediscovered insight -- that a team of agents can coordinate via common knowledge -- has given rise to algorithms capable of finding optimal joint policies in small common-payoff games. The Bayesian action decoder (BAD) leverages this insight and deep reinforcement learning to scale to games as large as two-player Hanabi. However, the approximations it uses to do so prevent it from discovering optimal joint policies even in games small enough to brute force optimal solutions. This work proposes CAPI, a novel algorithm which, like BAD, combines common knowledge with deep reinforcement learning. However, unlike BAD, CAPI prioritizes the propensity to discover optimal joint policies over scalability. While this choice precludes CAPI from scaling to games as large as Hanabi, empirical results demonstrate that, on the games to which CAPI does scale, it is capable of discovering optimal joint policies even when other modern multi-agent reinforcement learning algorithms are unable to do so. Code is available at https://github.com/ssokota/capi .Comment: AAAI 202

Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey

The ongoing amalgamation of UAV and ML techniques is creating a significant synergy and empowering UAVs with unprecedented intelligence and autonomy. This survey aims to provide a timely and comprehensive overview of ML techniques used in UAV operations and communications and identify the potential growth areas and research gaps. We emphasise the four key components of UAV operations and communications to which ML can significantly contribute, namely, perception and feature extraction, feature interpretation and regeneration, trajectory and mission planning, and aerodynamic control and operation. We classify the latest popular ML tools based on their applications to the four components and conduct gap analyses. This survey also takes a step forward by pointing out significant challenges in the upcoming realm of ML-aided automated UAV operations and communications. It is revealed that different ML techniques dominate the applications to the four key modules of UAV operations and communications. While there is an increasing trend of cross-module designs, little effort has been devoted to an end-to-end ML framework, from perception and feature extraction to aerodynamic control and operation. It is also unveiled that the reliability and trust of ML in UAV operations and applications require significant attention before full automation of UAVs and potential cooperation between UAVs and humans come to fruition.Comment: 36 pages, 304 references, 19 Figure

Training of Crisis Mappers and Map Production from Multi-sensor Data: Vernazza Case Study (Cinque Terre National Park, Italy)

This aim of paper is to presents the development of a multidisciplinary project carried out by the cooperation between Politecnico di Torino and ITHACA (Information Technology for Humanitarian Assistance, Cooperation and Action). The goal of the project was the training in geospatial data acquiring and processing for students attending Architecture and Engineering Courses, in order to start up a team of "volunteer mappers". Indeed, the project is aimed to document the environmental and built heritage subject to disaster; the purpose is to improve the capabilities of the actors involved in the activities connected in geospatial data collection, integration and sharing. The proposed area for testing the training activities is the Cinque Terre National Park, registered in the World Heritage List since 1997. The area was affected by flood on the 25th of October 2011. According to other international experiences, the group is expected to be active after emergencies in order to upgrade maps, using data acquired by typical geomatic methods and techniques such as terrestrial and aerial Lidar, close-range and aerial photogrammetry, topographic and GNSS instruments etc.; or by non conventional systems and instruments such us UAV, mobile mapping etc. The ultimate goal is to implement a WebGIS platform to share all the data collected with local authorities and the Civil Protectio