Network coding for function computation

Many applications such as parallel processing, distributed data analytics and sensor networks often need to compute functions of data that are observed in a distributed manner over a network. A network can be modeled as a directed graph, each vertex of which denotes a node that can carry out computations and communicate with its neighbors. The edges of the graph denote one-way noiseless communication links. A subset of nodes - called sources - observe independent messages, and a possibly different subset of nodes - called terminals - wish to compute a particular demand function of the messages. The information transmitted on the edges are specified by a set of functions, one for each edge; this set of functions is called a network code. We are interested in network codes that allow each terminal to compute the demanded function with zero-error. In the first part of this thesis, we assume that the message random variables are independent and uniformly distributed over a finite field. The demand function is set to be the finite field sum of all the messages observed in the network. A valid network code for this \textit{sum-network} problem allows each terminal to compute the sum, and has an associated computation rate. We wish to find the best possible computation rate for a given sum-network; this value is called its \textit{computation capacity}. Finding the computation capacity of a sum-network is known to be a difficult problem. Here we are able to evaluate it for certain systematically constructed sum-network problem instances. The construction procedure uses incidence structures, whose combinatorial properties allow us to be able to evaluate the computation capacity of the constructed sum-networks. An important aspect of the problem that we uncover is the strong dependence of the computation capacity on the finite field over which the sum is to computed. This is shown by a sum-network, whose computation capacity is $1$ over a finite field and close to $0$ over a different finite field. We also construct sum-networks whose computation capacity can take on arbitrarily many different values over different finite field alphabets. In the second part of the thesis, we focus on a particular directed acyclic network that has four nodes and four edges. It is the simplest instance of a network that does not have a tree structure. Three of the nodes are sources that observe independent messages that are uniformly distributed over a finite discrete alphabet. The fourth node is a terminal which wants to compute a demand function of the three messages. The demand function is an arbitrary discrete-valued function. We focus on network codes that have different rates on each of the four edges, thus we have a rate tuple associated with every valid network code. The collection of rate tuples for all valid network codes form a \textit{rate region}, and we describe a procedure to obtain an outer bound to this rate region. We illustrate our approach through different example demand functions. When the demand function is the finite field sum over $GF(2)$, we give a network code whose rate tuple matches the outer bound

Automated decision making and problem solving. Volume 2: Conference presentations

Related topics in artificial intelligence, operations research, and control theory are explored. Existing techniques are assessed and trends of development are determined

Fault-tolerant Stochastic Distributed Systems

The present doctoral thesis discusses the design of fault-tolerant distributed systems, placing emphasis in addressing the case where the actions of the nodes or their interactions are stochastic. The main objective is to detect and identify faults to improve the resilience of distributed systems to crash-type faults, as well as detecting the presence of malicious nodes in pursuit of exploiting the network. The proposed analysis considers malicious agents and computational solutions to detect faults. Crash-type faults, where the affected component ceases to perform its task, are tackled in this thesis by introducing stochastic decisions in deterministic distributed algorithms. Prime importance is placed on providing guarantees and rates of convergence for the steady-state solution. The scenarios of a social network (state-dependent example) and consensus (time- dependent example) are addressed, proving convergence. The proposed algorithms are capable of dealing with packet drops, delays, medium access competition, and, in particular, nodes failing and/or losing network connectivity. The concept of Set-Valued Observers (SVOs) is used as a tool to detect faults in a worst-case scenario, i.e., when a malicious agent can select the most unfavorable sequence of communi- cations and inject a signal of arbitrary magnitude. For other types of faults, it is introduced the concept of Stochastic Set-Valued Observers (SSVOs) which produce a confidence set where the state is known to belong with at least a pre-specified probability. It is shown how, for an algorithm of consensus, it is possible to exploit the structure of the problem to reduce the computational complexity of the solution. The main result allows discarding interactions in the model that do not contribute to the produced estimates. The main drawback of using classical SVOs for fault detection is their computational burden. By resorting to a left-coprime factorization for Linear Parameter-Varying (LPV) systems, it is shown how to reduce the computational complexity. By appropriately selecting the factorization, it is possible to consider detectable systems (i.e., unobservable systems where the unobservable component is stable). Such a result plays a key role in the domain of Cyber-Physical Systems (CPSs). These techniques are complemented with Event- and Self-triggered sampling strategies that enable fewer sensor updates. Moreover, the same triggering mechanisms can be used to make decisions of when to run the SVO routine or resort to over-approximations that temporarily compromise accuracy to gain in performance but maintaining the convergence characteristics of the set-valued estimates. A less stringent requirement for network resources that is vital to guarantee the applicability of SVO-based fault detection in the domain of Networked Control Systems (NCSs)

Information Bottleneck

The celebrated information bottleneck (IB) principle of Tishby et al. has recently enjoyed renewed attention due to its application in the area of deep learning. This collection investigates the IB principle in this new context. The individual chapters in this collection: • provide novel insights into the functional properties of the IB; • discuss the IB principle (and its derivates) as an objective for training multi-layer machine learning structures such as neural networks and decision trees; and • offer a new perspective on neural network learning via the lens of the IB framework. Our collection thus contributes to a better understanding of the IB principle specifically for deep learning and, more generally, of information–theoretic cost functions in machine learning. This paves the way toward explainable artificial intelligence