Analysis and Design of Robust and High-Performance Complex Dynamical Networks

In the first part of this dissertation, we develop some basic principles to investigate performance deterioration of dynamical networks subject to external disturbances. First, we propose a graph-theoretic methodology to relate structural specifications of the coupling graph of a linear consensus network to its performance measure. Moreover, for this class of linear consensus networks, we introduce new insights into the network centrality based not only on the network graph but also on a more structured model of network uncertainties. Then, for the class of generic linear networks, we show that the H_2-norm, as a performance measure, can be tightly bounded from below and above by some spectral functions of state and output matrices of the system. Finally, we study nonlinear autocatalytic networks and exploit their structural properties to characterize their existing hard limits and essential tradeoffs. In the second part, we consider problems of network synthesis for performance enhancement. First, we propose an axiomatic approach for the design and performance analysis of linear consensus networks by introducing a notion of systemic performance measure. We build upon this new notion and investigate a general form of combinatorial problem of growing a linear consensus network via minimizing a given systemic performance measure. Two efficient polynomial-time approximation algorithms are devised to tackle this network synthesis problem. Then, we investigate the optimal design problem of distributed system throttlers. A throttler is a mechanism that limits the flow rate of incoming metrics, e.g., byte per second, network bandwidth usage, capacity, traffic, etc. Finally, a framework is developed to produce a sparse approximation of a given large-scale network with guaranteed performance bounds using a nearly-linear time algorithm

Consensus problems and the effects of graph topology in collaborative control

In this dissertation, several aspects of design for networked systems are addressed. The main focus is on combining approaches from system theory and graph theory to characterize graph topologies that result in efficient decision making and control. In this framework, modelling and design of sparse graphs that are robust to failures and provide high connectivity are considered. A decentralized approach to path generation in a collaborative system is modelled using potential functions. Taking inspiration from natural swarms, various behaviors of the system such as target following, moving in cohesion and obstacle avoidance are addressed by appropriate encoding of the corresponding costs in the potential function and using gradient descent for minimizing the energy function. Different emergent behaviors emerge as a result of varying the weights attributed with different components of the potential function. Consensus problems are addressed as a unifying theme in many collaborative control problems and their robustness and convergence properties are studied. Implications of the continuous convergence property of consensus problems on their reachability and robustness are studied. The effects of link and agent faults on consensus problems are also investigated. In particular the concept of invariant nodes has been introduced to model the effect of nodes with different behaviors from regular nodes. A fundamental association is established between the structural properties of a graph and the performance of consensus algorithms running on them. This leads to development of a rigorous evaluation of the topology effects and determination of efficient graph topologies. It is well known that graphs with large diameter are not efficient as far as the speed of convergence of distributed algorithms is concerned. A challenging problem is to determine a minimum number of long range links (shortcuts), which guarantees a level of enhanced performance. This problem is investigated here in a stochastic framework. Specifically, the small world model of Watts and Strogatz is studied and it is shown that adding a few long range edges to certain graph topologies can significantly increase both the rate of convergence for consensus algorithms and the number of spanning trees in the graph. The simulations are supported by analytical stochastic methods inspired from perturbations of Markov chains. This approach is further extended to a probabilistic framework for understanding and quantifying the small world effect on consensus convergence rates: Time varying topologies, in which each agent nominally communicates according to a predefined topology, and switching with non-neighboring agents occur with small probability is studied. A probabilistic framework is provided along with fundamental bounds on the convergence speed of consensus problems with probabilistic switching. The results are also extended to the design of robust topologies for distributed algorithms. The design of a semi-distributed two-level hierarchical network is also studied, leading to improvement in the performance of distributed algorithms. The scheme is based on the concept of social degree and local leader selection and the use of consensus-type algorithms for locally determining topology information. Future suggestions include adjusting our algorithm towards a fully distributed implementation. Another important aspect of performance in collaborative systems is for the agents to send and receive information in a manner that minimizes process costs, such as estimation error and the cost of control. An instance of this problem is addressed by considering a collaborative sensor scheduling problem. It is shown that in finding the optimal joint estimates, the general tree-search solution can be efficiently solved by devising a method that utilizes the limited processing capabilities of agents to significantly decrease the number of search hypotheses

Energy-aware Sparse Sensing of Spatial-temporally Correlated Random Fields

This dissertation focuses on the development of theories and practices of energy aware sparse sensing schemes of random fields that are correlated in the space and/or time domains. The objective of sparse sensing is to reduce the number of sensing samples in the space and/or time domains, thus reduce the energy consumption and complexity of the sensing system. Both centralized and decentralized sensing schemes are considered in this dissertation. Firstly we study the problem of energy efficient Level set estimation (LSE) of random fields correlated in time and/or space under a total power constraint. We consider uniform sampling schemes of a sensing system with a single sensor and a linear sensor network with sensors distributed uniformly on a line where sensors employ a fixed sampling rate to minimize the LSE error probability in the long term. The exact analytical cost functions and their respective upper bounds of these sampling schemes are developed by using an optimum thresholding-based LSE algorithm. The design parameters of these sampling schemes are optimized by minimizing their respective cost functions. With the analytical results, we can identify the optimum sampling period and/or node distance that can minimize the LSE error probability. Secondly we propose active sparse sensing schemes with LSE of a spatial-temporally correlated random field by using a limited number of spatially distributed sensors. In these schemes a central controller is designed to dynamically select a limited number of sensing locations according to the information revealed from past measurements,and the objective is to minimize the expected level set estimation error.The expected estimation error probability is explicitly expressed as a function of the selected sensing locations, and the results are used to formulate the optimal sensing location selection problem as a combinatorial problem. Two low complexity greedy algorithms are developed by using analytical upper bounds of the expected estimation error probability. Lastly we study the distributed estimations of a spatially correlated random field with decentralized wireless sensor networks (WSNs). We propose a distributed iterative estimation algorithm that defines the procedures for both information propagation and local estimation in each iteration. The key parameters of the algorithm, including an edge weight matrix and a sample weight matrix, are designed by following the asymptotically optimum criteria. It is shown that the asymptotically optimum performance can be achieved by distributively projecting the measurement samples into a subspace related to the covariance matrices of data and noise samples

Autoregressive process parameters estimation from Compressed Sensing measurements and Bayesian dictionary learning

The main contribution of this thesis is the introduction of new techniques which allow to perform signal processing operations on signals represented by means of compressed sensing. Exploiting autoregressive modeling of the original signal, we obtain a compact yet representative description of the signal which can be estimated directly in the compressed domain. This is the key concept on which the applications we introduce rely on. In fact, thanks to proposed the framework it is possible to gain information about the original signal given compressed sensing measurements. This is done by means of autoregressive modeling which can be used to describe a signal through a small number of parameters. We develop a method to estimate these parameters given the compressed measurements by using an ad-hoc sensing matrix design and two different coupled estimators that can be used in different scenarios. This enables centralized and distributed estimation of the covariance matrix of a process given the compressed sensing measurements in a efficient way at low communication cost. Next, we use the characterization of the original signal done by means of few autoregressive parameters to improve compressive imaging. In particular, we use these parameters as a proxy to estimate the complexity of a block of a given image. This allows us to introduce a novel compressive imaging system in which the number of allocated measurements is adapted for each block depending on its complexity, i.e., spatial smoothness. The result is that a careful allocation of the measurements, improves the recovery process by reaching higher recovery quality at the same compression ratio in comparison to state-of-the-art compressive image recovery techniques. Interestingly, the parameters we are able to estimate directly in the compressed domain not only can improve the recovery but can also be used as feature vectors for classification. In fact, we also propose to use these parameters as more general feature vectors which allow to perform classification in the compressed domain. Remarkably, this method reaches high classification performance which is comparable with that obtained in the original domain, but with a lower cost in terms of dataset storage. In the second part of this work, we focus on sparse representations. In fact, a better sparsifying dictionary can improve the Compressed Sensing recovery performance. At first, we focus on the original domain and hence no dimensionality reduction by means of Compressed Sensing is considered. In particular, we develop a Bayesian technique which, in a fully automated fashion, performs dictionary learning. More in detail, using the uncertainties coming from atoms selection in the sparse representation step, this technique outperforms state-of-the-art dictionary learning techniques. Then, we also address image denoising and inpainting tasks using the aforementioned technique with excellent results. Next, we move to the compressed domain where a better dictionary is expected to provide improved recovery. We show how the Bayesian dictionary learning model can be adapted to the compressive case and the necessary assumptions that must be made when considering random projections. Lastly, numerical experiments confirm the superiority of this technique when compared to other compressive dictionary learning techniques

Opportunistic communications in large uncoordinated networks

(English) The increase of wireless devices offering high data rate services limits the coexistence of wireless systems sharing the same resources in a given geographical area because of inter-system interference. Therefore, interference management plays a key role in permitting the coexistence of several heterogeneous communication services. However, classical interference management strategies require lateral information giving rise to the need for inter-system coordination and cooperation, which is not always practical. Opportunistic communications offer a potential solution to the problem of inter-system interference management. The basic principle of opportunistic communications is to efficiently and robustly exploit the resources available in a wireless network and adapt the transmitted signals to the state of the network to avoid inter-system interference. Therefore, opportunistic communications depend on inferring the available network resources that can be safely exploited without inducing interference in coexisting communication nodes. Once the available network resources are identified, the most prominent opportunistic communication techniques consist in designing scenario-adapted precoding/decoding strategies to exploit the so-called null space. Despite this, classical solutions in the literature suffer from two main drawbacks: the lack of robustness to detection errors and the need for intra-system cooperation. This thesis focuses on the design of a null space-based opportunistic communication scheme that addresses the drawbacks exhibited by existing methodologies under the assumption that opportunistic nodes do not cooperate. For this purpose, a generalized detection error model independent of the null-space identification mechanism is introduced that allows the design of solutions that exhibit minimal inter-system interference in the worst case. These solutions respond to a maximum signal-to-interference ratio (SIR) criterion, which is optimal under non-cooperative conditions. The proposed methodology allows the design of a family of orthonormal waveforms that perform a spreading of the modulated symbols within the detected null space, which is key to minimizing the induced interference density. The proposed solutions are invariant within the inferred null space, allowing the removal of the feedback link without giving up coherent waveform detection. In the absence of coordination, the waveform design relies solely on locally sensed network state information, inducing a mismatch between the null spaces identified by the transmitter and receiver that may worsen system performance. Although the proposed solution is robust to this mismatch, the design of enhanced receivers using active subspace detection schemes is also studied. When the total number of network resources increases arbitrarily, the proposed solutions tend to be linear combinations of complex exponentials, providing an interpretation in the frequency domain. This asymptotic behavior allows us to adapt the proposed solution to frequency-selective channels by means of a cyclic prefix and to study an efficient modulation similar to the time division multiplexing scheme but using circulant waveforms. Finally, the impact of the use of multiple antennas in opportunistic null space-based communications is studied. The performed analysis reveals that, in any case, the structure of the antenna clusters affects the opportunistic communication, since the proposed waveform mimics the behavior of a single-antenna transmitter. On the other hand, the number of sensors employed translates into an improvement in terms of SIR.(Català) El creixement incremental dels dispositius sense fils que requereixen serveis d'alta velocitat de dades limita la coexistència de sistemes sense fils que comparteixen els mateixos recursos en una àrea geogràfica donada a causa de la interferència entre sistemes. Conseqüentment, la gestió d'interferència juga un paper fonamental per a facilitar la coexistència de diversos serveis de comunicació heterogenis. No obstant això, les estratègies clàssiques de gestió d'interferència requereixen informació lateral originant la necessitat de coordinació i cooperació entre sistemes, que no sempre és pràctica. Les comunicacions oportunistes ofereixen una solució potencial al problema de la gestió de les interferències entre sistemes. El principi bàsic de les comunicacions oportunistes és explotar de manera eficient i robusta els recursos disponibles en una xarxa sense fils i adaptar els senyals transmesos a l'estat de la xarxa per evitar interferències entre sistemes. Per tant, les comunicacions oportunistes depenen de la inferència dels recursos de xarxa disponibles que poden ser explotats de manera segura sense induir interferència en els nodes de comunicació coexistents. Una vegada que s'han identificat els recursos de xarxa disponibles, les tècniques de comunicació oportunistes més prominents consisteixen en el disseny d'estratègies de precodificació/descodificació adaptades a l'escenari per explotar l'anomenat espai nul. Malgrat això, les solucions clàssiques en la literatura sofreixen dos inconvenients principals: la falta de robustesa als errors de detecció i la necessitat de cooperació intra-sistema. Aquesta tesi tracta el disseny d'un esquema de comunicació oportunista basat en l'espai nul que afronta els inconvenients exposats per les metodologies existents assumint que els nodes oportunistes no cooperen. Per a aquest propòsit, s'introdueix un model generalitzat d'error de detecció independent del mecanisme d'identificació de l'espai nul que permet el disseny de solucions que exhibeixen interferències mínimes entre sistemes en el cas pitjor. Aquestes solucions responen a un criteri de màxima relació de senyal a interferència (SIR), que és òptim en condicions de no cooperació. La metodologia proposada permet dissenyar una família de formes d'ona ortonormals que realitzen un spreading dels símbols modulats dins de l'espai nul detectat, que és clau per minimitzar la densitat d’interferència induïda. Les solucions proposades són invariants dins de l'espai nul inferit, permetent suprimir l'enllaç de retroalimentació i, tot i així, realitzar una detecció coherent de forma d'ona. Sota l’absència de coordinació, el disseny de la forma d'ona es basa únicament en la informació de l'estat de la xarxa detectada localment, induint un desajust entre els espais nuls identificats pel transmissor i receptor que pot empitjorar el rendiment del sistema. Tot i que la solució proposada és robusta a aquest desajust, també s'estudia el disseny de receptors millorats fent ús de tècniques de detecció de subespai actiu. Quan el nombre total de recursos de xarxa augmenta arbitràriament, les solucions proposades tendeixen a ser combinacions lineals d'exponencials complexes, proporcionant una interpretació en el domini freqüencial. Aquest comportament asimptòtic permet adaptar la solució proposada a entorns selectius en freqüència fent ús d'un prefix cíclic i estudiar una modulació eficient derivada de l'esquema de multiplexat per divisió de temps emprant formes d'ona circulant. Finalment, s’estudia l'impacte de l'ús de múltiples antenes en comunicacions oportunistes basades en l'espai nul. L'anàlisi realitzada permet concloure que, en cap cas, l'estructura de les agrupacions d'antenes tenen un impacte sobre la comunicació oportunista, ja que la forma d'ona proposada imita el comportament d'un transmissor mono-antena. D'altra banda, el nombre de sensors emprat es tradueix en una millora en termes de SIR.(Español) El incremento de los dispositivos inalámbricos que ofrecen servicios de alta velocidad de datos limita la coexistencia de sistemas inalámbricos que comparten los mismos recursos en un área geográfica dada a causa de la interferencia inter-sistema. Por tanto, la gestión de interferencia juega un papel fundamental para facilitar la coexistencia de varios servicios de comunicación heterogéneos. Sin embargo, las estrategias clásicas de gestión de interferencia requieren información lateral originando la necesidad de coordinación y cooperación entre sistemas, que no siempre es práctica. Las comunicaciones oportunistas ofrecen una solución potencial al problema de la gestión de las interferencias entre sistemas. El principio básico de las comunicaciones oportunistas es explotar de manera eficiente y robusta los recursos disponibles en una red inalámbricas y adaptar las señales transmitidas al estado de la red para evitar interferencias entre sistemas. Por lo tanto, las comunicaciones oportunistas dependen de la inferencia de los recursos de red disponibles que pueden ser explotados de manera segura sin inducir interferencia en los nodos de comunicación coexistentes. Una vez identificados los recursos disponibles, las técnicas de comunicación oportunistas más prominentes consisten en el diseño de estrategias de precodificación/descodificación adaptadas al escenario para explotar el llamado espacio nulo. A pesar de esto, las soluciones clásicas en la literatura sufren dos inconvenientes principales: la falta de robustez a los errores de detección y la necesidad de cooperación intra-sistema. Esta tesis propone diseñar un esquema de comunicación oportunista basado en el espacio nulo que afronta los inconvenientes expuestos por las metodologías existentes asumiendo que los nodos oportunistas no cooperan. Para este propósito, se introduce un modelo generalizado de error de detección independiente del mecanismo de identificación del espacio nulo que permite el diseño de soluciones que exhiben interferencias mínimas entre sistemas en el caso peor. Estas soluciones responden a un criterio de máxima relación de señal a interferencia (SIR), que es óptimo en condiciones de no cooperación. La metodología propuesta permite diseñar una familia de formas de onda ortonormales que realizan un spreading de los símbolos modulados dentro del espacio nulo detectado, que es clave para minimizar la densidad de interferencia inducida. Las soluciones propuestas son invariantes dentro del espacio nulo inferido, permitiendo suprimir el enlace de retroalimentación sin renunciar a la detección coherente de forma de onda. En ausencia de coordinación, el diseño de la forma de onda se basa únicamente en la información del estado de la red detectada localmente, induciendo un desajuste entre los espacios nulos identificados por el transmisor y receptor que puede empeorar el rendimiento del sistema. A pesar de que la solución propuesta es robusta a este desajuste, también se estudia el diseño de receptores mejorados usando técnicas de detección de subespacio activo. Cuando el número total de recursos de red aumenta arbitrariamente, las soluciones propuestas tienden a ser combinaciones lineales de exponenciales complejas, proporcionando una interpretación en el dominio frecuencial. Este comportamiento asintótico permite adaptar la solución propuesta a canales selectivos en frecuencia mediante un prefijo cíclico y estudiar una modulación eficiente derivada del esquema de multiplexado por división de tiempo empleando formas de onda circulante. Finalmente, se estudia el impacto del uso de múltiples antenas en comunicaciones oportunistas basadas en el espacio nulo. El análisis realizado revela que la estructura de las agrupaciones de antenas no afecta la comunicación oportunista, ya que la forma de onda propuesta imita el comportamiento de un transmisor mono-antena. Por otro lado, el número de sensores empleado se traduce en una mejora en términos de SIR.Postprint (published version

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more