On the Convergence of Decentralized Gradient Descent

Consider the consensus problem of minimizing $f(x)=\sum_{i=1}^n f_i(x)$ where each $f_i$ is only known to one individual agent $i$ out of a connected network of $n$ agents. All the agents shall collaboratively solve this problem and obtain the solution subject to data exchanges restricted to between neighboring agents. Such algorithms avoid the need of a fusion center, offer better network load balance, and improve data privacy. We study the decentralized gradient descent method in which each agent $i$ updates its variable $x_{(i)}$, which is a local approximate to the unknown variable $x$, by combining the average of its neighbors' with the negative gradient step $-\alpha \nabla f_i(x_{(i)})$. The iteration is $$x_{(i)}(k+1) \gets \sum_{\text{neighbor} j \text{of} i} w_{ij} x_{(j)}(k) - \alpha \nabla f_i(x_{(i)}(k)),\quad\text{for each agent} i,$$ where the averaging coefficients form a symmetric doubly stochastic matrix $W=[w_{ij}] \in \mathbb{R}^{n \times n}$. We analyze the convergence of this iteration and derive its converge rate, assuming that each $f_i$ is proper closed convex and lower bounded, $\nabla f_i$ is Lipschitz continuous with constant $L_{f_i}$, and stepsize $\alpha$ is fixed. Provided that $\alpha < O(1/L_h)$ where $L_h=\max_i\{L_{f_i}\}$, the objective error at the averaged solution, $f(\frac{1}{n}\sum_i x_{(i)}(k))-f^*$, reduces at a speed of $O(1/k)$ until it reaches $O(\alpha)$. If $f_i$ are further (restricted) strongly convex, then both $\frac{1}{n}\sum_i x_{(i)}(k)$ and each $x_{(i)}(k)$ converge to the global minimizer $x^*$ at a linear rate until reaching an $O(\alpha)$-neighborhood of $x^*$. We also develop an iteration for decentralized basis pursuit and establish its linear convergence to an $O(\alpha)$-neighborhood of the true unknown sparse signal

Constructing living buildings: a review of relevant technologies for a novel application of biohybrid robotics

Biohybrid robotics takes an engineering approach to the expansion and exploitation of biological behaviours for application to automated tasks. Here, we identify the construction of living buildings and infrastructure as a high-potential application domain for biohybrid robotics, and review technological advances relevant to its future development. Construction, civil infrastructure maintenance and building occupancy in the last decades have comprised a major portion of economic production, energy consumption and carbon emissions. Integrating biological organisms into automated construction tasks and permanent building components therefore has high potential for impact. Live materials can provide several advantages over standard synthetic construction materials, including self-repair of damage, increase rather than degradation of structural performance over time, resilience to corrosive environments, support of biodiversity, and mitigation of urban heat islands. Here, we review relevant technologies, which are currently disparate. They span robotics, self-organizing systems, artificial life, construction automation, structural engineering, architecture, bioengineering, biomaterials, and molecular and cellular biology. In these disciplines, developments relevant to biohybrid construction and living buildings are in the early stages, and typically are not exchanged between disciplines. We, therefore, consider this review useful to the future development of biohybrid engineering for this highly interdisciplinary application.publishe

Compressive Spectrum Sensing in Cognitive IoT

PhDWith the rising of new paradigms in wireless communications such as Internet of things (IoT), current static frequency allocation policy faces a primary challenge of spectrum scarcity, and thus encourages the IoT devices to have cognitive capabilities to access the underutilised spectrum in the temporal and spatial dimensions. Wideband spectrum sensing is one of the key functions to enable dynamic spectrum access, but entails a major implementation challenge in terms of sampling rate and computation cost since the sampling rate of analog-to-digital converters (ADCs) should be higher than twice of the spectrum bandwidth based on the Nyquist-Shannon sampling theorem. By exploiting the sparse nature of wideband spectrum, sub-Nyquist sampling and sparse signal recovery have shown potential capabilities in handling these problems, which are directly related to compressive sensing (CS) from the viewpoint of its origin. To invoke sub-Nyquist wideband spectrum sensing in IoT, blind signal acquisition with low-complexity sparse recovery is desirable on compact IoT devices. Moreover, with cooperation among distributed IoT devices, the complexity of sampling and reconstruc- tion can be further reduced with performance guarantee. Specifically, an adaptively- regularized iterative reweighted least squares (AR-IRLS) reconstruction algorithm is proposed to speed up the convergence of reconstruction with less number of iterations. Furthermore, a low-complexity compressive spectrum sensing algorithm is proposed to reduce computation complexity in each iteration of IRLS-based reconstruction algorithm, from cubic time to linear time. Besides, to transfer computation burden from the IoT devices to the core network, a joint iterative reweighted sparse recovery scheme with geo-location database is proposed to adopt the occupied channel information from geo- location database to reduce the complexity in the signal reconstruction. Since numerous IoT devices access or release the spectrum randomly, the sparsity levels of wideband spec-trum signals are varying and unknown. A blind CS-based sensing algorithm is proposed to enable the local secondary users (SUs) to adaptively adjust the sensing time or sam- pling rate without knowledge of spectral sparsity. Apart from the signal reconstruction at the back-end, a distributed sub-Nyquist sensing scheme is proposed by utilizing the surrounding IoT devices to jointly sample the spectrum based on the multi-coset sam- pling theory, in which only the minimum number of low-rate ADCs on the IoT devices are required to form coset samplers without the prior knowledge of the number of occu- pied channels and signal-to-noise ratios. The models of the proposed algorithms are derived and verified by numerical analyses and tested on both real-world and simulated TV white space signals

Sense, Predict, Adapt, Repeat: A Blueprint for Design of New Adaptive AI-Centric Sensing Systems

As Moore's Law loses momentum, improving size, performance, and efficiency of processors has become increasingly challenging, ending the era of predictable improvements in hardware performance. Meanwhile, the widespread incorporation of high-definition sensors in consumer devices and autonomous technologies has fueled a significant upsurge in sensory data. Current global trends reveal that the volume of generated data already exceeds human consumption capacity, making AI algorithms the primary consumers of data worldwide. To address this, a novel approach to designing AI-centric sensing systems is needed that can bridge the gap between the increasing capabilities of high-definition sensors and the limitations of AI processors. This paper provides an overview of efficient sensing and perception methods in both AI and sensing domains, emphasizing the necessity of co-designing AI algorithms and sensing systems for dynamic perception. The proposed approach involves a framework for designing and analyzing dynamic AI-in-the-loop sensing systems, suggesting a fundamentally new method for designing adaptive sensing systems through inference-time AI-to-sensor feedback and end-to-end efficiency and performance optimization