Coordinated Deep Neural Networks: A Versatile Edge Offloading Algorithm

As artificial intelligence (AI) applications continue to expand, there is a growing need for deep neural network (DNN) models. Although DNN models deployed at the edge are promising to provide AI as a service with low latency, their cooperation is yet to be explored. In this paper, we consider the DNN service providers share their computing resources as well as their models' parameters and allow other DNNs to offload their computations without mirroring. We propose a novel algorithm called coordinated DNNs on edge (\textbf{CoDE}) that facilitates coordination among DNN services by creating multi-task DNNs out of individual models. CoDE aims to find the optimal path that results in the lowest possible cost, where the cost reflects the inference delay, model accuracy, and local computation workload. With CoDE, DNN models can make new paths for inference by using their own or other models' parameters. We then evaluate the performance of CoDE through numerical experiments. The results demonstrate a $75\%$ reduction in the local service computation workload while degrading the accuracy by only $2\%$ and having the same inference time in a balanced load condition. Under heavy load, CoDE can further decrease the inference time by $30\%$ while the accuracy is reduced by only $4\%$

Cooperative Transmission for Wireless Relay Networks Using Limited Feedback

To achieve the available performance gains in half-duplex wireless relay networks, several cooperative schemes have been earlier proposed using either distributed space-time coding or distributed beamforming for the transmitter without and with channel state information (CSI), respectively. However, these schemes typically have rather high implementation and/or decoding complexities, especially when the number of relays is high. In this paper, we propose a simple low-rate feedback-based approach to achieve maximum diversity with a low decoding and implementation complexity. To further improve the performance of the proposed scheme, the knowledge of the second-order channel statistics is exploited to design long-term power loading through maximizing the receiver signal-to-noise ratio (SNR) with appropriate constraints. This maximization problem is approximated by a convex feasibility problem whose solution is shown to be close to the optimal one in terms of the error probability. Subsequently, to provide robustness against feedback errors and further decrease the feedback rate, an extended version of the distributed Alamouti code is proposed. It is also shown that our scheme can be generalized to the differential transmission case, where it can be applied to wireless relay networks with no CSI available at the receiver.Comment: V1: 27 pages, 1 column, 6 figures. Submitted to IEEE Transactions on Signal Processing, February 2, 2009. V2: 30 pages, 1 column, 8 figures. Revised version submitted to IEEE Transactions on Signal Processing, July 23, 200

Out-Of-Domain Unlabeled Data Improves Generalization

We propose a novel framework for incorporating unlabeled data into semi-supervised classification problems, where scenarios involving the minimization of either i) adversarially robust or ii) non-robust loss functions have been considered. Notably, we allow the unlabeled samples to deviate slightly (in total variation sense) from the in-domain distribution. The core idea behind our framework is to combine Distributionally Robust Optimization (DRO) with self-supervised training. As a result, we also leverage efficient polynomial-time algorithms for the training stage. From a theoretical standpoint, we apply our framework on the classification problem of a mixture of two Gaussians in $\mathbb{R}^d$, where in addition to the $m$ independent and labeled samples from the true distribution, a set of $n$ (usually with $n\gg m$) out of domain and unlabeled samples are given as well. Using only the labeled data, it is known that the generalization error can be bounded by $\propto\left(d/m\right)^{1/2}$. However, using our method on both isotropic and non-isotropic Gaussian mixture models, one can derive a new set of analytically explicit and non-asymptotic bounds which show substantial improvement on the generalization error compared to ERM. Our results underscore two significant insights: 1) out-of-domain samples, even when unlabeled, can be harnessed to narrow the generalization gap, provided that the true data distribution adheres to a form of the ``cluster assumption", and 2) the semi-supervised learning paradigm can be regarded as a special case of our framework when there are no distributional shifts. We validate our claims through experiments conducted on a variety of synthetic and real-world datasets.Comment: Published at ICLR 2024 (Spotlight), 29 pages, no figure

Adaptive access and rate control of CSMA for energy, rate and delay optimization

In this article, we present a cross-layer adaptive algorithm that dynamically maximizes the average utility function. A per stage utility function is defined for each link of a carrier sense multiple access-based wireless network as a weighted concave function of energy consumption, smoothed rate, and smoothed queue size. Hence, by selecting weights we can control the trade-off among them. Using dynamic programming, the utility function is maximized by dynamically adapting channel access, modulation, and coding according to the queue size and quality of the time-varying channel. We show that the optimal transmission policy has a threshold structure versus the channel state where the optimal decision is to transmit when the wireless channel state is better than a threshold. We also provide a queue management scheme where arrival rate is controlled based on the link state. Numerical results show characteristics of the proposed adaptation scheme and highlight the trade-off among energy consumption, smoothed data rate, and link delay.This study was supported in part by the Spanish Government, Ministerio de Ciencia e Innovación (MICINN), under projects COMONSENS (CSD2008-00010, CONSOLIDER-INGENIO 2010 program) and COSIMA (TEC2010-19545-C04-03), in part by Iran Telecommunication Research Center under contract 6947/500, and in part by Iran National Science Foundation under grant number 87041174. This study was completed while M. Khodaian was at CEIT and TECNUN (University of Navarra)