Latency-Distortion Tradeoffs in Communicating Classification Results over Noisy Channels

In this work, the problem of communicating decisions of a classifier over a noisy channel is considered. With machine learning based models being used in variety of time-sensitive applications, transmission of these decisions in a reliable and timely manner is of significant importance. To this end, we study the scenario where a probability vector (representing the decisions of a classifier) at the transmitter, needs to be transmitted over a noisy channel. Assuming that the distortion between the original probability vector and the reconstructed one at the receiver is measured via f-divergence, we study the trade-off between transmission latency and the distortion. We completely analyze this trade-off using uniform, lattice, and sparse lattice-based quantization techniques to encode the probability vector by first characterizing bit budgets for each technique given a requirement on the allowed source distortion. These bounds are then combined with results from finite-blocklength literature to provide a framework for analyzing the effects of both quantization distortion and distortion due to decoding error probability (i.e., channel effects) on the incurred transmission latency. Our results show that there is an interesting interplay between source distortion (i.e., distortion for the probability vector measured via f-divergence) and the subsequent channel encoding/decoding parameters; and indicate that a joint design of these parameters is crucial to navigate the latency-distortion tradeoff. We study the impact of changing different parameters (e.g. number of classes, SNR, source distortion) on the latency-distortion tradeoff and perform experiments on AWGN and fading channels. Our results indicate that sparse lattice-based quantization is the most effective at minimizing latency across various regimes and for sparse, high-dimensional probability vectors (i.e., high number of classes).Comment: Submitted to IEEE Transactions on Communication

Trustworthy Actionable Perturbations

Counterfactuals, or modified inputs that lead to a different outcome, are an important tool for understanding the logic used by machine learning classifiers and how to change an undesirable classification. Even if a counterfactual changes a classifier's decision, however, it may not affect the true underlying class probabilities, i.e. the counterfactual may act like an adversarial attack and ``fool'' the classifier. We propose a new framework for creating modified inputs that change the true underlying probabilities in a beneficial way which we call Trustworthy Actionable Perturbations (TAP). This includes a novel verification procedure to ensure that TAP change the true class probabilities instead of acting adversarially. Our framework also includes new cost, reward, and goal definitions that are better suited to effectuating change in the real world. We present PAC-learnability results for our verification procedure and theoretically analyze our new method for measuring reward. We also develop a methodology for creating TAP and compare our results to those achieved by previous counterfactual methods.Comment: Accepted at the 41st International Conference on Machine Learning (ICML) 202

Generalization Bounds for Neural Belief Propagation Decoders

Machine learning based approaches are being increasingly used for designing decoders for next generation communication systems. One widely used framework is neural belief propagation (NBP), which unfolds the belief propagation (BP) iterations into a deep neural network and the parameters are trained in a data-driven manner. NBP decoders have been shown to improve upon classical decoding algorithms. In this paper, we investigate the generalization capabilities of NBP decoders. Specifically, the generalization gap of a decoder is the difference between empirical and expected bit-error-rate(s). We present new theoretical results which bound this gap and show the dependence on the decoder complexity, in terms of code parameters (blocklength, message length, variable/check node degrees), decoding iterations, and the training dataset size. Results are presented for both regular and irregular parity-check matrices. To the best of our knowledge, this is the first set of theoretical results on generalization performance of neural network based decoders. We present experimental results to show the dependence of generalization gap on the training dataset size, and decoding iterations for different codes.Comment: Published in IEEE Transactions on Information Theory (2024

Collagen and Elastic Fiber Content Correlation Analysis between Horizontal and Vertical Orientations of Skin Samples of Human Body

Background. Unequal distribution of dermal collagen and elastic fibers in different orientations of skin is reported to be one of the multifocal causes of scar related complications. Present study is to understand the correlation pattern between collagen in horizontal (CH) and in vertical (CV) directions as well as that of elastic in horizontal (EH) and vertical (EV) directions.Materials and Method. A total of 320 skin samples were collected in two orientations from suprascapular, anterior chest, lateral chest, anterior abdominal wall, and inguinal regions of 32 human cadavers. Spearman correlation coefficient (r) was calculated between the variables (CH,CV,EH, andEV).Results. Significant positive correlation betweenCHandCV, and betweenEHandEVobserved in all 5 areas tested. A negative correlation betweenCVandEVat suprascapular, lateral chest, and inguinal regions and negative correlation betweenCHandEHat anterior chest and anterior abdominal wall have been identified.Conclusion. Knowledge of asymmetric content of dermal collagen and elastic fibers together with the varied strength and degree of association in the given area provides guidelines to the dermatologists and aesthetic surgeons in placing elective incisions in the direction maximally utilizing the anatomical facts for aesthetically pleasing result.</jats:p

Deep Learning for Generative Adversarial Networks and Change Detection

The generalization capability of deep neural networks has led to an increase in its utilization for complex tasks across a wide array of applications, ranging from image classification, computer vision, cybersecurity, and healthcare. In this thesis, we look at applications of deep learning based techniques for change detection and generative modeling. The goal of this thesis is two-fold: (a) to provide quantitative measures for evaluating the performance of generative models, and; (b) to develop unsupervised algorithms to detect changes in time series data. Generative Adversarial Networks (GANs) are a popular framework that train two neural networks in an adversarial manner to generate synthetic samples that follow the distribution of input data. While the performance of GANs has been found to be better than other generative models in terms of the quality of the samples, they often suffer from the problem of mode collapse, i.e., synthetic samples tend to lack the diversity present in original data. Many approaches have been proposed to alleviate this phenomenon in GANs. The first contribution of this thesis are quantitative metrics that capture the extent of mode collapse, as well as the sample quality. The second contribution of this thesis is to devise an unsupervised algorithm for change detection. The proposed approach leverages deep learning based methods to estimate likelihood ratios between samples from two distributions. Subsequently, this methodology is used to devise an unsupervised change detection statistic. We also provide generalization of this framework to detect multiple changes, and for the online setting. We validate the performance of our approach using both synthetic and real-world datasets.Release after 05/21/202

Detection and Decoding: New Learning-Based Algorithms and Their Fundamental Limits

The ever-increasing demand for low-latency, higher reliability, and enhanced coverage in next-generation mobile communications, particularly for applications such as the Internet of Things and autonomous driving, necessitates the development of new algorithms that improve performance or reduce the complexity of contemporary approaches. Achieving these objectives relies on advancements in areas such as low-complexity modulation, coding and decoding schemes, detection algorithms, and beamforming methods. This thesis aims to address these needs through innovations in machine learning for channel coding and detection. First, we theoretically investigate neural belief propagation (NBP) decoders' ability to enhance decoding performance. Our theoretical and empirical analysis reveals the dependence of NBP decoders' generalization capabilities on factors such as code parameters, decoding iterations, and training dataset size. These insights are crucial for optimizing the design and training of NBP decoders for practical applications. Second, we propose Sparse Matrix Codes (SMC), a novel channel coding technique tailored for ultra-reliable low-latency communications. SMC maps message bits to a sparse matrix, which is then multiplied by a spreading matrix and transmitted over the communication channel. By adopting tools from compressed sensing, we derive a low-complexity decoding algorithm to effectively recover the message from the channel output. Lastly, we introduce DRE-CUSUM, a machine learning-based change detection method for high-dimensional data with unknown distribution parameters. The core idea is to estimate the density ratio before and after a split point in a time series, using a non-parametric model to detect changes. We provide theoretical justification and accuracy guarantees for the proposed approach, highlighting its robustness and reliability