Graphical model driven methods in adaptive system identification

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution September 2016Identifying and tracking an unknown linear system from observations of its inputs and outputs is a problem at the heart of many different applications. Due to the complexity and rapid variability of modern systems, there is extensive interest in solving the problem with as little data and computation as possible. This thesis introduces the novel approach of reducing problem dimension by exploiting statistical structure on the input. By modeling the input to the system of interest as a graph-structured random process, it is shown that a large parameter identification problem can be reduced into several smaller pieces, making the overall problem considerably simpler. Algorithms that can leverage this property in order to either improve the performance or reduce the computational complexity of the estimation problem are developed. The first of these, termed the graphical expectation-maximization least squares (GEM-LS) algorithm, can utilize the reduced dimensional problems induced by the structure to improve the accuracy of the system identification problem in the low sample regime over conventional methods for linear learning with limited data, including regularized least squares methods. Next, a relaxation of the GEM-LS algorithm termed the relaxed approximate graph structured least squares (RAGS-LS) algorithm is obtained that exploits structure to perform highly efficient estimation. The RAGS-LS algorithm is then recast into a recursive framework termed the relaxed approximate graph structured recursive least squares (RAGSRLS) algorithm, which can be used to track time-varying linear systems with low complexity while achieving tracking performance comparable to much more computationally intensive methods. The performance of the algorithms developed in the thesis in applications such as channel identification, echo cancellation and adaptive equalization demonstrate that the gains admitted by the graph framework are realizable in practice. The methods have wide applicability, and in particular show promise as the estimation and adaptation algorithms for a new breed of fast, accurate underwater acoustic modems. The contributions of the thesis illustrate the power of graphical model structure in simplifying difficult learning problems, even when the target system is not directly structured.The work in this thesis was supported primarily by the Office of Naval Research through an ONR Special Research Award in Ocean Acoustics; and at various times by the National Science Foundation, the WHOI Academic Programs Office and the MIT Presidential Fellowship Program

Advanced receivers for distributed cooperation in mobile ad hoc networks

Mobile ad hoc networks (MANETs) are rapidly deployable wireless communications systems, operating with minimal coordination in order to avoid spectral efficiency losses caused by overhead. Cooperative transmission schemes are attractive for MANETs, but the distributed nature of such protocols comes with an increased level of interference, whose impact is further amplified by the need to push the limits of energy and spectral efficiency. Hence, the impact of interference has to be mitigated through with the use PHY layer signal processing algorithms with reasonable computational complexity. Recent advances in iterative digital receiver design techniques exploit approximate Bayesian inference and derivative message passing techniques to improve the capabilities of well-established turbo detectors. In particular, expectation propagation (EP) is a flexible technique which offers attractive complexity-performance trade-offs in situations where conventional belief propagation is limited by computational complexity. Moreover, thanks to emerging techniques in deep learning, such iterative structures are cast into deep detection networks, where learning the algorithmic hyper-parameters further improves receiver performance. In this thesis, EP-based finite-impulse response decision feedback equalizers are designed, and they achieve significant improvements, especially in high spectral efficiency applications, over more conventional turbo-equalization techniques, while having the advantage of being asymptotically predictable. A framework for designing frequency-domain EP-based receivers is proposed, in order to obtain detection architectures with low computational complexity. This framework is theoretically and numerically analysed with a focus on channel equalization, and then it is also extended to handle detection for time-varying channels and multiple-antenna systems. The design of multiple-user detectors and the impact of channel estimation are also explored to understand the capabilities and limits of this framework. Finally, a finite-length performance prediction method is presented for carrying out link abstraction for the EP-based frequency domain equalizer. The impact of accurate physical layer modelling is evaluated in the context of cooperative broadcasting in tactical MANETs, thanks to a flexible MAC-level simulato

Recent Advances in Research on Island Phenomena

In natural languages, filler-gap dependencies can straddle across an unbounded distance. Since the 1960s, the term “island” has been used to describe syntactic structures from which extraction is impossible or impeded. While examples from English are ubiquitous, attested counterexamples in the Mainland Scandinavian languages have continuously been dismissed as illusory and alternative accounts for the underlying structure of such cases have been proposed. However, since such extractions are pervasive in spoken Mainland Scandinavian, these languages may not have been given the attention that they deserve in the syntax literature. In addition, recent research suggests that extraction from certain types of island structures in English might not be as unacceptable as previously assumed either. These findings break new empirical ground, question perceived knowledge, and may indeed have substantial ramifications for syntactic theory. This volume provides an overview of state-of-the-art research on island phenomena primarily in English and the Scandinavian languages, focusing on how languages compare to English, with the aim to shed new light on the nature of island constraints from different theoretical perspectives

Blind source separation for interference cancellation in CDMA systems

Communication is the science of "reliable" transfer of information between two parties, in the sense that the information reaches the intended party with as few errors as possible. Modern wireless systems have many interfering sources that hinder reliable communication. The performance of receivers severely deteriorates in the presence of unknown or unaccounted interference. The goal of a receiver is then to combat these sources of interference in a robust manner while trying to optimize the trade-off between gain and computational complexity. Conventional methods mitigate these sources of interference by taking into account all available information and at times seeking additional information e.g., channel characteristics, direction of arrival, etc. This usually costs bandwidth. This thesis examines the issue of developing mitigating algorithms that utilize as little as possible or no prior information about the nature of the interference. These methods are either semi-blind, in the former case, or blind in the latter case. Blind source separation (BSS) involves solving a source separation problem with very little prior information. A popular framework for solving the BSS problem is independent component analysis (ICA). This thesis combines techniques of ICA with conventional signal detection to cancel out unaccounted sources of interference. Combining an ICA element to standard techniques enables a robust and computationally efficient structure. This thesis proposes switching techniques based on BSS/ICA effectively to combat interference. Additionally, a structure based on a generalized framework termed as denoising source separation (DSS) is presented. In cases where more information is known about the nature of interference, it is natural to incorporate this knowledge in the separation process, so finally this thesis looks at the issue of using some prior knowledge in these techniques. In the simple case, the advantage of using priors should at least lead to faster algorithms.reviewe

Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases

Cardiothoracic and pulmonary diseases are a significant cause of mortality and morbidity worldwide. The COVID-19 pandemic has highlighted the lack of access to clinical care, the overburdened medical system, and the potential of artificial intelligence (AI) in improving medicine. There are a variety of diseases affecting the cardiopulmonary system including lung cancers, heart disease, tuberculosis (TB), etc., in addition to COVID-19-related diseases. Screening, diagnosis, and management of cardiopulmonary diseases has become difficult owing to the limited availability of diagnostic tools and experts, particularly in resource-limited regions. Early screening, accurate diagnosis and staging of these diseases could play a crucial role in treatment and care, and potentially aid in reducing mortality. Radiographic imaging methods such as computed tomography (CT), chest X-rays (CXRs), and echo ultrasound (US) are widely used in screening and diagnosis. Research on using image-based AI and machine learning (ML) methods can help in rapid assessment, serve as surrogates for expert assessment, and reduce variability in human performance. In this Special Issue, “Artificial Intelligence in Image-Based Screening, Diagnostics, and Clinical Care of Cardiopulmonary Diseases”, we have highlighted exemplary primary research studies and literature reviews focusing on novel AI/ML methods and their application in image-based screening, diagnosis, and clinical management of cardiopulmonary diseases. We hope that these articles will help establish the advancements in AI

Geodetic Sciences

Space geodetic techniques, e.g., global navigation satellite systems (GNSS), Very Long Baseline Interferometry (VLBI), satellite gravimetry and altimetry, and GNSS Reflectometry & Radio Occultation, are capable of measuring small changes of the Earth�s shape, rotation, and gravity field, as well as mass changes in the Earth system with an unprecedented accuracy. This book is devoted to presenting recent results and development in space geodetic techniques and sciences, including GNSS, VLBI, gravimetry, geoid, geodetic atmosphere, geodetic geophysics and geodetic mass transport associated with the ocean, hydrology, cryosphere and solid-Earth. This book provides a good reference for geodetic techniques, engineers, scientists as well as user community

Character Recognition

Character recognition is one of the pattern recognition technologies that are most widely used in practical applications. This book presents recent advances that are relevant to character recognition, from technical topics such as image processing, feature extraction or classification, to new applications including human-computer interfaces. The goal of this book is to provide a reference source for academic research and for professionals working in the character recognition field

Design and Analysis of A New Illumination Invariant Human Face Recognition System

In this dissertation we propose the design and analysis of a new illumination invariant face recognition system. We show that the multiscale analysis of facial structure and features of face images leads to superior recognition rates for images under varying illumination. We assume that an image I ( x,y ) is a black box consisting of a combination of illumination and reflectance. A new approximation is proposed to enhance the illumination removal phase. As illumination resides in the low-frequency part of images, a high-performance multiresolution transformation is employed to accurately separate the frequency contents of input images. The procedure is followed by a fine-tuning process. After extracting a mask, feature vector is formed and the principal component analysis (PCA) is used for dimensionality reduction which is then proceeded by the extreme learning machine (ELM) as a classifier. We then analyze the effect of the frequency selectivity of subbands of the transformation on the performance of the proposed face recognition system. In fact, we first propose a method to tune the characteristics of a multiresolution transformation, and then analyze how these specifications may affect the recognition rate. In addition, we show that the proposed face recognition system can be further improved in terms of the computational time and accuracy. The motivation for this progress is related to the fact that although illumination mostly lies in the low-frequency part of images, these low-frequency components may have low- or high-resonance nature. Therefore, for the first time, we introduce the resonance based analysis of face images rather than the traditional frequency domain approaches. We found that energy selectivity of the subbands of the resonance based decomposition can lead to superior results with less computational complexity. The method is free of any prior information about the face shape. It is systematic and can be applied separately on each image. Several experiments are performed employing the well known databases such as the Yale B, Extended-Yale B, CMU-PIE, FERET, AT&T, and LFW. Illustrative examples are given and the results confirm the effectiveness of the method compared to the current results in the literature