Optimal design of an unsupervised adaptive classifier with unknown priors

An adaptive detection scheme for M hypotheses was analyzed. It was assumed that the probability density function under each hypothesis was known, and that the prior probabilities of the M hypotheses were unknown and sequentially estimated. Each observation vector was classified using the current estimate of the prior probabilities. Using a set of nonlinear transformations, and applying stochastic approximation theory, an optimally converging adaptive detection and estimation scheme was designed. The optimality of the scheme lies in the fact that convergence to the true prior probabilities is ensured, and that the asymptotic error variance is minimum, for the class of nonlinear transformations considered. An expression for the asymptotic mean square error variance of the scheme was also obtained

Direct-form adaptive equalization for underwater acoustic communication

Submitted in partial fulfillment of the requirements for the degree of Master of Science at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2012Adaptive equalization is an important aspect of communication systems in various environments. It is particularly important in underwater acoustic communication systems, as the channel has a long delay spread and is subject to the effects of time- varying multipath fading and Doppler spreading. The design of the adaptation algorithm has a profound influence on the performance of the system. In this thesis, we explore this aspect of the system. The emphasis of the work presented is on applying concepts from inference and decision theory and information theory to provide an approach to deriving and analyzing adaptation algorithms. Limited work has been done so far on rigorously devising adaptation algorithms to suit a particular situation, and the aim of this thesis is to concretize such efforts and possibly to provide a mathematical basis for expanding it to other applications. We derive an algorithm for the adaptation of the coefficients of an equalizer when the receiver has limited or no information about the transmitted symbols, which we term the Soft-Decision Directed Recursive Least Squares algorithm. We will demonstrate connections between the Expectation-Maximization (EM) algorithm and the Recursive Least Squares algorithm, and show how to derive a computationally efficient, purely recursive algorithm from the optimal EM algorithm. Then, we use our understanding of Markov processes to analyze the performance of the RLS algorithm in hard-decision directed mode, as well as of the Soft-Decision Directed RLS algorithm. We demonstrate scenarios in which the adaptation procedures fail catastrophically, and discuss why this happens. The lessons from the analysis guide us on the choice of models for the adaptation procedure. We then demonstrate how to use the algorithm derived in a practical system for underwater communication using turbo equalization. As the algorithm naturally incorporates soft information into the adaptation process, it becomes easy to fit it into a turbo equalization framework. We thus provide an instance of how to use the information of a turbo equalizer in an adaptation procedure, which has not been very well explored in the past. Experimental data is used to prove the value of the algorithm in a practical context.Support from the agencies that funded this research- the Academic Programs Office at WHOI and the Office of Naval Research (through ONR Grant #N00014-07-10738 and #N00014-10-10259)

Linear MMSE-Optimal Turbo Equalization Using Context Trees

Formulations of the turbo equalization approach to iterative equalization and decoding vary greatly when channel knowledge is either partially or completely unknown. Maximum aposteriori probability (MAP) and minimum mean square error (MMSE) approaches leverage channel knowledge to make explicit use of soft information (priors over the transmitted data bits) in a manner that is distinctly nonlinear, appearing either in a trellis formulation (MAP) or inside an inverted matrix (MMSE). To date, nearly all adaptive turbo equalization methods either estimate the channel or use a direct adaptation equalizer in which estimates of the transmitted data are formed from an expressly linear function of the received data and soft information, with this latter formulation being most common. We study a class of direct adaptation turbo equalizers that are both adaptive and nonlinear functions of the soft information from the decoder. We introduce piecewise linear models based on context trees that can adaptively approximate the nonlinear dependence of the equalizer on the soft information such that it can choose both the partition regions as well as the locally linear equalizer coefficients in each region independently, with computational complexity that remains of the order of a traditional direct adaptive linear equalizer. This approach is guaranteed to asymptotically achieve the performance of the best piecewise linear equalizer and we quantify the MSE performance of the resulting algorithm and the convergence of its MSE to that of the linear minimum MSE estimator as the depth of the context tree and the data length increase.Comment: Submitted to the IEEE Transactions on Signal Processin

Joint Structure Learning of Multiple Non-Exchangeable Networks

Several methods have recently been developed for joint structure learning of multiple (related) graphical models or networks. These methods treat individual networks as exchangeable, such that each pair of networks are equally encouraged to have similar structures. However, in many practical applications, exchangeability in this sense may not hold, as some pairs of networks may be more closely related than others, for example due to group and sub-group structure in the data. Here we present a novel Bayesian formulation that generalises joint structure learning beyond the exchangeable case. In addition to a general framework for joint learning, we (i) provide a novel default prior over the joint structure space that requires no user input; (ii) allow for latent networks; (iii) give an efficient, exact algorithm for the case of time series data and dynamic Bayesian networks. We present empirical results on non-exchangeable populations, including a real data example from biology, where cell-line-specific networks are related according to genomic features.Comment: To appear in Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics (AISTATS

Joint estimation of multiple related biological networks

Graphical models are widely used to make inferences concerning interplay in multivariate systems. In many applications, data are collected from multiple related but nonidentical units whose underlying networks may differ but are likely to share features. Here we present a hierarchical Bayesian formulation for joint estimation of multiple networks in this nonidentically distributed setting. The approach is general: given a suitable class of graphical models, it uses an exchangeability assumption on networks to provide a corresponding joint formulation. Motivated by emerging experimental designs in molecular biology, we focus on time-course data with interventions, using dynamic Bayesian networks as the graphical models. We introduce a computationally efficient, deterministic algorithm for exact joint inference in this setting. We provide an upper bound on the gains that joint estimation offers relative to separate estimation for each network and empirical results that support and extend the theory, including an extensive simulation study and an application to proteomic data from human cancer cell lines. Finally, we describe approximations that are still more computationally efficient than the exact algorithm and that also demonstrate good empirical performance.Comment: Published in at http://dx.doi.org/10.1214/14-AOAS761 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org