Optimization of Information Rate Upper and Lower Bounds for Channels with Memory

We consider the problem of minimizing upper bounds and maximizing lower bounds on information rates of stationary and ergodic discrete-time channels with memory. The channels we consider can have a finite number of states, such as partial response channels, or they can have an infinite state-space, such as time-varying fading channels. We optimize recently-proposed information rate bounds for such channels, which make use of auxiliary finite-state machine channels (FSMCs). Our main contribution in this paper is to provide iterative expectation-maximization (EM) type algorithms to optimize the parameters of the auxiliary FSMC to tighten these bounds. We provide an explicit, iterative algorithm that improves the upper bound at each iteration. We also provide an effective method for iteratively optimizing the lower bound. To demonstrate the effectiveness of our algorithms, we provide several examples of partial response and fading channels, where the proposed optimization techniques significantly tighten the initial upper and lower bounds. Finally, we compare our results with an improved variation of the \emph{simplex} local optimization algorithm, called \emph{Soblex}. This comparison shows that our proposed algorithms are superior to the Soblex method, both in terms of robustness in finding the tightest bounds and in computational efficiency. Interestingly, from a channel coding/decoding perspective, optimizing the lower bound is related to increasing the achievable mismatched information rate, i.e., the information rate of a communication system where the decoder at the receiver is matched to the auxiliary channel, and not to the original channel.Comment: Submitted to IEEE Transactions on Information Theory, November 24, 200

A communications system perspective for dynamic mode atomic force microscopy, with applications to high-density storage and nanoimaging

In recent times, the atomic force microscope (AFM) has been used in various fields like biology, chemistry, physics and medicine for obtaining atomic level images. The AFM is a high-resolution microscope which can provide the resolution on the order of fractions of a nanometer. It has applications in the field of material characterization, probe based data storage, nano-imaging etc. The prevalent mode of using the AFM is the static mode where the cantilever is in continuous contact with the sample. This is harsh on the probe and the sample. The problem of probe and sample wear can be partly addressed by using the dynamic mode operation with the high quality factor cantilevers. In the dynamic mode operation, the cantilever is forced sinusoidally using a dither piezo. The oscillating cantilever gently taps the sample which reduces the probe-sample wear. In this dissertation, we demonstrate that viewing the dynamic mode operation from a communication systems perspective can yield huge gains in nano-interrogation speed and fidelity. In the first part of the dissertation, we have considered a data storage system that operates by encoding information as topographic profiles on a polymer medium. A cantilever probe with a sharp tip (few nm radius) is used to create and sense the presence of topographic profiles, resulting in a density of few Tb per square inch. The usage of the static mode is harsh on the probe and the media. In this work, the high quality factor dynamic mode operation, which alleviates the probe-media wear, is analyzed. The read operation is modeled as a communication channel which incorporates system memory due to inter-symbol interference and the cantilever state. We demonstrate an appropriate level of abstraction of this complex nanoscale system that obviates the need for an involved physical model. Next, a solution to the maximum likelihood sequence detection problem based on the Viterbi algorithm is devised. Experimental and simulation results demonstrate that the performance of this detector is several orders of magnitude better than the performance of other existing schemes. In the second part of the dissertation, we have considered another interesting application of the dynamic mode AFM in the field of nano-imaging. Nano-imaging has played a vital role in biology, chemistry and physics as it enables interrogation of material with sub-nanometer resolution. However, current nano-imaging techniques are too slow to be useful in the high speed applications of interest such as studying the evolution of certain biological processes over time that involve very small time scales. In this work, we present a high speed one-bit imaging technique using the dynamic mode AFM with a high quality factor cantilever. We propose a communication channel model for the cantilever based nano-imaging system. Next, we devise an imaging algorithm that incorporates a learned prior from the previous scan line while detecting the features on the current scan line. Experimental results demonstrate that our proposed algorithm provides significantly better image resolution compared to current nano-imaging techniques at high scanning speed. While modeling the probe-based data storage system and the cantilever based nano-imaging system, it has been observed that the channel models exhibit the behavior similar to intersymbol-interference (ISI) channel with data dependent time-correlated noise. The Viterbi algorithm can be adapted for performing maximum likelihood sequence detection in such channels. However, the problem of finding an analytical upper bound on the bit error rate of the Viterbi detector in this case has not been fully investigated. In the third part of the dissertation, we have considered a subset of the class of ISI channels with data dependent Gauss-Markov noise. We derive an upper bound on the pairwise error probability (PEP) between the transmitted bit sequence and the decoded bit sequence that can be expressed as a product of functions depending on current and previous states in the (incorrect) decoded sequence and the (correct) transmitted sequence. In general, the PEP is asymmetric. The average BER over all possible bit sequences is then determined using a pairwise state diagram. Simulations results demonstrate that analytic bound on BER is tight in high SNR regime

EM-Based iterative channel estimation and sequence detection for space-time coded modulation

Reliable detection of signals transmitted over a wireless communication channel requires knowledge of the channel estimate. In this work, the application of expectationmaximization (EM) algorithm to estimation of unknown channel and detection of space-time coded modulation (STCM) signals is investigated. An STCM communication system is presented which includes symbol interleaving at the transmitter and iterative EM-based soft-output decoding at the receiver. The channel and signal model are introduced with a quasi-static and time-varying Rayleigh fading channels considered to evaluate the performance of the communication system. Performance of the system employing Kalman filter with per-survivor processing to do the channel estimation and Viterbi algorithm for sequence detection is used as a reference. The first approach to apply the EM algorithm to channel estimation presents a design of an online receiver with sliding data window. Next, a block-processing EM-based iterative receiver is presented which utilizes soft values of a posteriori probabilities (APP) with maximum a posteriori probability (MAP) as the criterion of optimality in both: detection and channel estimation stages (APP-EM receiver). In addition, a symbol interleaver is introduced at the transmitter which has a great desirable impact on system performance. First, it eliminates error propagation between the detection and channel estimation stages in the receiver EM loop. Second, the interleaver increases the diversity advantage to combat deep fades of a fast fading channel. In the first basic version of the APP-EM iterative receiver, it is assumed that noise variance at the receiver input is known. Then a modified version of the receiver is presented where such assumption is not made. In addition to sequence detection and channel estimation, the EM iteration loop includes the estimation of unknown additive white Gaussian noise variance. Finally, different properties of the APP-EM iterative receiver are investigated including the effects of training sequence length on system performance, interleaver and channel correlation length effects and the performance of the system at different Rayleigh channel fading rates

Fundamental limits in Gaussian channels with feedback: confluence of communication, estimation, and control

The emerging study of integrating information theory and control systems theory has attracted tremendous attention, mainly motivated by the problems of control under communication constraints, feedback information theory, and networked systems. An often overlooked element is the estimation aspect; however, estimation cannot be studied isolatedly in those problems. Therefore, it is natural to investigate systems from the perspective of unifying communication, estimation, and control;This thesis is the first work to advocate such a perspective. To make Matters concrete, we focus on communication systems over Gaussian channels with feedback. For some of these channels, their fundamental limits for communication have been studied using information theoretic methods and control-oriented methods but remain open. In this thesis, we address the problems of characterizing and achieving the fundamental limits for these Gaussian channels with feedback by applying the unifying perspective;We establish a general equivalence among feedback communication, estimation, and feedback stabilization over the same Gaussian channels. As a consequence, we see that the information transmission (communication), information processing (estimation), and information utilization (control), seemingly different and usually separately treated, are in fact three sides of the same entity. We then reveal that the fundamental limitations in feedback communication, estimation, and control coincide: The achievable communication rates in the feedback communication problems can be alternatively given by the decay rates of the Cramer-Rao bounds (CRB) in the associated estimation problems or by the Bode sensitivity integrals in the associated control problems. Utilizing the general equivalence, we design optimal feedback communication schemes based on the celebrated Kalman filtering algorithm; these are the first deterministic, optimal communication schemes for these channels with feedback (except for the degenerated AWGN case). These schemes also extend the Schalkwijk-Kailath (SK) coding scheme and inherit its useful features, such as reduced coding complexity and improved performance. Hence, this thesis demonstrates that the new perspective plays a significant role in gaining new insights and new results in studying Gaussian feedback communication systems. We anticipate that the perspective could be extended to more general problems and helpful in building a theoretically and practically sound paradigm that unifies information, estimation, and control

Neural-Kalman Schemes for Non-Stationary Channel Tracking and Learning

This Thesis focuses on channel tracking in Orthogonal Frequency-Division Multiplexing (OFDM), a widely-used method of data transmission in wireless communications, when abrupt changes occur in the channel. In highly mobile applications, new dynamics appear that might make channel tracking non-stationary, e.g. channels might vary with location, and location rapidly varies with time. Simple examples might be the di erent channel dynamics a train receiver faces when it is close to a station vs. crossing a bridge vs. entering a tunnel, or a car receiver in a route that grows more tra c-dense. Some of these dynamics can be modelled as channel taps dying or being reborn, and so tap birth-death detection is of the essence. In order to improve the quality of communications, we delved into mathematical methods to detect such abrupt changes in the channel, such as the mathematical areas of Sequential Analysis/ Abrupt Change Detection and Random Set Theory (RST), as well as the engineering advances in Neural Network schemes. This knowledge helped us nd a solution to the problem of abrupt change detection by informing and inspiring the creation of low-complexity implementations for real-world channel tracking. In particular, two such novel trackers were created: the Simpli- ed Maximum A Posteriori (SMAP) and the Neural-Network-switched Kalman Filtering (NNKF) schemes. The SMAP is a computationally inexpensive, threshold-based abrupt-change detector. It applies the three following heuristics for tap birth-death detection: a) detect death if the tap gain jumps into approximately zero (memoryless detection); b) detect death if the tap gain has slowly converged into approximately zero (memory detection); c) detect birth if the tap gain is far from zero. The precise parameters for these three simple rules can be approximated with simple theoretical derivations and then ne-tuned through extensive simulations. The status detector for each tap using only these three computationally inexpensive threshold comparisons achieves an error reduction matching that of a close-to-perfect path death/birth detection, as shown in simulations. This estimator was shown to greatly reduce channel tracking error in the target Signal-to-Noise Ratio (SNR) range at a very small computational cost, thus outperforming previously known systems. The underlying RST framework for the SMAP was then extended to combined death/birth and SNR detection when SNR is dynamical and may drift. We analyzed how di erent quasi-ideal SNR detectors a ect the SMAP-enhanced Kalman tracker's performance. Simulations showed SMAP is robust to SNR drift in simulations, although it was also shown to bene t from an accurate SNR detection. The core idea behind the second novel tracker, NNKFs, is similar to the SMAP, but now the tap birth/death detection will be performed via an arti cial neuronal network (NN). Simulations show that the proposed NNKF estimator provides extremely good performance, practically identical to a detector with 100% accuracy. These proposed Neural-Kalman schemes can work as novel trackers for multipath channels, since they are robust to wide variations in the probabilities of tap birth and death. Such robustness suggests a single, low-complexity NNKF could be reusable over di erent tap indices and communication environments. Furthermore, a di erent kind of abrupt change was proposed and analyzed: energy shifts from one channel tap to adjacent taps (partial tap lateral hops). This Thesis also discusses how to model, detect and track such changes, providing a geometric justi cation for this and additional non-stationary dynamics in vehicular situations, such as road scenarios where re ections on trucks and vans are involved, or the visual appearance/disappearance of drone swarms. An extensive literature review of empirically-backed abrupt-change dynamics in channel modelling/measuring campaigns is included. For this generalized framework of abrupt channel changes that includes partial tap lateral hopping, a neural detector for lateral hops with large energy transfers is introduced. Simulation results suggest the proposed NN architecture might be a feasible lateral hop detector, suitable for integration in NNKF schemes. Finally, the newly found understanding of abrupt changes and the interactions between Kalman lters and neural networks is leveraged to analyze the neural consequences of abrupt changes and brie y sketch a novel, abrupt-change-derived stochastic model for neural intelligence, extract some neuro nancial consequences of unstereotyped abrupt dynamics, and propose a new portfolio-building mechanism in nance: Highly Leveraged Abrupt Bets Against Failing Experts (HLABAFEOs). Some communication-engineering-relevant topics, such as a Bayesian stochastic stereotyper for hopping Linear Gauss-Markov (LGM) models, are discussed in the process. The forecasting problem in the presence of expert disagreements is illustrated with a hopping LGM model and a novel structure for a Bayesian stereotyper is introduced that might eventually solve such problems through bio-inspired, neuroscienti cally-backed mechanisms, like dreaming and surprise (biological Neural-Kalman). A generalized framework for abrupt changes and expert disagreements was introduced with the novel concept of Neural-Kalman Phenomena. This Thesis suggests mathematical (Neural-Kalman Problem Category Conjecture), neuro-evolutionary and social reasons why Neural-Kalman Phenomena might exist and found signi cant evidence for their existence in the areas of neuroscience and nance. Apart from providing speci c examples, practical guidelines and historical (out)performance for some HLABAFEO investing portfolios, this multidisciplinary research suggests that a Neural- Kalman architecture for ever granular stereotyping providing a practical solution for continual learning in the presence of unstereotyped abrupt dynamics would be extremely useful in communications and other continual learning tasks.Programa de Doctorado en Multimedia y Comunicaciones por la Universidad Carlos III de Madrid y la Universidad Rey Juan CarlosPresidente: Luis Castedo Ribas.- Secretaria: Ana García Armada.- Vocal: José Antonio Portilla Figuera

Exploiting channel symmetry in two-way channels

Wireless communication is frequently employed for the bi-directional exchange of information between devices with disparate capabilities. Mobile devices such as cellular phones and laptops are subject to stringent size weight and power constraints relative to the base stations or access points they communicate with. Asymmetry of constraints results in asymmetry of the cost of computation. This disparity in cost motivates efforts to shift signal processing such as channel estimation and equalization from cost sensitive devices to those with more relaxed constraints. In this work we develop a two-way training channel estimation and precoding scheme that shifts all the computation involved in dealing with multiplicative channel gain as well as frequency selective channels to the base station. This work demonstrates the first tractable implementation of a precoding based transceiver scheme of this class on the Rice University Wireless Open Access Research Platform (WARP)