A Nonstochastic Information Theory for Communication and State Estimation

In communications, unknown variables are usually modelled as random variables, and concepts such as independence, entropy and information are defined in terms of the underlying probability distributions. In contrast, control theory often treats uncertainties and disturbances as bounded unknowns having no statistical structure. The area of networked control combines both fields, raising the question of whether it is possible to construct meaningful analogues of stochastic concepts such as independence, Markovness, entropy and information without assuming a probability space. This paper introduces a framework for doing so, leading to the construction of a maximin information functional for nonstochastic variables. It is shown that the largest maximin information rate through a memoryless, error-prone channel in this framework coincides with the block-coding zero-error capacity of the channel. Maximin information is then used to derive tight conditions for uniformly estimating the state of a linear time-invariant system over such a channel, paralleling recent results of Matveev and Savkin

Robust Hypothesis Testing with a Relative Entropy Tolerance

This paper considers the design of a minimax test for two hypotheses where the actual probability densities of the observations are located in neighborhoods obtained by placing a bound on the relative entropy between actual and nominal densities. The minimax problem admits a saddle point which is characterized. The robust test applies a nonlinear transformation which flattens the nominal likelihood ratio in the vicinity of one. Results are illustrated by considering the transmission of binary data in the presence of additive noise.Comment: 14 pages, 5 figures, submitted to the IEEE Transactions on Information Theory, July 2007, revised April 200

On the Equivalence of f-Divergence Balls and Density Bands in Robust Detection

The paper deals with minimax optimal statistical tests for two composite hypotheses, where each hypothesis is defined by a non-parametric uncertainty set of feasible distributions. It is shown that for every pair of uncertainty sets of the f-divergence ball type, a pair of uncertainty sets of the density band type can be constructed, which is equivalent in the sense that it admits the same pair of least favorable distributions. This result implies that robust tests under $f$-divergence ball uncertainty, which are typically only minimax optimal for the single sample case, are also fixed sample size minimax optimal with respect to the equivalent density band uncertainty sets.Comment: 5 pages, 1 figure, accepted for publication in the Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 201

Robust Techniques for Signal Processing: A Survey

Coordinated Science Laboratory was formerly known as Control Systems LaboratoryU.S. Army Research Office / DAAG29-81-K-0062U.S. Air Force Office of Scientific Research / AFOSR 82-0022Joint Services Electronics Program / N00014-84-C-0149National Science Foundation / ECS-82-12080U.S. Office of Naval Research / N00014-80-K-0945 and N00014-81-K-001

Advanced Coding And Modulation For Ultra-wideband And Impulsive Noises

The ever-growing demand for higher quality and faster multimedia content delivery over short distances in home environments drives the quest for higher data rates in wireless personal area networks (WPANs). One of the candidate IEEE 802.15.3a WPAN proposals support data rates up to 480 Mbps by using punctured convolutional codes with quadrature phase shift keying (QPSK) modulation for a multi-band orthogonal frequency-division multiplexing (MB-OFDM) system over ultra wideband (UWB) channels. In the first part of this dissertation, we combine more powerful near-Shannon-limit turbo codes with bandwidth efficient trellis coded modulation, i.e., turbo trellis coded modulation (TTCM), to further improve the data rates up to 1.2 Gbps. A modified iterative decoder for this TTCM coded MB-OFDM system is proposed and its bit error rate performance under various impulsive noises over both Gaussian and UWB channel is extensively investigated, especially in mismatched scenarios. A robust decoder which is immune to noise mismatch is provided based on comparison of impulsive noises in time domain and frequency domain. The accurate estimation of the dynamic noise model could be very difficult or impossible at the receiver, thus a significant performance degradation may occur due to noise mismatch. In the second part of this dissertation, we prove that the minimax decoder in \cite, which instead of minimizing the average bit error probability aims at minimizing the worst bit error probability, is optimal and robust to certain noise model with unknown prior probabilities in two and higher dimensions. Besides turbo codes, another kind of error correcting codes which approach the Shannon capacity is low-density parity-check (LDPC) codes. In the last part of this dissertation, we extend the density evolution method for sum-product decoding using mismatched noises. We will prove that as long as the true noise type and the estimated noise type used in the decoder are both binary-input memoryless output symmetric channels, the output from mismatched log-likelihood ratio (LLR) computation is also symmetric. We will show the Shannon capacity can be evaluated for mismatched LLR computation and it can be reduced if the mismatched LLR computation is not an one-to-one mapping function. We will derive the Shannon capacity, threshold and stable condition of LDPC codes for mismatched BIAWGN and BIL noise types. The results show that the noise variance estimation errors will not affect the Shannon capacity and stable condition, but the errors do reduce the threshold. The mismatch in noise type will only reduce Shannon capacity when LLR computation is based on BIL

Bayesian Cognitive Science, Monopoly, and Neglected Frameworks

A widely shared view in the cognitive sciences is that discovering and assessing explanations of cognitive phenomena whose production involves uncertainty should be done in a Bayesian framework. One assumption supporting this modelling choice is that Bayes provides the best approach for representing uncertainty. However, it is unclear that Bayes possesses special epistemic virtues over alternative modelling frameworks, since a systematic comparison has yet to be attempted. Currently, it is then premature to assert that cognitive phenomena involving uncertainty are best explained within the Bayesian framework. As a forewarning, progress in cognitive science may be hindered if too many scientists continue to focus their efforts on Bayesian modelling, which risks to monopolize scientific resources that may be better allocated to alternative approaches