On the Minimization of Convex Functionals of Probability Distributions Under Band Constraints

The problem of minimizing convex functionals of probability distributions is solved under the assumption that the density of every distribution is bounded from above and below. A system of sufficient and necessary first-order optimality conditions as well as a bound on the optimality gap of feasible candidate solutions are derived. Based on these results, two numerical algorithms are proposed that iteratively solve the system of optimality conditions on a grid of discrete points. Both algorithms use a block coordinate descent strategy and terminate once the optimality gap falls below the desired tolerance. While the first algorithm is conceptually simpler and more efficient, it is not guaranteed to converge for objective functions that are not strictly convex. This shortcoming is overcome in the second algorithm, which uses an additional outer proximal iteration, and, which is proven to converge under mild assumptions. Two examples are given to demonstrate the theoretical usefulness of the optimality conditions as well as the high efficiency and accuracy of the proposed numerical algorithms.Comment: 13 pages, 5 figures, 2 tables, published in the IEEE Transactions on Signal Processing. In previous versions, the example in Section VI.B contained some mistakes and inaccuracies, which have been fixed in this versio

Adaptive multicoding and robust linear-quadratic receivers for uncertain CDMA frequency-selective fading channels

Wideband Code Division Multiple Access (WCDMA) communications in the presence of channel uncertainty poses a challenging problem with many practical applications in the wireless communications filed. In this dissertation, robust linear-quadratic (LQ) receivers for time-varying, frequency-selective CDMA channels in the presence of uncertainty regarding instantaneous channel state information are proposed and studied. In order to enhance the performance of the LQ receivers, a novel modulation technique adaptive multicoding is employed. We proposed a simple, intuitively appealing cost function the modified deflection ratio that can be maximized to find signal constellations and associated LQ receivers that are optimal in a certain sense. We discuss the properties of the proposed LQ cost function and derive a related adaptive algorithm for the simultaneous design of signals and receivers based on a simple multicoding technique. The Chernoff bound for the LQ receivers is also derived to compensate for the analytical intractability of the probability of bit error. Finally, in order to achieve higher data rate transmission in favorable channels, we extend our approach from binary signals to M-ary signal constellations in a multi-dimension subspace

Signal processing and noisy systems

We try to place noisy systems in signal processing, to evoke some basic ideas and to emphasize some main evolution ways in signal modelization, decision modelization, complex decision systems, a priori information acquisition, decision algorithms.On tente de situer la place des systèmes bruités dans le traitement du signal, d'évoquer quelques concepts de base, et de dégager quelques voies importantes d'évolution dans les domaines de modélisation des signaux, modélisation des décisions, systèmes de décision complexe, acquisition d'information a priori, algorithmes décisionnels

Robust Techniques for Signal Processing: A Survey

Coordinated Science Laboratory was formerly known as Control Systems LaboratoryU.S. Army Research Office / DAAG29-81-K-0062U.S. Air Force Office of Scientific Research / AFOSR 82-0022Joint Services Electronics Program / N00014-84-C-0149National Science Foundation / ECS-82-12080U.S. Office of Naval Research / N00014-80-K-0945 and N00014-81-K-001

Divergence Measures

Data science, information theory, probability theory, statistical learning and other related disciplines greatly benefit from non-negative measures of dissimilarity between pairs of probability measures. These are known as divergence measures, and exploring their mathematical foundations and diverse applications is of significant interest. The present Special Issue, entitled “Divergence Measures: Mathematical Foundations and Applications in Information-Theoretic and Statistical Problems”, includes eight original contributions, and it is focused on the study of the mathematical properties and applications of classical and generalized divergence measures from an information-theoretic perspective. It mainly deals with two key generalizations of the relative entropy: namely, the R_ényi divergence and the important class of f -divergences. It is our hope that the readers will find interest in this Special Issue, which will stimulate further research in the study of the mathematical foundations and applications of divergence measures

Design and Analysis of Optimal and Minimax Robust Sequential Hypothesis Tests

In this dissertation a framework for the design and analysis of optimal and minimax robust sequential hypothesis tests is developed. It provides a coherent theory as well as algorithms for the implementation of optimal and minimax robust sequential tests in practice. After introducing some fundamental concepts of sequential analysis and optimal stopping theory, the optimal sequential test for stochastic processes with Markovian representations is derived. This is done by formulating the sequential testing problem as an optimal stopping problem whose cost function is given by a weighted sum of the expected run-length and the error probabilities of the test. Based on this formulation, a cost minimizing testing policy can be obtained by solving a nonlinear integral equation. It is then shown that the partial generalized derivatives of the optimal cost function are, up to a constant scaling factor, identical to the error probabilities of the cost minimizing test. This relation is used to formulate the problem of designing optimal sequential tests under constraints on the error probabilities as a problem of solving an integral equation under constraints on the partial derivatives of its solution function. Finally, it is shown that the latter problem can be solved by means of standard linear programming techniques without the need to calculate the partial derivatives explicitly. Numerical examples are given to illustrate this procedure. The second half of the dissertation is concerned with the design of minimax robust sequential hypothesis tests. First, the minimax principle and a general model for distributional uncertainties is introduced. Subsequently, sufficient conditions are derived for distributions to be least favorable with respect to the expected run-length and error probabilities of a sequential test. Combining the results on optimal sequential tests and least favorable distributions yields a sufficient condition for a sequential test to be minimax optimal under general distributional uncertainties. The cost function of the minimax optimal test is further identified as a convex statistical similarity measure and the least favorable distributions as the distributions that are most similar with respect to this measure. In order to obtain more specific results, the density band model is introduced as an example for a nonparametric uncertainty model. The corresponding least favorable distributions are stated in an implicit form, based on which a simple algorithm for their numerical calculation is derived. Finally, the minimax robust sequential test under density band uncertainties is discussed and shown to admit the characteristic minimax property of a maximally flat performance profile over its state space. A numerical example for a minimax optimal sequential test completes the dissertation