A Sphere-Packing Error Exponent for Mismatched Decoding

We derive a sphere-packing error exponent for coded transmission over discrete memoryless channels with a fixed decoding metric. By studying the error probability of the code over an auxiliary channel, we find a lower bound to the probability of error of mismatched decoding. The bound is shown to decay exponentially for coding rates smaller than a new upper bound to the mismatch capacity. For rates higher than the new upper bound, the error probability is shown to be bounded away from zero. The new upper bound is shown to improve over previous upper bounds to the mismatch capacity

Let's be Honest: An Optimal No-Regret Framework for Zero-Sum Games

We revisit the problem of solving two-player zero-sum games in the decentralized setting. We propose a simple algorithmic framework that simultaneously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game. We achieve this goal in three steps. First, we provide a novel analysis of the optimistic mirror descent (OMD), showing that it can be modified to guarantee fast convergence for both honest regret and value of the game, when the players are playing collaboratively. Second, we propose a new algorithm, dubbed as robust optimistic mirror descent (ROMD), which attains optimal adversarial regret without knowing the time horizon beforehand. Finally, we propose a simple signaling scheme, which enables us to bridge OMD and ROMD to achieve the best of both worlds. Numerical examples are presented to support our theoretical claims and show that our non-adaptive ROMD algorithm can be competitive to OMD with adaptive step-size selection.Comment: Proceedings of the 35th International Conference on Machine Learnin

Let’s be honest: An optimal no-regret framework for zero-sum games

We revisit the problem of solving two-player zero- sum games in the decentralized setting. We pro- pose a simple algorithmic framework that simulta- neously achieves the best rates for honest regret as well as adversarial regret, and in addition resolves the open problem of removing the logarithmic terms in convergence to the value of the game. We achieve this goal in three steps. First, we provide a novel analysis of the optimistic mirror descent (OMD), showing that it can be modified to guarantee fast convergence for both honest re- gret and value of the game, when the players are playing collaboratively. Second, we propose a new algorithm, dubbed as robust optimistic mir- ror descent (ROMD), which attains optimal ad- versarial regret without knowing the time horizon beforehand. Finally, we propose a simple signal- ing scheme, which enables us to bridge OMD and ROMD to achieve the best of both worlds. Numerical examples are presented to support our theoretical claims and show that our non-adaptive ROMD algorithm can be competitive to OMD with adaptive step-size selection

Mismatched Decoding: Capacity and Error Exponent Upper Bounds

Mismatched decoding models the coded communication problem in scenarios where a sub-optimal decoder is used. These situations arise when optimal maximum-likelihood decoding cannot be used: i) the channel transition is unknown and imperfectly estimated or, ii) when, for complexity reasons, the channel likelihood is too difficult to compute and an alternative decoding metric is needed. In addition, important problems in information theory such as zero-error capacity or erasures-only capacity can be cast as instances of mismatched decoding. In the mismatched decoding problem, the optimal maximum-likelihood decoder is replaced by a maximum metric decoder, where the metric is not necessarily the channel likelihood. This thesis studies the problem of channel coding with mismatched decoding. Single-letter characterization of mismatch capacity is a long standing open problem in information theory. Lower bounds for the mismatch capacity have been studied extensively using random coding techniques. On the other hand, upperbounds for the mismatch capacity were not explored until a few years ago. In this thesis we study a novel technique for tackling this problem based on analysing the probability of error of a given codebook using an auxiliary channel and a hypothetical decoder. Careful design of the auxiliary channel leads to a lower bound on the probability of error of the codebook in the original mismatched decoding problem. The first version of our upper bound based on the mentioned technique studies the case where we design the auxiliary channel with the following property. When a codeword is sent over the auxiliary channel and the hypothetical decoder makes an error then the mismatched decoder makes an error for sure. The results is this part are mainly derived using the method of types and graph theory. Moreover, the resulting bound is shown to be computable using convex optimization techniques. A simple algorithm is introduced and is shown to converge to the derived bound. A multiletter version of the bound is studied and is shown to yield no improvement over the single-letter bound. An implication of the bound for binary-input channels is derived which provides a simple sufficient condition for the mismatch capacity being strictly less than the matched capacity. Finally, using some examples the application of the bound is illustrated. A generalized version of the mentioned method is used to derive an upper bound to the reliability function. In contrast to the previous part, when a codeword is sent over the auxiliary channel and the hypothetical decoder makes an error, the mismatched decoder makes an error with a positive probability. Moreover, this probability is conditioned to be bounded away from zero by the inverse of a polynomial function of the block length. This is a strictly weaker design assumption for the auxiliary channel and the hypothetical decoder compared to the previous one. Nevertheless, the rate that the reliability functions becomes equal to zero is shown to be an upper bound to the mismatch capacity. To prove the validity of this upper bound on the reliability function the method of types and graph theory are mainly used. The optimization problem for computing this upper bound is shown to be non-convex. This upper bound is compared to the existing upper bounds on the mismatch capacity and is shown to improve over the recent ones. Using several numerical examples the application of the bound is illustrated

A sphere-packing exponent for mismatched decoding

Comunicació presentada a 2021 IEEE International Symposium on Information Theory (ISIT), celebrat del 12 al 20 de juliol de 2021 de manera virtual.We derive a sphere-packing error exponent for mismatched decoding over discrete memoryless channels. We find a lower bound to the probability of error of mismatched decoding that decays exponentially for coding rates smaller than a new upper bound to the mismatch capacity. For rates higher than the new upper bound, the error probability is shown to be bounded away from zero. The new upper bound is shown to improve over previous upper bounds to the mismatch capacity.This work was supported in part by the European Research Council under Grant 725411

Minimum probability of error of list M-ary hypothesis testing

We study a variation of Bayesian M-ary hypothesis testing in which the test outputs a list of L candidates out of the M possible upon processing the observation. We study the minimum error probability of list hypothesis testing, where an error is defined as the event where the true hypothesis is not in the list output by the test. We derive two exact expressions of the minimum probability or error. The first is expressed as the error probability of a certain non-Bayesian binary hypothesis test and is reminiscent of the meta-converse bound by Polyanskiy, Poor and Verdú (2010). The second, is expressed as the tail probability of the likelihood ratio between the two distributions involved in the aforementioned non-Bayesian binary hypothesis test. Hypothesis testing, error probability, information theory.European Research Council (Grant 725411); Spanish Ministry of Economy and Competitiveness (Grant PID2020-116683GB-C22)