Novel Lower Bounds on the Entropy Rate of Binary Hidden Markov Processes

Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here, this result is applied for deriving novel lower bounds on the entropy rate of binary hidden Markov processes. For symmetric underlying Markov processes, our bound improves upon the best known bound in the very noisy regime. The nonsymmetric case is also considered, and explicit bounds are derived for Markov processes that satisfy the $(1,\infty)$-RLL constraint

How to Quantize nn Outputs of a Binary Symmetric Channel to n−1n-1 Bits?

Suppose that $Y^n$ is obtained by observing a uniform Bernoulli random vector $X^n$ through a binary symmetric channel with crossover probability $\alpha$. The "most informative Boolean function" conjecture postulates that the maximal mutual information between $Y^n$ and any Boolean function $\mathrm{b}(X^n)$ is attained by a dictator function. In this paper, we consider the "complementary" case in which the Boolean function is replaced by $f:\left\{0,1\right\}^n\to\left\{0,1\right\}^{n-1}$, namely, an $n-1$ bit quantizer, and show that $I(f(X^n);Y^n)\leq (n-1)\cdot\left(1-h(\alpha)\right)$ for any such $f$. Thus, in this case, the optimal function is of the form $f(x^n)=(x_1,\ldots,x_{n-1})$.Comment: 5 pages, accepted ISIT 201

Boolean functions: noise stability, non-interactive correlation distillation, and mutual information

Let $T_{\epsilon}$ be the noise operator acting on Boolean functions $f:\{0, 1\}^n\to \{0, 1\}$, where $\epsilon\in[0, 1/2]$ is the noise parameter. Given $\alpha>1$ and fixed mean $\mathbb{E} f$, which Boolean function $f$ has the largest $\alpha$-th moment $\mathbb{E}(T_\epsilon f)^\alpha$? This question has close connections with noise stability of Boolean functions, the problem of non-interactive correlation distillation, and Courtade-Kumar's conjecture on the most informative Boolean function. In this paper, we characterize maximizers in some extremal settings, such as low noise ($\epsilon=\epsilon(n)$ is close to 0), high noise ($\epsilon=\epsilon(n)$ is close to 1/2), as well as when $\alpha=\alpha(n)$ is large. Analogous results are also established in more general contexts, such as Boolean functions defined on discrete torus $(\mathbb{Z}/p\mathbb{Z})^n$ and the problem of noise stability in a tree model.Comment: Corrections of some inaccuracie