Lower bounds for the average probability of error of estimating a hidden
variable X given an observation of a correlated random variable Y, and Fano's
inequality in particular, play a central role in information theory. In this
paper, we present a lower bound for the average estimation error based on the
marginal distribution of X and the principal inertias of the joint distribution
matrix of X and Y. Furthermore, we discuss an information measure based on the
sum of the largest principal inertias, called k-correlation, which generalizes
maximal correlation. We show that k-correlation satisfies the Data Processing
Inequality and is convex in the conditional distribution of Y given X. Finally,
we investigate how to answer a fundamental question in inference and privacy:
given an observation Y, can we estimate a function f(X) of the hidden random
variable X with an average error below a certain threshold? We provide a
general method for answering this question using an approach based on
rate-distortion theory.Comment: Allerton 2013 with extended proof, 10 page