3 research outputs found
An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation
Information theory plays an indispensable role in the development of
algorithm-independent impossibility results, both for communication problems
and for seemingly distinct areas such as statistics and machine learning. While
numerous information-theoretic tools have been proposed for this purpose, the
oldest one remains arguably the most versatile and widespread: Fano's
inequality. In this chapter, we provide a survey of Fano's inequality and its
variants in the context of statistical estimation, adopting a versatile
framework that covers a wide range of specific problems. We present a variety
of key tools and techniques used for establishing impossibility results via
this approach, and provide representative examples covering group testing,
graphical model selection, sparse linear regression, density estimation, and
convex optimization.Comment: Chapter in upcoming book "Information-Theoretic Methods in Data
Science" (Cambridge University Press) edited by Yonina Eldar and Miguel
Rodrigues. (v2 & v3) Minor corrections and edit
An Introductory Guide to Fano's Inequality with Applications in Statistical Estimation
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous information-theoretic tools have been proposed for this purpose, the oldest one remains arguably the most versatile and widespread: Fano's inequality. In this chapter, we provide a survey of Fano's inequality and its variants in the context of statistical estimation, adopting a versatile framework that covers a wide range of specic problems. We present a variety of key tools and techniques used for establishing impossibility results via this approach, and provide representative examples covering group testing, graphical model selection, sparse linear regression, density estimation, and convex optimization