Interpretability is a pressing issue for machine learning. Common approaches
to interpretable machine learning constrain interactions between features of
the input, rendering the effects of those features on a model's output
comprehensible but at the expense of model complexity. We approach
interpretability from a new angle: constrain the information about the features
without restricting the complexity of the model. Borrowing from information
theory, we use the Distributed Information Bottleneck to find optimal
compressions of each feature that maximally preserve information about the
output. The learned information allocation, by feature and by feature value,
provides rich opportunities for interpretation, particularly in problems with
many features and complex feature interactions. The central object of analysis
is not a single trained model, but rather a spectrum of models serving as
approximations that leverage variable amounts of information about the inputs.
Information is allocated to features by their relevance to the output, thereby
solving the problem of feature selection by constructing a learned continuum of
feature inclusion-to-exclusion. The optimal compression of each feature -- at
every stage of approximation -- allows fine-grained inspection of the
distinctions among feature values that are most impactful for prediction. We
develop a framework for extracting insight from the spectrum of approximate
models and demonstrate its utility on a range of tabular datasets.Comment: project page: https://distributed-information-bottleneck.github.i