2 research outputs found
Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations
Maximum entropy models are increasingly being used to describe the collective
activity of neural populations with measured mean neural activities and
pairwise correlations, but the full space of probability distributions
consistent with these constraints has not been explored. We provide upper and
lower bounds on the entropy for the {\em minimum} entropy distribution over
arbitrarily large collections of binary units with any fixed set of mean values
and pairwise correlations. We also construct specific low-entropy distributions
for several relevant cases. Surprisingly, the minimum entropy solution has
entropy scaling logarithmically with system size for any set of first- and
second-order statistics consistent with arbitrarily large systems. We further
demonstrate that some sets of these low-order statistics can only be realized
by small systems. Our results show how only small amounts of randomness are
needed to mimic low-order statistical properties of highly entropic
distributions, and we discuss some applications for engineered and biological
information transmission systems.Comment: 34 pages, 7 figure