The capacity with which a system of independent neuron-like units represents
a given set of stimuli is studied by calculating the mutual information between
the stimuli and the neural responses. Both discrete noiseless and continuous
noisy neurons are analyzed. In both cases, the information grows monotonically
with the number of neurons considered. Under the assumption that neurons are
independent, the mutual information rises linearly from zero, and approaches
exponentially its maximum value. We find the dependence of the initial slope on
the number of stimuli and on the sparseness of the representation.Comment: 19 pages, 6 figures, Phys. Rev. E, vol 63, 11910 - 11924 (2000
