Information Theory’s failure in neuroscience: on the limitations of cybernetics

In Cybernetics (1961 Edition), Professor Norbert Wiener noted that “The role of information and the technique of measuring and transmitting information constitute a whole discipline for the engineer, for the neuroscientist, for the psychologist, and for the sociologist”. Sociology aside, the neuroscientists and the psychologists inferred “information transmitted” using the discrete summations from Shannon Information Theory. The present author has since scrutinized the psychologists’ approach in depth, and found it wrong. The neuroscientists’ approach is highly related, but remains unexamined. Neuroscientists quantified “the ability of [physiological sensory] receptors (or other signal-processing elements) to transmit information about stimulus parameters”. Such parameters could vary along a single continuum (e.g., intensity), or along multiple dimensions that altogether provide a Gestalt – such as a face. Here, unprecedented scrutiny is given to how 23 neuroscience papers computed “information transmitted” in terms of stimulus parameters and the evoked neuronal spikes. The computations relied upon Shannon’s “confusion matrix”, which quantifies the fidelity of a “general communication system”. Shannon’s matrix is square, with the same labels for columns and for rows. Nonetheless, neuroscientists labelled the columns by “stimulus category” and the rows by “spike-count category”. The resulting “information transmitted” is spurious, unless the evoked spike-counts are worked backwards to infer the hypothetical evoking stimuli. The latter task is probabilistic and, regardless, requires that the confusion matrix be square. Was it? For these 23 significant papers, the answer is No

Logarithmic distributions prove that intrinsic learning is Hebbian

In this paper, we present data for the lognormal distributions of spike rates, synaptic weights and intrinsic excitability (gain) for neurons in various brain areas, such as auditory or visual cortex, hippocampus, cerebellum, striatum, midbrain nuclei. We find a remarkable consistency of heavy-tailed, specifically lognormal, distributions for rates, weights and gains in all brain areas examined. The difference between strongly recurrent and feed-forward connectivity (cortex vs. striatum and cerebellum), neurotransmitter (GABA (striatum) or glutamate (cortex)) or the level of activation (low in cortex, high in Purkinje cells and midbrain nuclei) turns out to be irrelevant for this feature. Logarithmic scale distribution of weights and gains appears to be a general, functional property in all cases analyzed. We then created a generic neural model to investigate adaptive learning rules that create and maintain lognormal distributions. We conclusively demonstrate that not only weights, but also intrinsic gains, need to have strong Hebbian learning in order to produce and maintain the experimentally attested distributions. This provides a solution to the long-standing question about the type of plasticity exhibited by intrinsic excitability

Emergence of Tuning to Natural Stimulus Statistics along the Central Auditory Pathway

We have previously shown that neurons in primary auditory cortex (A1) of anaesthetized (ketamine/medetomidine) ferrets respond more strongly and reliably to dynamic stimuli whose statistics follow "natural" 1/f dynamics than to stimuli exhibiting pitch and amplitude modulations that are faster (1/f(0.5)) or slower (1/f(2)) than 1/f. To investigate where along the central auditory pathway this 1/f-modulation tuning arises, we have now characterized responses of neurons in the central nucleus of the inferior colliculus (ICC) and the ventral division of the mediate geniculate nucleus of the thalamus (MGV) to 1/f(gamma) distributed stimuli with gamma varying between 0.5 and 2.8. We found that, while the great majority of neurons recorded from the ICC showed a strong preference for the most rapidly varying (1/f(0.5) distributed) stimuli, responses from MGV neurons did not exhibit marked or systematic preferences for any particular gamma exponent. Only in A1 did a majority of neurons respond with higher firing rates to stimuli in which gamma takes values near 1. These results indicate that 1/f tuning emerges at forebrain levels of the ascending auditory pathway

Estimating the Amount of Information Conveyed by a Population of Neurons

Recent advances in electrophysiological recording technology have allowed for the collection of data from large populations of neurons simultaneously. Yet despite these advances, methods for the estimation of the amount of information conveyed by multiple neurons have been stymied by the “curse of dimensionality”–as the number of included neurons increases, so too does the dimensionality of the data necessary for such measurements, leading to an exponential and, therefore, intractible increase in the amounts of data required for valid measurements. Here we put forth a novel method for the estimation of the amount of information transmitted by the discharge of a large population of neurons, a method which exploits the little-known fact that (under certain constraints) the Fourier coefficients of variables such as neural spike trains follow a Gaussian distribution. This fact enables an accurate measure of information even with limited data. The method, which we call the Fourier Method, is presented in detail, tested for robustness, and its application is demonstrated with both simulated and real spike trains. ii

Cortical coding

The encoding of information in the primate inferior temporal visual cortex, hippocampus, orbitofrontal cortex, and insula is described. All these areas have sparse distributed graded firing rate representations. The firing rate probability distribution is close to exponential. The information increases approximately linearly with the number of neurons. Consistent with this relative independence, there is little extra information that is available because of stimulus-dependent synchrony, and little redundancy. The code can be read very fast, in 20–50 ms, by dot-product, biologically plausible decoding. The advantages of this code include high capacity, generalisation, graceful degradation, and rapid readout of the information by biologically plausible dot-product decoding. None of these are properties of local or “grandmother-cell’ representations. Consistent evidence is becoming available for humans. These sparse distributed graded firing rate representations have major computational advantages for the brain including for language that are not met by local or ‘grandmother-cell’ representations

Neurons with stereotyped and rapid responses provide a reference frame for relative temporal coding in primate auditory cortex

The precise timing of spikes of cortical neurons relative to stimulus onset carries substantial sensory information. To access this information the sensory systems would need to maintain an internal temporal reference that reflects the precise stimulus timing. Whether and how sensory systems implement such reference frames to decode time-dependent responses, however, remains debated. Studying the encoding of naturalistic sounds in primate (Macaca mulatta) auditory cortex we here investigate potential intrinsic references for decoding temporally precise information. Within the population of recorded neurons, we found one subset responding with stereotyped fast latencies that varied little across trials or stimuli, while the remaining neurons had stimulus-modulated responses with longer and variable latencies. Computational analysis demonstrated that the neurons with stereotyped short latencies constitute an effective temporal reference for relative coding. Using the response onset of a simultaneously recorded stereotyped neuron allowed decoding most of the stimulus information carried by onset latencies and the full spike train of stimulus-modulated neurons. Computational modeling showed that few tens of such stereotyped reference neurons suffice to recover nearly all information that would be available when decoding the same responses relative to the actual stimulus onset. These findings reveal an explicit neural signature of an intrinsic reference for decoding temporal response patterns in the auditory cortex of alert animals. Furthermore, they highlight a role for apparently unselective neurons as an early saliency signal that provides a temporal reference for extracting stimulus information from other neurons

The Role of Thalamic Population Synchrony in the Emergence of Cortical Feature Selectivity

In a wide range of studies, the emergence of orientation selectivity in primary visual cortex has been attributed to a complex interaction between feed-forward thalamic input and inhibitory mechanisms at the level of cortex. Although it is well known that layer 4 cortical neurons are highly sensitive to the timing of thalamic inputs, the role of the stimulus-driven timing of thalamic inputs in cortical orientation selectivity is not well understood. Here we show that the synchronization of thalamic firing contributes directly to the orientation tuned responses of primary visual cortex in a way that optimizes the stimulus information per cortical spike. From the recorded responses of geniculate X-cells in the anesthetized cat, we synthesized thalamic sub-populations that would likely serve as the synaptic input to a common layer 4 cortical neuron based on anatomical constraints. We used this synchronized input as the driving input to an integrate-and-fire model of cortical responses and demonstrated that the tuning properties match closely to those measured in primary visual cortex. By modulating the overall level of synchronization at the preferred orientation, we show that efficiency of information transmission in the cortex is maximized for levels of synchronization which match those reported in thalamic recordings in response to naturalistic stimuli, a property which is relatively invariant to the orientation tuning width. These findings indicate evidence for a more prominent role of the feed-forward thalamic input in cortical feature selectivity based on thalamic synchronization