A learning rule balancing energy consumption and information maximization in a feed-forward neuronal network

Information measures are often used to assess the efficacy of neural networks, and learning rules can be derived through optimization procedures on such measures. In biological neural networks, computation is restricted by the amount of available resources. Considering energy restrictions, it is thus reasonable to balance information processing efficacy with energy consumption. Here, we studied networks of non-linear Hawkes neurons and assessed the information flow through these networks using mutual information. We then applied gradient descent for a combination of mutual information and energetic costs to obtain a learning rule. Through this procedure, we obtained a rule containing a sliding threshold, similar to the Bienenstock-Cooper-Munro rule. The rule contains terms local in time and in space plus one global variable common to the whole network. The rule thus belongs to so-called three-factor rules and the global variable could be related to a number of biological processes. In neural networks using this learning rule, frequent inputs get mapped onto low energy orbits of the network while rare inputs aren't learned

Sum rate analysis of multiple-access neuro-spike communication channel with dynamic spiking threshold

© 2019 Elsevier B.V. The information from outside world is encoded into spikes by the sensory neurons. These spikes are further propagated to different brain regions through various neural pathways. In the cortical region, each neuron receives inputs from multiple neurons that change its membrane potential. If the accumulated change in the membrane potential is more than a threshold value, a spike is generated. According to various studies in neuroscience, this spiking threshold adapts with time depending on the previous spike. This causes short-term changes in the neural responses giving rise to short-term plasticity. Therefore, in this paper, we analyze a multiple-input single-output (MISO) neuro-spike communication channel and study the effects of dynamic spiking threshold on mutual information and maximum achievable sum rate of the channel. Since spike generation consumes a generous portion of the metabolic energy provided to the brain, we further put metabolic constraint in calculating the mutual information and find a trade-off between maximum achievable sum rate and metabolic energy consumed. Moreover, we analyze three types of neurons present in the cortical region, i.e., Regular spiking, Intrinsic bursting and Fast spiking neurons. We aim to characterize these neurons in terms of encoding/transmission rates and energy expenditure. It will provide a guideline for the practical implementation of bio-inspired nanonetworks as well as for the development of ICT-based diagnosis and treatment techniques for neural diseases.This work was supported in part by European Research Council (ERC) under grant ERC-2013-CoG 616922 (Project MINERVA) and ERC-2017-PoC 780645 (ERC Proof of Concept project MINRGRACE)

Information Theory is abused in neuroscience

In 1948, Claude Shannon introduced his version of a concept that was core to Norbert Wiener's cybernetics, namely, information theory. Shannon's formalisms include a physical framework, namely a general communication system having six unique elements. Under this framework, Shannon information theory offers two particularly useful statistics, channel capacity and information transmitted. Remarkably, hundreds of neuroscience laboratories subsequently reported such numbers. But how (and why) did neuroscientists adapt a communications-engineering framework? Surprisingly, the literature offers no clear answers. To therefore first answer "how", 115 authoritative peer-reviewed papers, proceedings, books and book chapters were scrutinized for neuroscientists' characterizations of the elements of Shannon's general communication system. Evidently, many neuroscientists attempted no identification of the system's elements. Others identified only a few of Shannon's system's elements. Indeed, the available neuroscience interpretations show a stunning incoherence, both within and across studies. The interpretational gamut implies hundreds, perhaps thousands, of different possible neuronal versions of Shannon's general communication system. The obvious lack of a definitive, credible interpretation makes neuroscience calculations of channel capacity and information transmitted meaningless. To now answer why Shannon's system was ever adapted for neuroscience, three common features of the neuroscience literature were examined: ignorance of the role of the observer, the presumption of "decoding" of neuronal voltage-spike trains, and the pursuit of ingrained analogies such as information, computation, and machine. Each of these factors facilitated a plethora of interpretations of Shannon's system elements. Finally, let us not ignore the impact of these "informational misadventures" on society at large. It is the same impact as scientific fraud