Neuron dynamics in the presence of 1/f noise

Interest in understanding the interplay between noise and the response of a non-linear device cuts across disciplinary boundaries. It is as relevant for unmasking the dynamics of neurons in noisy environments as it is for designing reliable nanoscale logic circuit elements and sensors. Most studies of noise in non-linear devices are limited to either time-correlated noise with a Lorentzian spectrum (of which the white noise is a limiting case) or just white noise. We use analytical theory and numerical simulations to study the impact of the more ubiquitous "natural" noise with a 1/f frequency spectrum. Specifically, we study the impact of the 1/f noise on a leaky integrate and fire model of a neuron. The impact of noise is considered on two quantities of interest to neuron function: The spike count Fano factor and the speed of neuron response to a small step-like stimulus. For the perfect (non-leaky) integrate and fire model, we show that the Fano factor can be expressed as an integral over noise spectrum weighted by a (low pass) filter function. This result elucidates the connection between low frequency noise and disorder in neuron dynamics. We compare our results to experimental data of single neurons in vivo, and show how the 1/f noise model provides much better agreement than the usual approximations based on Lorentzian noise. The low frequency noise, however, complicates the case for information coding scheme based on interspike intervals by introducing variability in the neuron response time. On a positive note, the neuron response time to a step stimulus is, remarkably, nearly optimal in the presence of 1/f noise. An explanation of this effect elucidates how the brain can take advantage of noise to prime a subset of the neurons to respond almost instantly to sudden stimuli.Comment: Phys. Rev. E in pres

Neural encoding of behaviourally relevant visual-motion information in the fly

Egelhaaf M, Kern R, Krapp HG, Kretzberg J, Kurtz R, Warzecha A-K. Neural encoding of behaviourally relevant visual-motion information in the fly. Trends in Neurosciences. 2002;25(2):96-102.Information processing in visual systems is constrained by the spatial and temporal characteristics of the sensory input and by the biophysical properties of the neuronal circuits. Hence, to understand how visual systems encode behaviourally relevant information, we need to know about both the computational capabilities of the nervous system and the natural conditions under which animals normally operate. By combining behavioural, neurophysiological and computational approaches, it is now possible in the fly to assess adaptations that process visual-motion information under the constraints of its natural input. It is concluded that neuronal operating ranges and coding strategies appear to be closely matched to the inputs the animal encounters under behaviourally relevant conditions

Effects of Adaptation in a Somatosensory Thalamocortical Circuit

In the mammalian brain, thalamocortical circuits perform the initial stage of processing before information is sent to higher levels of the cerebral cortex. Substantial changes in receptive field properties are produced in the thalamocortical response transformation. In the whisker-to-barrel thalamocortical pathway, the response magnitude of barrel excitatory cells is sensitive to the velocity of whisker deflections, whereas in the thalamus, velocity is only encoded by firing synchrony. The behavior of this circuit can be captured in a model which contains a window of opportunity for thalamic firing synchrony to engage intra-barrel recurrent excitation before being 'damped' by slightly delayed, but strong, local feedforward inhibition. Some remaining aspects of the model that require investigation are: (1) how does adaptation with ongoing and repetitive sensory stimulation affect processing in this circuit and (2) what are the rules governing intra-barrel interactions. By examining sensory processing in thalamic barreloids and cortical barrels, before and after adaptation with repetitive high-frequency whisker stimulation, I have determined that adaptation modifies the operations of the thalamocortical circuit without fundamentally changing it. In the non-adapted state, higher velocities produce larger responses in barrel cells than lower velocities. Similarly, in the adapted barrel, putative excitatory and inhibitory neurons can respond with temporal fidelity to high-frequency whisker deflections if they are of sufficient velocity. Additionally, before and after adaptation, relative to putative excitatory cells, inhibitory cells produce larger responses and are more broadly-tuned for stimulus parameters (e.g., the angle of whisker deflection). In barrel excitatory cells, adaptation is angularly-nonspecific; that is, response suppression is not specific to the angle of the adapting stimulus. The angular tuning of barrel excitatory cells is sharpened and the original angular preference is maintained. This is consistent with intra-barrel interactions being angularly-nonspecific. The maintenance of the original angular preference also suggests that the same thalamocortical inputs determine angular tuning before and after adaptation. In summary, the present findings suggest that adaptation narrows the window of opportunity for synchronous thalamic inputs to engage recurrent excitation so that it can withstand strong, local inhibition. These results from the whisker-to-barrel thalamocortical response transformation are likely to have parallels in other systems

Temporal integration in cochlear implants and the effect of high pulse rates

Although cochlear implants (CIs) have proven to be an invaluable help for many people afflicted with severe hearing loss, there are still many hurdles left before a full restoration of hearing. A better understanding of how individual stimuli in a pulse train interact temporally to form a conjoined percept, and what effects the stimulation rate has on the percept of loudness will be beneficial for further improvements in the development of new coding strategies and thus in the quality of life of CI-wearers. Two experiments presented here deal on the topic of temporal integration with CIs, and raise the question of the effects of the high stimulation rates made possible by the broad spread of stimulation. To this effect, curves of equal loudness were measured as a function of pulse train length for different stimulation characteristics. In the first exploratory experiment, threshold and maximum acceptable loudness (MAL) were measured, and the existence and behaviour of the critical duration of integration in cochlear implants is discussed. In the second experiment, the effect of level was further investigated by including MAL measurements at shorter durations, as well as a line of equal loudness at a comfortable level. It is found that the amount of temporal integration (the slope of integration as a function of duration) is greatly decreased in electrical hearing compared to acoustic hearing. The higher stimulation rates seem to have a compensating effect on this, increasing the slope with increasing rate. The highest rates investigated here lead to slopes that are even comparable to those found in persons with normal hearing and hearing impaired. The rate also has an increasing effect on the dynamic range, which is otherwise taken to be a correlate of good performance. The values presented here point towards larger effects of rate on dynamic range than what has been found so far in the literature for more moderate ranges. While rate effects on threshold, dynamic range and integration slope seem to act uniformly for the different test subjects, the critical duration of integration varies strongly but in a non-consistent way, possibly reflecting more central, individual-specific effects. Additionally, measurements on the voltage spread of human CI-wearers are presented which are used to validate a 3D computational model of the human cochlea developed in our group. The theoretical model falls squarely inside of the distribution of measurements. A single, implant dependent voltage-offset seems to adequately explain most of the variability

Temporal integration in cochlear implants and the effect of high pulse rates

Although cochlear implants (CIs) have proven to be an invaluable help for many people afflicted with severe hearing loss, there are still many hurdles left before a full restoration of hearing. A better understanding of how individual stimuli in a pulse train interact temporally to form a conjoined percept, and what effects the stimulation rate has on the percept of loudness will be beneficial for further improvements in the development of new coding strategies and thus in the quality of life of CI-wearers. Two experiments presented here deal on the topic of temporal integration with CIs, and raise the question of the effects of the high stimulation rates made possible by the broad spread of stimulation. To this effect, curves of equal loudness were measured as a function of pulse train length for different stimulation characteristics. In the first exploratory experiment, threshold and maximum acceptable loudness (MAL) were measured, and the existence and behaviour of the critical duration of integration in cochlear implants is discussed. In the second experiment, the effect of level was further investigated by including MAL measurements at shorter durations, as well as a line of equal loudness at a comfortable level. It is found that the amount of temporal integration (the slope of integration as a function of duration) is greatly decreased in electrical hearing compared to acoustic hearing. The higher stimulation rates seem to have a compensating effect on this, increasing the slope with increasing rate. The highest rates investigated here lead to slopes that are even comparable to those found in persons with normal hearing and hearing impaired. The rate also has an increasing effect on the dynamic range, which is otherwise taken to be a correlate of good performance. The values presented here point towards larger effects of rate on dynamic range than what has been found so far in the literature for more moderate ranges. While rate effects on threshold, dynamic range and integration slope seem to act uniformly for the different test subjects, the critical duration of integration varies strongly but in a non-consistent way, possibly reflecting more central, individual-specific effects. Additionally, measurements on the voltage spread of human CI-wearers are presented which are used to validate a 3D computational model of the human cochlea developed in our group. The theoretical model falls squarely inside of the distribution of measurements. A single, implant dependent voltage-offset seems to adequately explain most of the variability

Efficient hardware implementations of bio-inspired networks

The human brain, with its massive computational capability and power efficiency in small form factor, continues to inspire the ultimate goal of building machines that can perform tasks without being explicitly programmed. In an effort to mimic the natural information processing paradigms observed in the brain, several neural network generations have been proposed over the years. Among the neural networks inspired by biology, second-generation Artificial or Deep Neural Networks (ANNs/DNNs) use memoryless neuron models and have shown unprecedented success surpassing humans in a wide variety of tasks. Unlike ANNs, third-generation Spiking Neural Networks (SNNs) closely mimic biological neurons by operating on discrete and sparse events in time called spikes, which are obtained by the time integration of previous inputs. Implementation of data-intensive neural network models on computers based on the von Neumann architecture is mainly limited by the continuous data transfer between the physically separated memory and processing units. Hence, non-von Neumann architectural solutions are essential for processing these memory-intensive bio-inspired neural networks in an energy-efficient manner. Among the non-von Neumann architectures, implementations employing non-volatile memory (NVM) devices are most promising due to their compact size and low operating power. However, it is non-trivial to integrate these nanoscale devices on conventional computational substrates due to their non-idealities, such as limited dynamic range, finite bit resolution, programming variability, etc. This dissertation demonstrates the architectural and algorithmic optimizations of implementing bio-inspired neural networks using emerging nanoscale devices. The first half of the dissertation focuses on the hardware acceleration of DNN implementations. A 4-layer stochastic DNN in a crossbar architecture with memristive devices at the cross point is analyzed for accelerating DNN training. This network is then used as a baseline to explore the impact of experimental memristive device behavior on network performance. Programming variability is found to have a critical role in determining network performance compared to other non-ideal characteristics of the devices. In addition, noise-resilient inference engines are demonstrated using stochastic memristive DNNs with 100 bits for stochastic encoding during inference and 10 bits for the expensive training. The second half of the dissertation focuses on a novel probabilistic framework for SNNs using the Generalized Linear Model (GLM) neurons for capturing neuronal behavior. This work demonstrates that probabilistic SNNs have comparable perform-ance against equivalent ANNs on two popular benchmarks - handwritten-digit classification and human activity recognition. Considering the potential of SNNs in energy-efficient implementations, a hardware accelerator for inference is proposed, termed as Spintronic Accelerator for Probabilistic SNNs (SpinAPS). The learning algorithm is optimized for a hardware friendly implementation and uses first-to-spike decoding scheme for low latency inference. With binary spintronic synapses and digital CMOS logic neurons for computations, SpinAPS achieves a performance improvement of 4x in terms of GSOPS/W/mm$^2$ when compared to a conventional SRAM-based design. Collectively, this work demonstrates the potential of emerging memory technologies in building energy-efficient hardware architectures for deep and spiking neural networks. The design strategies adopted in this work can be extended to other spike and non-spike based systems for building embedded solutions having power/energy constraints

The role of noise in sensorimotor control

Goal-directed arm movements show stereotypical trajectories, despite the infinite possible ways to reach a given end point. This thesis examines the hypothesis that this stereotypy arises because movements are optimised to reduce the consequences of signal-dependent noise on the motor command. Both experimental and modelling studies demonstrate that signal-dependent noise arises from the normal behaviour of the muscle and motor neuron pool, and has a particular distribution across muscles of different sizes. Specifically, noise decreases in a systematic fashion with increasing muscle strength and motor unit number. Simulations of obstacle avoidance performance in the presence of signal-dependent noise demonstrate that the optimal trajectory for reaching the target accurately and without collision matches the observed trajectories. Isometric force generation is also shown to have systematic changes in variability with posture, which can be explained by the presence of signal-dependent noise in the muscles of the arm. These results confirm the tested hypothesis and imply that consideration of the statistics of action is crucial to human movement planning. To investigate the importance of feedback in the motor system, the impact of static position on motor excitability was examined using transcranial magnetic stimulation and systematic changes in motor evoked potentials were observed. Force generated at the wrist following stimulation was analysed in terms of different possible movement representations, and the differences between force fields arising from stimulation over the cervical spinal cord and from stimulation over primary motor cortex are determined. These results demonstrate the structured influence of proprioceptive feedback on the human motor system. All the experiments are discussed in relation to current theories describing the control of human movements and the impact of noise in the motor system

Neurophysiological mechanisms of sensorimotor recovery from stroke

Ischemic stroke often results in the devastating loss of nervous tissue in the cerebral cortex, leading to profound motor deficits when motor territory is lost, and ultimately resulting in a substantial reduction in quality of life for the stroke survivor. The International Classification of Functioning, Disability and Health (ICF) was developed in 2002 by the World Health Organization (WHO) and provides a framework for clinically defining impairment after stroke. While the reduction of burdens due to neurological disease is stated as a mission objective of the National Institute of Neurological Disorders and Stroke (NINDS), recent clinical trials have been unsuccessful in translating preclinical research breakthroughs into actionable therapeutic treatment strategies with meaningful progress towards this goal. This means that research expanding another NINDS mission is now more important than ever: improving fundamental knowledge about the brain and nervous system in order to illuminate the way forward. Past work in the monkey model of ischemic stroke has suggested there may be a relationship between motor improvements after injury and the ability of the animal to reintegrate sensory and motor information during behavior. This relationship may be subserved by sprouting cortical axonal processes that originate in the spared premotor cortex after motor cortical injury in squirrel monkeys. The axons were observed to grow for relatively long distances (millimeters), significantly changing direction so that it appears that they specifically navigate around the injury site and reorient toward the spared sensory cortex. Critically, it remains unknown whether such processes ever form functional synapses, and if they do, whether such synapses perform meaningful calculations or other functions during behavior. The intent of this dissertation was to study this phenomenon in both intact rats and rats with a focal ischemia in primary motor cortex (M1) contralateral to the preferred forelimb during a pellet retrieval task. As this proved to be a challenging and resource-intensive endeavor, a primary objective of the dissertation became to provide the tools to facilitate such a project to begin with. This includes the creation of software, hardware, and novel training and behavioral paradigms for the rat model. At the same time, analysis of previous experimental data suggested that plasticity in the neural activity of the bilateral motor cortices of rats performing pellet retrievals after focal M1 ischemia may exhibit its most salient changes with respect to functional changes in behavior via mechanisms that were different than initially hypothesized. Specifically, a major finding of this dissertation is the finding that evidence of plasticity in the unit activity of bilateral motor cortical areas of the reaching rat is much stronger at the level of population features. These features exhibit changes in dynamics that suggest a shift in network fixed points, which may relate to the stability of filtering performed during behavior. It is therefore predicted that in order to define recovery by comparison to restitution, a specific type of fixed point dynamics must be present in the cortical population state. A final suggestion is that the stability or presence of these dynamics is related to the reintegration of sensory information to the cortex, which may relate to the positive impact of physical therapy during rehabilitation in the postacute window. Although many more rats will be needed to state any of these findings as a definitive fact, this line of inquiry appears to be productive for identifying targets related to sensorimotor integration which may enhance the efficacy of future therapeutic strategies

A bio-inspired computational model for motion detection

Tese de Doutoramento (Programa Doutoral em Engenharia Biomédica)Last years have witnessed a considerable interest in research dedicated to show that solutions to challenges in autonomous robot navigation can be found by taking inspiration from biology. Despite their small size and relatively simple nervous systems, insects have evolved vision systems able to perform the computations required for a safe navigation in dynamic and unstructured environments, by using simple, elegant and computationally efficient strategies. Thus, invertebrate neuroscience provides engineers with many neural circuit diagrams that can potentially be used to solve complicated engineering control problems. One major and yet unsolved problem encountered by visually guided robotic platforms is collision avoidance in complex, dynamic and inconstant light environments. In this dissertation, the main aim is to draw inspiration from recent and future findings on insect’s collision avoidance in dynamic environments and on visual strategies of light adaptation applied by diurnal insects, to develop a computationally efficient model for robotic control, able to work even in adverse light conditions. We first present a comparative analysis of three leading collision avoidance models based on a neural pathway responsible for signing collisions, the Lobula Giant Movement Detector/Desceding Contralateral Movement Detector (LGMD/DCMD), found in the locust visual system. Models are described, simulated and results are compared with biological data from literature. Due to the lack of information related to the way this collision detection neuron deals with dynamic environments, new visual stimuli were developed. Locusts Lo- custa Migratoria were stimulated with computer-generated discs that traveled along a combination of non-colliding and colliding trajectories, placed over a static and two distinct moving backgrounds, while simultaneously recording the DCMD activity extracellularly. Based on these results, an innovative model was developed. This model was tested in specially designed computer simulations, replicating the same visual conditions used for the biological recordings. The proposed model is shown to be sufficient to give rise to experimentally observed neural insect responses. Using a different approach, and based on recent findings, we present a direct approach to estimate potential collisions through a sequential computation of the image’s power spectra. This approach has been implemented in a real robotic platform, showing that distant dependent variations on image statistics are likely to be functional significant. Maintaining the collision detection performance at lower light levels is not a trivial task. Nevertheless, some insect visual systems have developed several strategies to help them to optimize visual performance over a wide range of light intensities. In this dissertation we address the neural adaptation mechanisms responsible to improve light capture on a day active insect, the bumblebee Bombus Terrestris. Behavioral analyses enabled us to investigate and infer about the spatial and temporal neural summation extent applied by those insects to improve image reliability at the different light levels. As future work, the collision avoidance model may be coupled with a bio-inspired light adaptation mechanism and used for robotic autonomous navigation.Os últimos anos têm testemunhado um aumento progressivo da investigação dedicada a demonstrar que possíveis soluções, para problemas existentes na navegação autónoma de robôs, podem ser encontradas buscando inspiração na biologia. Apesar do reduzido tamanho e da simplicidade do seu sistema nervoso, os insectos possuem sistemas de visão capazes de realizar os cálculos necessários para uma navegação segura em ambientes dinâmicos e não estruturados, por meio de estratégias simples, elegantes e computacionalmente eficientes. Assim, a área da neurociência que se debruça sobre o estudo dos invertebrados fornece, à area da engenharia, uma vasta gama de diagramas de circuitos neurais, que podem ser usados como base para a resolução de problemas complexos. Um atual e notável problema, cujas plataformas robóticas baseadas em sistemas de visão estão sujeitas, é o problema de deteção de colisões em ambientes complexos, dinâmicos e de intensidade luminosa variável. Assim, o objetivo principal do trabalho aqui apresentado é o de procurar inspiração em recentes e futuras descobertas relacionadas com os mecanismos que possibilitam a deteção de colisões em ambientes dinâmicos, bem como nas estratégias visuais de adaptação à luz, aplicadas por insectos diurnos. Numa primeira abordagem é feita uma análise comparativa dos três principais modelos, propostos na literatura, de deteção de colisões, que têm por base o funcionamento dos neurónios Lobular Gigante Detector de Movimento/ Detector de Movimento Descendente Contralateral (LGMD / DCMD), que fazem parte do sistema visual do gafanhoto. Os modelos são descritos, simulados e os resultados são comparados com os dados biológicos existentes, descritos na literatura. Devido à falta de informação relacionada com a forma como estes neurónios detectores de colisões lidam com ambientes dinâmicos, foram desenvolvidos novos estímulos visuais. A estimulação de gafanhotos Locusta Migratoria foi realizada usando-se estímulos controlados, gerados por computador, efectuando diferentes combinações de trajectórias de não-colisão e colisão, colocados sobre um fundo estático e dois fundos dinâmicos. extracelulares do neurónio DCMD. Com base nos resultados obtidos foi possível desenvolver um modelo inovador. Este foi testado sob estímulos visuais desenvolvidos computacionalmente, recriando as mesmas condições visuais usadas aquando dos registos neuronais biológicos. O modelo proposto mostrou ser capaz de reproduzir os resultados neuronais dos gafanhotos, experimentalmente obtidos. Usando uma abordagem diferente, e com base em descobertas recentes, apresentamos uma metodologia mais direta, que possibilita estimar possíveis colisões através de cálculos sequenciais dos espetros de potência das imagens captadas. Esta abordagem foi implementada numa plataforma robótica real, mostrando que, variações estatísticas nas imagens captadas, são susceptíveis de serem funcionalmente significativas. Manter o desempenho da deteção de colisões, em níveis de luz reduzida, não é uma tarefa trivial. No entanto, alguns sistemas visuais de insectos desenvolveram estratégias de forma a optimizar o seu desempenho visual numa larga gama de intensidades luminosas. Nesta dissertação, os mecanismos de adaptação neuronais, responsáveis pela melhoraria de captação de luz num inseto diurno, a abelha Bombus Terrestris, serviram como uma base de estudo. Adaptando análises comportamentais, foi-nos permitido investigar e inferir acerca da extensão dos somatórios neuronais, espaciais e temporais, aplicados por estes insetos, por forma a melhorar a qualidade das imagens captadas a diferentes níveis de luz. Como trabalho futuro, o modelo de deteção de colisões deverá ser acoplado com um mecanismo de adaptação à luz, sendo ambos bio-inspirados, e que possam ser utilizados na navegação robótica autónoma