SuperSpike: Supervised learning in multi-layer spiking neural networks

A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico. Here we revisit the problem of supervised learning in temporally coding multi-layer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three factor learning rule capable of training multi-layer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike-time patterns

Spatial release from masking in children with and without auditory processing disorder in real and virtual auditory environments

Auditory Processing Disorder (APD) is a developmental disorder characterised by difficulties in listening to speech-in-noise despite normal audiometric thresholds. It is still poorly understood and much disputed and there is a need for better diagnostic tools. One promising finding is that some children referred for APD assessment have a reduced spatial release from masking (SRM). Current clinical tests measure SRM in virtual auditory environments created from head-related transfer functions (HRTFs) of a standardised adult head. Adults and children, however, have different head dimensions and mismatched HRTFs are known to affect aspects of binaural hearing like localisation. There has been little research on HRTFs in children and it is unclear whether a large mismatch can impact speech perception, especially for children with APD who have difficulties with accurately processing auditory information. In this project, we examined the effect of nonindividualised virtual auditory environments on the SRM in adults and children with and without APD. The first study with normal-hearing adults compared environments created from individually measured HRTFs and two nonindividualised sets of HRTFs to a real anechoic environment. Speech reception thresholds (SRTs) were measured for target sentences at 0° and two symmetric speech maskers at 0° or ±90° azimuth. No significant effect of auditory environment on SRTs and SRM could be observed. A larger study was then conducted with APD and typically-developing children aged 7 to 12 years. Individual HRTFs were measured for each child. The SRM was measured in environments created from these individualised HRTFs or artificial head HRTFs and in the real anechoic environment. To assess the influence of spectral cues, SRTs were also measured for HRTFs from a spherical head model that only contains interaural time and level differences. Additionally, the study included an extended high-frequency audiogram, a receptive language test and two parental questionnaires. The SRTs of children with APD were worse than those of typically-developing children in all conditions but SRMs were similar. Only small differences in SRTs were found across environments, mainly for the spherical head HRTFs. SRTs in children were higher than in adults but improved with age. APD children also had higher hearing thresholds and performed worse in the language test

Transferrin-polycation conjugates as carriers for DNA uptake into cells.

We have developed a high-efficiency nucleic acid delivery system that uses receptor-mediated endocytosis to carry DNA macromolecules into cells. We accomplished this by conjugating the iron-transport protein transferrin to polycations that bind nucleic acids. Human transferrin, as well as the chicken homologue conalbumin, has been covalently linked to the small DNA-binding protein protamine or to polylysines of various sizes through a disulfide linkage. These modified transferrin molecules maintain their ability to bind their cognate receptor and to mediate efficient iron transport into the cell. The transferrin-polycation molecules form electrophoretically stable complexes with double-stranded DNA, single-stranded DNA, and modified RNA molecules independent of nucleic acid size (from short oligonucleotides to DNA of 21 kilobase pairs). When complexes of transferrin-polycation and a bacterial plasmid DNA containing the gene for Photinus pyralis luciferase are supplied to eukaryotic cells, high-level expression of the luciferase gene occurs, demonstrating transferrin receptor-mediated endocytosis and expression of the imported DNA. We refer to this delivery system as "transferrinfection.

Spatial release of masking in children and adults in non-individualized virtual environments

The spatial release of masking (SRM) is often measured in virtual auditory environments created from head-related transfer functions (HRTFs) of a standardized adult head. Adults and children, however, differ in head dimensions and mismatched HRTFs are known to affect some aspects of binaural hearing. So far, there has been little research on HRTFs in children and it is unclear whether a large mismatch of spatial cues can degrade speech perception in complex environments. In two studies, the effect of non-individualized virtual environments on SRM accuracy in adults and children was examined. The SRMs were measured in virtual environments created from individual and non-individualized HRTFs and the equivalent real anechoic environment. Speech reception thresholds (SRTs) were measured for frontal target sentences and symmetrical speech maskers at 0° or ±90° azimuth. No significant difference between environments was observed for adults. In 7 to 12-year-old children, SRTs and SRMs improved with age, with SRMs approaching adult levels. SRTs differed slightly between environments and were significantly worse in a virtual environment based on HRTFs from a spherical head. Adult HRTFs seem sufficient to accurately measure SRTs in children even in complex listening conditions

Improving equilibrium propagation without weight symmetry through Jacobian homeostasis

Equilibrium propagation (EP) is a compelling alternative to the backpropagation of error algorithm (BP) for computing gradients of neural networks on biological or analog neuromorphic substrates. Still, the algorithm requires weight symmetry and infinitesimal equilibrium perturbations, i.e., nudges, to estimate unbiased gradients efficiently. Both requirements are challenging to implement in physical systems. Yet, whether and how weight asymmetry affects its applicability is unknown because, in practice, it may be masked by biases introduced through the finite nudge. To address this question, we study generalized EP, which can be formulated without weight symmetry, and analytically isolate the two sources of bias. For complex-differentiable non-symmetric networks, we show that the finite nudge does not pose a problem, as exact derivatives can still be estimated via a Cauchy integral. In contrast, weight asymmetry introduces bias resulting in low task performance due to poor alignment of EP's neuronal error vectors compared to BP. To mitigate this issue, we present a new homeostatic objective that directly penalizes functional asymmetries of the Jacobian at the network's fixed point. This homeostatic objective dramatically improves the network's ability to solve complex tasks such as ImageNet 32x32. Our results lay the theoretical groundwork for studying and mitigating the adverse effects of imperfections of physical networks on learning algorithms that rely on the substrate's relaxation dynamics