Temporal Convolution in Spiking Neural Networks: a Bio-mimetic Paradigm

Abstract Recent spectacular advances in Artificial Intelligence (AI), in large, be attributed to developments in Deep Learning (DL). In essence, DL is not a new concept. In many respects, DL shares characteristics of “traditional” types of Neural Network (NN). The main distinguishing feature is that it uses many more layers in order to learn increasingly complex features. Each layer convolutes into the previous by simplifying and applying a function upon a subsection of that layer. Deep Learning’s fantastic success can be attributed to dedicated researchers experimenting with many different groundbreaking techniques, but also some of its triumph can also be attributed to fortune. It was the right technique at the right time. To function effectively, DL mainly requires two things: (a) vast amounts of training data and (b) a very specific type of computational capacity. These two respective requirements have been amply met with the growth of the internet and the rapid development of GPUs. As such DL is an almost perfect fit for today’s technologies. However, DL is only a very rough approximation of how the brain works. More recently, Spiking Neural Networks (SNNs) have tried to simulate biological phenomena in a more realistic way. In SNNs information is transmitted as discreet spikes of data rather than a continuous weight or a differentiable activation function. In practical terms this means that far more nuanced interactions can occur between neurons and that the network can run far more efficiently (e.g. in terms of calculations needed and therefore overall power requirements). Nevertheless, the big problem with SNNs is that unlike DL it does not “fit” well with existing technologies. Worst still is that no one has yet come up with definitive way to make SNNs function at a “deep” level. The difficulty is that in essence "deep" and "spiking" refer to fundamentally different characteristics of a neural network: "spiking" focuses on the activation of individual neurons, whereas "deep" concerns itself to the network architecture itself [1]. However, these two methods are in fact not contradictory, but have so far been developed in isolation from each other due to the prevailing technology driving each technique and the fundamental conceptual distance between each of the two biological paradigms. If advances in AI are to continue at the present rate that new technologies are going to be developed and the contradictory aspects of DL and SNN are going to have to be reconciled. Very recently, there have been a handful of attempts to amalgamate DL and SNN in a variety of ways [2]-one of the most exciting being the creation of a specific hierarchical learning paradigm in Recurrent SNN (RSNNs) called e-prop [3]. However, this paper posits that this has been made problematic because a fundamental agent in the way the biological brain functions has been missing from each paradigm, and that if this is included in a new model then the union between DL and RSNN can be made in a more harmonious manner. The missing piece to the jigsaw, in fact, is the glial cell and the unacknowledged function it plays in neural processing. In this context, this paper examines how DL and SNN can be combined, and how glial dynamics cannot only address outstanding issues with the existing individual paradigms - for example the “weight transport” problem - but also act as the “glue” – e.g. pun intended - between these two paradigms. This idea has direct parallel with the idea of convolution in DL but has the added dimension of time. It is important not only where events happen but also when events occur in this new paradigm. The synergy between these two powerful paradigms give hints at the direction and potential of what could be an important part of the next wave of development in AI

Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons

An important open problem of computational neuroscience is the generic organization of computations in networks of neurons in the brain. We show here through rigorous theoretical analysis that inherent stochastic features of spiking neurons, in combination with simple nonlinear computational operations in specific network motifs and dendritic arbors, enable networks of spiking neurons to carry out probabilistic inference through sampling in general graphical models. In particular, it enables them to carry out probabilistic inference in Bayesian networks with converging arrows (“explaining away”) and with undirected loops, that occur in many real-world tasks. Ubiquitous stochastic features of networks of spiking neurons, such as trial-to-trial variability and spontaneous activity, are necessary ingredients of the underlying computational organization. We demonstrate through computer simulations that this approach can be scaled up to neural emulations of probabilistic inference in fairly large graphical models, yielding some of the most complex computations that have been carried out so far in networks of spiking neurons

PDGF-C Induces Maturation of Blood Vessels in a Model of Glioblastoma and Attenuates the Response to Anti-VEGF Treatment

Recent clinical trials of VEGF inhibitors have shown promise in the treatment of recurrent glioblastomas (GBM). However, the survival benefit is usually short-lived as tumors escape anti-VEGF therapies. Here we tested the hypothesis that Platelet Derived Growth Factor-C (PDGF-C), an isoform of the PDGF family, affects GBM progression independent of VEGF pathway and hinders anti-VEGF therapy.We first showed that PDGF-C is present in human GBMs. Then, we overexpressed or downregulated PDGF-C in a human GBM cell line, U87MG, and grew them in cranial windows in nude mice to assess vessel structure and function using intravital microscopy. PDGF-C overexpressing tumors had smaller vessel diameters and lower vascular permeability compared to the parental or siRNA-transfected tumors. Furthermore, vessels in PDGF-C overexpressing tumors had more extensive coverage with NG2 positive perivascular cells and a thicker collagen IV basement membrane than the controls. Treatment with DC101, an anti-VEGFR-2 antibody, induced decreases in vessel density in the parental tumors, but had no effect on the PDGF-C overexpressing tumors.These results suggest that PDGF-C plays an important role in glioma vessel maturation and stabilization, and that it can attenuate the response to anti-VEGF therapy, potentially contributing to escape from vascular normalization

Why Can't Rodents Vomit? A Comparative Behavioral, Anatomical, and Physiological Study

The vomiting (emetic) reflex is documented in numerous mammalian species, including primates and carnivores, yet laboratory rats and mice appear to lack this response. It is unclear whether these rodents do not vomit because of anatomical constraints (e.g., a relatively long abdominal esophagus) or lack of key neural circuits. Moreover, it is unknown whether laboratory rodents are representative of Rodentia with regards to this reflex. Here we conducted behavioral testing of members of all three major groups of Rodentia; mouse-related (rat, mouse, vole, beaver), Ctenohystrica (guinea pig, nutria), and squirrel-related (mountain beaver) species. Prototypical emetic agents, apomorphine (sc), veratrine (sc), and copper sulfate (ig), failed to produce either retching or vomiting in these species (although other behavioral effects, e.g., locomotion, were noted). These rodents also had anatomical constraints, which could limit the efficiency of vomiting should it be attempted, including reduced muscularity of the diaphragm and stomach geometry that is not well structured for moving contents towards the esophagus compared to species that can vomit (cat, ferret, and musk shrew). Lastly, an in situ brainstem preparation was used to make sensitive measures of mouth, esophagus, and shoulder muscular movements, and phrenic nerve activity-key features of emetic episodes. Laboratory mice and rats failed to display any of the common coordinated actions of these indices after typical emetic stimulation (resiniferatoxin and vagal afferent stimulation) compared to musk shrews. Overall the results suggest that the inability to vomit is a general property of Rodentia and that an absent brainstem neurological component is the most likely cause. The implications of these findings for the utility of rodents as models in the area of emesis research are discussed. © 2013 Horn et al

Acute treatment with valproic acid and L-thyroxine ameliorates clinical signs of experimental autoimmune encephalomyelitis and prevents brain pathology in DA rats

This work was supported by grants from the Swedish Research Council (MJ (K2008-66X-20776-01-4 and K2012-99X-20776-05-3)), OH (2011-3457) and GCB (K2011-80P-21816-01-4 and K2011-80X- 21817-01-4)), Harald and Greta Jeanssons Foundation (MJ), Swedish Association for Persons with Neurological Disabilities (MJ), ÅkeWibergs Foundation (MJ), Åke Löwnertz Foundation (MJ), Swedish Brain Foundation (MJ and GCB), David and Astrid Hagélen Foundation (GCB), Swedish Society for Medical Research (GCB), Swedish Society of Medicine (GCB), Socialstyrelsen (MJ), Karolinska Institutet funds (MJ and GCB), Marie Curie Integration Grant, Seventh Framework Programme, European Union (GCB, PCIG12-GA-2012-333713)), Neuropromise LSHM-CT-2005-018637 (MZA, HL) and Theme Center for Regenerative Medicine at Karolinska Institutet (OH)

Evidence of Sex-specific Differences in Masticatory Jaw Movement Patterns

The complexity of human oral functional movements has not been studied in detail quantitatively, and only recently have studies begun to evaluate whether such movements contain sex-specific characteristics. Therefore, the purposes of this study were: (1) to quantify in detail the jaw movements and associated masticatory electromyographic activity occurring during gum chewing, and (2) to explore these data for evidence of sex specificity. Fourteen male and 17 female subjects participated in the study. Approximately 11 right- and 11 left-sided chewing cycles and associated masticatory electromyographic activity were sampled from each subject. The samples were quantified into 165 variables per chewing cycle, averaged to create a single multivariate vector for each subject, and then analyzed by a step-wise discriminant analysis. With a combination of 6 variables, a jackknifed cross-validation test found the probability of correct classification to be 93.5%. These findings support the hypothesis that masticatory jaw movements contain sex-specific features.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/66870/2/10.1177_00220345970760031301.pd