Estimating Post-Synaptic Effects for Online Training of Feed-Forward SNNs

Facilitating online learning in spiking neural networks (SNNs) is a key step in developing event-based models that can adapt to changing environments and learn from continuous data streams in real-time. Although forward-mode differentiation enables online learning, its computational requirements restrict scalability. This is typically addressed through approximations that limit learning in deep models. In this study, we propose Online Training with Postsynaptic Estimates (OTPE) for training feed-forward SNNs, which approximates Real-Time Recurrent Learning (RTRL) by incorporating temporal dynamics not captured by current approximations, such as Online Training Through Time (OTTT) and Online Spatio-Temporal Learning (OSTL). We show improved scaling for multi-layer networks using a novel approximation of temporal effects on the subsequent layer's activity. This approximation incurs minimal overhead in the time and space complexity compared to similar algorithms, and the calculation of temporal effects remains local to each layer. We characterize the learning performance of our proposed algorithms on multiple SNN model configurations for rate-based and time-based encoding. OTPE exhibits the highest directional alignment to exact gradients, calculated with backpropagation through time (BPTT), in deep networks and, on time-based encoding, outperforms other approximate methods. We also observe sizeable gains in average performance over similar algorithms in offline training of Spiking Heidelberg Digits with equivalent hyper-parameters (OTTT/OSTL - 70.5%; OTPE - 75.2%; BPTT - 78.1%)

Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks

The large computing and memory cost of deep neural networks (DNNs) often precludes their use in resource-constrained devices. Quantizing the parameters and operations to lower bit-precision offers substantial memory and energy savings for neural network inference, facilitating the use of DNNs on edge computing platforms. Recent efforts at quantizing DNNs have employed a range of techniques encompassing progressive quantization, step-size adaptation, and gradient scaling. This paper proposes a new quantization approach for mixed precision convolutional neural networks (CNNs) targeting edge-computing. Our method establishes a new pareto frontier in model accuracy and memory footprint demonstrating a range of quantized models, delivering best-in-class accuracy below 4.3 MB of weights (wgts.) and activations (acts.). Our main contributions are: (i) hardware-aware heterogeneous differentiable quantization with tensor-sliced learned precision, (ii) targeted gradient modification for wgts. and acts. to mitigate quantization errors, and (iii) a multi-phase learning schedule to address instability in learning arising from updates to the learned quantizer and model parameters. We demonstrate the effectiveness of our techniques on the ImageNet dataset across a range of models including EfficientNet-Lite0 (e.g., 4.14MB of wgts. and acts. at 67.66% accuracy) and MobileNetV2 (e.g., 3.51MB wgts. and acts. at 65.39% accuracy)

Memory Organization for Energy-Efficient Learning and Inference in Digital Neuromorphic Accelerators

The energy efficiency of neuromorphic hardware is greatly affected by the energy of storing, accessing, and updating synaptic parameters. Various methods of memory organisation targeting energy-efficient digital accelerators have been investigated in the past, however, they do not completely encapsulate the energy costs at a system level. To address this shortcoming and to account for various overheads, we synthesize the controller and memory for different encoding schemes and extract the energy costs from these synthesized blocks. Additionally, we introduce functional encoding for structured connectivity such as the connectivity in convolutional layers. Functional encoding offers a 58% reduction in the energy to implement a backward pass and weight update in such layers compared to existing index-based solutions. We show that for a 2 layer spiking neural network trained to retain a spatio-temporal pattern, bitmap (PB-BMP) based organization can encode the sparser networks more efficiently. This form of encoding delivers a 1.37x improvement in energy efficiency coming at the cost of a 4% degradation in network retention accuracy as measured by the van Rossum distance.Comment: submitted to ISCAS202

Pulmonary ORMDL3 is critical for induction of Alternaria -induced allergic airways disease

Genome-wide association studies have identified the ORM (yeast)-like protein isoform 3 (ORMDL3) gene locus on human chromosome 17q to be a highly significant risk factor for childhood-onset asthma. Objective We sought to investigate in vivo the functional role of ORMDL3 in disease inception. Methods An Ormdl3-deficient mouse was generated and the role of ORMDL3 in the generation of allergic airways disease to the fungal aeroallergen Alternaria alternata was determined. An adeno-associated viral vector was also used to reconstitute ORMDL3 expression in airway epithelial cells of Ormdl3 knockout mice. Results Ormdl3 knockout mice were found to be protected from developing allergic airways disease and showed a marked decrease in pathophysiology, including lung function and airway eosinophilia induced by Alternaria. Alternaria is a potent inducer of cellular stress and the unfolded protein response, and ORMDL3 was found to play a critical role in driving the activating transcription factor 6–mediated arm of this response through Xbp1 and downstream activation of the endoplasmic reticulum–associated degradation pathway. In addition, ORMDL3 mediated uric acid release, another marker of cellular stress. In the knockout mice, reconstitution of Ormdl3 transcript levels specifically in the bronchial epithelium resulted in reinstatement of susceptibility to fungal allergen–induced allergic airways disease. Conclusions This study demonstrates that ORMDL3, an asthma susceptibility gene identified by genome-wide association studies, contributes to key pathways that promote changes in airway physiology during allergic immune responses

Orientation cues for high-flying nocturnal insect migrants: do turbulence-induced temperature and velocity fluctuations indicate the mean wind flow?

Migratory insects flying at high altitude at night often show a degree of common alignment, sometimes with quite small angular dispersions around the mean. The observed orientation directions are often close to the downwind direction and this would seemingly be adaptive in that large insects could add their self-propelled speed to the wind speed, thus maximising their displacement in a given time. There are increasing indications that high-altitude orientation may be maintained by some intrinsic property of the wind rather than by visual perception of relative ground movement. Therefore, we first examined whether migrating insects could deduce the mean wind direction from the turbulent fluctuations in temperature. Within the atmospheric boundary-layer, temperature records show characteristic ramp-cliff structures, and insects flying downwind would move through these ramps whilst those flying crosswind would not. However, analysis of vertical-looking radar data on the common orientations of nocturnally migrating insects in the UK produced no evidence that the migrants actually use temperature ramps as orientation cues. This suggests that insects rely on turbulent velocity and acceleration cues, and refocuses attention on how these can be detected, especially as small-scale turbulence is usually held to be directionally invariant (isotropic). In the second part of the paper we present a theoretical analysis and simulations showing that velocity fluctuations and accelerations felt by an insect are predicted to be anisotropic even when the small-scale turbulence (measured at a fixed point or along the trajectory of a fluid-particle) is isotropic. Our results thus provide further evidence that insects do indeed use turbulent velocity and acceleration cues as indicators of the mean wind direction

Generalising unit-refutation completeness and SLUR via nested input resolution

We introduce two hierarchies of clause-sets, SLUR_k and UC_k, based on the classes SLUR (Single Lookahead Unit Refutation), introduced in 1995, and UC (Unit refutation Complete), introduced in 1994. The class SLUR, introduced in [Annexstein et al, 1995], is the class of clause-sets for which unit-clause-propagation (denoted by r_1) detects unsatisfiability, or where otherwise iterative assignment, avoiding obviously false assignments by look-ahead, always yields a satisfying assignment. It is natural to consider how to form a hierarchy based on SLUR. Such investigations were started in [Cepek et al, 2012] and [Balyo et al, 2012]. We present what we consider the "limit hierarchy" SLUR_k, based on generalising r_1 by r_k, that is, using generalised unit-clause-propagation introduced in [Kullmann, 1999, 2004]. The class UC, studied in [Del Val, 1994], is the class of Unit refutation Complete clause-sets, that is, those clause-sets for which unsatisfiability is decidable by r_1 under any falsifying assignment. For unsatisfiable clause-sets F, the minimum k such that r_k determines unsatisfiability of F is exactly the "hardness" of F, as introduced in [Ku 99, 04]. For satisfiable F we use now an extension mentioned in [Ansotegui et al, 2008]: The hardness is the minimum k such that after application of any falsifying partial assignments, r_k determines unsatisfiability. The class UC_k is given by the clause-sets which have hardness <= k. We observe that UC_1 is exactly UC. UC_k has a proof-theoretic character, due to the relations between hardness and tree-resolution, while SLUR_k has an algorithmic character. The correspondence between r_k and k-times nested input resolution (or tree resolution using clause-space k+1) means that r_k has a dual nature: both algorithmic and proof theoretic. This corresponds to a basic result of this paper, namely SLUR_k = UC_k.Comment: 41 pages; second version improved formulations and added examples, and more details regarding future directions, third version further examples, improved and extended explanations, and more on SLUR, fourth version various additional remarks and editorial improvements, fifth version more explanations and references, typos corrected, improved wordin

Visual ecology of aphids – a critical review on the role of colours in host finding

We review the rich literature on behavioural responses of aphids (Hemiptera: Aphididae) to stimuli of different colours. Only in one species there are adequate physiological data on spectral sensitivity to explain behaviour crisply in mechanistic terms. Because of the great interest in aphid responses to coloured targets from an evolutionary, ecological and applied perspective, there is a substantial need to expand these studies to more species of aphids, and to quantify spectral properties of stimuli rigorously. We show that aphid responses to colours, at least for some species, are likely based on a specific colour opponency mechanism, with positive input from the green domain of the spectrum and negative input from the blue and/or UV region. We further demonstrate that the usual yellow preference of aphids encountered in field experiments is not a true colour preference but involves additional brightness effects. We discuss the implications for agriculture and sensory ecology, with special respect to the recent debate on autumn leaf colouration. We illustrate that recent evolutionary theories concerning aphid–tree interactions imply far-reaching assumptions on aphid responses to colours that are not likely to hold. Finally we also discuss the implications for developing and optimising strategies of aphid control and monitoring

The Complexity of Drawing a Graph in a Polygonal Region

We prove that the following problem is complete for the existential theory of the reals: Given a planar graph and a polygonal region, with some vertices of the graph assigned to points on the boundary of the region, place the remaining vertices to create a planar straight-line drawing of the graph inside the region. This strengthens an NP-hardness result by Patrignani on extending partial planar graph drawings. Our result is one of the first showing that a problem of drawing planar graphs with straight-line edges is hard for the existential theory of the reals. The complexity of the problem is open in the case of a simply connected region. We also show that, even for integer input coordinates, it is possible that drawing a graph in a polygonal region requires some vertices to be placed at irrational coordinates. By contrast, the coordinates are known to be bounded in the special case of a convex region, or for drawing a path in any polygonal region.Comment: Appears in the Proceedings of the 26th International Symposium on Graph Drawing and Network Visualization (GD 2018

Variations in Pastors’ Perceptions of the Etiology of Depression By Race and Religious Affiliation

Depression is a major, preventable problem in the United States, yet relatively few individuals seek care in traditional mental health settings. Instead, many choose to confide in friends, family, or clergy. Thus, it is important to discover how clergy perceive the definition of and etiology of depression. The author conducted a survey with 204 Protestant pastors in California. Multinomial logistic regression revealed a statistically significant difference in how depression is perceived based on race. Caucasian American pastors more readily agreed with the statement that depression was a biological mood disorder, while African American pastors more readily agreed that depression was a moment of weakness when dealing with trials and tribulations. Also, mainline Protestants more frequently disagreed with statements about spiritual causes of depression than Pentecostals and non-denominational pastors. The findings suggest that racial and religious affiliational influences shape how pastors view, and ultimately intervene, in the area of depression