Training a General Spiking Neural Network with Improved Efficiency and Minimum Latency

Spiking Neural Networks (SNNs) that operate in an event-driven manner and employ binary spike representation have recently emerged as promising candidates for energy-efficient computing. However, a cost bottleneck arises in obtaining high-performance SNNs: training a SNN model requires a large number of time steps in addition to the usual learning iterations, hence this limits their energy efficiency. This paper proposes a general training framework that enhances feature learning and activation efficiency within a limited time step, providing a new solution for more energy-efficient SNNs. Our framework allows SNN neurons to learn robust spike feature from different receptive fields and update neuron states by utilizing both current stimuli and recurrence information transmitted from other neurons. This setting continuously complements information within a single time step. Additionally, we propose a projection function to merge these two stimuli to smoothly optimize neuron weights (spike firing threshold and activation). We evaluate the proposal for both convolution and recurrent models. Our experimental results indicate state-of-the-art visual classification tasks, including CIFAR10, CIFAR100, and TinyImageNet.Our framework achieves 72.41% and 72.31% top-1 accuracy with only 1 time step on CIFAR100 for CNNs and RNNs, respectively. Our method reduces 10x and 3x joule energy than a standard ANN and SNN, respectively, on CIFAR10, without additional time steps.Comment: Accepted by ACML 202

The growth and shrinkage of water droplets at the oil-solid interface.

HYPOTHESIS: The mechanism for the spontaneous formation of water droplets at oil/solid interfaces immersed in water is currently unclear. We hypothesize that growth and shrinkage of droplets are kinetically controlled by diffusion of water through the oil, driven by differences in chemical potential between the solid substrate and the aqueous reservoir. EXPERIMENTS: The formation, growth and shrinkage of water droplets at an immersed oil/solid interface are investigated theoretically and experimentally with three silicone oils. The surface is hydrophobic and the droplets formed are truncated spheres with radius, a, less than 10 μm. The expansion and contraction of the droplets can be controlled by adjusting the difference in chemical potential. The growth kinetics are modelled in terms of water migration through the oil layer which predicts a2∝t. FINDINGS: This is the first study of possible mechanisms for the formation of such interfacial droplets. Several possible causes are shown to be unfavourable, negligible, or are eliminated by careful experiments controlling key parameters (such as oil viscosity, substrate chemistry). The rate constant for mass transport is proportional to difference in chemical potential and an estimate shows dissociation of surface groups on the substrate provides a driving chemical potential of the right magnitude.National Natural Science Foundation of China (21872078

Diverse drug delivery systems for the enhancement of cancer immunotherapy: an overview

Despite the clear benefits demonstrated by immunotherapy, there is still an inevitable off-target effect resulting in serious adverse immune reactions. In recent years, the research and development of Drug Delivery System (DDS) has received increased prominence. In decades of development, DDS has demonstrated the ability to deliver drugs in a precisely targeted manner to mitigate side effects and has the advantages of flexible control of drug release, improved pharmacokinetics, and drug distribution. Therefore, we consider that combining cancer immunotherapy with DDS can enhance the anti-tumor ability. In this paper, we provide an overview of the latest drug delivery strategies in cancer immunotherapy and briefly introduce the characteristics of DDS based on nano-carriers (liposomes, polymer nano-micelles, mesoporous silica, extracellular vesicles, etc.) and coupling technology (ADCs, PDCs and targeted protein degradation). Our aim is to show readers a variety of drug delivery platforms under different immune mechanisms, and analyze their advantages and limitations, to provide more superior and accurate targeting strategies for cancer immunotherapy

Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey

Transformer-based Large Language Models (LLMs) have been applied in diverse areas such as knowledge bases, human interfaces, and dynamic agents, and marking a stride towards achieving Artificial General Intelligence (AGI). However, current LLMs are predominantly pretrained on short text snippets, which compromises their effectiveness in processing the long-context prompts that are frequently encountered in practical scenarios. This article offers a comprehensive survey of the recent advancement in Transformer-based LLM architectures aimed at enhancing the long-context capabilities of LLMs throughout the entire model lifecycle, from pre-training through to inference. We first delineate and analyze the problems of handling long-context input and output with the current Transformer-based models. We then provide a taxonomy and the landscape of upgrades on Transformer architecture to solve these problems. Afterwards, we provide an investigation on wildly used evaluation necessities tailored for long-context LLMs, including datasets, metrics, and baseline models, as well as optimization toolkits such as libraries, frameworks, and compilers to boost the efficacy of LLMs across different stages in runtime. Finally, we discuss the challenges and potential avenues for future research. A curated repository of relevant literature, continuously updated, is available at https://github.com/Strivin0311/long-llms-learning.Comment: 40 pages, 3 figures, 4 table

DiME : a scalable disease module identification algorithm with application to glioma progression

Disease module is a group of molecular components that interact intensively in the disease specific biological network. Since the connectivity and activity of disease modules may shed light on the molecular mechanisms of pathogenesis and disease progression, their identification becomes one of the most important challenges in network medicine, an emerging paradigm to study complex human disease. This paper proposes a novel algorithm, DiME (Disease Module Extraction), to identify putative disease modules from biological networks. We have developed novel heuristics to optimise Community Extraction, a module criterion originally proposed for social network analysis, to extract topological core modules from biological networks as putative disease modules. In addition, we have incorporated a statistical significance measure, B-score, to evaluate the quality of extracted modules. As an application to complex diseases, we have employed DiME to investigate the molecular mechanisms that underpin the progression of glioma, the most common type of brain tumour. We have built low (grade II)--and high (GBM)--grade glioma co-expression networks from three independent datasets and then applied DiME to extract potential disease modules from both networks for comparison. Examination of the interconnectivity of the identified modules have revealed changes in topology and module activity (expression) between low- and high- grade tumours, which are characteristic of the major shifts in the constitution and physiology of tumour cells during glioma progression. Our results suggest that transcription factors E2F4, AR and ETS1 are potential key regulators in tumour progression. Our DiME compiled software, R/C++ source code, sample data and a tutorial are available at http://www.cs.bham.ac.uk/~szh/DiME

Support Strength Criteria and Intelligent Design of Underground Powerhouses

The proper design of underground powerhouse support is the key engineering technique to guarantee the safe construction and operation of underground works. By regression analysis of the surrounding rock support parameters of 29 underground powerhouses with a span of 18.0–34.0 m, the empirical formula of the relationship between the support strength of anchor bar, strength-stress ratio, and plant span and the relationship among the support strength of the anchor cable, strength-stress ratio, and plant span are proposed. Furthermore, an intelligent design model for the anchor support of the underground powerhouse was trained by a BP (back propagation) neural network. Research shows that the support strength index of the anchor bolt and the anchor cable of these 29 plants are all distributed around 1.0. Therefore, a support strength index of 0.8–1.2 can be used as a reference for practical engineering support design. Finally, the reliability of the intelligent design model for the anchor support of the underground powerhouse was verified by comparison with actual engineering and support strength index. This shows that the intelligent design model can provide a reference for engineering design and construction