SparseSpikformer: A Co-Design Framework for Token and Weight Pruning in Spiking Transformer

As the third-generation neural network, the Spiking Neural Network (SNN) has the advantages of low power consumption and high energy efficiency, making it suitable for implementation on edge devices. More recently, the most advanced SNN, Spikformer, combines the self-attention module from Transformer with SNN to achieve remarkable performance. However, it adopts larger channel dimensions in MLP layers, leading to an increased number of redundant model parameters. To effectively decrease the computational complexity and weight parameters of the model, we explore the Lottery Ticket Hypothesis (LTH) and discover a very sparse ($\ge$90%) subnetwork that achieves comparable performance to the original network. Furthermore, we also design a lightweight token selector module, which can remove unimportant background information from images based on the average spike firing rate of neurons, selecting only essential foreground image tokens to participate in attention calculation. Based on that, we present SparseSpikformer, a co-design framework aimed at achieving sparsity in Spikformer through token and weight pruning techniques. Experimental results demonstrate that our framework can significantly reduce 90% model parameters and cut down Giga Floating-Point Operations (GFLOPs) by 20% while maintaining the accuracy of the original model

Spiking Inception Module for Multi-layer Unsupervised Spiking Neural Networks

Spiking Neural Network (SNN), as a brain-inspired approach, is attracting attention due to its potential to produce ultra-high-energy-efficient hardware. Competitive learning based on Spike-Timing-Dependent Plasticity (STDP) is a popular method to train an unsupervised SNN. However, previous unsupervised SNNs trained through this method are limited to a shallow network with only one learnable layer and cannot achieve satisfactory results when compared with multi-layer SNNs. In this paper, we eased this limitation by: 1)We proposed a Spiking Inception (Sp-Inception) module, inspired by the Inception module in the Artificial Neural Network (ANN) literature. This module is trained through STDP-based competitive learning and outperforms the baseline modules on learning capability, learning efficiency, and robustness. 2)We proposed a Pooling-Reshape-Activate (PRA) layer to make the Sp-Inception module stackable. 3)We stacked multiple Sp-Inception modules to construct multi-layer SNNs. Our algorithm outperforms the baseline algorithms on the hand-written digit classification task, and reaches state-of-the-art results on the MNIST dataset among the existing unsupervised SNNs.Comment: Published at the 2020 International Joint Conference on Neural Networks (IJCNN); Extended from arXiv:2001.0168

Numerical analysis of pressure fluctuation in a multiphase rotodynamic pump with air–water two-phase flow

International audiencePressure fluctuation in single-phase pumps has been studied widely, while less attention has been paid to research on multiphase pumps that are commonly used in the petroleum chemical industry. Therefore, this study investigates the pressure fluctuation for a multiphase rotodynamic pump handling air–water two-phase flow. Simulations based on the Euler two-fluid model were carried out using ANSYS_CFX16.0 at different Inlet Gas Void Fractions (IGVFs) and various flow rate values. Under conditions of IGVF = 0% (pure water) and IGVF = 15%, the accuracy of the numerical method was tested by comparing the experimental data. The results showed that the rotor–stator interaction was still the main generation driver of pressure fluctuation in gas–liquid two-phase pumps. However, the fluctuation near the impeller outlet ascribe to the rotor–stator interaction was weakened by the complex gas–liquid flow. For the different IGVF, the variation trend of fluctuation was similar along the streamwise direction. That is, the fluctuation in the impeller increased before decreasing, while in the guide vane it decreased gradually. Also, the fluctuation in the guide vane was generally greater than for the impeller and the maximum amplitude appeared in the vicinity of guide vane inlet

Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models

The ability to quickly learn a new task with minimal instruction - known as few-shot learning - is a central aspect of intelligent agents. Classical few-shot benchmarks make use of few-shot samples from a single modality, but such samples may not be sufficient to characterize an entire concept class. In contrast, humans use cross-modal information to learn new concepts efficiently. In this work, we demonstrate that one can indeed build a better ${\bf visual}$ dog classifier by ${\bf read}$ing about dogs and ${\bf listen}$ing to them bark. To do so, we exploit the fact that recent multimodal foundation models such as CLIP are inherently cross-modal, mapping different modalities to the same representation space. Specifically, we propose a simple cross-modal adaptation approach that learns from few-shot examples spanning different modalities. By repurposing class names as additional one-shot training samples, we achieve SOTA results with an embarrassingly simple linear classifier for vision-language adaptation. Furthermore, we show that our approach can benefit existing methods such as prefix tuning, adapters, and classifier ensembling. Finally, to explore other modalities beyond vision and language, we construct the first (to our knowledge) audiovisual few-shot benchmark and use cross-modal training to improve the performance of both image and audio classification.Comment: CVPR 2023. Project website: https://linzhiqiu.github.io/papers/cross_modal

Lasing- Encoded Microsensor Driven by Interfacial Cavity Resonance Energy Transfer

Microlasers are emerging tools for biomedical applications. In particular, whispering- gallery- mode (WGM) microlasers are promising candidates for sensing at the biointerface owing to their high quality- factor and potential in molecular assays, and intracellular and extracellular detection. However, lasing particles with sensing functionality remain challenging since the overlap between the WGM optical mode and external gain medium is much lower compared to internal gain inside the cavity. To overcome this problem, the concept of Förster resonant energy transfer (FRET) is exploited on WGM droplet microlaser by separating donor and acceptor molecules at the cavity- surface interface. It is first discovered that the interfacial FRET laser not only originates from conventional FRET but utilizes coherent radiative energy transfer (CRET) to excite acceptor molecules by inducing light- harvesting effect near the cavity interface. Simulations and experiments have revealed that the absorption spectrum of individual analyte plays a crucial role in interfacial FRET laser. Distinct lasing spectra can therefore distinguish molecules of different absorption properties upon binding. Finally, detection of small fluorescent molecules and photosynthetic protein is performed. The results presented here not only demonstrate the wide- ranging potential of microlaser external cavity implementation in molecular sensing applications, but also provide comprehensive insights into cavity energy transfer in laser physics.A novel concept is proposed to achieve active lasing- encoded biosensors by taking advantage of light- harvesting effect at the cavity interface, where interfacial molecular lasers based on cavity resonant energy transfer are demonstrated. This work marks a critical step of realizing whispering- gallery- mode (WGM) laser probes for biosensing, opening a new avenue in laser- based molecular sensing.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/154969/1/adom201901596-sup-0001-SuppMat.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/154969/2/adom201901596_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/154969/3/adom201901596.pd