Spikformer: When Spiking Neural Network Meets Transformer

We consider two biologically plausible structures, the Spiking Neural Network (SNN) and the self-attention mechanism. The former offers an energy-efficient and event-driven paradigm for deep learning, while the latter has the ability to capture feature dependencies, enabling Transformer to achieve good performance. It is intuitively promising to explore the marriage between them. In this paper, we consider leveraging both self-attention capability and biological properties of SNNs, and propose a novel Spiking Self Attention (SSA) as well as a powerful framework, named Spiking Transformer (Spikformer). The SSA mechanism in Spikformer models the sparse visual feature by using spike-form Query, Key, and Value without softmax. Since its computation is sparse and avoids multiplication, SSA is efficient and has low computational energy consumption. It is shown that Spikformer with SSA can outperform the state-of-the-art SNNs-like frameworks in image classification on both neuromorphic and static datasets. Spikformer (66.3M parameters) with comparable size to SEW-ResNet-152 (60.2M,69.26%) can achieve 74.81% top1 accuracy on ImageNet using 4 time steps, which is the state-of-the-art in directly trained SNNs models

Characterization of particle size and composition of respirable coal mine dust

Respirable coal mine dust (RCMD) particles, particularly the nano-sized fraction (\u3c1 \u3eμm) of the RCMD if present, can cause severe lung diseases in coal miners. Characterization of both the particle size and chemical composition of such RCMD particles remains a work in progress, in par-ticular, with respect to the nano-sized fraction of RCMD. In this work, various methods were sur-veyed and used to obtain both the size and chemical composition of RCMD particles, including scanning electron microscopy (SEM), scanning transmission electron microscopy (S-TEM), dynamic light scattering (DLS), and asymmetric flow field-flow fractionation (AsFIFFF). It was found that the micron-sized fraction (\u3e1 μm) of RCMD particles collected at the miner location, from an underground coal mine, contained more coal particles, while those collected at the bolter location contained more rock dust particles. Two image processing procedures were developed to determine the size of individual RCMD particles. The particle size distribution (PSD) results showed that a significant amount (~80% by number) of nano-sized particles were present in the RCMD sample collected in an underground coal mine. The presence of nano-sized RCMD particles was confirmed by bulk sample analysis, using both DLS and AsFIFFF. The mode particle size at the peak frequency of the size distribution was found to be 300–400 nm, which was consistent with the result obtained from SEM analysis. The chemical composition data of the nano-sized RCMD showed that not only diesel particles, but also both coal and rock dust particles were present in the nano-sized fraction of the RCMD. The presence of the nano-sized fraction of RCMD particles may be site and location dependent, and a detailed analysis of the entire size range of RCMD particles in different underground coal mines is needed