68 research outputs found
Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition
The prevalence of violence in daily life poses significant threats to
individuals' physical and mental well-being. Using surveillance cameras in
public spaces has proven effective in proactively deterring and preventing such
incidents. However, concerns regarding privacy invasion have emerged due to
their widespread deployment. To address the problem, we leverage Dynamic Vision
Sensors (DVS) cameras to detect violent incidents and preserve privacy since it
captures pixel brightness variations instead of static imagery. We introduce
the Bullying10K dataset, encompassing various actions, complex movements, and
occlusions from real-life scenarios. It provides three benchmarks for
evaluating different tasks: action recognition, temporal action localization,
and pose estimation. With 10,000 event segments, totaling 12 billion events and
255 GB of data, Bullying10K contributes significantly by balancing violence
detection and personal privacy persevering. And it also poses a challenge to
the neuromorphic dataset. It will serve as a valuable resource for training and
developing privacy-protecting video systems. The Bullying10K opens new
possibilities for innovative approaches in these domains.Comment: Accepted at the 37th Conference on Neural Information Processing
Systems (NeurIPS 2023) Track on Datasets and Benchmark
FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks
Spiking neural networks (SNNs) have been widely used due to their strong
biological interpretability and high energy efficiency. With the introduction
of the backpropagation algorithm and surrogate gradient, the structure of
spiking neural networks has become more complex, and the performance gap with
artificial neural networks has gradually decreased. However, most SNN hardware
implementations for field-programmable gate arrays (FPGAs) cannot meet
arithmetic or memory efficiency requirements, which significantly restricts the
development of SNNs. They do not delve into the arithmetic operations between
the binary spikes and synaptic weights or assume unlimited on-chip RAM
resources by using overly expensive devices on small tasks. To improve
arithmetic efficiency, we analyze the neural dynamics of spiking neurons,
generalize the SNN arithmetic operation to the multiplex-accumulate operation,
and propose a high-performance implementation of such operation by utilizing
the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory
efficiency, we design a memory system to enable efficient synaptic weights and
membrane voltage memory access with reasonable on-chip RAM consumption.
Combining the above two improvements, we propose an FPGA accelerator that can
process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is
implemented on several FPGA edge devices with limited resources but still
guarantees a peak performance of 5.53TSOP/s at 300MHz. As a lightweight
accelerator, FireFly achieves the highest computational density efficiency
compared with existing research using large FPGA devices
Is Conventional SNN Really Efficient? A Perspective from Network Quantization
Spiking Neural Networks (SNNs) have been widely praised for their high energy
efficiency and immense potential. However, comprehensive research that
critically contrasts and correlates SNNs with quantized Artificial Neural
Networks (ANNs) remains scant, often leading to skewed comparisons lacking
fairness towards ANNs. This paper introduces a unified perspective,
illustrating that the time steps in SNNs and quantized bit-widths of activation
values present analogous representations. Building on this, we present a more
pragmatic and rational approach to estimating the energy consumption of SNNs.
Diverging from the conventional Synaptic Operations (SynOps), we champion the
"Bit Budget" concept. This notion permits an intricate discourse on
strategically allocating computational and storage resources between weights,
activation values, and temporal steps under stringent hardware constraints.
Guided by the Bit Budget paradigm, we discern that pivoting efforts towards
spike patterns and weight quantization, rather than temporal attributes,
elicits profound implications for model performance. Utilizing the Bit Budget
for holistic design consideration of SNNs elevates model performance across
diverse data types, encompassing static imagery and neuromorphic datasets. Our
revelations bridge the theoretical chasm between SNNs and quantized ANNs and
illuminate a pragmatic trajectory for future endeavors in energy-efficient
neural computations
Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks
Spiking Neural Networks (SNNs) have received considerable attention not only
for their superiority in energy efficient with discrete signal processing, but
also for their natural suitability to integrate multi-scale biological
plasticity. However, most SNNs directly adopt the structure of the
well-established DNN, rarely automatically design Neural Architecture Search
(NAS) for SNNs. The neural motifs topology, modular regional structure and
global cross-brain region connection of the human brain are the product of
natural evolution and can serve as a perfect reference for designing
brain-inspired SNN architecture. In this paper, we propose a Multi-Scale
Evolutionary Neural Architecture Search (MSE-NAS) for SNN, simultaneously
considering micro-, meso- and macro-scale brain topologies as the evolutionary
search space. MSE-NAS evolves individual neuron operation, self-organized
integration of multiple circuit motifs, and global connectivity across motifs
through a brain-inspired indirect evaluation function, Representational
Dissimilarity Matrices (RDMs). This training-free fitness function could
greatly reduce computational consumption and NAS's time, and its
task-independent property enables the searched SNNs to exhibit excellent
transferbility and scalability. Extensive experiments demonstrate that the
proposed algorithm achieves state-of-the-art (SOTA) performance with shorter
simulation steps on static datasets (CIFAR10, CIFAR100) and neuromorphic
datasets (CIFAR10-DVS and DVS128-Gesture). The thorough analysis also
illustrates the significant performance improvement and consistent
bio-interpretability deriving from the topological evolution at different
scales and the RDMs fitness function
FireFly v2: Advancing Hardware Support for High-Performance Spiking Neural Network with a Spatiotemporal FPGA Accelerator
Spiking Neural Networks (SNNs) are expected to be a promising alternative to
Artificial Neural Networks (ANNs) due to their strong biological
interpretability and high energy efficiency. Specialized SNN hardware offers
clear advantages over general-purpose devices in terms of power and
performance. However, there's still room to advance hardware support for
state-of-the-art (SOTA) SNN algorithms and improve computation and memory
efficiency. As a further step in supporting high-performance SNNs on
specialized hardware, we introduce FireFly v2, an FPGA SNN accelerator that can
address the issue of non-spike operation in current SOTA SNN algorithms, which
presents an obstacle in the end-to-end deployment onto existing SNN hardware.
To more effectively align with the SNN characteristics, we design a
spatiotemporal dataflow that allows four dimensions of parallelism and
eliminates the need for membrane potential storage, enabling on-the-fly spike
processing and spike generation. To further improve hardware acceleration
performance, we develop a high-performance spike computing engine as a backend
based on a systolic array operating at 500-600MHz. To the best of our
knowledge, FireFly v2 achieves the highest clock frequency among all FPGA-based
implementations. Furthermore, it stands as the first SNN accelerator capable of
supporting non-spike operations, which are commonly used in advanced SNN
algorithms. FireFly v2 has doubled the throughput and DSP efficiency when
compared to our previous version of FireFly and it exhibits 1.33 times the DSP
efficiency and 1.42 times the power efficiency compared to the current most
advanced FPGA accelerators
Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks
The evolution of the human brain has led to the development of complex
synaptic plasticity, enabling dynamic adaptation to a constantly evolving
world. This progress inspires our exploration into a new paradigm for Spiking
Neural Networks (SNNs): a Plasticity-Driven Learning Framework (PDLF). This
paradigm diverges from traditional neural network models that primarily focus
on direct training of synaptic weights, leading to static connections that
limit adaptability in dynamic environments. Instead, our approach delves into
the heart of synaptic behavior, prioritizing the learning of plasticity rules
themselves. This shift in focus from weight adjustment to mastering the
intricacies of synaptic change offers a more flexible and dynamic pathway for
neural networks to evolve and adapt. Our PDLF does not merely adapt existing
concepts of functional and Presynaptic-Dependent Plasticity but redefines them,
aligning closely with the dynamic and adaptive nature of biological learning.
This reorientation enhances key cognitive abilities in artificial intelligence
systems, such as working memory and multitasking capabilities, and demonstrates
superior adaptability in complex, real-world scenarios. Moreover, our framework
sheds light on the intricate relationships between various forms of plasticity
and cognitive functions, thereby contributing to a deeper understanding of the
brain's learning mechanisms. Integrating this groundbreaking plasticity-centric
approach in SNNs marks a significant advancement in the fusion of neuroscience
and artificial intelligence. It paves the way for developing AI systems that
not only learn but also adapt in an ever-changing world, much like the human
brain
Preparation and Innovative Design Applications of Paper-Based Aluminized Film
The growing demand for sustainable and innovative materials in product design has spurred interest in unconventional resources. Despite this, a gap persists in the effective utilization of paper-based materials, particularly with metallic coatings, for creative applications. This study aims to address this by exploring the technical methods for applying Aluminum (Al) coatings to paper substrates. We developed paper-based aluminum coatings and combined them with corrugated cardboard to create a novel material for product development. Utilizing high-strength specialty paper as the substrate, an orthogonal experiment was conducted to identify key process parameters. Factors such as target–substrate distance, working pressure, current intensity, and coating duration were evaluated for their impact on the properties of the Al film. Our research culminated in the production of high-quality Al-plated corrugated cardboard. Capitalizing on its unique attributes, we employed a design approach that led to the creation of innovative furniture featuring structural forms like folding and insertion. This study not only introduces a new range of Al-plated corrugated cardboard products but also expands the potential applications of paper-based aluminized film in material-based product design.</jats:p
Spatio–Temporal Relationship and Evolvement of Socioeconomic Factors and PM<sub>2.5</sub> in China During 1998–2016
A comprehensive understanding of the relationships between PM2.5 concentration and socioeconomic factors provides new insight into environmental management decision-making for sustainable development. In order to identify the contributions of socioeconomic development to PM2.5, their spatial interaction and temporal variation of long time series are analyzed in this paper. Unary linear regression method, Spearman’s rank and bivariate Moran’s I methods were used to investigate spatio–temporal variations and relationships of socioeconomic factors and PM2.5 concentration in 31 provinces of China during the period of 1998–2016. Spatial spillover effect of PM2.5 concentration and the impact of socioeconomic factors on PM2.5 concentration were analyzed by spatial lag model. Results demonstrated that PM2.5 concentration in most provinces of China increased rapidly along with the increase of socioeconomic factors, while PM2.5 presented a slow growth trend in Southwest China and a descending trend in Northwest China along with the increase of socioeconomic factors. Long time series analysis revealed the relationships between PM2.5 concentration and four socioeconomic factors. PM2.5 concentration was significantly positive spatial correlated with GDP per capita, industrial added value and private car ownership, while urban population density appeared a negative spatial correlation since 2006. GDP per capita and industrial added values were the most important factors to increase PM2.5, followed by private car ownership and urban population density. The findings of the study revealed spatial spillover effects of PM2.5 between different provinces, and can provide a theoretical basis for sustainable development and environmental protection
- …