125 research outputs found
Bullying10K: A Large-Scale Neuromorphic Dataset towards Privacy-Preserving Bullying Recognition
The prevalence of violence in daily life poses significant threats to
individuals' physical and mental well-being. Using surveillance cameras in
public spaces has proven effective in proactively deterring and preventing such
incidents. However, concerns regarding privacy invasion have emerged due to
their widespread deployment. To address the problem, we leverage Dynamic Vision
Sensors (DVS) cameras to detect violent incidents and preserve privacy since it
captures pixel brightness variations instead of static imagery. We introduce
the Bullying10K dataset, encompassing various actions, complex movements, and
occlusions from real-life scenarios. It provides three benchmarks for
evaluating different tasks: action recognition, temporal action localization,
and pose estimation. With 10,000 event segments, totaling 12 billion events and
255 GB of data, Bullying10K contributes significantly by balancing violence
detection and personal privacy persevering. And it also poses a challenge to
the neuromorphic dataset. It will serve as a valuable resource for training and
developing privacy-protecting video systems. The Bullying10K opens new
possibilities for innovative approaches in these domains.Comment: Accepted at the 37th Conference on Neural Information Processing
Systems (NeurIPS 2023) Track on Datasets and Benchmark
FireFly: A High-Throughput and Reconfigurable Hardware Accelerator for Spiking Neural Networks
Spiking neural networks (SNNs) have been widely used due to their strong
biological interpretability and high energy efficiency. With the introduction
of the backpropagation algorithm and surrogate gradient, the structure of
spiking neural networks has become more complex, and the performance gap with
artificial neural networks has gradually decreased. However, most SNN hardware
implementations for field-programmable gate arrays (FPGAs) cannot meet
arithmetic or memory efficiency requirements, which significantly restricts the
development of SNNs. They do not delve into the arithmetic operations between
the binary spikes and synaptic weights or assume unlimited on-chip RAM
resources by using overly expensive devices on small tasks. To improve
arithmetic efficiency, we analyze the neural dynamics of spiking neurons,
generalize the SNN arithmetic operation to the multiplex-accumulate operation,
and propose a high-performance implementation of such operation by utilizing
the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory
efficiency, we design a memory system to enable efficient synaptic weights and
membrane voltage memory access with reasonable on-chip RAM consumption.
Combining the above two improvements, we propose an FPGA accelerator that can
process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is
implemented on several FPGA edge devices with limited resources but still
guarantees a peak performance of 5.53TSOP/s at 300MHz. As a lightweight
accelerator, FireFly achieves the highest computational density efficiency
compared with existing research using large FPGA devices
Is Conventional SNN Really Efficient? A Perspective from Network Quantization
Spiking Neural Networks (SNNs) have been widely praised for their high energy
efficiency and immense potential. However, comprehensive research that
critically contrasts and correlates SNNs with quantized Artificial Neural
Networks (ANNs) remains scant, often leading to skewed comparisons lacking
fairness towards ANNs. This paper introduces a unified perspective,
illustrating that the time steps in SNNs and quantized bit-widths of activation
values present analogous representations. Building on this, we present a more
pragmatic and rational approach to estimating the energy consumption of SNNs.
Diverging from the conventional Synaptic Operations (SynOps), we champion the
"Bit Budget" concept. This notion permits an intricate discourse on
strategically allocating computational and storage resources between weights,
activation values, and temporal steps under stringent hardware constraints.
Guided by the Bit Budget paradigm, we discern that pivoting efforts towards
spike patterns and weight quantization, rather than temporal attributes,
elicits profound implications for model performance. Utilizing the Bit Budget
for holistic design consideration of SNNs elevates model performance across
diverse data types, encompassing static imagery and neuromorphic datasets. Our
revelations bridge the theoretical chasm between SNNs and quantized ANNs and
illuminate a pragmatic trajectory for future endeavors in energy-efficient
neural computations
Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks
Spiking Neural Networks (SNNs) have received considerable attention not only
for their superiority in energy efficient with discrete signal processing, but
also for their natural suitability to integrate multi-scale biological
plasticity. However, most SNNs directly adopt the structure of the
well-established DNN, rarely automatically design Neural Architecture Search
(NAS) for SNNs. The neural motifs topology, modular regional structure and
global cross-brain region connection of the human brain are the product of
natural evolution and can serve as a perfect reference for designing
brain-inspired SNN architecture. In this paper, we propose a Multi-Scale
Evolutionary Neural Architecture Search (MSE-NAS) for SNN, simultaneously
considering micro-, meso- and macro-scale brain topologies as the evolutionary
search space. MSE-NAS evolves individual neuron operation, self-organized
integration of multiple circuit motifs, and global connectivity across motifs
through a brain-inspired indirect evaluation function, Representational
Dissimilarity Matrices (RDMs). This training-free fitness function could
greatly reduce computational consumption and NAS's time, and its
task-independent property enables the searched SNNs to exhibit excellent
transferbility and scalability. Extensive experiments demonstrate that the
proposed algorithm achieves state-of-the-art (SOTA) performance with shorter
simulation steps on static datasets (CIFAR10, CIFAR100) and neuromorphic
datasets (CIFAR10-DVS and DVS128-Gesture). The thorough analysis also
illustrates the significant performance improvement and consistent
bio-interpretability deriving from the topological evolution at different
scales and the RDMs fitness function
FireFly v2: Advancing Hardware Support for High-Performance Spiking Neural Network with a Spatiotemporal FPGA Accelerator
Spiking Neural Networks (SNNs) are expected to be a promising alternative to
Artificial Neural Networks (ANNs) due to their strong biological
interpretability and high energy efficiency. Specialized SNN hardware offers
clear advantages over general-purpose devices in terms of power and
performance. However, there's still room to advance hardware support for
state-of-the-art (SOTA) SNN algorithms and improve computation and memory
efficiency. As a further step in supporting high-performance SNNs on
specialized hardware, we introduce FireFly v2, an FPGA SNN accelerator that can
address the issue of non-spike operation in current SOTA SNN algorithms, which
presents an obstacle in the end-to-end deployment onto existing SNN hardware.
To more effectively align with the SNN characteristics, we design a
spatiotemporal dataflow that allows four dimensions of parallelism and
eliminates the need for membrane potential storage, enabling on-the-fly spike
processing and spike generation. To further improve hardware acceleration
performance, we develop a high-performance spike computing engine as a backend
based on a systolic array operating at 500-600MHz. To the best of our
knowledge, FireFly v2 achieves the highest clock frequency among all FPGA-based
implementations. Furthermore, it stands as the first SNN accelerator capable of
supporting non-spike operations, which are commonly used in advanced SNN
algorithms. FireFly v2 has doubled the throughput and DSP efficiency when
compared to our previous version of FireFly and it exhibits 1.33 times the DSP
efficiency and 1.42 times the power efficiency compared to the current most
advanced FPGA accelerators
Transformation-Based Fuzzy Rule Interpolation With Mahalanobis Distance Measures Supported by Choquet Integral
Fuzzy rule interpolation (FRI) strongly supports approximate inference when a new observation matches no rules, through selecting and subsequently interpolating appropriate rules close to the observation from the given (sparse) rule base. Traditional ways of implementing the critical rule selection process are typically based on the exploitation of Euclidean distances between the observation and rules. It is conceptually straightforward for implementation but applying this distance metric may systematically lead to inferior results because it fails to reflect the variations of the relevance or significance levels amongst different domain features. To address this important issue, a novel transformation-based FRI approach is presented, on the basis of utilising the Mahalanobis distance metric. The new FRI method works by transforming a given sparse rule base into a coordinates system where the distance between instances of the same category becomes closer while that between different categories becomes further apart. In so doing, when an observation is present that matches no rules, the most relevant neighbouring rules to implement the required interpolation are more likely to be selected. Following this, the scale and move factors within the classical transformation-based FRI procedure are also modified by Choquet integral. Systematic experimental investigation over a range of classification problems demonstrates that the proposed approach remarkably outperforms the existing state-of-the-art FRI methods in both accuracy and efficiency
Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks
The evolution of the human brain has led to the development of complex
synaptic plasticity, enabling dynamic adaptation to a constantly evolving
world. This progress inspires our exploration into a new paradigm for Spiking
Neural Networks (SNNs): a Plasticity-Driven Learning Framework (PDLF). This
paradigm diverges from traditional neural network models that primarily focus
on direct training of synaptic weights, leading to static connections that
limit adaptability in dynamic environments. Instead, our approach delves into
the heart of synaptic behavior, prioritizing the learning of plasticity rules
themselves. This shift in focus from weight adjustment to mastering the
intricacies of synaptic change offers a more flexible and dynamic pathway for
neural networks to evolve and adapt. Our PDLF does not merely adapt existing
concepts of functional and Presynaptic-Dependent Plasticity but redefines them,
aligning closely with the dynamic and adaptive nature of biological learning.
This reorientation enhances key cognitive abilities in artificial intelligence
systems, such as working memory and multitasking capabilities, and demonstrates
superior adaptability in complex, real-world scenarios. Moreover, our framework
sheds light on the intricate relationships between various forms of plasticity
and cognitive functions, thereby contributing to a deeper understanding of the
brain's learning mechanisms. Integrating this groundbreaking plasticity-centric
approach in SNNs marks a significant advancement in the fusion of neuroscience
and artificial intelligence. It paves the way for developing AI systems that
not only learn but also adapt in an ever-changing world, much like the human
brain
Participatory location fingerprinting through stationary crowd in a public or commercial indoor environment
The training phase of indoor location fingerprinting has been traditionally performed by dedicated surveyors in a manner that is time and labour intensive. Crowdsourcing process is more efficient, but is impractical in public or commercial buildings because it requires occasional location fix provided explicitly by the participant, the availability of an indoor map for correlating the traces, and the existence of landmarks throughout the area. Here, we address these issues for the first time in this context by leveraging the existence of stationary crowd that have timetabled roles, such as desk-bound employees, lecturers and students. We propose a scalable and effortless positioning system in the context of a public/commercial building by using Wi-Fi sensor readings from its stationary occupants' smartphones combined with their timetabling information. Most significantly, the entropy concept of information theory is utilised to differentiate between good and spurious measurements in a manner that does not rely on the existence of known trusted users. Our analysis and experimental results show that, regardless of such participants' unpredictable behaviour, including not following their timetabling information, hiding their location or purposefully generating wrong data, our entropy-based filtering approach ensures the creation of a radio-map incrementally from their measurements. Its effectiveness is validated experimentally with two well-known machine learning algorithms
- …