The human brain exhibits remarkable abilities in integrating temporally
distant sensory inputs for decision-making. However, existing brain-inspired
spiking neural networks (SNNs) have struggled to match their biological
counterpart in modeling long-term temporal relationships. To address this
problem, this paper presents a novel Contextual Embedding Leaky
Integrate-and-Fire (CE-LIF) spiking neuron model. Specifically, the CE-LIF
model incorporates a meticulously designed contextual embedding component into
the adaptive neuronal firing threshold, thereby enhancing the memory storage of
spiking neurons and facilitating effective sequential modeling. Additionally,
theoretical analysis is provided to elucidate how the CE-LIF model enables
long-term temporal credit assignment. Remarkably, when compared to
state-of-the-art recurrent SNNs, feedforward SNNs comprising the proposed
CE-LIF neurons demonstrate superior performance across extensive sequential
modeling tasks in terms of classification accuracy, network convergence speed,
and memory capacity