16 June 2023 in Frontiers in Neuroscience
The spiking neural network (SNN), as a promising brain-inspired computational model with binary spike information transmission mechanism, rich spatially-temporal dynamics, and event-driven characteristics, has received extensive attention. However, its intricately discontinuous spike mechanism brings difficulty to the optimization of the deep SNN. Since the surrogate gradient method can greatly mitigate the optimization difficulty and shows great potential in directly training deep SNNs, a variety of direct learning-based deep SNN works have been proposed and achieved satisfying progress in recent years. In this paper, we present a comprehensive survey of these direct learning-based deep SNN works, mainly categorized into accuracy improvement methods, efficiency improvement methods, and temporal dynamics utilization methods. In addition, we also divide these categorizations into finer granularities further to better organize and introduce them. Finally, the challenges and trends that may be faced in future research are prospected.
15 July 2025 in Neural Networks
1 January 2024 in IEEE Signal Processing Letters
Recently, the neuromorphic vision sensor has received more and more interest. However, the neuromorphic data consists of asynchronous event spikes, which makes it difficult to construct a big benchmark to train a power general neural network model, thus limiting the neuromorphic data understanding for “unseen” objects by deep learning. While for the frame image, since the training data can be obtained easily, the zero-shot and few-shot learning for “unseen” task via the large Contrastive Vision-Language Pre-training (CLIP) model, which is pre-trained by large-scale image-text pairs in 2D, have shown inspirational performance. We wonder whether the CLIP could be transferred to neuromorphic data recognition to handle the “unseen” problem. To this end, we materialize this idea with NeuroCLIP in the paper. The NeuroCLIP consists of 2D CLIP and two specially designed modules for neuromorphic data understanding. First, an event-frame module that could convert the event spikes to the sequential frame image with a simple discrimination strategy. Second, an inter-timestep adapter, which is a simple fine-tuned adapter based on a spiking neural network (SNN) for the sequential features coming from the visual encoder of CLIP to improve the few-shot performance. Various experiments on neuromorphic datasets including N-MNIST, CIFAR10-DVS, and ES-ImageNet demonstrate the effectiveness of NeuroCLIP.
17 January 2025 in Complex & Intelligent Systems
Achieving accurate and generalized autonomous navigation in unknown environments poses a significant challenge in robotics and artificial intelligence. Animals exhibits superlative navigation capabilities by combining the representation of internal neurals and sensory cues of self-motion and external information. This paper proposes a brain-inspired navigation method based upon the spiking neural networks (SNN) and reinforcement learning, integrated with a lidar system that serves as the local environment explorer, by which realizes high performance of obstacle avoidance and target arrival in mapless circumstances. An asymptotic gradient method is introduced to optimize the backpropagation during training, which facilitates the improvement of model robustness. The results of our experiments conducted on the Gazebo platform showcase how our approach effectively improves navigation performance in various intricate environments. Our approach yielded a higher success navigation rate ranging from 2% to 5%, depending on the SNN timesteps. Considering the inherent lower computational cost of SNN, this work contributes to advancing the fusion of SNN and reinforcement learning techniques for energy-efficient autonomous navigation tasks in real-world mapless scenarios.