Dongcheng Zhao

Dongcheng Zhao

Auto-generated profile, based on publicly available information

Dongcheng Zhao

Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 June 2023 in Neural Networks
Verified
Journal Article
Open access
YDYiting Dong
Yiting Dong
DZDongcheng Zhao
Dongcheng Zhao
YLYang Li
Yang Li
YZYi Zeng
Yi Zeng
The backpropagation algorithm has promoted the rapid development of deep learning, but it relies on a large amount of labeled data and still has a large gap with how humans learn. The human brain can quickly learn various conceptual knowledge in a self-organized and unsupervised manner, accomplished through coordinating various learning rules and structures in the human brain. Spike-timing-dependent plasticity (STDP) is a general learning rule in the brain, but spiking neural networks (SNNs) trained with STDP alone is inefficient and perform poorly. In this paper, taking inspiration from short-term synaptic plasticity, we design an adaptive synaptic filter and introduce the adaptive spiking threshold as the neuron plasticity to enrich the representation ability of SNNs. We also introduce an adaptive lateral inhibitory connection to adjust the spikes balance dynamically to help the network learn richer features. To speed up and stabilize the training of unsupervised spiking neural networks, we design a samples temporal batch STDP (STB-STDP), which updates weights based on multiple samples and moments. By integrating the above three adaptive mechanisms and STB-STDP, our model greatly accelerates the training of unsupervised spiking neural networks and improves the performance of unsupervised SNNs on complex tasks. Our model achieves the current state-of-the-art performance of unsupervised STDP-based SNNs in the MNIST and FashionMNIST datasets. Further, we tested on the more complex CIFAR10 dataset, and the results fully illustrate the superiority of our algorithm. Our model is also the first work to apply unsupervised STDP-based SNNs to CIFAR10. At the same time, in the small-sample learning scenario, it will far exceed the supervised ANN using the same structure.

5 June 2023 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Verified
Journal Article
Open access
JLJindong Li
Jindong Li
GSGuobin Shen
Guobin Shen
DZDongcheng Zhao
Dongcheng Zhao
QZQian Zhang
Qian Zhang
YZYi Zeng
Yi Zeng
Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high-energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of SNNs has become more complex, and the performance gap with artificial neural networks (ANNs) has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neurons on-the-fly (FireFly). FireFly is the first SNN accelerator that incorporates DSP optimization techniques into SNN synaptic operations. FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53 TOP/s at 300 MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

2 June 2022 in Patterns
Verified
Journal Article
Open access
GSGuobin Shen
Guobin Shen
DZDongcheng Zhao
Dongcheng Zhao
YZYi Zeng
Yi Zeng
The spiking neural network (SNN) mimics the information-processing operation in the human brain. Directly applying backpropagation to the training of the SNN still has a performance gap compared with traditional deep neural networks. To address the problem, we propose a biologically plausible spatial adjustment that rethinks the relationship between membrane potential and spikes and realizes a reasonable adjustment of gradients to different time steps. It precisely controls the backpropagation of the error along the spatial dimension. Secondly, we propose a biologically plausible temporal adjustment to make the error propagate across the spikes in the temporal dimension, which overcomes the problem of the temporal dependency within a single spike period of traditional spiking neurons. We have verified our algorithm on several datasets, and the experimental results have shown that our algorithm greatly reduces network latency and energy consumption while also improving network performance.

11 July 2022 in Neural Networks
Verified
Journal Article
Open access
DZDongcheng Zhao
Dongcheng Zhao
YZYi Zeng
Yi Zeng
YLYang Li
Yang Li
Spiking neural networks (SNNs) transmit information through discrete spikes that perform well in processing spatial-temporal information. Owing to their nondifferentiable characteristic, difficulties persist in designing SNNs that deliver good performance. SNNs trained with backpropagation have recently exhibited impressive performance by using gradient approximation. However, their performance on complex tasks remains significantly inferior to that of deep neural networks. By taking inspiration from autapses in the brain that connect spiking neurons with a self-feedback connection, we apply adaptive time-delayed self-feedback to the membrane potential to regulate the precision of the spikes. We also strike a balance between the excitatory and inhibitory mechanisms of neurons to dynamically control the output of spiking neurons. By combining these two mechanisms, we propose a deep SNN with adaptive self-feedback and balanced excitatory and inhibitory neurons (BackEISNN). The results of experiments on several standard datasets show that the two modules not only accelerate the convergence of the network but also increase its accuracy. Our model achieved state-of-the-art performance on the MNIST, Fashion-MNIST, and N-MNIST datasets. The proposed BackEISNN also achieved remarkably good performance on the CIFAR10 dataset while using a relatively light structure that competes against state-of-the-art SNNs.

20 September 2023 in Proceedings of the National Academy of Sciences
Verified
Journal Article
Open access
GSGuobin Shen
Guobin Shen
DZDongcheng Zhao
Dongcheng Zhao
YDYiting Dong
Yiting Dong
YZYi Zeng
Yi Zeng
In biological neural systems, different neurons are capable of self-organizing to form different neural circuits for achieving a variety of cognitive functions. However, the current design paradigm of spiking neural networks is based on structures derived from deep learning. Such structures are dominated by feedforward connections without taking into account different types of neurons, which significantly prevent spiking neural networks from realizing their potential on complex tasks. It remains an open challenge to apply the rich dynamical properties of biological neural circuits to model the structure of current spiking neural networks. This paper provides a more biologically plausible evolutionary space by combining feedforward and feedback connections with excitatory and inhibitory neurons. We exploit the local spiking behavior of neurons to adaptively evolve neural circuits such as forward excitation, forward inhibition, feedback inhibition, and lateral inhibition by the local law of spike-timing-dependent plasticity and update the synaptic weights in combination with the global error signals. By using the evolved neural circuits, we construct spiking neural networks for image classification and reinforcement learning tasks. Using the brain-inspired Neural circuit Evolution strategy (NeuEvo) with rich neural circuit types, the evolved spiking neural network greatly enhances capability on perception and reinforcement learning tasks. NeuEvo achieves state-of-the-art performance on CIFAR10, DVS-CIFAR10, DVS-Gesture, and N-Caltech101 datasets and achieves advanced performance on ImageNet. Combined with on-policy and off-policy deep reinforcement learning algorithms, it achieves comparable performance with artificial neural networks. The evolved spiking neural circuits lay the foundation for the evolution of complex networks with functions.

25 March 2024 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Verified
Journal Article
Closed access
JLJindong Li
Jindong Li
GSGuobin Shen
Guobin Shen
DZDongcheng Zhao
Dongcheng Zhao
QZQian Zhang
Qian Zhang
YZYi Zeng
Yi Zeng
Spiking neural networks (SNNs) are expected to be a promising alternative to artificial neural networks (ANNs) due to their strong biological interpretability and high energy efficiency. Specialized SNN hardware offers clear advantages over general-purpose devices in terms of power and performance. However, there is still room to advance hardware support for state-of-the-art (SOTA) SNN algorithms and improve computation and memory efficiency. As a further step in supporting high-performance SNNs on specialized hardware, we introduce FireFly v2, an field-programmable gate array (FPGA) SNN accelerator that can address the issue of nonspike operation in current SOTA SNN algorithms, which presents an obstacle in the end-to-end deployment onto existing SNN hardware. To more effectively align with the SNN characteristics, we design a spatiotemporal dataflow that allows four dimensions of parallelism and eliminates the need for membrane potential storage, enabling on-the-fly spike processing and spike generation. To further improve hardware acceleration performance, we develop a high-performance spike computing engine as a backend based on a systolic array operating at 500–600 MHz. To the best of our knowledge, FireFly v2 achieves the highest clock frequency among all FPGA-based implementations. Furthermore, it stands as the first SNN accelerator capable of supporting nonspike operations, which are commonly used in advanced SNN algorithms. FireFly v2 has doubled the throughput and DSP efficiency when compared to our previous version of FireFly and it exhibits $\times 1.33$ the DSP efficiency and $\times 1.42$ the power efficiency compared to the current most advanced FPGA accelerators.