4 December 2024 in IEEE Journal of Solid-State Circuits
Deep spiking neural networks (DSNNs), such as spiking transformers, have demonstrated comparable performance to artificial neural networks (ANNs). With higher spike input sparsity and the utilization of accumulation (AC)-only operations, DSNNs have great potential for achieving high energy efficiency. Many researchers have proposed neuromorphic processors to accelerate spiking neural networks (SNNs) with dedicated architectures. However, three problems still exist when processing DSNNs, including redundant memory access among timesteps, inefficiency in exploiting unstructured sparsity in spikes, and the lack of optimizations for new operators involved in DSNNs. In this work, an accelerator for deep and sparse SNNs is proposed with three design features: a 3-D computation array that allows parallel computation of multiple timesteps to maximize weight data reuse and reduce external memory access; a parallel non-zero data fetcher that efficiently searches non-zero spike positions and fetches corresponding weights to reduce computation latency; and a multimode unified computation scheduler that can be configured to maximize energy efficiency for spiking convolution (SCONV), spiking $Q, K,~\text {and}~V$ matrix generation, and spiking self-attention (SSA). The accelerator is implemented and fabricated using 40-nm CMOS technology. When compared with state-of-the-art sparse processors, it achieves the best energy efficiency of 0.078 pJ/SOP and the highest recognition accuracy of 77.6% on ImageNet using the spiking transformer algorithm.
24 May 2023 in IEEE Transactions on Biomedical Circuits and Systems
Implementing neural networks (NN) on edge devices enables AI to be applied in many daily scenarios. The stringent area and power budget on edge devices impose challenges on conventional NNs with massive energy-consuming Multiply Accu- mulation (MAC) operations and offer an opportunity for Spiking Neural Networks (SNN), which can be implemented within sub- mW power budget. However, mainstream SNN topologies varies from Spiking Feedforward Neural Network (SFNN), Spiking Recurrent Neural Network (SRNN), to Spiking Convolutional Neural Network (SCNN), and it is challenging for the edge SNN processor to adapt to different topologies. Besides, online learning ability is critical for edge devices to adapt to local environments but comes with dedicated learning modules, further increasing area and power consumption burdens. To alleviate these prob- lems, this work proposed RAINE, a reconfigurable neuromorphic engine supporting multiple SNN topologies and a dedicated trace- based rewarded spike-timing-dependent plasticity (TR-STDP) learning algorithm. Sixteen Unified-Dynamics Learning-Engines (UDLEs) are implemented in RAINE to realize a compact and reconfigurable implementation of different SNN operations. Three topology-aware data reuse strategies are proposed and analyzed to optimize the mapping of different SNNs on RAINE. A 40-nm prototype chip is fabricated, achieving energy-per- synaptic-operation (SOP) of 6.2 pJ/SOP at 0.51V, and power consumption of 510 μW at 0.45V. Finally, three examples with dif- ferent SNN topologies, including SRNN-based ECG arrhythmia detection, SCNN-based 2D image classification, and end-to-end on-chip learning for MNIST digit recognition, are demonstrated on RAINE with ultra-low energy consumption of 97.7nJ/step, 6.28μJ/sample, and 42.98μJ/sample respectively. These results show the feasibility of obtaining high reconfigurability and low power consumption simultaneously on a SNN processor.
1 February 2023 in Journal of Neural Engineering
Objective. Retinal prostheses are promising devices to restore vision for patients with severe age-related macular degeneration or retinitis pigmentosa disease. The visual processing mechanism embodied in retinal prostheses play an important role in the restoration effect. Its performance depends on our understanding of the retina’s working mechanism and the evolvement of computer vision models. Recently, remarkable progress has been made in the field of processing algorithm for retinal prostheses where the new discovery of the retina’s working principle and state-of-the-arts computer vision models are combined together. Approach. We investigated the related research on artificial intelligence techniques for retinal prostheses. The processing algorithm in these studies could be attributed to three types: computer vision-related methods, biophysical models, and deep learning models. Main results. In this review, we first illustrate the structure and function of the normal and degenerated retina, then demonstrate the vision rehabilitation mechanism of three representative retinal prostheses. It is necessary to summarize the computational frameworks abstracted from the normal retina. In addition, the development and feature of three types of different processing algorithms are summarized. Finally, we analyze the bottleneck in existing algorithms and propose our prospect about the future directions to improve the restoration effect. Significance. This review systematically summarizes existing processing models for predicting the response of the retina to external stimuli. What’s more, the suggestions for future direction may inspire researchers in this field to design better algorithms for retinal prostheses.
17 April 2024 in IEEE Transactions on Biomedical Circuits and Systems
Cloud-based training and edge-based inference modes for Artificial Intelligence of Medical Things (AIoMT) applications suffer from accuracy degradation due to physiological signal variations among patients. On-chip learning can overcome this issue by online adaptation of neural network parameters for user-specific tasks. However, existing on-chip learning processors have limitations in terms of versatility, resource utilization, and energy efficiency. We propose HybMED, which is a novel neural signal processor that supports on-chip hybrid neural network training using a composite direct feedback alignment-based paradigm. HybMED is suitable for general-purpose health monitoring AIoMT devices. It improves resource utilization and area efficiency by the reconfigurable homogeneous core with heterogeneous data flow and enhances energy efficiency by exploiting sparsity at different granularities. The chip was fabricated by TSMC 40nm process and tested in multiple physiological signal processing tasks, demonstrating an average improvement in accuracy of 41.16% following online few-shot learning. The chip demonstrates an area efficiency of 1.17 GOPS/mm 2 and an energy efficiency of 1.58 TOPS/W. Compared to the previous state-of-the-art physiological signal processors with on-chip learning, the chip achieves a 65× improvement in area efficiency and 1.48× improvement in energy efficiency, respectively.
5 May 2023 in Neural Networks