28 December 2023 in IEEE Journal of Solid-State Circuits
Edge vision devices face real 3-D scenes with lighting fluctuations. They require a vision chip capable of acquiring 2-D/3-D vision with a high dynamic range and performing real-time intelligent in situ processing, while maintaining low end-to-end latency. Aimed at this versatile edge vision, this article presents a bio-inspired full-spiking vision chip that consists of a single-photon-avalanche-diode (SPAD) image sensor and a spiking vision processor monolithically. To decrease latency while realizing versatile functions, we build a bio-inspired full spiking vision on chip by directly coupling spike-like SPAD imaging with a neuroinspired spike-computing paradigm. By modeling this spiking visual flow, we theoretically demonstrate direct coupling and derive a spike preprocessing method for dim-vision enhancement. The SPAD image sensor equipped with a configurable exposure gating scheme can realize three modes: 2-D, 3-D depth, and dim-vision imaging. The spiking vision processor contains a parallel processing element (PE) array and can be reconfigured as a preprocessor or a spiking neural network (SNN) processor. A prototype chip integrated with a $128\ttimes 128$ SPAD image sensor with a reconfigurable spiking vision processor (256 PEs) was implemented in a 0.18- $\mu$ m CMOS technology. The chip experimentally demonstrates three imaging modes, 99.30% classification accuracy on the MNIST dataset with 2-D vision, only 3.9% accuracy loss under 0.02 lux with dim-vision, 1.68 cm obstacle detection error with 3-D depth vision, and a 3.58 $\mu$ s latency for light adaptation. The chip realizes a 100 dB dynamic range, 100 000 spiking maps per second (SMps), and 81.92 GSOPS at an 80-MHz clock.
5 June 2023 in IEEE Transactions on Circuits and Systems II: Express Briefs
Spiking Neural Networks (SNNs), inspired by the biological brain, attract extensive attention for their simplified computation and the ability to process spatiotemporal data. With the development of retina-like sensors, there is a growing demand for edge SNN accelerators. The structural dataflow and great spiking sparsity of SNNs provide much exploration space for energy-efficient accelerators. However, current SNN accelerators did not co-optimize the two features together. This work proposes an energy-efficient edge accelerator architecture to support typical spiking neural networks. We propose a Weight Stationary-Local Output Stationary (WS-LOS) dataflow for SNNs and maximize the data reuse with a hierarchical memory structure. Three methods, including address skipping (AS), dynamic workload scheduling (DWS), and reconfigurable adder tree (RAT), are proposed to exploit the spiking sparsity. The accelerator is synthesized under 65nm technology, running at 250MHz. We prove our accelerator’s ability to process spatiotemporal data based on the DVS-Gesture dataset, achieving 92.7% accuracy and 24.3μJ/image energy. This design achieves 349.6KFPS recognition throughput on the MNIST dataset and 0.52 μJ/image energy, which realizes top performance among edge SNN accelerators.
7 March 2022 in IEEE Transactions on Circuits and Systems II: Express Briefs
Nowadays, the vision chip bridging sensing and processing has been extensively employed in high-speed image processing, owing to its excellent performance, low power consumption, and economical cost. However, there is a dilemma in designing processors to support conventional computer vision algorithms and neural networks since the two algorithms have a non-trivial trade-off in proposing a unified architecture. By analyzing computation properties, we propose a novel hierarchical parallel vision processor (ViP) for hybrid vision chips to accelerate both traditional computer vision (CV) and neural network (NN). The ViP architecture includes three parallelism levels: PE for pixel-centric, computing core (CC) for block, and vision core (VC) for global. PEs contain dedicated computing units and data paths for convolution operations without degrading its flexibility. Each CC is driven by customized SIMD instructions and can be dynamically connected for meeting block parallelism requirements. ViP is fabricated in 65nm CMOS technology and achieves a peak performance of 614.4 GOPS and an energy efficiency of 640 GOPS/W at 200 MHz clock frequency. Notably, several experiments on CV and NN are performed, illustrating an ultra-low latency in executing hybrid algorithms.