Always-on intelligent visual perception applications are widely deployed in edges in the AIoT era. In order to eliminate power costs of data conversion and transmission, this paper proposes Senputing, an ultra-low-power processing-in-sensor chip that completely fuses sensing and computing together for a BNN-based hierarchical processing system. This chip could operate in two modes. In computation mode, photocurrents are directly utilized for computing without being converted into voltages, and the computation results of 1-st BNN layer are directly sent out to subsequent BNN processors for an always-on coarse classification, eliminating conversion power and storage cost of raw images. Once an interested objected is detected, this chip switches to sensor mode and sends raw images to potential full-precision processors or cloud servers for fine-grained recognition or segmentation. A $32\times 32$ prototype is fabricated with 180nm CMOS process. It accomplishes MNIST dataset classification task with the accuracy of 93.76% and the power consumption of 147nW at 156fps, achieving $13.1\times $ energy efficiency compared with state-of-the-art work.

With the rising importance of energy efficiency, zero leakage power and instant-on capability are highly desired features in energy harvesting sensors, as well as “normally off” high performance processors. However, intermittent power in such systems requires nonvolatile memory (NVM) to hold intermediate data and avoid rollbacks. Previous work has adopted FeRAM and STT-MRAM to achieve zero-standby power and fast-restore nonvolatile processors (NVPs) [1-3]. Previous NVPs, however, suffer from several drawbacks: 1) Various power interrupt periods are not considered; 2) the 2-macro memory architecture slows access speed; 3) worst-case store/restore operations are always performed. We present a 65nm fully-CMOS-logic-compatible ReRAM-based NVP achieving time/space-adaptive data retention. A 1-macro nvSRAM with self-write-termination (SWT) is integrated to boost clock frequency and reduce store energy. The adaptive retention and SWT strategy relieve the ReRAM write endurance challenge (106-1012), making it sufficient for most applications. The NVP operates at 100MHz with 20ns/0.45nJ restore time (TRESTORE)/energy (ERESTORE), realizing 6× reduction in TRESTORE, >6000× reduction in ERESTORE and 4× higher clock frequency compared with existing designs.

In the era of Artificial Intelligence (AI), bio-inspired perceptual computing system design brings favorable opportunities, while still facing considerable challenges in the meantime. Especially for tasks of image recognition in power-limited vision-based Internet of Things (IoT) devices, energy constraints due to the end of Dennard scaling limit the performance of Neural Network (NN) algorithms on popular digital platforms, which would not reach the energy efficiency requirement for embedded AI applications. In this paper, a processing near sensor architecture in mixed-signal domain with CMOS Image Sensor (CIS) of convolutional-kernel-readout method is proposed. Visual data is collected from a smart CIS, which can realize maximum $5 x5$ kernel-readout with minimum one slide step for convolutional operations. The outputs of CIS are directly processed by analog processing units locating near CIS without the constraint of digital clock and bottleneck of Analog-to-Digital Converter (ADC). By analyzing the effects of analog noise on classification accuracy, we further evaluate the fault-tolerance of the system to circuit noise and the device imperfection, such as mismatch and process variation. A mixed-signal visual perception chip is fabricated with a 32 x 32 image sensor and a Binarized Neural Network (BNN) processing array integrated with SMIC 180nm standard CMOS mixed-signal process. Measurement results show up to 545.4 GOPS/W energy efficiency with 1.8mW power consumption taking the advantages of ADC-free processing architecture. This work provides a promising alternative for low-power vision-based IoT intelligent applications.

Current BNN-based visual system wastes lots of energy in data conversion and movement, hindering its deployment on battery-powered devices. This paper proposes MACSen, an ultra-low-power processing-in-sensor (PIS) architecture which integrates sensing with computing and directly outputs the computation results. The multiply-and-accumulation (MAC) operation in BNN is fused with the Correlated Double Sampling (CDS) procedure together to save data conversion power. A 4×4 MACSen prototype of 180nm process was fabricated for demonstration, and it achieves the frame rate of 1000fps and the energy efficiency of 1.32TOP/s/W in computation mode. Furthermore, the system demonstration on MNIST dataset classification task shows that the hardware BNN implementation integrating MACSen incurs no accuracy degradation and gains 61% energy saving compared with state-of-the-art work.

In recent years, Neural networks (NNs) present vast potential for innovative applications. However, energy efficiency continues to remain a challenge in deploying NNs on the edge. In this context, computation-in-memory (CIM) architecture becomes an emerging trend in the area of energy-efficient hardware design, because it reduces data movement of multiply-accumulate (MAC) computation significantly. However, many recent works employ massive data converters to feed input data and transform output results, which may counteract the benefits of in-memory processing. To tackle this limitation, we propose a combined architecture cooperating sensor with CIM macro to achieve local processing of sensory signals. Current-mode computing techniques are exploited to achieve high energy efficiency while eliminating data conversion overhead. Moreover, we thoroughly analyze the non-idealities of the proposed mixed-signal circuits and present a co-design scheme to mitigate these imperfections. We have fabricated a 2K-bit CIM macro in the proposed architecture with TSMC 65-nm technology. The fabricated chip achieved 60.6 TOPS/W energy efficiency while consuming 845.5 μW power and 0.3 mm² core area, presenting a promising solution for energy-constrained edge devices.