27 June 2023 in IEEE Journal of Solid-State Circuits
This article presents an intelligent vision sensor (IVS) with embedded tiny convolutional neural network (CNN) model and programmable processing-in-sensor (PIS) circuit for real-time inference applications of low-power edge devices. The proposed imager realizes the full computing functions of a customized three-layers tiny network, which includes a $3 \times 3$ convolution layer (stride $=$ 3) with activation function of rectified linear unit (ReLU), a $2 \times 2$ maximum pooling (MP) layer (stride $=$ 2), and a $1 \times 1$ fully connected (FC) layer for inference. A 0.8 V $128 \times 128$ IVS prototype was fabricated and verified in TSMC 0.18 $\mu$ m standard CMOS technology. In normal image mode, it consumed 76.4 $\mu$ W with full-resolution ( $126 \times 126$ active resolution) image output at 125 f/s. In CNN mode, it consumed 134.5 $\mu$ W at 250 f/s and an achieved iFoMs of 33.8 pJ/pixel $\cdot$ frame. Using the proposed mixed-mode PIS circuits, the prototype is configured to demonstrate a “human face or not detection” task with an achieved accuracy of 93.6%.
27 September 2023 in IEEE Journal of Solid-State Circuits
Advanced artificial intelligence edge devices are expected to support floating-point (FP) multiply and accumulation operations while ensuring high energy efficiency and high inference accuracy. This work presents an FP compute-in-memory (CIM) macro that exploits the advantages of computing in the time, digital, and analog-voltage domain for high energy efficiency and accuracy. This work employs: 1) a hybrid-domain macrostructure to enable the computation of both the exponent and mantissa within the same CIM macro; 2) a time-domain computing scheme for energy-efficient exponent computation; 3) a product-exponent-based input-mantissa alignment scheme to enable the accumulation of the product mantissa in the same column; and 4) a place-value-dependent digital–analog-hybrid computing scheme to enable energy-efficient mantissa computations of sufficient accuracy. A 22-nm 832-kB FP-CIM macro fabricated using foundry-provided compact 6T-static random access memory (SRAM) cells achieved a high energy efficiency of 72.14 tera-floating-point operations per second (TFLOPS)/W while performing FP-multiply-and-accumulate (MAC) operations involving BF16-input, BF16-weight, FP32-output, and 128 accumulations.
2 October 2023 in IEEE Journal of Solid-State Circuits
On-chip non-volatile compute-in-memory (nvCIM) enables artificial intelligence (AI)-edge processors to perform multiply-and-accumulate (MAC) operations while enabling the non-volatile storage of weight data in power-off mode to enhance energy efficiency. However, the design challenges of nvCIM-based AI-edge processors include: 1) lack of a nvCIM-friendly computing flow; 2) a tradeoff between usage of memory devices versus process variations, computing yield and area overhead; 3) long computing latency and low energy efficiency; and 4) small-signal margin and large bitline current. This article presents an nvCIM-friendly AI-edge processor that uses a hybrid-mode resistive random access memory nvCIM (hmRe-nvCIM) macro to overcome the abovementioned challenges by three processor-level schemes: 1) a multimode nvCIM engine controller (mmCIM-EC); 2) a bitwise-input-sparsity and place-value-aware dynamic accumulation (BIS-PVA-DA); and 3) a bitwise weight column inversion (BWCI) and two macro-level schemes: 1) a dynamic-accumulation-aware current quantization (DACQ) and 2) a current–voltage-hybrid analog-to-digital converter (CVH-ADC). The proposed AI-edge processor fabricated using 22-nm technology achieved 51.4 TOPS/W and 472.7- $\mu \text{s}$ wake-up to response time, while the hmRe-nvCIM macro achieved 67.2 TOPS/W under 8-bit input, 8-bit weight, and 22- or 24-bit output precision.
29 December 2023 in IEEE Journal of Solid-State Circuits
This article presents a novel static random access memory computing-in-memory (SRAM-CIM) structure designed for high-precision multiply-and-accumulate (MAC) operations with high energy efficiency (EF), high readout accuracy, and short compute latency. The proposed device employs 1) a time-domain incremental-accumulation (TDIA) scheme to enable high-accumulation MAC operations while maintaining a large signal margin across MAC values (MACVs), 2) a dynamic differential-reference (D2REF) scheme based on software-hardware co-design to reduce read energy consumption, and 3) a low-dMACV-aware recursive time-to-digital converter (LMAR-TDC) for implementation with the D2REF scheme to further suppress readout energy consumption. A 28 nm 1 Mb SRAM-CIM macro fabricated using foundry-provided compact 6T-SRAM cells achieved EF of 39.31 TOPS/W and compute latency of 6.6 ns for 8b-MAC operations with 64 accumulations per cycle and near-full output precision (22b).
24 October 2023 in IEEE Journal of Solid-State Circuits
Nonvolatile near-memory-compute (nvNMC) macros are promising candidates for edge artificial intelligence (AI) devices requiring high energy efficiency, short wakeup-to-compute latency, and robust inference accuracy with high precision of inputs (IN), weights ( $W)$ , and outputs (OUT). Nonetheless, the practical application of nvNMC macros is hindered by inherent design challenges: 1) high energy consumption in reading repetitious weight data, 2) low energy efficiency due to high bitstream toggling rate in digital multiply-and-accumulate (MAC) circuits, 3) narrow signal margin and high memory readout latency, and 4) redundant MAC computation in digital MAC circuits under input-stationary flow. In addressing these challenges, we developed a number of schemes based on the concept of system-circuit co-design, including 1) a weight-feature aware read (WFAR) scheme; 2) a toggling-aware weight-tuning (TAWT) scheme; 3) a differential charge-accumulating margin-enhanced voltage-sensing amplifier (DCME-VSA); and 4) an input-sparsity aware pre-computing unit (ISAPU). An 8-Mb spin-transfer torque magnetic random access memory (STT-MRAM) nvNMC macro fabricated using foundry-provided 22 nm STT-MRAM achieved read bandwidth of 436 GB/s, MAC computing latency of 20 ns, and energy efficiency of 53.6–190.2 TOPS/W when performing eight-bit input and eight-bit weight MAC operations with 26-bit output and 576 accumulations.
22 October 2024 in IEEE Journal of Solid-State Circuits
Artificial intelligence (AI) edge devices increasingly require the enhanced accuracy of floating-point (FP) multiply-and-accumulate (MAC) operations as well as nonvolatile on-chip memory to minimize the movement of weight data in power-off mode. Designing non-volatile compute-in-memory (nvCIM) macros for FP operations imposes several challenges, including: 1) a tradeoff between inference accuracy and weight bit-width following pre-alignment; 2) long computing latency and high energy consumption; 3) large cell array current during computation; and 4) high multi-bit readout energy consumption. In this study, we devised four schemes to address these issues, including: 1) a kernel-wise weight pre-alignment (K-WPA); 2) a rescheduled multi-bit input compression (RS-MIC); 3) HRS-favored dual-sign-bit (HF-DSB); and 4) residue-shared analog-to-digital converter (RS-ADC). A 16 Mb resistive random access memory (ReRAM) nvCIM macro fabricated for FP operations using foundry-provided ReRAM (22 nm CMOS technology) achieved an efficiency of 34.2 TFLOPS/W under BF16-input, BF16-weight, and FP32-output and 31.4 TFLOPS/W under FP16-input, FP16-weight, and FP32-output.