8 August 2023 in IEEE Journal of Solid-State Circuits
Accurate identification of the target and tracking it at high speeds using drone-mounted cameras and compute hardware finds military and commercial applications. Conventional frame-based cameras and convolutional neural networks (CNNs) extract detailed spatial information to show high accuracy but suffer from lower throughput caused by large models. Alternatively, event cameras capture the motion information as an asynchronous event stream with high temporal resolution. Spiking neural networks (SNNs) can be used to process these data at high speed, but the sparse sensing and difficulty in training SNN limit the accuracy. Fusing the complementary spatial and temporal advantages of the frame and event-based pipelines allows high-speed identification and tracking while preserving accuracy. The SNN processes the event stream continuously to provide high-speed target estimates with lower accuracy, while periodic anchors provided by the reliable CNN restore the accuracy. In this work, we present a heterogeneous programmable ARM Cortex-based system-on-a-chip (SoC) in 40-nm Taiwan Semiconductor Manufacturing Company (TSMC) ultra low power (ULP) technology with power-efficient RRAM compute-in-memory (CIM) for CNN and high-speed SRAM compute-near-memory (CNM) for SNN for the modality-matched acceleration of the hybrid vision. Our SoC incorporates: 1) two levels of power gating to save 91.8% of total chip power with non-volatile RRAM-CIM; 2) embedded triple error correction (TEC) within RRAM CIM macro to suppress the raw bit errors in reading by $>$ 5 orders of magnitude; and 3) parallelly operating CNN and SNN modules to provide $>$ 100 outputs/s. Such cross-layer hybrid approaches can mitigate fundamental tradeoffs in sensing and processing.
19 November 2020 in IEEE Transactions on Circuits and Systems I: Regular Papers
The increase in the of edge devices has led to the emergence of edge computing where the computations are performed on the device. In recent years, deep neural networks (DNNs) have become the state-of-the-art method in a broad range of applications, from image recognition, to cognitive tasks to control. However, neural network models are typically large and computationally expensive and therefore not deployable on power and memory constrained edge devices. Sparsification techniques have been proposed to reduce the memory foot-print of neural network models. However, they typically lead to substantial hardware and memory overhead. In this article, we propose a hardware-aware pruning method using linear feedback shift register (LFSRs) to generate the locations of non-zero weights in real-time during inference. We call this LFSR-generated pseudo-random sequence based sparsity (LGPS) technique. We explore two different architectures for our hardware-friendly LGPS technique, based on (1) row/column indexing with LFSRs and (2) column-wise indexing with nested LFSRs, respectively. Using the proposed method, we present a total saving of energy and area up to 37.47% and 49.93% respectively and speed up of 1.53x w.r.t the baseline pruning method, for the VGG-16 network on down-sampled ImageNet.
1 December 2023 in IEEE Solid-State Circuits Letters
This letter describes an analog current-summing compute-in-memory macro using resistive random-access memory (RRAM). The readout transimpedance amplifiers use offset cancelling with differential inputs from added sensing paths for the bitline (BL) and sourceline (SL) to minimize channel-to-channel (ch./ch.) gain error while mitigating IR drop in the BL, SL, and multiplexors (MUXes). The analog-to-digital converters (ADCs) use dynamic offset cancellation to remove ch./ch. ADC intrinsic offset and error due to RRAM off-state current. The 64Kb macro implemented with foundry RRAM in 40nm CMOS has an area of 0.0263mm2, ch./ch. gain std. dev. of 1.9%, IR drop per-wordline of 0.004%, and 1.1V efficiency of 7.8-58.8 TOPS/W.
14 August 2020 in IEEE Design & Test
Since the beginning of the third wave of artificial intelligence and in particular, neural networks, there has been tremendous progress in creating hardware accelerators and software tools enabling even faster research and product development. Despite this progress, we eventually face the same problem that has long plagued the computing industry: the von Neumann bottleneck. To surpass this bottleneck we must look into alternative technologies that can avoid both the memory bottleneck and the inherent limitations of the silicon complementary MOS (CMOS) technology. In this paper, we focus on in-memory computing enabled by an emerging class of non-volatile memory devices that exhibit exciting new properties that can expand the capabilities of the next-generation of hardware design. We review new computing architectures, circuits techniques, and four promising device technologies, namely, resistive random access memory (RRAM), spintransfer- torque RAM (STT-MRAM), ferroelectric-FET RAM (FeFET-RAM), and phase change RAM (PCRAM).
19 September 2024 in IEEE Journal of Solid-State Circuits
Accelerators for miniaturized robots addressing tasks such as autonomous surveillance need to balance their compute capabilities against the requirements for low energy use and a compact form factor imposed by the small size of the platforms. Many applications require machine learning (ML) inference for perception tasks as well as estimation of the robot’s own trajectory for localization. The paradigm of using large on-die memories to store deep neural network (DNN) weights on-chip has the potential to yield improved efficiency by reducing off-chip memory accesses. By implementing these large weight stores on-die using an embedded nonvolatile memory (eNVM) technology, density can be improved while leakage can be reduced using power-down modes. Furthermore, the localization workflow requires the evaluation of state equations with concurrent addition operations. This presents a potential bottleneck, motivating a dedicated localization block. We introduce an accelerator combining a resistive random access memory (RRAM)-based inference subsection and a localization accelerator block using an SRAM-like cross-coupled structure. The inference subsection combines INT8 matrix datapaths with 5 MB of RRAM (2.07 $\mathrm{Mb/mm}^2$ considering the $20.25\text{-}\mathrm{mm}^2$ die) at 0.256 pJ/bit and 12.8 GB/s, and supports an SRAM-retentive power-down mode consuming $110~\mathrm{\mu W}$ . At full utilization, at $V_{\text{MIN}}$ , throughput is 102.4 GOPS and efficiency is 0.84 TOPS/W. The localization block allows voltage-pulse-driven data updates to support concurrent in-place addition to address the related bottleneck.