Self-attention-based transformers have outperformed recurrent and convolutional neural networks (RNN/ CNNs) in many applications. Despite the effectiveness, calculating self-attention is prohibitively costly due to quadratic computation and memory requirements. To solve this challenge, this article proposes a hybrid analog-ReRAM and digital-SRAM in-memory computing accelerator (HARDSEA), a computing-in-memory (CIM) accelerator supporting self-attention in transformer applications. To trade off between energy efficiency and algorithm accuracy, HARDSEA features an algorithm-architecture-circuit codesign. A product-quantization-based scheme dynamically facilitates self-attention sparsity by predicting lightweight token relevance. A hybrid in-memory computing architecture employs both high-efficiency analog ReRAM-CIM and high-precision digital SRAM-CIM to implement the proposed new scheme. The ReRAM-CIM, whose precision is sensitive to circuit nonidealities, takes charge of token relevance prediction where only computing monotonicity is demanded. The SRAM-CIM, utilized for exact sparse attention computing, is reorganized as an on-memory-boundary computing scheme, thus adapting to irregular sparsity patterns. In addition, we propose a time-domain winner-take-all (WTA) circuit to replace the expensive ADCs in ReRAM-CIM macros. Experimental results show that HARDSEA prunes BERT and GPT-2 models to 12%–33% sparsity without accuracy loss, achieving $13.5\times $ – $28.5\times $ speedup and $291.6\times $ – $1894.3\times $ energy efficiency over GPU. Compared to state-of-the-art transformer accelerators, HARDSEA has $1.2\times $ – $14.9\times $ better energy efficiency at the same level of throughput.
In the last decade, HfO2-based ferroelectric capacitors (FeCaps) have undergone significant advancements, particularly within the realm of nonvolatile ferroelectric random access memories (FeRAMs). Nonetheless, the READ operation in FeRAMs is inherently destructive, rendering it unsuitable for neuromorphic computing. In this study, we have engineered tunable nonvolatile capacitances within FeCaps, featuring nondestructive readout functionality. Robust capacitance states can be read at a zero d.c. bias (Vbias) with different a.c. signals, not only preventing the alteration of their stored state but also benefiting to the low power consumption. Moreover, the capacitance memory window (CMW) at Vbias of zero can be effectively modulated through electrode engineering, leading to a larger CMW when there is a greater disparity in work functions between the electrodes. Furthermore, we provide a comprehensive investigation into synaptic behavior of TiN/Hf0.5Zr0.5O2/Pt FeCaps, demonstrating their excellent cycle-to-cycle uniformity, retention, and endurance characteristics, which confirm their high reliability in maintaining nonvolatile capacitance states. These findings underscore the significant potential of FeCaps in advancing low-power neuromorphic computing.
1 October 2020 in IEEE Journal of Solid-State Circuits
By virtue of hardware parallelism, ternary content addressable memory (TCAM) is attractive for low-latency search for packet forwarding in routers for network communications in the upcoming fifth-generation (5G) era. However, the demand for high-density TCAM encounters remarkable costs in silicon area and power consumption. In this work, a 16-kb nonvolatile ternary content addressable memory (nvTCAM) test chip based on resistive memory (ReRAM) is demonstrated in 28-nm process with two techniques to deal with the aforementioned issues. First, the crossbar array with 2-diode-2-ReRAM (2D2R) nvTCAM cell is proposed to realize >3x improvement in storage density. The back-end-of-line integration of both diode and ReRAM resistor also allows for further 3-D stacking to realize larger storage density. Second, the machine learning concept is exploited to realize intelligent search operation. K-means clustering is employed to allocate the entry storage and then the search of destination IP address can be targeted to a specific bank for low power. The evaluation shows >70% reduction in search energy with 2% overhead in silicon area for bank count of four. The test chip also achieves a match delay of 2 ns under nominal operating conditions.
12 March 2021 in IEEE Transactions on Circuits and Systems II: Express Briefs
In this work, a ReRAM-based energy-efficient CIM accelerator is presented with two techniques for edge AI applications. Firstly, a circuit-algorithm co-design scheme is proposed to realize fully analog processing, which improves the energy efficiency and the throughput of neural network. To deal with the I-V nonlinearity of ReRAM, a nonlinear-aware training algorithm is proposed to improve the network accuracy. Secondly, a 1T2R cell is proposed to replace previous 2T2R for weight storage with 35% area saving. For evaluation, a neural network with two fully connected layers and one ReLU layer is built for the MNIST dataset. The error rate can be reduced by >46% and the energy efficiency is 99 TOPS/W@200 MHz, 2.6X improvement over the digital method.
16 October 2023 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Neuromorphic computing based on spike neural networks (SNNs) exhibits great potential in reducing energy consumption in hardware systems. Resistive random-access memory (ReRAM) is regarded as a promising candidate to construct neuromorphic hardware, attributing to their high-density, nonvolatile, and compute-in-memory capability. However, the ReRAM-based neuromorphic chips are still in their infancy, cannot support multicore or with limited neuron configurability. To alleviate these problems, we propose a hybrid multicore SNN chip based on 60 K-ReRAM synapses and 480-digital neurons in the 180 nm node, achieving a synaptic density of 20K bit/mm 2 per core. To improve the efficiency of inter-core communication, we adopt a network-on-chip architecture with a bit character encoding strategy. In addition, an adaptive multiplier-less digital neuron is designed to support both Izhikevich and leaky integrate-and-fire models through register bit control, meeting different application scenarios. Finally, we evaluate the performance of our chip on the MNIST dataset recognition tasks, achieving 97.65% accuracy. Also, a minimum energy per synaptic operation (SOP) of 6.6 pJ in the 180 nm node is obtained, outperforming the TrueNorth’s 26 pJ in 28 nm. These results show that our design has a great potential for large-scale SNN implementations and may pave the way for designing high-efficient neuromorphic hardware with ReRAM technology.