Keji Zhou

Dr. Keji Zhou


Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 December 2023 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Verified
Journal Article
Closed access
SLShiwei Liu
Shiwei Liu
CMChen Mu
Chen Mu
HJHao Jiang
Hao Jiang
YWYunzhengmao Wang
Yunzhengmao Wang
JZJinshan Zhang
Jinshan Zhang
CCChixiao Chen
Chixiao Chen
Self-attention-based transformers have outperformed recurrent and convolutional neural networks (RNN/ CNNs) in many applications. Despite the effectiveness, calculating self-attention is prohibitively costly due to quadratic computation and memory requirements. To solve this challenge, this article proposes a hybrid analog-ReRAM and digital-SRAM in-memory computing accelerator (HARDSEA), a computing-in-memory (CIM) accelerator supporting self-attention in transformer applications. To trade off between energy efficiency and algorithm accuracy, HARDSEA features an algorithm-architecture-circuit codesign. A product-quantization-based scheme dynamically facilitates self-attention sparsity by predicting lightweight token relevance. A hybrid in-memory computing architecture employs both high-efficiency analog ReRAM-CIM and high-precision digital SRAM-CIM to implement the proposed new scheme. The ReRAM-CIM, whose precision is sensitive to circuit nonidealities, takes charge of token relevance prediction where only computing monotonicity is demanded. The SRAM-CIM, utilized for exact sparse attention computing, is reorganized as an on-memory-boundary computing scheme, thus adapting to irregular sparsity patterns. In addition, we propose a time-domain winner-take-all (WTA) circuit to replace the expensive ADCs in ReRAM-CIM macros. Experimental results show that HARDSEA prunes BERT and GPT-2 models to 12%–33% sparsity without accuracy loss, achieving $13.5\times $ – $28.5\times $ speedup and $291.6\times $ – $1894.3\times $ energy efficiency over GPU. Compared to state-of-the-art transformer accelerators, HARDSEA has $1.2\times $ – $14.9\times $ better energy efficiency at the same level of throughput.

12 June 2022 in Chip
Verified
Journal Article
Open access
KZKeji Zhou
Keji Zhou
CMChen Mu
Chen Mu
BWBo Wen
Bo Wen
XZXu-Meng Zhang
Xu-Meng Zhang
GWGuang-Jian Wu
Guang-Jian Wu
QLQi Liu
Qi Liu

4 March 2024 in Applied Physics Letters
Verified
Journal Article
Closed access
PDF file
SWShuyu Wu
Shuyu Wu
XZXumeng Zhang
Xumeng Zhang
Rongrong CaoRongrong Cao
Rongrong Cao
KZKeji Zhou
Keji Zhou
JLJikai Lu
Jikai Lu
QLQi Liu
Qi Liu
In the last decade, HfO2-based ferroelectric capacitors (FeCaps) have undergone significant advancements, particularly within the realm of nonvolatile ferroelectric random access memories (FeRAMs). Nonetheless, the READ operation in FeRAMs is inherently destructive, rendering it unsuitable for neuromorphic computing. In this study, we have engineered tunable nonvolatile capacitances within FeCaps, featuring nondestructive readout functionality. Robust capacitance states can be read at a zero d.c. bias (Vbias) with different a.c. signals, not only preventing the alteration of their stored state but also benefiting to the low power consumption. Moreover, the capacitance memory window (CMW) at Vbias of zero can be effectively modulated through electrode engineering, leading to a larger CMW when there is a greater disparity in work functions between the electrodes. Furthermore, we provide a comprehensive investigation into synaptic behavior of TiN/Hf0.5Zr0.5O2/Pt FeCaps, demonstrating their excellent cycle-to-cycle uniformity, retention, and endurance characteristics, which confirm their high reliability in maintaining nonvolatile capacitance states. These findings underscore the significant potential of FeCaps in advancing low-power neuromorphic computing.

1 October 2020 in IEEE Journal of Solid-State Circuits
Verified
Journal Article
Closed access
KZKeji Zhou
Keji Zhou
XXXiaoyong Xue
Xiaoyong Xue
Jianguo YangJianguo Yang
Jianguo Yang
XXXiaoxin Xu
Xiaoxin Xu
HLHangbing Lv
Hangbing Lv
MLMing Liu
Ming Liu
By virtue of hardware parallelism, ternary content addressable memory (TCAM) is attractive for low-latency search for packet forwarding in routers for network communications in the upcoming fifth-generation (5G) era. However, the demand for high-density TCAM encounters remarkable costs in silicon area and power consumption. In this work, a 16-kb nonvolatile ternary content addressable memory (nvTCAM) test chip based on resistive memory (ReRAM) is demonstrated in 28-nm process with two techniques to deal with the aforementioned issues. First, the crossbar array with 2-diode-2-ReRAM (2D2R) nvTCAM cell is proposed to realize >3x improvement in storage density. The back-end-of-line integration of both diode and ReRAM resistor also allows for further 3-D stacking to realize larger storage density. Second, the machine learning concept is exploited to realize intelligent search operation. K-means clustering is employed to allocate the entry storage and then the search of destination IP address can be targeted to a specific bank for low power. The evaluation shows >70% reduction in search energy with 2% overhead in silicon area for bank count of four. The test chip also achieves a match delay of 2 ns under nominal operating conditions.

12 March 2021 in IEEE Transactions on Circuits and Systems II: Express Briefs
Verified
Journal Article
Closed access
KZKeji Zhou
Keji Zhou
CZChenyang Zhao
Chenyang Zhao
JFJinbei Fang
Jinbei Fang
JJJingwen Jiang
Jingwen Jiang
DCDeyang Chen
Deyang Chen
XZXiaoyang Zeng
Xiaoyang Zeng
In this work, a ReRAM-based energy-efficient CIM accelerator is presented with two techniques for edge AI applications. Firstly, a circuit-algorithm co-design scheme is proposed to realize fully analog processing, which improves the energy efficiency and the throughput of neural network. To deal with the I-V nonlinearity of ReRAM, a nonlinear-aware training algorithm is proposed to improve the network accuracy. Secondly, a 1T2R cell is proposed to replace previous 2T2R for weight storage with 35% area saving. For evaluation, a neural network with two fully connected layers and one ReLU layer is built for the MNIST dataset. The error rate can be reduced by >46% and the energy efficiency is 99 TOPS/W@200 MHz, 2.6X improvement over the digital method.

16 October 2023 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
Verified
Journal Article
Closed access
HJHao Jiang
Hao Jiang
JLJikai Lu
Jikai Lu
CZChenggao Zhang
Chenggao Zhang
STShuangzhu Tang
Shuangzhu Tang
JAJunjie An
Junjie An
QLQi Liu
Qi Liu
Neuromorphic computing based on spike neural networks (SNNs) exhibits great potential in reducing energy consumption in hardware systems. Resistive random-access memory (ReRAM) is regarded as a promising candidate to construct neuromorphic hardware, attributing to their high-density, nonvolatile, and compute-in-memory capability. However, the ReRAM-based neuromorphic chips are still in their infancy, cannot support multicore or with limited neuron configurability. To alleviate these problems, we propose a hybrid multicore SNN chip based on 60 K-ReRAM synapses and 480-digital neurons in the 180 nm node, achieving a synaptic density of 20K bit/mm 2 per core. To improve the efficiency of inter-core communication, we adopt a network-on-chip architecture with a bit character encoding strategy. In addition, an adaptive multiplier-less digital neuron is designed to support both Izhikevich and leaky integrate-and-fire models through register bit control, meeting different application scenarios. Finally, we evaluate the performance of our chip on the MNIST dataset recognition tasks, achieving 97.65% accuracy. Also, a minimum energy per synaptic operation (SOP) of 6.6 pJ in the 180 nm node is obtained, outperforming the TrueNorth’s 26 pJ in 28 nm. These results show that our design has a great potential for large-scale SNN implementations and may pave the way for designing high-efficient neuromorphic hardware with ReRAM technology.