Xuyi Cai

Xuyi Cai

Auto-generated profile, based on publicly available information

Xuyi Cai

Institute of Computing Technology, Chinese Academy of Sciences & University of Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 March 2022 in CCF Transactions on High Performance Computing
Verified
Journal Article
Closed access
PDF file
XMXiaohan Ma
Xiaohan Ma
YWYing Wang
Ying Wang
YWYujie Wang
Yujie Wang
XCXuyi Cai
Xuyi Cai
YHYinhe Han
Yinhe Han

27 April 2024
Verified
Conference Paper
Open access
SQSongyun Qu
Songyun Qu
SZShixin Zhao
Shixin Zhao
BLBing Li
Bing Li
YHYintao He
Yintao He
XCXuyi Cai
Xuyi Cai
YWYing Wang
Ying Wang

22 June 2021
Verified
Conference Paper
Open access
XCXuyi Cai
Xuyi Cai
YWYing Wang
Ying Wang
LZLei Zhang
Lei Zhang
Neural network layer fusion has been proposed to parallelize the inference of neural layers and thus significantly reduces the feature-induced memory accesses. However, how to fuse the neural layers is still a challenging issue that heavily depends on both the network architecture and the specific DNN processor configuration. This work formalizes the layer fusion problem for DNN processors, proves that prior fusion solutions cannot guarantee memory-level optimality, and presents a novel neural network fusion framework, Optimus. Optimus includes an accurate memory cost model to evaluate fusion schemes, and a Computing-Graph (CG) based layer fusion algorithm, which generates high-efficiency layer-fusion schemes for arbitrary network architectures on DNN processors. The proposed off-line and on-line graph-based fusion algorithms can reduce 10.1% - 72.2% off-chip memory traffic and obtain 1.71x - 3.94x energy efficiency over SOTA baselines on DNN workloads, and they bring significant power-efficiency boost to the DNN processors of different architectures and dataflows.

1 August 2022 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
XCXuyi Cai
Xuyi Cai
YWYing Wang
Ying Wang
KTKaijie Tu
Kaijie Tu
CGChengsi Gao
Chengsi Gao
LZLei Zhang
Lei Zhang
In DNN processors, main memory consumes much more energy than arithmetic operations. Therefore, many memory-oriented network scheduling (MONS) techniques are introduced to exploit on-chip data reuse opportunities and reduce accesses to memory. However, to derive the theoretical lower bound of memory overhead for DNNs is still a significant challenge, which also sheds light on how to reach memory-level optimality by means of network scheduling. Prior work on MONS mainly focused on disparate optimization techniques or missed some of the data reusing opportunities in diverse network models, thus their results are likely to deviate from the true memory-optimality that can be achieved in processors. This paper introduces Olympus, which comprehensively considers the entire memory-level DNN scheduling space, formally analyzes the true memory-optimality and also how to reach the memory-optimal schedules for an arbitrary DNN running on a DNN processor. The key idea behind Olympus is to derive a true memory lower-bound regarding both the intra-layer and inter-layer reuse opportunities, which has not been simultaneously explored by prior works. Evaluation on SOTA DNN processors of different architectures shows that Olympus can guarantee the minimum off-chip memory access, and it reduces 12.3-85.6% DRAM access and saves 7.4-70.3% energy on the latest network models.