Xing Su

Xing Su


Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 May 2024
Verified
Conference Paper
Open access
XFXiao Fu
Xiao Fu
WYWeiling Yang
Weiling Yang
DDDezun Dong
Dezun Dong
XSXing Su
Xing Su
Transformers reign supreme in natural language processing, representing a milestone innovation in deep learning. For high-performance model inference, optimizing the time-consuming attention module is crucial. Owing to the irregular-shaped matrix workloads and intricate data access patterns, the attention operator is bounded by memory bandwidth. Existing works utilize kernel fusion to reduce memory access overhead, resulting in promising performance enhancements. However, these efforts primarily focus on GPU or X86 architectures, leaving ARM multi-cores, commonly encountered in emerging HPC systems, insufficiently explored. We present MEATTEN, a memory-efficient attention fusion scheme and batched approach to exploit ARM multi-core CPUs effectively. It builds on fused micro-kernels and a new data layout suitable for SIMD vectorization. An analytic model is used to guide loop permutation, tiling, and batched parallelization according to the on-chip hierarchical memory architecture and workload characterization. We apply MEATTEN to three representative ARM multi-cores against state-of-the-art libraries and compilers. Experimental results demonstrate that our approach consistently outperforms prior approaches across various evaluation scenarios and platforms.

10 January 2024 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
WYWeiling Yang
Weiling Yang
Jianbin FangJianbin Fang
Jianbin Fang
DDDezun Dong
Dezun Dong
XSXing Su
Xing Su
ZWZheng Wang
Zheng Wang
General Matrix Multiplication (GEMM) is a key subroutine in high-performance computing. While the mainstream Basic Linear Algebra Subprograms (BLAS) libraries can deliver good performance on large and regular-shaped GEMMs, they are inadequate for optimizing small and irregular-shaped GEMMs, which are commonly seen in emerging HPC applications. Recent research has focused on improving GEMM performance on GPUs, but there is still significant room for improvement on emerging HPC hardware based on multi-core CPUs. We present LibShalom2 , an open-source library to optimize full-spectrum GEMMs, taking small, irregular-shaped, and large-scale regular-shaped matrices. LibShalom2 explicitly targets the ARMv8 architecture, which is becoming common in HPC systems. LibShalom2 is designed to minimize the expensive memory accessing overhead for data packing and processing small matrices. It uses analytic methods to determine GEMM kernel optimization parameters, enhancing the computation and parallelization efficiency of the GEMM kernels. We evaluate LibShalom2 by applying it to three ARMv8 multi-core architectures and comparing it against five mainstream linear algebra libraries. Experimental results show that LibShalom2 consistently outperforms existing solutions across full-spectrum GEMM workloads and hardware architectures. We also show that LibShalom2 delivers an average speedup of 2.2x for real-life neural network workloads.

10 October 2018 in ACM Transactions on Architecture and Code Optimization
Verified
Journal Article
Open access
XSXing Su
Xing Su
XLXiangke Liao
Xiangke Liao
HJHao Jiang
Hao Jiang
CYCanqun Yang
Canqun Yang
JXJingling Xue
Jingling Xue
GEneral Matrix Multiply (GEMM) is the most fundamental computational kernel routine in the BLAS library. To achieve high performance, in-memory data must be prefetched into fast on-chip caches before they are used. Two techniques, software prefetching and data packing, have been used to effectively exploit the capability of on-chip least recent used (LRU) caches, which are popular in traditional high-performance processors used in high-end servers and supercomputers. However, the market has recently witnessed a new diversity in processor design, resulting in high-performance processors equipped with shared caches with non-LRU replacement policies. This poses a challenge to the development of high-performance GEMM in a multithreaded context. As several threads try to load data into a shared cache simultaneously, interthread cache conflicts will increase significantly. We present a Shared Cache Partitioning (SCP) method to eliminate interthread cache conflicts in the GEMM routines, by partitioning a shared cache into physically disjoint sets and assigning different sets to different threads. We have implemented SCP in the OpenBLAS library and evaluated it on Phytium 2000+, a 64-core AArch64 processor with private LRU L1 caches and shared pseudo-random L2 caches (per four-core cluster). Our evaluation shows that SCP has effectively reduced the conflict misses in both L1 and L2 caches in a highly optimized GEMM implementation, resulting in an improvement of its performance by 2.75% to 6.91%.

27 November 2018 in Electronics
Verified
Journal Article
Open access
PDF file
XSXing Su
Xing Su
FLFei Lei
Fei Lei
The Basic Linear Algebra Subprograms (BLAS) is a fundamental numerical software and GEneral Matrix Multiply (GEMM) is the most important computational kernel routine in the BLAS library. On multi-core and many-core processors, the whole workload of GEMM is partitioned and scheduled to multiple threads to exploit the parallel hardware. Generally, the workload is equally partitioned among threads and all threads are expected to accomplish their work in roughly the same time. However, this is not the case on Non-Uniform Memory Access (NUMA) architectures. The NUMA effect may cause threads to run at different speeds, and the overall executing times of GEMM is determined by the slowest thread. In this paper, we propose a hybrid-grained dynamic load-balancing method to reduce the harm of the NUMA effect by allowing fast threads to steal work from slow ones. We evaluate the proposed method on Phytium 2000+, an emerging 64-core high-performance processor based on Arm’s AArch64 architecture. Results show that our method reduces the synchronization overhead by 51.5% and achieves an improvement of GEMM performance by 1.9%.

10 November 2026 in ACM Transactions on Architecture and Code Optimization
Verified
Journal Article
Open access
YWYanghai Wang
Yanghai Wang
EYEnda Yu
Enda Yu
XSXing Su
Xing Su
DDDezun Dong
Dezun Dong
ZLZiyu Liu
Ziyu Liu
YTYuhua Tang
Yuhua Tang
The rapid advancement of deep learning models has led to their widespread application across various domains; however, this has resulted in a significant increase in both hardware and software costs, necessitating optimizations for resource efficiency. To address this issue, performance prediction techniques evaluate the performance impact of corresponding optimizations on deep learning systems to maximize performance gains. This paper presents a comprehensive survey of performance prediction techniques for deep learning systems, categorizing them into allocation, cluster, and accelerator levels based on the system level at which their corresponding optimizations are situated, ranging from high to low. Allocation-level techniques primarily construct the relationships between resource configurations and their performance, cluster-level techniques model the optimizations within deep learning frameworks at given configurations, while accelerator-level techniques assess the efficiency of specific hardware. We point out that prediction techniques at different levels exhibit a hierarchical relationship rather than operating in isolation. High-level prediction techniques can incorporate the results from low-level techniques into their performance models to accommodate more low-level optimizations and improve their prediction accuracy. However, this will slow down the construction and execution speed of performance models, requiring trade-offs for specific use cases. Finally, we discuss future challenges and research directions in performance prediction techniques for deep learning systems.