Jingwei Sun

Jingwei Sun

Auto-generated profile, based on publicly available information

Jingwei Sun

University of Science and Technology of China, Hefei, China

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 January 2020 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
JSJingwei Sun
Jingwei Sun
GSGuangzhong Sun
Guangzhong Sun
SZShiyan Zhan
Shiyan Zhan
JZJiepeng Zhang
Jiepeng Zhang
YCYong Chen
Yong Chen
Automated performance modeling and performance prediction of parallel programs are highly valuable in many use cases, such as in guiding task management and job scheduling, offering insights of application behaviors, and assisting resource requirement estimation. The performance of parallel programs is affected by numerous factors, including but not limited to hardware, applications, algorithms, and input parameters, thus an accurate performance prediction is often a challenging and daunting task. In this article, we focus on automatically predicting the execution time of parallel programs (more specifically, MPI programs) with different inputs, at different scales, and without domain knowledge. We model the correlation between the execution time and domain-independent runtime features. These features include values of variables, counters of branches, loops, and MPI communications. Through automatically instrumenting an MPI program, each execution of the program will output a feature vector and its corresponding execution time. After collecting data from executions with different inputs, a random forest machine learning approach is used to build an empirical performance model, which can predict the execution time of the program given a new input. A transfer learning method is used to reuse an existing performance model and improve the prediction accuracy on a new platform that lacks historical execution data. Our experiments and analyses of three parallel applications, Graph500, GalaxSee, and SMG2000, on three different systems confirm that our method performs well, with less than 20 percent prediction error on average.

19 November 2024 in ACM Transactions on Architecture and Code Optimization
Verified
Journal Article
Open access
JLJunqing Lin
Junqing Lin
JSJingwei Sun
Jingwei Sun
XSXiaolong Shi
Xiaolong Shi
HZHonghe Zhang
Honghe Zhang
XYXianzhi Yu
Xianzhi Yu
GSGuangzhong Sun
Guangzhong Sun
As deep neural networks (DNNs) become increasingly large and complicated, pruning techniques are proposed for lower memory footprint and more efficient inference. The most critical kernel to execute pruned sparse DNNs on GPUs is Sparse-dense Matrix Multiplication (SpMM). To maximize the performance of SpMM, despite the high-performance implementation generated from advanced tensor compilers, they often take a long time to iteratively search tuning configurations. Such a long time slows down the cycle of exploring better DNN architectures or pruning algorithms. In this article, we propose LO-SpMM to efficiently generate high-performance SpMM implementations for sparse DNN inference. Based on the analysis of nonzero elements’ layout, the characterization of the GPU architecture, and a rank-based cost model, LO-SpMM can effectively reduce the search space and eliminate possibly low-performance candidates. Besides, rather than generating complete SpMM implementations for evaluation, LO-SpMM constructs simplified proxies to quickly estimate performance, thereby substantially reducing compilation and execution costs. Experimental results show that LO-SpMM can reduce the search time by 281× at most, while the performance of generated SpMM implementations is comparable to or better than the state-of-the-art sparse tensor compiling solutions.

1 November 2022 in IEEE Transactions on Parallel and Distributed Systems
Verified
Journal Article
Closed access
JSJingwei Sun
Jingwei Sun
TYTao Yan
Tao Yan
HSHao Sun
Hao Sun
HLHuancheng Lin
Huancheng Lin
GSGuangzhong Sun
Guangzhong Sun
In high performance computing (HPC) systems, collecting and replaying communication traces are fundamental approaches to analyze performance. With increasingly large-scale HPC systems and applications, tracing tools can produce huge trace data that is costly and challenging to store and analyze. Due to the inherent repetition of behaviors of HPC applications, domain-aware data compression methods can effectively reduce the storage cost of trace data. This study proposes LCR (Lossy Compression and Replay), a framework that aggressively compresses and replays MPI communication traces. Differing from existing trace compression methods, which explicitly identify loop and synchronization structures of communication events, LCR models traces as time series and compactly represents them by lightweight recurrent neural networks. Experimental results demonstrate that LCR can further reduce the size of irregular traces by three orders of magnitude at most, compared with existing structural methods. Meanwhile, LCR accurately reproduces performance and communication patterns of original MPI programs.


7 August 2023
Verified
Conference Paper
Open access
JLJunqing Lin
Junqing Lin
HZHonghe Zhang
Honghe Zhang
XSXiaolong Shi
Xiaolong Shi
JSJingwei Sun
Jingwei Sun
XYXianzhi Yu
Xianzhi Yu
GSGuangzhong Sun
Guangzhong Sun
As deep neural networks (DNNs) become increasingly large and complicated, pruning techniques are proposed for lower memory footprint and more efficient inference. The most critical kernel to execute pruned sparse DNNs on GPUs is Sparse-dense Matrix Multiplication (SpMM). To maximize the performance of SpMM, despite the high-performance code generated from recent tensor compilers, they often take a long time for iteratively searching candidate configurations. Such a long time slows down the cycle of exploring better DNN architectures or pruning algorithms. In this paper, we propose EC-SpMM to efficiently generate high-performance SpMM kernels for sparse DNN inference. Based on the analysis of nonzero elements’ layout, the characterization of GPU architecture, and a rank-based cost model, EC-SpMM can effectively reduce the search space and eliminate possibly low-performance candidates. Experimental results show that EC-SpMM can reduce the compilation time by a factor of 35 ×, while the performance of generated SpMM kernels is comparable or even better, compared with the state-of-the-art sparse tensor compiling solution.

21 October 2024
Verified
Conference Paper
Open access
YHYu He
Yu He
YXYiyu Xu
Yiyu Xu
JSJingwei Sun
Jingwei Sun
GSGuangzhong Sun
Guangzhong Sun
In educational data mining, concept prerequisite relations extraction determines which concepts need to be learned before learning another concept. It plays a crucial role in pedagogical practices, such as learning path planning and curriculum design. Deep neural networks, especially graph neural networks, have recently made significant strides in concept prerequisite relations extraction. However, existing methods face two primary limitations. (1) Methods with better performance construct heterogeneous complete graphs, leading to higher model complexity and training cost. Meanwhile, the performance of low-complexity methods is inferior to the former. (2) A disregard for temporal context, essential for learning, limits both the performance and the application of these methods. To address these issues, we propose a novel graph-based approach, called Learning-path based Concept Prerequisite Relations Extraction (LCPRE). LCPRE constructs a lightweight sparse graph in a simple manner, which reduces complexity from quadratic to linear and captures the temporal feature through learning-path, a comprehensible learning approach from one concept to another. Experimental results on three benchmark datasets demonstrate that LCPRE outperforms existing methods, establishing a new state-of-the-art in concept prerequisite relations extraction.