Kexin Pei

Kexin Pei

Auto-generated profile, based on publicly available information

Kexin Pei

University of Chicago, Chicago, USA

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 February 2024
Verified
Conference Paper
Open access
YDYangruibo Ding
Yangruibo Ding
BSBenjamin Steenhoek
Benjamin Steenhoek
KPKexin Pei
Kexin Pei
GKGail Kaiser
Gail Kaiser
WLWei Le
Wei Le
BRBaishakhi Ray
Baishakhi Ray
Most existing pre-trained language models for source code focus on learning the static code text, typically augmented with static code structures (abstract syntax tree, dependency graphs, etc.). However, program semantics will not be fully exposed before the real execution. Without an understanding of the program execution, statically pre-trained models fail to comprehensively capture the dynamic code properties, such as the branch coverage and the runtime variable values, and they are consequently less effective at code understanding tasks, such as retrieving semantic clones and detecting software vulnerabilities. To close the gap between the static nature of language models and the dynamic characteristics of programs, we introduce TRACED, an execution-aware pre-training strategy for source code. Specifically, we pre-train code language models with a combination of source code, executable inputs, and corresponding execution traces. Our goal is to teach code models the complicated execution logic during the pre-training, enabling the model to statically estimate the dynamic code properties without repeatedly executing code during task-specific fine-tuning. To illustrate the effectiveness of our proposed approach, we fine-tune and evaluate TRACED on three downstream tasks: static execution estimation, clone retrieval, and vulnerability detection. The empirical results show that TRACED relatively improves the statically pre-trained code models by 12.4% for complete execution path prediction and by 25.2% for runtime variable value predictions. TRACED also significantly outperforms statically pre-trained models in clone retrieval and vulnerability detection across four public benchmarks.

28 December 2022 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
KPKexin Pei
Kexin Pei
ZXZhou Xuan
Zhou Xuan
JYJunfeng Yang
Junfeng Yang
SJSuman Jana
Suman Jana
BRBaishakhi Ray
Baishakhi Ray
Detecting semantically similar binary functions – a crucial capability with broad security usages including vulnerability detection, malware analysis, and forensics – requires understanding function behaviors and intentions. This task is challenging as semantically similar functions can be compiled to run on different architectures and with diverse compiler optimizations or obfuscations. Most existing approaches match functions based on syntactic features without understanding the functions' execution semantics. We present Trex , a transfer-learning-based framework, to automate learning approximate execution semantics explicitly from functions' traces collected via forced-execution ( i.e., by violating the control flow semantics) and transfer the learned knowledge to match semantically similar functions. While it is known that forced-execution traces are too imprecise to be directly used to detect semantic similarity, our key insight is that these traces can instead be used to teach an ML model approximate execution semantics of diverse instructions and their compositions. We thus design a pretraining task, which trains the model to learn approximate execution semantics from the two modalities ( i.e., forced-executed code and traces) of the function. We then finetune the pretrained model to match semantically similar functions. We evaluate Trex on 1,472,066 functions from 13 popular software projects, compiled to run on 4 architectures (x86, x64, ARM, and MIPS), and with 4 optimizations ( O0 - O3 ) and 5 obfuscations. Trex outperforms the state-of-the-art solutions by 7.8%, 7.2%, and 14.3% in cross-architecture, optimization, and obfuscation function matching, respectively, while running 8× faster. Ablation studies suggest that the pretraining significantly boosts the function matching performance, underscoring the importance of learning execution semantics. Our case studies demonstrate the practical use-cases of Trex – on 180 real-world firmware images, Trex uncovers 14 vulnerabilities not disclosed by previous studies. We release the code and dataset of Trex at https://github.com/CUMLSec/trex .

12 April 2026
Verified
Conference Paper
Open access
JZJie Zhu
Jie Zhu
CSChihao Shen
Chihao Shen
ZLZiyang Li
Ziyang Li
JYJiahao Yu
Jiahao Yu
YCYizheng Chen
Yizheng Chen
KPKexin Pei
Kexin Pei