Mengshi Zhang

Mengshi Zhang

Auto-generated profile, based on publicly available information

Mengshi Zhang

Facebook, Menlo Park, CA, USA

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 April 2019 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
MZMengshi Zhang
Mengshi Zhang
YLYaoxian Li
Yaoxian Li
XLXia Li
Xia Li
LCLingchao Chen
Lingchao Chen
YZYuqun Zhang
Yuqun Zhang
SKSarfraz Khurshid
Sarfraz Khurshid
Manual debugging is notoriously tedious and time-consuming. Therefore, various automated fault localization techniques have been proposed to help with manual debugging. Among the existing fault localization techniques, spectrum-based fault localization (SBFL) is one of the most widely studied techniques due to being lightweight. The focus of the existing SBFL techniques is to consider how to differentiate program entities (i.e., one dimension in program spectra); indeed, this focus is aligned with the ultimate goal of finding the faulty lines of code. Our key insight is to enhance the existing SBFL techniques by additionally considering how to differentiate tests (i.e., the other dimension in program spectra), which, to the best of our knowledge, has not been studied in prior work. We present our basic approach, PRFL, a lightweight technique that boosts SBFL by differentiating tests using PageRank algorithm. Specifically, given the original program spectrum information, PRFL uses PageRank to recompute the spectrum by considering the contributions of different tests. Next, traditional SBFL techniques are applied to the recomputed spectrum to achieve more effective fault localization. On top of PRFL, we explore PRFL+ and PRFLMA, two variants which extend PRFL by optimizing its components and integrating Method-level Aggregation technique, respectively. Though being simple and lightweight, PRFL has been demonstrated to outperform state-of-the-art SBFL techniques significantly (e.g., ranking 39.2% / 82.3% more real/artificial faults within Top-1 compared with the most effective traditional SBFL technique) with low overhead on 395 real faults from 6 Defects4J projects and 96925 artificial faults from 240 GitHub projects. To further validate PRFL's effectiveness, we compare PRFL with multiple recent proposed fault localization techniques (e.g., Multric, Metallaxis and MBFL-hybrid-avg), and the experimental results show that PRFL outperforms them as well. Furthermore, we study the performance of PRFLMA, and the experimental results present it can locate 137 real faults (73.4% / 24.5% more compared with the most effective SBFL/PRFL technique) and 35058 artificial faults (159.6% / 28.1% more than SBFL/PRFL technique) at Top-1. At last, we study the generalizability of PRFL on another benchmark Bugs.jar, and the result shows PRFL can help locate around 30% more faults at Top 1.

30 June 2023 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
CXChenhao Xu
Chenhao Xu
JGJiaqi Ge
Jiaqi Ge
YLYong Li
Yong Li
YDYao Deng
Yao Deng
LGLongxiang Gao
Longxiang Gao
Xi ZhengXi Zheng
Xi Zheng
Federated learning (FL) enables collaborative training of a shared model on edge devices while maintaining data privacy. FL is effective when dealing with independent and identically distributed (iid) datasets, but struggles with non-iid datasets. Various personalized approaches have been proposed, but such approaches fail to handle underlying shifts in data distribution, such as data distribution skew commonly observed in real-world scenarios (e.g., driver behavior in smart transportation systems changing across time and location). Additionally, trust concerns among unacquainted devices and security concerns with the centralized aggregator pose additional challenges. To address these challenges, this paper presents a dynamically optimized personal deep learning scheme based on blockchain and federated learning. Specifically, the innovative smart contract implemented in the blockchain allows distributed edge devices to reach a consensus on the optimal weights of personalized models. Experimental evaluations using multiple models and real-world datasets demonstrate that the proposed scheme achieves higher accuracy and faster convergence compared to traditional federated and personalized learning approaches.

26 May 2023 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
CZChenguang Zhu
Chenguang Zhu
MZMengshi Zhang
Mengshi Zhang
XWXiuheng Wu
Xiuheng Wu
XXXiufeng Xu
Xiufeng Xu
YLYi Li
Yi Li
Modern software systems are complex, and they heavily rely on external libraries developed by different teams and organizations. Such systems suffer from higher instability due to incompatibility issues caused by library upgrades. In this article, we address the problem by investigating the impact of a library upgrade on the behaviors of its clients. We developed CompCheck , an automated upgrade compatibility checking framework that generates incompatibility-revealing tests based on previous examples. CompCheck first establishes an offline knowledge base of incompatibility issues by mining from open source projects and their upgrades. It then discovers incompatibilities for a specific client project, by searching for similar library usages in the knowledge base and generating tests to reveal the problems. We evaluated CompCheck on 202 call sites of 37 open source projects and the results show that CompCheck successfully revealed incompatibility issues on 76 call sites, 72.7% and 94.9% more than two existing techniques, confirming CompCheck ’s applicability and effectiveness.

12 March 2024 in IEEE Transactions on Dependable and Secure Computing
Verified
Journal Article
Closed access
CXChenhao Xu
Chenhao Xu
JGJiaqi Ge
Jiaqi Ge
YDYao Deng
Yao Deng
LGLongxiang Gao
Longxiang Gao
MZMengshi Zhang
Mengshi Zhang
Xi ZhengXi Zheng
Xi Zheng
Federated learning (FL) is a distributed framework for machine learning that enables collaborative training of a shared model across data silos while preserving data privacy. However, the FL aggregation server faces a challenge in waiting for a large volume of model parameters from selected nodes before generating a global model, which leads to inefficient communication and aggregation. Although transmitting only the signs of stochastic gradient descent (SignSGD) reduces the transmission load, it decreases model accuracy, and the time waiting for local model collection remains substantial. Moreover, the security of FL is severely compromised by prevalent poisoning, backdoor, and DDoS attacks, causing ineffective and inaccurate model training. To overcome these challenges, this paper proposes a B lockchain-based A synchronous S ign S GD (BASS) architecture for efficient and secure federated learning. By integrating a blockchain-based semi-asynchronous aggregation scheme with sign-based gradient compression, BASS considerably improves communication and aggregation efficiency, while providing resistance against attacks. Besides, a novel node-summarized sign aggregation algorithm is developed for the blockchain leaders to ensure the convergence and accuracy of the global model. An open-source prototype is developed, on top of which extensive experiments are conducted. The results validate the superiority of BASS in terms of efficiency, model accuracy, and security.

8 October 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
SFSen Fang
Sen Fang
WDWeiyuan Ding
Weiyuan Ding
MZMengshi Zhang
Mengshi Zhang
ZCZihao Chen
Zihao Chen
BXBowen Xu
Bowen Xu
Motivation: Evaluating the programming robustness of large language models (LLMs) is paramount for ensuring their reliability in AI-based software development. Existing robustness assessments commonly employ exogenous adversarial perturbations, where modifications are crafted externally and applied to the model input. Two limitations follow. First, different exogenous attack strategies can produce different rankings across models, making conclusions dependent on the perturbation setting. Second, these assessments do not directly characterize stability when a model's own outputs are transformed and reused as inputs. This setting is relevant to iterative software engineering workflows. Solution: We introduce EvaLooop , a novel assessment framework that evaluates robustness from a self-consistency perspective, leveraging the natural duality inherent in software engineering tasks ( e.g., code generation and code summarization). EvaLooop establishes a self-contained feedback loop where the same LLM iteratively transforms between code and natural language, producing endogenous transformations from its own output distribution. Robustness under these self-generated transformations is quantified by the Average Sustainable Loops (ASL) metric, a normalized score combining quadratically weighted successful-loop counts with semantic similarity. This cyclical strategy identifies when generated code first fails functional tests and provides a complementary view of robustness within the evaluated loops, without requiring external attack configurations. Results: We evaluate 96 popular LLMs, ranging from 0.5B to 685B parameters, on EvaLooop equipped with the MBPP Plus benchmark, and find that EvaLooop typically induces a 2.65%–47.62% absolute drop in Pass@1 accuracy within ten loops. Stability under self-generated transformations does not always align with initial performance ( i.e., a one-shot query); for instance, Qwen3-235B-A22B-Instruct-2507, despite inferior initial code generation compared to OpenAI's o-series models and DeepSeek-V3, achieves a higher ASL score. Impact: EvaLooop reveals distinct degradation patterns across different models, providing developers with a practical framework for evaluating LLM functional coherence through iterative transformations and offering complementary insights that support more informed model selection decisions. A living leaderboard: https://evalooop.github.io/ .

7 March 2018 in arXiv
Verified
Preprint
Open access
PDF file
MZMengshi Zhang
Mengshi Zhang
YZYuqun Zhang
Yuqun Zhang
LZLingming Zhang
Lingming Zhang
CLCong Liu
Cong Liu
SKSarfraz Khurshid
Sarfraz Khurshid
While Deep Neural Networks (DNNs) have established the fundamentals of DNN-based autonomous driving systems, they may exhibit erroneous behaviors and cause fatal accidents. To resolve the safety issues of autonomous driving systems, a recent set of testing techniques have been designed to automatically generate test cases, e.g., new input images transformed from the original ones. Unfortunately, many such generated input images often render inferior authenticity, lacking accurate semantic information of the driving scenes and hence compromising the resulting efficacy and reliability. In this paper, we propose DeepRoad, an unsupervised framework to automatically generate large amounts of accurate driving scenes to test the consistency of DNN-based autonomous driving systems across different scenes. In particular, DeepRoad delivers driving scenes with various weather conditions (including those with rather extreme conditions) by applying the Generative Adversarial Networks (GANs) along with the corresponding real-world weather scenes. Moreover, we have implemented DeepRoad to test three well-recognized DNN-based autonomous driving systems. Experimental results demonstrate that DeepRoad can detect thousands of behavioral inconsistencies in these systems.