Huang Dong

Huang Dong


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 December 2025 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
DHDong Huang
Dong Huang
JZJie M. Zhang
Jie M. Zhang
QBQingwen Bu
Qingwen Bu
XXXiaofei Xie
Xiaofei Xie
JCJunjie Chen
Junjie Chen
HCHeming Cui
Heming Cui
As the adoption of LLMs becomes more widespread in software coding ecosystems, a pressing issue has emerged: does the generated code contain social bias and unfairness, such as those related to age, gender, and race? This issue concerns the integrity, fairness, and ethical foundation of software applications that depend on the code generated by these models but are underexplored in the literature. This article presents a novel bias testing framework that is specifically designed for code generation tasks. Based on this framework, we conduct an extensive empirical study on the biases in code generated by five widely studied LLMs (i.e., PALM-2-CodeChat-bison, Claude-instant-1, GPT-3.5-turbo, GPT-4-turbo, and GPT-4). Our findings reveal that biases are prevalent. For example, 13.47% to 49.10% of the codes generated by these LLMs have biased behaviors towards gender. Moreover, we study five bias mitigation prompt strategies that are commonly used in current code generation scenarios, i.e., zero-shot, one-shot, few-shot, and two Chain-of-Thought (CoT) prompts, with and without provided feedback-driven refinement. Our evaluation results illustrate that using direct prompt engineering strategies has limited effectiveness in mitigating bias, but our test execution feedback can help to reduce the ratio of code biases to a large extent (e.g., from 59.88% to 4.79% for GPT-4).

1 June 2021 in Journal of Materials Processing Technology
Verified
Journal Article
Closed access
XJXiaoyuan Ji
Xiaoyuan Ji
QYQiuyu Yan
Qiuyu Yan
DHDong Huang
Dong Huang
BWBo Wu
Bo Wu
XXXiaojing Xu
Xiaojing Xu
MWMenghuai Wu
Menghuai Wu

18 November 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
DHDong Huang
Dong Huang
JZJie M. Zhang
Jie M. Zhang
MHMark Harman
Mark Harman
QZQianru Zhang
Qianru Zhang
MDMingzhe Du
Mingzhe Du
SNSee-Kiong Ng
See-Kiong Ng
Recently, large language models (LLMs) have shown great promise in automating unit test generation, significantly reducing the manual effort required by developers. To effectively evaluate the capabilities of LLMs in this domain, it is crucial to have a well-designed benchmark that accurately reflects real-world scenarios and mitigates common pitfalls. Existing LLM test generation benchmarks are limited by two critical drawbacks: data contamination and structurally simple function code. As a result, scientific conclusions drawn from empirical studies using these benchmarks may be compromised. The evidence presented may be biased due to contamination and may fail to generalize beyond toy programs due to structural simplicity. To address these problems, we introduce ULT (UnLeakedTestbench), a new benchmark specifically designed for function-level unit test generation from real-world Python functions. ULT is constructed through a multi-stage curation process that ensures high cyclomatic complexity and mitigates test case contamination. With 3,909 carefully selected function-level tasks, ULT provides a more realistic and challenging evaluation of LLMs’ test generation capabilities. We also provide PLT (PreLeakedTestbench), a pair of ULT benchmarks with leaked tests designed to enable a controlled analysis of memorization versus reasoning in test generation. Based on the two datasets, we conduct a large-scale empirical study involving 12 state-of-the-art LLMs, comparing their performance against established benchmarks. Our evaluation results demonstrate that ULT is significantly more challenging. For example, test cases generated by LLMs only achieve 41.32%, 45.10%, 30.22%, and 40.21% for accuracy, statement coverage, branch coverage, and mutation score on average for all LLMs, respectively. These results are substantially lower than the corresponding metrics on TestEval (91.79%, 92.18%, 82.04%, and 49.69%) and PLT (47.07%, 55.13%, 40.07%, and 50.80%). In addition, different from existing benchmarks, ULT shows a strong correlation between test generation performance and code generation performance. For example, the correlation coefficient between the coding ability and test generation performance ( \(Pass@1\) ) on ULT is 0.79 (p = 0.002), while it is only 0.56 (p = 0.059) and 0.52 (p = 0.080) on TestEval and PLT, respectively. This indicates that ULT more effectively measures the generalization ability of LLMs. We also make ULT and evaluation results publicly available to foster further research (To preserve the integrity of the benchmark and prevent test case leakage into future LLM training sets, we do not release the ground-truth tests. Instead, we provide the complete evaluation results for all models benchmarked in this article. This allows researchers to compare new models against our findings without compromising the benchmark). ULT is available at https://github.com/huangd1999/UnLeakedTestBench .

27 September 2024 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
DHDong Huang
Dong Huang
QBQingwen Bu
Qingwen Bu
YFYichao Fu
Yichao Fu
YQYuhao Qing
Yuhao Qing
XXXiaofei Xie
Xiaofei Xie
HCHeming Cui
Heming Cui
Deep neural networks (DNNs) have been widely deployed in software to address various tasks (e.g., autonomous driving, medical diagnosis). However, they can also produce incorrect behaviors that result in financial losses and even threaten human safety. To reveal and repair incorrect behaviors in DNNs, developers often collect rich, unlabeled datasets from the natural world and label them to test DNN models. However, properly labeling a large number of datasets is a highly expensive and time-consuming task. To address the above-mentioned problem, we propose neuron sensitivity-guided test case selection (NSS), which can reduce the labeling time by selecting valuable test cases from unlabeled datasets. NSS leverages the information of the internal neuron induced by the test cases to select valuable test cases, which have high confidence in causing the model to behave incorrectly. We evaluated NSS with four widely used datasets and four well-designed DNN models compared to the state-of-the-art (SOTA) baseline methods. The results show that NSS performs well in assessing the probability of failure triggering in test cases and in the improvement capabilities of the model. Specifically, compared to the baseline approaches, NSS achieves a higher fault detection rate (e.g., when selecting 5% of the test cases from the unlabeled dataset in the MNIST and LeNet1 experiment, NSS can obtain an 81.8% fault detection rate, which is a 20% increase compared with SOTA baseline strategies).

21 August 2025 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZZZongyuan Zhang
Zongyuan Zhang
TDTianyang Duan
Tianyang Duan
ZLZheng Lin
Zheng Lin
DHDong Huang
Dong Huang
ZFZihan Fang
Zihan Fang
YCYong Cui
Yong Cui
Recently, deep reinforcement learning (DRL) has emerged as a promising approach for robotic control. However, the deployment of DRL in real-world robots is hindered by its sensitivity to environmental perturbations. While existing white-box adversarial attacks rely on local gradient information and apply uniform perturbations across all states to evaluate DRL robustness, they fail to account for temporal dynamics and state-specific vulnerabilities. To combat the above challenge, we first conduct a theoretical analysis of white-box attacks in DRL by establishing the Adversarial Victim Dynamics Markov Decision Process (AVD-MDP), to derive the necessary and sufficient conditions for a successful attack. Based on this, we propose the Selective State-Aware Reinforcement adversarial attack (STAR), to optimize perturbation stealthiness and state visitation dispersion. STAR first employs a soft mask-based state-targeting mechanism to minimize redundant perturbations, enhancing stealthiness and attack effectiveness. Then, it incorporates an information-theoretic optimization objective to maximize mutual information between perturbations, environmental states, and victim actions, ensuring a dispersed state-visitation distribution that steers the victim agent into vulnerable states for maximum return reduction. Extensive experiments demonstrate that STAR outperforms state-of-the-art benchmarks.

10 November 2025
Verified
Conference Paper
Open access
QZQianru Zhang
Qianru Zhang
XGXinyi Gao
Xinyi Gao
HWHaixin Wang
Haixin Wang
DHDong Huang
Dong Huang
SYSiu Ming Yiu
Siu Ming Yiu
HYHongzhi Yin
Hongzhi Yin