Yue Zhang

Ms. Yue Zhang


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 September 2021 in Applied Sciences
Verified
Journal Article
Open access
PDF file
SZShicheng Zhou
Shicheng Zhou
JLJingju Liu
Jingju Liu
DHDongdong Hou
Dongdong Hou
XZXiaofeng Zhong
Xiaofeng Zhong
YZYue Zhang
Yue Zhang
Penetration testing is an effective way to test and evaluate cybersecurity by simulating a cyberattack. However, the traditional methods deeply rely on domain expert knowledge, which requires prohibitive labor and time costs. Autonomous penetration testing is a more efficient and intelligent way to solve this problem. In this paper, we model penetration testing as a Markov decision process problem and use reinforcement learning technology for autonomous penetration testing in large scale networks. We propose an improved deep Q-network (DQN) named NDSPI-DQN to address the sparse reward problem and large action space problem in large-scale scenarios. First, we reasonably integrate five extensions to DQN, including noisy nets, soft Q-learning, dueling architectures, prioritized experience replay, and intrinsic curiosity model to improve the exploration efficiency. Second, we decouple the action and split the estimators of the neural network to calculate two elements of action separately, so as to decrease the action space. Finally, the performance of algorithms is investigated in a range of scenarios. The experiment results demonstrate that our methods have better convergence and scaling performance.

14 October 2022 in Applied Sciences
Verified
Journal Article
Open access
PDF file
YZYue Zhang
Yue Zhang
JLJingju Liu
Jingju Liu
SZShicheng Zhou
Shicheng Zhou
DHDongdong Hou
Dongdong Hou
XZXiaofeng Zhong
Xiaofeng Zhong
CLCanju Lu
Canju Lu
With the development of technology, people’s daily lives are closely related to networks. The importance of cybersecurity protection draws global attention. Automated penetration testing is the novel method to protect the security of networks, which enhances efficiency and reduces costs compared with traditional manual penetration testing. Previous studies have provided many ways to obtain a better policy for penetration testing paths, but many studies are based on ideal penetration testing scenarios. In order to find potential vulnerabilities from the perspective of hackers in the real world, this paper models the process of black-box penetration testing as a Partially Observed Markov Decision Process (POMDP). In addition, we propose a new algorithm named ND3RQN, which is applied to the automated black-box penetration testing. In the POMDP model, an agent interacts with a network environment to choose a better policy without insider information about the target network, except for the start points. To handle this problem, we utilize a Long Short-Term Memory (LSTM) structure empowering agent to make decisions based on historical memory. In addition, this paper enhances the current algorithm using the structure of the neural network, the calculation method of the Q-value, and adding noise parameters to the neural network to advance the generalization and efficiency of this algorithm. In the last section, we conduct comparison experiments of the ND3RQN algorithm and other recent state-of-the-art (SOTA) algorithms. The experimental results vividly show that this novel algorithm is able to find a greater attack-path strategy for all vulnerable hosts in the automated black-box penetration testing. Additionally, the generalization and robustness of this algorithm are far superior to other SOTA algorithms in different size simulation scenarios based on the CyberBattleSim simulation developed by Microsoft.

16 December 2024 in IEEE Transactions on Dependable and Secure Computing
Verified
Journal Article
Closed access
SZShicheng Zhou
Shicheng Zhou
JLJingju Liu
Jingju Liu
YLYuliang Lu
Yuliang Lu
JYJiahai Yang
Jiahai Yang
DHDongdong Hou
Dongdong Hou
SHShulong Hu
Shulong Hu
Penetration testing (pentesting) assesses cybersecurity through simulated attacks, while the conventional manual-based method is costly, time-consuming, and personnelconstrained. Reinforcement learning (RL) provides an agentenvironment interaction learning paradigm, making it a promising way for autonomous pentesting. However, agents' scalability in large action spaces and policy transferability across scenarios limit the applicability of RL-based autonomous pentesting. To address these challenges, we present a novel autonomous pentesting framework based on reinforcement learning (namely APRIL) to train agents that are scalable and transferable in large action spaces. In APRIL, we construct realistic, bounded, host-level state space via embedding techniques to avoid the complexities of dealing with unbounded network-level information. We employ semantic correlations between pentesting actions as prior knowledge to represent discrete action space into a continuous and semantically meaningful embedding space. Agents are then trained to reason over actions within the action embedding space, where two key methods are applied: an upper-confidence bound-based action refinement method to encourage efficient exploration, and a distance-aware loss to improve learning efficiency and generalization performance. We conduct experiments in simulated scenarios constructed based on virtualized vulnerable environments. The results demonstrate APRIL's scalability in large action spaces and its ability to facilitate policy transfer across diverse scenarios.

8 May 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
SZShicheng Zhou
Shicheng Zhou
JLJingju Liu
Jingju Liu
YLYuliang Lu
Yuliang Lu
JYJiahai Yang
Jiahai Yang
YZYue Zhang
Yue Zhang
SHShulong Hu
Shulong Hu

11 November 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
JLJingju Liu
Jingju Liu
YZYue Zhang
Yue Zhang
SZShicheng Zhou
Shicheng Zhou
JYJiahai Yang
Jiahai Yang
YLYuliang Lu
Yuliang Lu
XZXiaofeng Zhong
Xiaofeng Zhong

12 December 2025 in Frontiers of Information Technology & Electronic Engineering
Verified
Journal Article
Closed access
PDF file
SZShicheng Zhou
Shicheng Zhou
JLJingju Liu
Jingju Liu
YLYuliang Lu
Yuliang Lu
JYJiahai Yang
Jiahai Yang
YZYue Zhang
Yue Zhang
JCJie Chen
Jie Chen
With the increasing number of vulnerabilities exposed on the Internet, autonomous penetration testing (pentesting) has emerged as a promising research area