Qianyue Hao

Qianyue Hao

Auto-generated profile, based on publicly available information

Qianyue Hao

Tsinghua University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 January 2026 in Nature
Verified
Journal Article
Closed access
PDF file
QHQianyue Hao
Qianyue Hao
FXFengli Xu
Fengli Xu
YLYong Li
Yong Li
JEJames Evans
James Evans

4 August 2023
Verified
Conference Paper
Open access
QHQianyue Hao
Qianyue Hao
WHWenzhen Huang
Wenzhen Huang
TFTao Feng
Tao Feng
JYJian Yuan
Jian Yuan
YLYong Li
Yong Li
Recent advancements in reinforcement learning have witnessed remarkable achievements by intelligent agents ranging from game-playing to industrial applications. Of particular interest is the area of multi-agent reinforcement learning (MARL), which holds significant potential for real-world scenarios. However, typical MARL methods are limited in their ability to handle tens of agents, leaving scenarios with up to hundreds or even thousands of agents almost unexplored. The scaling up of the number of agents presents two primary challenges: (1) agent-agent interactions are crucial in multi-agent systems while the number of interactions grows quadratically with the number of agents, resulting in substantial computational complexity and difficulty in strategies-learning; (2) the strengths of interactions among agents exhibit variations both across agents and over time, making it difficult to precisely model such interactions. In this paper, we propose a novel approach named Graph Attention Mean Field (GAT-MF). By converting agent-agent interactions into interactions between each agent and a weighted mean field, we achieve a substantial reduction in computational complexity. The proposed method offers a precise modeling of interaction dynamics with mathematical proofs of its correctness. Additionally, we design a graph attention mechanism to automatically capture the diverse and time-varying strengths of interactions, ensuring an accurate representation of agent interactions. Through extensive experimentation conducted in both manual and real-world scenarios involving over 3000 agents, we validate the efficacy of our method. The results demonstrate that our method outperforms the best baseline method with a remarkable improvement of 42.7%. Furthermore, our method saves 86.4% training time and 19.2% GPU memory compared to the best baseline method. For reproducibility, our source codes and data are available at https://github.com/tsinghua-fib-lab/Large-Scale-MARL-GATMF.

24 December 2024 in ACM Computing Surveys
Verified
Journal Article
Open access
YZYu Zheng
Yu Zheng
QHQianyue Hao
Qianyue Hao
JWJingwei Wang
Jingwei Wang
CGChangzheng Gao
Changzheng Gao
JCJinwei Chen
Jinwei Chen
YLYong Li
Yong Li
Developing smart cities is vital for ensuring sustainable development and improving human well-being. One critical aspect of building smart cities is designing intelligent methods to address various decision-making problems that arise in urban areas. As machine learning techniques continue to advance rapidly, a growing body of research has been focused on utilizing these methods to achieve intelligent urban decision-making. In this survey, we conduct a systematic literature review on the application of machine learning methods in urban decision-making, with a focus on planning, transportation, and healthcare. First, we provide a taxonomy based on typical applications of machine learning methods for urban decision-making. We then present background knowledge on these tasks and the machine learning techniques that have been adopted to solve them. Next, we examine the challenges and advantages of applying machine learning in urban decision-making, including issues related to urban complexity, urban heterogeneity, and computational cost. Afterward and primarily, we elaborate on the existing machine learning methods that aim at solving urban decision-making tasks in planning, transportation, and healthcare, highlighting their strengths and limitations. Finally, we discuss open problems and the future directions of applying machine learning to enable intelligent urban decision-making, such as developing foundation models and combining reinforcement learning algorithms with human feedback. We hope this survey can help researchers in related fields understand the recent progress made in existing works, and inspire novel applications of machine learning in smart cities.

24 August 2024
Verified
Conference Paper
Open access
JWJingwei Wang
Jingwei Wang
QHQianyue Hao
Qianyue Hao
WHWenzhen Huang
Wenzhen Huang
XFXiaochen Fan
Xiaochen Fan
ZTZhentao Tang
Zhentao Tang
YLYong Li
Yong Li
In large-scale metropolis, it is critical to efficiently allocate various resources such as electricity, medical care, and transportation to meet the living demands of citizens, according to the spatio-temporal distributions of resources and demands. Previous researchers have done plentiful work on such problems by leveraging Multi-Agent Reinforcement Learning (MARL) methods, where multiple agents cooperatively regulate and allocate the resources to meet the demands. However, facing the great number of agents in large cities, existing MARL methods lack efficient parameter sharing strategies among agents to reduce computational complexity. There remain two primary challenges in efficient parameter sharing: (1) during the RL training process, the behavior of agents changes significantly, limiting the performance of group parameter sharing based on fixed role division decided before training; (2) the behavior of agents forms complicated action trajectories, where their role characteristics are implicit, adding difficulty to dynamically adjusting agent role divisions during the training process. In this paper, we propose Dynamic Parameter Sharing (DyPS) to solve the above challenges. We design self-supervised learning tasks to extract the implicit behavioral characteristics from the action trajectories of agents. Based on the obtained behavioral characteristics, we propose a hierarchical MARL framework capable of dynamically revising the agent role divisions during the training process and thus shares parameters among agents with the same role, reducing computational complexity. In addition, our framework can be combined with various typical MARL algorithms, including IPPO, MAPPO, etc. We conduct 7 experiments in 4 representative resource allocation scenarios, where extensive results demonstrate our method's superior performance, outperforming the state-of-the-art baseline methods by up to 31%. Our source codes are available at https://github.com/tsinghua-fib-lab/DyPS.

9 November 2022 in ACM Transactions on Intelligent Systems and Technology
Verified
Journal Article
Open access
QHQianyue Hao
Qianyue Hao
FXFengli Xu
Fengli Xu
LCLin Chen
Lin Chen
PHPan Hui
Pan Hui
YLYong Li
Yong Li
With the advent of the COVID-19 pandemic, the shortage in medical resources became increasingly more evident. Therefore, efficient strategies for medical resource allocation are urgently needed. However, conventional rule-based methods employed by public health experts have limited capability in dealing with the complex and dynamic pandemic-spreading situation. In addition, model-based optimization methods such as dynamic programming (DP) fail to work since we cannot obtain a precise model in real-world situations most of the time. Model-free reinforcement learning (RL) is a powerful tool for decision-making; however, three key challenges exist in solving this problem via RL: (1) complex situations and countless choices for decision-making in the real world; (2) imperfect information due to the latency of pandemic spreading; and (3) limitations on conducting experiments in the real world since we cannot set up pandemic outbreaks arbitrarily. In this article, we propose a hierarchical RL framework with several specially designed components. We design a decomposed action space with a corresponding training algorithm to deal with the countless choices, ensuring efficient and real-time strategies. We design a recurrent neural network–based framework to utilize the imperfect information obtained from the environment. We also design a multi-agent voting method, which modifies the decision-making process considering the randomness during model training and, thus, improves the performance. We build a pandemic-spreading simulator based on real-world data, serving as the experimental platform. We then conduct extensive experiments. The results show that our method outperforms all baselines, which reduces infections and deaths by 14.25% on average without the multi-agent voting method and up to 15.44% with it.

20 December 2023 in Journal of Global Health
Verified
Journal Article
Open access
ZHZhenyu Han
Zhenyu Han
LCLin Chen
Lin Chen
QHQianyue Hao
Qianyue Hao
QHQiwei He
Qiwei He
KBKatherine Budeski
Katherine Budeski
YLYong Li
Yong Li
Vaccines are a key resource for curbing the global spread of infectious diseases [1] through effective epidemic control at a relatively low economic cost compared with lockdowns or quarantine measures. The scarcity of vaccine resources during the early stage of disease outbreaks makes them a highly-demanded resource hoarded by high-income countries under the influence of […]