Wei Pan

Dr. Wei Pan


Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 September 2021 in IEEE Transactions on Cognitive and Developmental Systems
Verified
Journal Article
Closed access
WPWei Pan
Wei Pan
NWNanding Wang
Nanding Wang
CXChenxi Xu
Chenxi Xu
KHKao-Shing Hwang
Kao-Shing Hwang
Multi-agent reinforcement learning is widely used and can successfully solve many problems in the real world. In the multi-agent reinforcement learning system, a global critic network is used to guide each agent’s strategy to update to learn the most beneficial strategy for the collective. However, the global critic network also makes the current agent’s learning be affected by other agents’ strategies, which leads to unstable learning. To solve this problem, we propose Dynamic Decomposed Multi-Agent Deep Deterministic Policy Gradient (DD-MADDPG): a new network that considers both global and local evaluations and adaptively adjusts the agent’s attention to the two evaluations. Besides, the use of the experience replay buffer by Multi-Agent Deep Deterministic Policy Gradient(MADDPG) produces outdated experience, and the outdated strategies of other agents further affect the learning of the current agent. To reduce the influence of other agents’ outdated experience, we propose TDError and Time-based experience sampling (T2-PER) based on DD-MADDPG. We evaluate the proposed algorithm’s performance according to the learning stability and the average return obtained by the agents. We have conducted experiments in the MPE environment. The results show that the proposed method has better stability and higher learning efficiency than MADDPG and has a certain generalization ability.


7 March 2023 in Studies in Informatics and Control
Verified
Journal Article
Open access
WPWei Pan
Wei Pan
ZSZepei Sun
Zepei Sun
HSHuanyu Sang
Huanyu Sang
ZWZihao Wang
Zihao Wang

9 February 2023 in International Journal of Computers Communications & Control
Verified
Journal Article
Open access
PDF file
WPWei Pan
Wei Pan
CLCheng Liu
Cheng Liu
Multi-Agent Reinforcement Learning (MARL) is widely used to solve various real-world problems. In MARL, the environment contains multiple agents. A good grasp of the environment can guide agents to learn cooperative strategies. In Centralized Training Decentralized Execution (CTDE), a centralized critic is used to guide cooperative strategies learning. However, having multiple agents in the environment leads to the curse of dimensionality and influence of other agents’ strategies, resulting in difficulties for centralized critics to learn good cooperative strategies. We propose a graph-based approach to overcome the above problems. It uses a graph neural network, which uses partial observations of agents as input, and information between agents is aggregated by graph methods to extract information about the whole environment. In this way, agents can improve their understanding of the overall state of the environment and other agents in the environment while avoiding dimensional explosion. Then we combine a dual critic dynamic decomposition method with soft actor-critic to train policy. The former uses individual and global rewards for learning, avoiding the influence of other agents’ strategies, and the latter help to learn an optional policy better. We call this approach Multi-Agent Graph-based soft Actor-Critic (MAGAC). We compare our proposed method with several classical MARL algorithms under the Multi-agent Particle Environment (MPE). The experimental results show that our method can achieve a faster learning speed while learning better policy.

9 November 2016 in Advances in Intelligent Systems and Computing
Verified
Book Chapter
Closed access
PDF file
WPWei Pan
Wei Pan
MWMengzi Wang
Mengzi Wang
YFYuanyuan Fu
Yuanyuan Fu
HSHaobin Shi
Haobin Shi

22 June 2017 in arXiv
Verified
Preprint
Open access
PDF file
QCQun Chen
Qun Chen
SBSong Bai
Song Bai
ZLZhanhuai Li
Zhanhuai Li
ZGZhiying Gou
Zhiying Gou
BSBo Suo
Bo Suo
WPWei Pan
Wei Pan
The Bulk Synchronous Parallel(BSP) computational model has emerged as the dominant distributed framework to build large-scale iterative graph processing systems. While its implementations(e.g., Pregel, Giraph, and Hama) achieve high scalability, frequent synchronization and communication among the workers can cause substantial parallel inefficiency. To help address this critical concern, this paper introduces the GraphHP(Graph Hybrid Processing) platform which inherits the friendly vertex-centric BSP programming interface and optimizes its synchronization and communication overhead. To achieve the goal, we first propose a hybrid execution model which differentiates between the computations within a graph partition and across the partitions, and decouples the computations within a partition from distributed synchronization and communication. By implementing the computations within a partition by pseudo-superstep iteration in memory, the hybrid execution model can effectively reduce synchronization and communication overhead while not requiring heavy scheduling overhead or graph-centric sequential algorithms. We then demonstrate how the hybrid execution model can be easily implemented within the BSP abstraction to preserve its simple programming interface. Finally, we evaluate our implementation of the GraphHP platform on classical BSP applications and show that it performs significantly better than the state-of-the-art BSP implementations. Our GraphHP implementation is based on Hama, but can easily generalize to other BSP platforms.