Haitao Yang

Haitao Yang


Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 April 2019 in Applied Sciences
Verified
Journal Article
Open access
PDF file
JZJian Zhang
Jian Zhang
YPYaozong Pan
Yaozong Pan
RWRuili Wang
Ruili Wang
YFYuqiang Fang
Yuqiang Fang
HYHaitao Yang
Haitao Yang
Decentralized partially observable Markov decision processes (Dec-POMDPs) are general multi-agent models for planning under uncertainty, but are intractable to solve. Doubly exponential growth of the search space as the horizon increases makes a brute-force search impossible. Heuristic methods can guide the search towards the right direction quickly and have been successful in different domains. In this paper, we propose a new Q-value function representation—Monte Carlo Q-value function Q MC , which is proved to be an upper bound of the optimal Q-value function Q * . We introduce two Monte Carlo tree search enhancements—heavy playout for a simulation policy and adaptive samples—to speed up computation of Q MC . Then, we present a clustering and expansion with Monte-Carlo algorithm (CEMC)—an offline planning algorithm using Q MC as Q-value function, which is based on the generalized multi-agent A* with incremental clustering and expansion (GMAA*-ICE or ICE). CEMC calculates Q-value functions as required, without computing and storing all Q-value functions. An extended policy pruning strategy is used in CEMC. Finally, we present empirical results demonstrating that CEMC outperforms the best heuristic algorithm with a compact Q-value presentation in term of runtime for the same horizon, and has less memory usage for larger problems.