Xiaoteng Ma

Xiaoteng Ma

Auto-generated profile, based on publicly available information

Xiaoteng Ma

Department of Automation, Tsinghua University, Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 February 2024 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
QZQiyuan Zhang
Qiyuan Zhang
SLShu Leng
Shu Leng
XMXiaoteng Ma
Xiaoteng Ma
QLQihan Liu
Qihan Liu
XWXueqian Wang
Xueqian Wang
JYJun Yang
Jun Yang
Current constrained reinforcement learning (RL) methods guarantee constraint satisfaction only in expectation, which is inadequate for safety-critical decision problems. Since a constraint satisfied in expectation remains a high probability of exceeding the cost threshold, solving constrained RL problems with high probabilities of satisfaction is critical for RL safety. In this work, we consider the safety criterion as a constraint on the conditional value-at-risk (CVaR) of cumulative costs, and propose the CVaR-constrained policy optimization algorithm (CVaR-CPO) to maximize the expected return while ensuring agents pay attention to the upper tail of constraint costs. According to the bound on the CVaR-related performance between two policies, we first reformulate the CVaR-constrained problem in augmented state space using the state extension procedure and the trust-region method. CVaR-CPO then derives the optimal update policy by applying the Lagrangian method to the constrained optimization problem. In addition, CVaR-CPO utilizes the distribution of constraint costs to provide an efficient quantile-based estimation of the CVaR-related value function. We conduct experiments on constrained control tasks to show that the proposed method can produce behaviors that satisfy safety constraints, and achieve comparable performance to most safe RL (SRL) methods.

5 July 2022 in Building and Environment
Verified
Journal Article
Closed access
KSkailai Sun
kailai Sun
XMXiaoteng Ma
Xiaoteng Ma
PLPeng Liu
Peng Liu
QZQianchuan Zhao
Qianchuan Zhao

16 June 2023 in European Journal of Operational Research
Verified
Journal Article
Closed access
SMShuai Ma
Shuai Ma
XMXiaoteng Ma
Xiaoteng Ma
LXLi Xia
Li Xia