Chenjia Bai

Chenjia Bai

Auto-generated profile, based on publicly available information

Chenjia Bai

Institute of Artificial Intelligence (TeleAI), China Telecom, Shanghai, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 September 2021 in IEEE Transactions on Cybernetics
Verified
Journal Article
Closed access
CBChenjia Bai
Chenjia Bai
LWLingxiao Wang
Lingxiao Wang
YWYixin Wang
Yixin Wang
ZWZhaoran Wang
Zhaoran Wang
RZRui Zhao
Rui Zhao
PLPeng Liu
Peng Liu
Multigoal reinforcement learning (RL) extends the typical RL with goal-conditional value functions and policies. One efficient multigoal RL algorithm is the hindsight experience replay (HER). By treating a hindsight goal from failed experiences as the original goal, HER enables the agent to receive rewards frequently. However, a key assumption of HER is that the hindsight goals do not change the likelihood of the sampled transitions and trajectories used in training, which is not the fact according to our analysis. More specifically, we show that using hindsight goals changes such a likelihood and results in a biased learning objective for multigoal RL. We analyze the hindsight bias due to this use of hindsight goals and propose the bias-corrected HER (BHER), an efficient algorithm that corrects the hindsight bias in training. We further show that BHER outperforms several state-of-the-art multigoal RL approaches in challenging robotics tasks.

1 September 2019 in Neurocomputing
Verified
Journal Article
Closed access
CBChenjia Bai
Chenjia Bai
PLPeng Liu
Peng Liu
WZWei Zhao
Wei Zhao
XTXianglong Tang
Xianglong Tang

4 November 2022 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
CBChenjia Bai
Chenjia Bai
TXTing Xiao
Ting Xiao
ZZZhoufan Zhu
Zhoufan Zhu
LWLingxiao Wang
Lingxiao Wang
FZFan Zhou
Fan Zhou
ZWZhaoran Wang
Zhaoran Wang
A key challenge in offline reinforcement learning (RL) is how to ensure the learned offline policy is safe, especially in safety-critical domains. In this article, we focus on learning a distributional value function in offline RL and optimizing a worst-case criterion of returns. However, optimizing a distributional value function in offline RL can be hard, since the crossing quantile issue is serious, and the distribution shift problem needs to be addressed. To this end, we propose monotonic quantile network (MQN) with conservative quantile regression (CQR) for risk-averse policy learning. First, we propose an MQN to learn the distribution over returns with non-crossing guarantees of the quantiles. Then, we perform CQR by penalizing the quantile estimation for out-of-distribution (OOD) actions to address the distribution shift in offline RL. Finally, we learn a worst-case policy by optimizing the conditional value-at-risk (CVaR) of the distributional value function. Furthermore, we provide theoretical analysis of the fixed-point convergence in our method. We conduct experiments in both risk-neutral and risk-sensitive offline settings, and the results show that our method obtains safe and conservative behaviors in robotic locomotion tasks.

1 December 2021 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
CBChenjia Bai
Chenjia Bai
PLPeng Liu
Peng Liu
KLKaiyu Liu
Kaiyu Liu
LWLingxiao Wang
Lingxiao Wang
YZYingnan Zhao
Yingnan Zhao
ZWZhaoran Wang
Zhaoran Wang
Efficient exploration remains a challenging problem in reinforcement learning, especially for tasks where extrinsic rewards from environments are sparse or even totally disregarded. Significant advances based on intrinsic motivation show promising results in simple environments but often get stuck in environments with multimodal and stochastic dynamics. In this work, we propose a variational dynamic model based on the conditional variational inference to model the multimodality and stochasticity. We consider the environmental state-action transition as a conditional generative process by generating the next-state prediction under the condition of the current state, action, and latent variable, which provides a better understanding of the dynamics and leads to a better performance in exploration. We derive an upper bound of the negative log likelihood of the environmental transition and use such an upper bound as the intrinsic reward for exploration, which allows the agent to learn skills by self-supervised exploration without observing extrinsic rewards. We evaluate the proposed method on several image-based simulation tasks and a real robotic manipulating task. Our method outperforms several state-of-the-art environment model-based exploration approaches.

30 October 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
ZDZhihong Deng
Zhihong Deng
ZFZuyue Fu
Zuyue Fu
LWLingxiao Wang
Lingxiao Wang
ZYZhuoran Yang
Zhuoran Yang
CBChenjia Bai
Chenjia Bai
JJJing Jiang
Jing Jiang
Offline reinforcement learning (RL) harnesses the power of massive datasets for resolving sequential decision problems. Most existing papers only discuss defending against out-of-distribution (OOD) actions while we investigate a broader issue, the false correlations between epistemic uncertainty and decision-making, an essential factor that causes suboptimality. In this paper, we propose falSe COrrelation REduction (SCORE) for offline RL, a practically effective and theoretically provable algorithm. We empirically show that SCORE achieves the SoTA performance with 3.1x acceleration on various tasks in a standard benchmark (D4RL). The proposed algorithm introduces an annealing behavior cloning regularizer to help produce a high-quality estimation of uncertainty which is critical for eliminating false correlations from suboptimality. Theoretically, we justify the rationality of the proposed method and prove its convergence to the optimal policy with a sublinear rate under mild assumptions.

20 November 2023 in Artificial Intelligence
Verified
Journal Article
Closed access
CBChenjia Bai
Chenjia Bai
LWLingxiao Wang
Lingxiao Wang
JHJianye Hao
Jianye Hao
ZYZhuoran Yang
Zhuoran Yang
BZBin Zhao
Bin Zhao
XLXuelong Li
Xuelong Li