Yao Mu

Yao Mu

Auto-generated profile, based on publicly available information

Yao Mu

Shanghai Jiao Tong University

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 May 2024 in IEEE Transactions on Intelligent Transportation Systems
Verified
Journal Article
Closed access
ZGZeyu Gao
Zeyu Gao
YMYao Mu
Yao Mu
CCChen Chen
Chen Chen
JDJingliang Duan
Jingliang Duan
Ping LuoPing Luo
Ping Luo
SLShengbo Li
Shengbo Li
End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to control signals. Recent advanced methods construct a latent world model to map the high dimensional observations into compact latent space. However, the latent states embedded by the world model proposed in previous works may contain a large amount of task-irrelevant information, resulting in low sampling efficiency and poor robustness to input perturbations. Meanwhile, the training data distribution is usually unbalanced, and the learned policy is challenging to cope with the corner cases during the driving process. To solve the above challenges, we present a SEM antic M asked recurrent world model ( SEM2 ), which introduces a semantic filter to extract key driving-relevant features and make decisions via the filtered features, and is trained with a multi-source data sampler, which aggregates common data and multiple corner case data in a single batch, to balance the data distribution. Extensive experiments on CARLA show our method outperforms the state-of-the-art approaches in terms of sample efficiency and robustness to input permutations.

15 May 2024 in IEEE Transactions on Industrial Informatics
Verified
Journal Article
Closed access
JWJunjie Wang
Junjie Wang
QZQichao Zhang
Qichao Zhang
YMYao Mu
Yao Mu
DLDong Li
Dong Li
DZDongbin Zhao
Dongbin Zhao
HJHao JianYe
Hao JianYe
The latent world model, which efficiently represents high-dimensional observations within a latent space, has shown promise in reinforcement learning-based policies for visual control tasks. Due to a lack of clear environmental context comprehension, its applicability in a variety of contexts with unknown dynamics is constrained. We propose a prototypical context- aware dynamics (ProtoCAD) model to address this issue. This model captures local dynamics using temporally consistent latent contexts and aids generalization in visual control tasks. By grouping prototypes over historical experiences, ProtoCAD collects useful contextual information that improves model-based reinforcement learning dynamics generalization in two ways. First, to guarantee the consistency of prototype assignments for various temporal segments of the same latent trajectory, a temporally consistent prototypes regularizer is used. Then, a context representation is devised to combine the aggregated prototype with the projection embedding of latent states. According to extensive trials, ProtoCAD outperforms competing approaches in terms of dynamics generalization for visual robotic control and autonomous driving applications.

10 July 2024 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
JMJi Ma
Ji Ma
HDHongming Dai
Hongming Dai
YMYao Mu
Yao Mu
PWPengying Wu
Pengying Wu
HWHao Wang
Hao Wang
CLChang Liu
Chang Liu
Zero-Shot Object Navigation (ZSON) requires agents to autonomously locate and approach unseen objects in unfamiliar environments and has emerged as a particularly challenging task within the domain of Embodied AI. Existing datasets for developing ZSON algorithms lack consideration of dynamic obstacles, object attribute diversity, and scene texts, thus exhibiting noticeable discrepancies from real-world situations. To address these issues, we propose a Dataset for Open-Vocabulary Zero-Shot Object Navigation in Dynamic Environments (DOZE) that comprises ten high-fidelity 3D scenes with over 18 k tasks, aiming to mimic complex, dynamic real-world scenarios. Specifically, DOZE scenes feature multiple moving humanoid obstacles, a wide array of open-vocabulary objects, diverse distinct-attribute objects, and valuable textual hints. Besides, different from existing datasets that only provide collision checking between the agent and static obstacles, we enhance DOZE by integrating capabilities for detecting collisions between the agent and moving obstacles. This novel functionality enables the evaluation of the agents' collision avoidance abilities in dynamic environments. We test four representative ZSON methods on DOZE, revealing substantial room for improvement in existing approaches concerning navigation efficiency, safety, and object recognition accuracy.

23 December 2025 in IEEE Robotics & Automation Magazine
Verified
Journal Article
Closed access
KZKangjie Zhou
Kangjie Zhou
YMYao Mu
Yao Mu
HSHaoyang Song
Haoyang Song
YZYi Zeng
Yi Zeng
PWPengying Wu
Pengying Wu
CLChang Liu
Chang Liu
Robotic navigation in complex environments remains a critical research challenge. Traditional navigation methods focus on optimal trajectory generation within fixed free workspace, therefore struggling in environments lacking viable paths to the goal, such as disaster zones or cluttered warehouses. To address this problem, we propose AINav, an adaptive interactive navigation approach that proactively interacts with environments to create feasible paths to achieve originally unreachable goals. Specifically, we present a primitive skill tree for task planning with large language models (LLMs), facilitating effective reasoning to determine interaction objects and sequences. To ensure robust subtask execution, we adopt reinforcement learning (RL) to pretrain a comprehensive skill library containing versatile locomotion and interaction behaviors for motion planning. Furthermore, we introduce an adaptive replanning approach featuring two LLM-based modules: an advisor serving as a flexible replanning trigger and an arborist for autonomous plan adjustment. Integrated with the tree structure, the replanning mechanism allows for convenient node addition and pruning, enabling rapid plan adaptation in a priori unknown environments. Comprehensive simulations and experiments have demonstrated AINav’s effectiveness and adaptivity in diverse scenarios. The supplementary video is available at https://youtu.be/CjXm5KFx9AI.

1 October 2026
Verified
Book Chapter
Closed access
PDF file
RLRuiying Li
Ruiying Li
YZYunlang Zhou
Yunlang Zhou
YZYuyao Zhu
Yuyao Zhu
KCKylin Chen
Kylin Chen
SWSukai Wang
Sukai Wang
YMYao Mu
Yao Mu

1 January 2026 in Journal of Image and Graphics
Verified
Journal Article
Closed access
MYMu Yao
Mu Yao
上海上海交通大学人工智能研究院ArtificialIntelligenceInstitute,ShanghaiJiaoTongUniversity
上海交通大学人工智能研究院ArtificialIntelligenceInstitute,ShanghaiJiaoTongUniversity
ZHZhao Hao
Zhao Hao
HRHu Ruizhen
Hu Ruizhen
ZLZhang Li
Zhang Li
北京北京大学PekingUniversity
北京大学PekingUniversity
具身智能作为人工智能发展的关键领域,正面临数据异构性、强物理约束及交互昂贵等挑战,难以直接复制大语言模型的“大规模预训练+规模定律”范式。本文从数据、模型、系统与评测四个维度全面梳理了具身智能的前沿技术演进。在数据层面,文章提出了“数据金字塔”结构,主张利用底层庞大的仿真与互联网视频数据构建物理常识,通过中层人类交互数据进行行为映射,最终以顶层少量真机数据实现技能落地。在模型层面,探讨了主流视觉-语言-动作模型(VLA)的扩展瓶颈,并指出“世界模型”作为具身预训练的新方向,能够通过模拟环境动力学与未来预演,赋予智能体更强的物理直觉与泛化能力。在系统层面,观察到架构正从单一端到端模型向类操作系统的“分层架构”演进,实现高层语义规划与底层运动控制的解耦。最后,本文审视了当前评测体系在真实性与可复现性上的挑战,并对行走与操作一体化及具身智能“ImageNet时刻”的到来进行了展望。