1 August 2025 in IEEE Robotics and Automation Letters
With the increasing presence of service robots and autonomous vehicles in human environments, navigation systems need to evolve beyond simple destination reach to incorporate social awareness. This letter introduces GSON, a novel group-based social navigation framework that leverages Large Multimodal Models (LMMs) to enhance robots' social perception capabilities. Our approach uses visual prompting to enable zero-shot extraction of social relationships among pedestrians and integrates these results with robust pedestrian detection and tracking pipelines to overcome the inherent inference speed limitations of LMMs. The planning system incorporates a mid-level planner that sits between global path planning and local motion planning, effectively preserving both global context and reactive responsiveness while avoiding disruption of the predicted social group. We validate GSON through extensive real-world mobile robot navigation experiments involving complex social scenarios such as queuing, conversations, and photo sessions. Comparative results show that our system significantly outperforms existing navigation approaches in minimizing social perturbations while maintaining comparable performance on traditional navigation metrics.
11 August 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharp loss landscape and have poor weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with improved stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. To further reduce the magnitude of random noise while achieving better performance, we propose DP-FedSAM-$\operatorname{top}_{k}$ by adopting the local update sparsification technique. From the theoretical perspective, we present the convergence analysis to investigate how our algorithms mitigate the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with Rényi DP, the sensitivity analysis of local updates, and generalization analysis. At last, we empirically confirm that our algorithms achieve state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.
29 May 2026 in IEEE Transactions on Aerospace and Electronic Systems
Continuum manipulators (CMs) hold significant promise for space applications due to their flexibility and adaptability in extreme environments. However, similar to traditional articulated space manipulators (A-SMs), CMs face challenges arising from the dynamic coupling between the manipulator and the spacecraft. A key difficulty with CMs lies in their infinite degrees of freedom (DOFs) and the absence of conventional joint structures, which complicates the development of differential kinematics models similar to those of A-SMs using Jacobian matrix. To address this issue, we introduce the concept of the Jacobian operator to overcome the difficulties in establishing the differential kinematics of CMs. Building on this foundation, we further introduce the concept of the generalized Jacobian operator, an extension of the generalized Jacobian matrix in A-SMs, to develop the differential kinematics of continuum space manipulators (C-SMs). Based on the derived differential kinematics, we then propose a trajectory planning method for C-SMs. Additionally, utilizing the null space of the Jacobian operator, a base disturbance-free trajectory planning approach is developed, ensuring smooth operation under dynamic coupling conditions. Finally, the validity and effectiveness of the proposed planning methods are demonstrated through simulation cases. This framework offers a robust solution to the kinematic challenges of C-SMs, paving the way for more efficient trajectory planning and control in space applications.
15 July 2025 in IEEE Robotics and Automation Letters
Deep reinforcement learning (DRL) has emerged as a powerful tool for controlling cable-driven continuum space robots (CDCSRs), offering a solution that bypasses complex system modeling. However, DRL based on dense reward functions (DRLDR) requires meticulous tuning of the reward structure, whereas DRL based on sparse reward functions (DRLSR) exhibits limited decision-making abilities, particularly in the space environments. To avoid extensive fine-tuning and enhance the performance in controlling CDCSRs, we propose the behavior cloning assisted twin delayed deep deterministic policy gradient (BATD3), a novel algorithm that integrates behavior cloning (BC) with DRLSR. Firstly, a DRLSR-based control framework is developed, which reformulates the control problem as a Markov decision process (MDP). Building on this, the BATD3 algorithm is proposed, comprising two training phases: the prior phase to train the BC model using demonstrations; the formal phase to pre-fill the RL replay buffer with demonstrations and successful BC-environment interaction trajectories, and optimize the RL model with the assistance of BC. Finally, extensive experiments are conducted in the MuJoCo environment to assess the performance of BATD3 in controlling CDCSRs. The results highlight the effectiveness, generalization, stability, robustness and potential of BATD3, along with the practicality and feasibility of the DRLSR-based control framework for CDCSRs.
1 October 2024 in Neural Networks
2 July 2025 in IEEE Transactions on Systems, Man, and Cybernetics: Systems
Space teleoperation significantly extends human reach in space missions. However, traditional approaches are constrained by factors, such as the reliance on accurate dynamic models and the risk of operator fatigue during prolonged tasks. Additionally, while data-driven intelligent approaches reduce the need for prior knowledge, they have yet to adequately address the time delay issues inherent in these systems. To overcome these challenges, we introduce the belief state actor-critic (BSAC) method, the first deep reinforcement learning approach tailored for space teleoperation capture tasks within a bilateral control framework. We first establish a generalized agent-based architecture for space teleoperation, shifting decision-making from human operators to autonomous agents. Following a comprehensive analysis of the time delay challenges, we propose the BSAC algorithm, which integrates state augmentation and belief state techniques to mitigate the effects of delays in teleoperated Markov decision processes. Extensive experiments are conducted on the MuJoCo simulation platform, modeling a real hardware system across various scenarios. The learned policies are then successfully transferred and validated in a real-world setup, demonstrating the effectiveness and robustness of BSAC. In summary, our results support the feasibility of agent-based frameworks capable of overcoming time delay challenges in space teleoperation.