In order to implement more accurate and fine-grained monitoring of robot-environment interaction (REI) in real time, this article extends the concept of digital twin (DT) to REI description and proposes a novel semantic-enhanced DT system with exteroceptive and proprioceptive modalities of robots as mechanisms for states monitoring and semantics inference associated with interactions. We create a 3-D generic graphic model embedded with contact dynamics based on domain descriptions and sensor data to mirror a real interactive scene and simultaneously extract the appearance and physical properties of real entities. In addition, we present a multifeature fused visual relations detector and introduce a lightweight spatial relation data set to measure the visual relationships between entity pairs. To keep the DT consistent with the response of a real system, the model is updated with interactive subprocess-triggered control. In our system, the whole framework is developed with virtual reality techniques, which provides a high degree of observability. The experiments on the objects sorting task in a kitchen scene are conducted, and the abilities for system monitoring in real time and semantic reasoning are evaluated; the results demonstrate the effectiveness and feasibility of our proposal.

Multielement decision-making is crucial for the robust deployment of human-robot collaboration (HRC) systems in flexible manufacturing environments with personalized tasks and dynamic scenes. Large Language Models (LLMs) have recently demonstrated remarkable reasoning capabilities in various robotic tasks, potentially offering this capability. However, the application of LLMs to actual HRC systems requires the timely and comprehensive capturing of real-scene information. In this study, we suggest incorporating real scene data into LLMs using digital twin (DT) technology and present a cognitive digital twin prototype system of HRC manipulation, known as HRC-CogiDT. Specifically, we initially construct a scene semantic graph encoding the geometric information of entities, spatial relations between entities, actions of humans and robots, and collaborative activities. Subsequently, we devise a prompt that merges scene semantics with prior knowledge of activities, linking the real scene with LLMs. To evaluate performance, we compile an HRC scene understanding dataset and set up a laboratory-level experimental platform. Empirical results indicate that HRC-CogiDT can swiftly perceive scene changes and make high-level decisions based on varying task requirements, such as task planning, anomaly detection, and schedule reasoning. This study provides promising insights for the future applications of LLMs in robotics. Note to Practitioners —Recently, LLMs have demonstrated significant success in various robotic tasks, suggesting their potential as a powerful tool for robotic decision-making. Motivated by this, to improve the production efficiency of HRC in flexible manufacturing, we innovatively combine LLMs with DT technology, and propose a cognitive DT system for HRC, aiming to integrate LLMs into the decision-making loop of HRC system. Experiments conducted in a laboratory-scale platform indicate that the proposed system can handle different decision-making needs in different HRC activities. This system can provide professional guidance to operators in a comprehensible form and serve as a medium for monitoring the safety and standardization of the manipulation process. Future work will explore the use of virtual space provided by the proposed system to optimize the decision outputs of LLMs to make the proposed system more broadly applicable.

In recent years, data-driven learning methods have been widely studied for autonomous robot skill learning. However, these methods rely on large amounts of robot-environment interaction data for training, which largely prevents them from being applied to real-world robots. To address this problem, this paper proposes a novel simulation-reality closed-loop learning framework for autonomous robot skill learning that can improve data efficiency, enhance policy stability and achieve effective policy simulation-to-reality (sim2real) transfer. First, a hybrid control model combining the asymmetric deep deterministic policy gradients (Asym-DDPG) model and the forward prediction control (FPC) model is proposed to learn vision-based manipulation policies in simulations, which can decompose complex tasks to improve learning efficiency. Second, a novel pixel-level domain adaptation method named Position-CycleGAN is designed to translate real images to simulated images while also preserving the task-related information. The policy trained in simulations can be directly migrated into real robots in a reverse reality-to-simulation manner using the Position-CycleGAN model. The experimental results validate the effectiveness of the proposed framework. This work provides an efficient and feasible path for achieving autonomous skill learning.

Reinforcement Learning from Human Feedback (RLHF) enables robots to learn cooperative strategies aligned with human expectations by incorporating feedback into the learning process. However, existing RLHF methods rely on explicit query-based feedback, which is limited for complex, long-horizon tasks due to feedback sparsity and delay. Additionally, active feedback collection disrupts task execution and increases human cognitive load. To address these challenges, we propose Neural Trust-guided Reinforcement Learning (NeuTRL), a novel human-in-the-loop framework that utilizes human trust, derived from electroencephalography (EEG) signals, as implicit feedback for reward shaping and policy optimization. NeuTRL incorporates a deep neural network to map EEG to trust states, enabling high-frequency evaluations of the robot's behavior. Additionally, we propose a Long Short-Term Memory (LSTM)-based reward model to capture the temporal dynamics of trust evolution and generate dense rewards. NeuTRL enables robots to learn collaborative policies using dense EEG-based trust feedback, addressing the feedback sparsity challenge while maintaining uninterrupted interaction. Experimental results demonstrate that NeuTRL achieves a 74.5% improvement in average collaborative task performance compared to baselines without human feedback and a 26.9% improvement compared to query-based RLHF. Qualitative analysis further reveals that NeuTRL significantly enhances cooperative adaptability and coordination. These results highlight NeuTRL's potential to advance trust-aware, adaptive human-robot collaboration in dynamic environments.