This paper presents the implementation of an obstacle avoidance algorithm on the UR5e collaborative robot. The algorithm, previously developed and verified in simulation, allows one to modify in real time the trajectory of the manipulator with three different modalities to avoid obstacles. Some test cases with fixed or dynamic obstacles affecting the robot’s motion were first simulated and then experimented on. The paper describes the hardware/software architecture of the robotic system: an external controller is realized by a standard PC that communicates with the robot controller by a TCP/IP protocol; algorithms and data processing are executed by Python/Matlab software that guarantees a duty cycle of at least 100 Hz. The error analysis between simulated and real data allows one to conclude that the developed algorithms revealed to be effectively applied to a real robotic system, showing behavior similar to what is expected by simulations.

This paper presents an obstacle avoidance strategy for mobile manipulators consisting of a robotic arm and a base with a non-holonomic differential wheel system. The algorithm makes it possible to avoid obstacles in a dynamic environment, without planning the path a priori. A series of examples are proposed in simulation using Matlab and analyzed to show how the algorithm works if the obstacle interferes with the manipulator or the base. In addition, the possibility of prioritizing the movement of certain parts of the system using the weighted pseudo-inverse matrix is introduced. In this way, it is possible to give movement priority to the base if it is necessary to move the robot over long distances while keeping the manipulator as still as possible. The use of null space to keep the end-effector stationary while it avoids obstacles is also explored, exploiting the system’s redundancy and allowing the rest of the kinematic chain and the mobile base to move accordingly. Finally, current standards are analyzed and a solution is shown that allows the robot to vary its behavior to avoid obstacles depending on the distance to the target point.

In Industry 5.0, the transition from fixed traditional automation to flexible human–robot collaboration (HRC) needs interfaces that are both intuitive and efficient. This paper introduces a novel, multimodal control system for autonomous object handling, specifically designed to enhance natural user interaction in dynamic work environments. The system integrates a 6-Degrees of Freedom (DoF) collaborative robot (UR5e) with a hand-eye RGB-D vision system to achieve robust autonomy. The core technical contribution lies in a vision pipeline utilizing deep learning for object detection and point cloud processing for accurate 6D pose estimation, enabling advanced tasks such as human-aware object handover directly onto the operator’s hand. Crucially, an Automatic Speech Recognition (ASR) is incorporated, providing a Natural Language Understanding (NLU) layer that allows operators to issue real-time commands for task modification, error correction and object selection. Experimental results demonstrate that this multimodal approach offers a streamlined workflow aiming to improve operational flexibility compared to traditional HMIs, while enhancing the perceived naturalness of the collaborative task. The system establishes a framework for highly responsive and intuitive human–robot workspaces, advancing the state of the art in natural interaction for collaborative object manipulation.