Recent advances in the field of collaborative robotics aim to endow industrial robots with prediction and anticipation abilities. In many shared tasks, the robot’s ability to accurately perceive and recognize the objects being manipulated by the human operator is crucial to make predictions about the operator’s intentions. In this context, this paper proposes a novel learning-based framework to enable an assistive robot to recognize the object grasped by the human operator based on the pattern of the hand and finger joints. The framework combines the strengths of the commonly available software MediaPipe in detecting hand landmarks in an RGB image with a deep multi-class classifier that predicts the manipulated object from the extracted keypoints. This study focuses on the comparison between two deep architectures, a convolutional neural network and a transformer, in terms of prediction accuracy, precision, recall and F1-score. We test the performance of the recognition system on a new dataset collected with different users and in different sessions. The results demonstrate the effectiveness of the proposed methods, while providing valuable insights into the factors that limit the generalization ability of the models.

This paper presents a collaborative manufacturing cell implemented in a laboratory setting, focusing on developing learning-based interaction abilities to enhance versatility and ease of use. The key components of the system include 3D real-time volumetric monitoring for safety, visual recognition of hand gestures for human-to-robot communication, classification of physical-contact-based interaction primitives during handover operations, and detection of hand–object interactions to anticipate human intentions. Due to the nature and complexity of perception, deep-learning-based techniques were used to enhance robustness and adaptability. The main components are integrated in a system containing multiple functionalities, coordinated through a dedicated state machine. This ensures appropriate actions and reactions based on events, enabling the execution of specific modules to complete a given multi-step task. An ROS-based architecture supports the software infrastructure among sensor interfacing, data processing, and robot and gripper controllers nodes. The result is demonstrated by a functional use case that involves multiple tasks and behaviors, paving the way for the deployment of more advanced collaborative cells in manufacturing contexts.

There are numerous technologies and tools to acquire data related to the evolution of spatial phenomena over time. These data are typically organized as sequences of 2D geometric shapes obtained from observations taken at different times. The transformation of such sequences of 2D geometric shapes into spatiotemporal data representations, which can be easily processed and interpreted, has the potential to enable novel applications in fields as diverse as environmental sciences, climate sciences, biology or medicine. This paper focuses on the representation of moving 2D geometric shapes acquired at discrete times using continuous models of time and space. Using morphing techniques based on compatible triangulations, issues regarding the representation of spatiotemporal data in databases, as well as the influence of different design strategies on the fidelity of the approximations with respect to the modelled phenomena, are investigated. An experimental study using synthetic and real data was performed. The findings show that the use of triangulation based interpolation is a promising approach, because it allows creating continuous spatiotemporal representations that are more realistic than those obtained using the solutions proposed in previous work. Open issues regarding the representation of spatiotemporal data in information systems are also highlighted.