The grasping of transparent objects is challenging but of significance to robots. In this article, a visual–tactile fusion framework for transparent object grasping in complex backgrounds is proposed, which synergizes the advantages of vision and touch, and greatly improves the grasping efficiency of transparent objects. First, we propose a multiscene synthetic grasping dataset named SimTrans12 K together with a Gaussian-mask annotation method. Next, based on the TaTa gripper, we propose a grasping network named transparent object-grasping convolutional neural network for grasping position detection, which shows good performance in both synthetic and real scenes. Inspired by human grasping, a tactile calibration method and a visual–tactile fusion classification method are designed, which improve the grasping success rate by 36.7% compared with direct grasping and the classification accuracy by 39.1%. Furthermore, the tactile height sensing module and the tactile position exploration module are added to solve the problem of grasping transparent objects in irregular and visually undetectable scenes. The experimental results demonstrate the validity of the framework.

Fully leveraging the loco-manipulation capabilities of a quadruped robot equipped with a robotic arm is non-trivial, as it requires controlling all degrees of freedom (DoFs) of the quadruped robot to achieve effective whole-body coordination. In this letter, we propose a novel framework RoboDuet, which employs two collaborative policies to realize locomotion and manipulation simultaneously, achieving whole-body control through mutual interactions. Beyond enabling large-range 6D pose tracking for manipulation, we find that the two-policy framework supports zero-shot transfer across quadruped robots with similar morphology and physical dimensions in the real world. Our experiments demonstrate that RoboDuet achieves a 23% improvement in success rate over the baseline in challenging loco-manipulation tasks employing whole-body control. To support further research, we provide open-source code and additional videos on our website: https://locomanip-duet.github.io/.

Transparent objects are a common part of daily life, but their unique optical properties make estimating their 6D pose a challenging task. In this letter, we propose TGF-Net, a monocular instance-level 6D pose estimation method for transparent objects based on geometric fusion. TGF-Net learns the edge features and surface fragments of transparent objects as intermediate features and reduces the influence of appearance changes on the 6D pose estimation of transparent objects by fusing rich geometric features in the network. Moreover, we propose an approach for generating high-fidelity large-scale synthetic datasets of transparent objects using Blender and use this approach to generate a synthetic dataset Trans6D-32 K. Trans6D-32 K contains rendered RGB images and poses information about transparent objects in a variety of different backgrounds, perspectives, and lighting conditions. To evaluate the performance of TGF-Net on 6D pose estimation of transparent objects, we compare with multiple related works on the dataset Trans6D-32 K. TGF-Net can be trained entirely on synthetic datasets without fine-tuning and applied directly to real-world scenarios. Multiple challenging real-scene experiments demonstrate the good performance of TGF-Net, while grasping experiments demonstrate the application value of TGF-Net in transparent object manipulation.

This paper proposes a novel material recognition method for robotic tactile sensing. The method is composed of two steps. Firstly, a human-touch-inspired short-duration (1 s) slide action is conducted by the robot to obtain the tactile data. Then, the tactile data is processed with a machine learning algorithm, where 11 bioinspired features were designed to imitate the mechanical stimuli towards the four main types of tactile receptors in the skin. In this paper, a material database consisting of 144,000 tactile images is used to train seven classifiers, and the most accurate classifier is selected to recognize 12 household objects according to their properties and materials. In the property recognition, the materials are classified into 4 categories according to their compliance and texture, and the best accuracy reaches 96% in 36 ms. In the material recognition, the specific materials are recognized, and the best accuracy reaches 90% in 37 ms. The results verify the effectiveness of the proposed method.

This paper proposes a multi-map path planning approach for rescue robots to address the challenges posed by complex obstacle information, high uncertainty, and the difficulties in mine disaster scenarios. Based on multiple possible environmental maps, each with associated subjective probabilities derived from prior knowledge and expert estimations, a mathematical model for multi-map path planning in mine disaster rescue scenarios is developed. An improved hybrid algorithm combining ant colony optimization (ACO) and genetic algorithm (GA) is then proposed to solve the established model. In the hybrid approach, the improved ACO is employed to overcome the limitations of traditional genetic algorithms, such as poor initial population quality, slow convergence, and suboptimal results. Additionally, a grid-based, rectangular-area, obstacle avoidance strategy is incorporated to precisely evaluate the obstacle avoidance path of each individual across different obstacle maps. Finally, the feasibility and effectiveness of the proposed hybrid algorithm are validated through simulations involving both single and multiple mine disaster maps. The results demonstrate the potential of the proposed approach for solving robot path optimization problems in complex multi-environment scenarios.

Four-wheel, independently driven skid-steer mobile robots have been widely used in some fields, such as indoor product shipping and outdoor inspection and exploration. Traditional skid-steer mobile robot controllers often use a kinematics controller to obtain the desired speed of each wheel, complete speed closed-loop control of each wheel and achieve the robot’s trajectory tracking control. However, the controller based on kinematics may lead to robot chattering and wheel spin from being directly driven by the motor on uneven grounds. To solve these problems, we developed a four-wheel, independently driven skid-steer mobile robot with a damping module for the timing-belt servo system and proposed a model-based coordinated trajectory tracking control method with the timing-belt servo system. First, the kinematics and dynamics of the mobile robot are established, including the chassis kinematics and dynamics, as well as the dynamics of the timing-belt servo system. Secondly, the hierarchical control law is designed, which has adaptive robust control of the upper-level robot chassis, middle-level control allocation approach, and adaptive robust control of the bottom-level timing-belt servo system. Finally, the proposed method is verified by the robot’s trajectory tracking control comparison simulation experiments and has a better control performance.