There is no denying the fact that LiDAR-camera online calibration is an extremely complex problem. For automatic driving technology development, a stable, accurate, and fast online LiDAR-camera calibration system becomes quite essential. As deep learning plays a vital role in many fields, learning-based methods promise to solve this problem. However, the existing methods do not fully consider the geometric constraints in the calibration process. In this study, we propose a new network: RGKCNet, a 2-D–3-D pose estimation network based on keypoints, which can be applied to calibrate the camera and LiDAR in real time. Using the declarative layer, extrinsic calibration is regarded as a bilevel optimization problem, which allows us to embed an optimizer based on geometric constraints in the end-to-end network and, thus, realize 2-D–3-D data association. With a trainable point weighting layer, the network can extract sparse keypoints and give the corresponding weights used in pose estimation, which further improves the robustness of the network. We verify the performance of the proposed method on the KITTI dataset to demonstrate the significant effect in practical applications.

Category-level grasping based on related tasks is an important research interest in robotic grasping, driven by substantial industrial demand. A robust evaluation mechanism and an accurate inference module are crucial components of the research. This article proposes a novel grasping algorithmic framework. This approach facilitates the acquisition of task-specific datasets and training without additional manual annotations. The algorithm establishes point-wise correspondence between grasp instances and the category-level prior model, achieving precise transfer of semantic information and grasp priors. Compared with previous similar work, this article truly realizes point-wise level knowledge transfer between grasp instances and priors, obtaining superior performance in category-level grasping tasks.

The perception and processing of transparent objects face significant challenges in various applications, primarily due to the limitations of traditional sensors. These sensors often struggle to capture the complete depth information of transparent objects, mainly due to the refraction and reflection of light on their surfaces, as well as the lack of visible texture. Previous research has explored the use of deep learning models to generate complete depth maps from RGB images and corrupted depth data obtained from depth sensors. However, existing methods suffer from design flaws that limit the effectiveness of depth completion. To address this issue, we propose TDCNet, a novel dual-branch CNN-Transformer parallel network specifically designed for depth completion of transparent objects. Our framework consists of two distinct branches: one focuses on feature extraction from partial depth maps, while the other processes RGB-D images. Experimental results demonstrate that our model achieves state-of-the-art performance on multiple public datasets. The code and pre-trained models are publicly available at https://github.com/XianghuiFan/TDCNet.

Homography estimation is a fundamental task in computer vision that involves obtaining the transformation between multi-view images for image alignment. Although convolutional neural networks (CNNs) have shown state-of-the-art performance in this task, few works have explored the use of transformer-based models, which have demonstrated superiority in high-level vision tasks. In this paper, we propose a strong baseline model for homography estimation that combines a swin transformer feature representation for global features and a CNN feature representation for local features. Additionally, we introduce a cross non-local layer to coarsely search for matched features within the feature maps. In the homography regression stage, we adopt an attention layer to drop out weak correlation feature points from the channels of the correlation volume. Our experiments show that our method outperforms the state-of-the-art methods in 8 Degree-of-Freedoms (DOFs) homography estimation. Code is available at https://github.com/mingxiaohuo/ABHE.

Torque-based impedance control, a kind of classical active compliant control, is widely required in human–robot interaction, medical rehabilitation, and other fields. Adaptive impedance control effectively tracks the force when the robot comes in contact with an unknown environment. Conventional adaptive impedance control (AIC) introduces the force tracking error of the last moment to adjust the controller parameters online, which is an indirect method. In this paper, joint friction in the robot system is first identified and compensated for to enable the excellent performance of torque-based impedance control. Second, neural networks are inserted into the torque-based impedance controller, and a neural adaptive impedance control (NAIC) scheme with directly online optimized parameters is proposed. In addition, NAIC can be deployed directly without the need for data collection and training. Simulation studies and real-world experiments with a six link rotary robot manipulator demonstrate the excellent performance of NAIC.