Visual-Inertial Odometry (VIO) provides a robust localization solution for Simultaneous Localization and Mapping (SLAM) systems. Self-supervised VIO, a leading approach, has the advantage of not requiring extensive ground-truth labels. Regrettably, this method still poses challenges for robotic applications, particularly unmanned aerial vehicles (UAVs), due to its computational complexity arising from inadequate model designs. To address this bottleneck, we introduce BotVIO (where “Bot” refers to “robotics”), a Transformer-based self-supervised VIO model, offering an excellent solution to alleviate computational burdens for robotics. Our lightweight backbone combines shallow CNNs with spatial-temporal-enhanced Transformers to replace conventional architectures, while the minimalist cross-fusion module uses single-layer cross-attention to enhance multimodal interaction. Extensive experiments show that, during pose estimation, BotVIO achieves a remarkable 70.37% reduction in trainable parameters and a 74.85% decrease in inference speed, reaching up to 57.80 fps on an NVIDIA Jetson NX (10W&2CORE), while improving pose accuracy and robustness. For the benefit of the community, we make public the source code.11https://github.com/wenhuiwei-ustc/BotVIO
1 August 2023 in IEEE Transactions on Instrumentation and Measurement
Recently, learning-based Visual Odometry (VO) has attained remarkable success in vision-based measurement, especially in indoor robotics. Unfortunately, existing methods usually under-explore geometric-semantic information, thus resulting in inefficient perception in unseen dynamic environments. Meanwhile, they are usually time-consuming, since they typically rely on high-complexity semantic segmentation models, resulting in concurrency reduction. In this paper, we develop a geometric-semantic information enhanced lightweight visual odometry (GSL-VO) that can work particularly well in dynamic environments. Specifically, on the one hand, to improve the robustness of VO through geometric-semantic information, we first come up with a novel image enhancement module to tackle motion blur, thus enabling accurate geometric and semantic information extraction. Second, we design an adaptive geometric-semantic information processing module that combines geometric and semantic information to retain reliable features for pose measurement. Moreover, semantic information is expressed via a probability framework for accurate and robust movable object extraction. On the other hand, we further propose a lightweight semantic segmentation model that enjoys an efficient multi-level feature aggregation capability to address the speed bottleneck of VO. A series of experiments on two well-known RGB-D dynamic datasets indicate that our proposed method is both accurate and fast: while achieving a significant average improvement of 70.5% in ATE over state-of-the-art learning-based VO on Bonn RGB-D Dynamic dataset, GSL-VO leads to high 22.3 FPS on a low-cost platform, which makes it well-suited for practical scenarios. Remarkably, on a challenging dynamic sequence of TUM RGB-D dataset, GSL-VO improves the baseline VO by 88.9% in ATE.
28 September 2015 in IEEE Transactions on Aerospace and Electronic Systems
This paper presents a new random weighting method to deal with the systematic error of the kinematic model for dynamic navigation. This method incorporates random weights in the kinematic model to control the systematic error of the kinematic model for improving the navigation accuracy. A theory of random weighting estimation is established, showing that 1) the random weighting estimation of the kinematic model's systematic error is unbiased and 2) the covariance matrix of the predicted state vector can be controlled by adjusting the covariance matrices of the predicted residual vector and estimated state vector to improve the accuracy of state prediction. Random weighting estimations are also constructed for the systematic error of the kinematic model as well as the covariance matrices of predicted residual vector, predicted state vector, and state noise vector. Experimental results demonstrate the effectiveness of the proposed random weighting method in resisting the disturbances of the kinematic model noise for improving the accuracy of dynamic navigation.
5 June 2024 in IEEE Transactions on Intelligent Transportation Systems
Self-supervised monocular visual odometry has a crucial advantage of not depending on labels and has shown significant performance in autonomous driving and robotics. However, recent methods suffer from limited feature representations as they depend on coarse semantic masks to handle dynamic objects, resulting in diminished accuracy in dynamic environments. In contrast to these coarse-grained methods, we present Fine-MVO, a novel self-supervised monocular visual odometry that aims to address dynamic objects using implicit fine-grained feature representations, thus achieving excellent accuracy and robustness in dynamic environments. First, Fine-MVO provides an efficient cross-feature augmentation module and a novel loss weight balance strategy to effectively leverage fine-grained features with implicit semantic information, leading to a great improvement in the depth estimation accuracy, especially on object boundaries in the scenes. Secondly, we design a novel pose-feature enhancement module and an effective two-stage training policy to empower the pose network to focus on robust static regions and temporal information, thereby enhancing the pose estimation performance in dynamic and long-term environments. Extensive experimental results demonstrate the excellent accuracy and generalization of Fine-MVO. Specifically, Fine-MVO achieves a remarkable $36.80\%$ improvement in pose accuracy over the state-of-the-art method on the KITTI dataset, which even breaks through the performance of loop closure within geometry-based visual odometry methods. Furthermore, Fine-MVO exhibits satisfactory generalization on the outdoor dataset AirDOS-Shibuya, attaining a notable improvement of $28.22\%$ over current advanced method. Excitingly, Fine-MVO also reveals outstanding generalization on the indoor dataset TUM-RGBD.
15 September 2010 in Journal of Chemical Crystallography
Micro-expression is a spontaneous facial expression with short duration, low intensity and facial partial action units. Micro-expression recognition plays an important role in psychological diagnosis, lie detection, and security systems. However, even the state-of-the-art recognition models suffer from the lack of micro-expression samples. In order to augment the training data, we propose a new method — an improved Generative adversarial network (GAN) for micro-expressions based on multi-label learning from action units. In the proposed model, action units (AUs) are added to GAN in the form of multi-labels. The designed loss function ensures to generate high quality images. The designed loss function for video can obtain a smooth action video with a trajectory of the AU. The designed loss function of optical flow guarantees low intensity for micro-expression generation and high similarity with the original training micro-expression samples. Moreover, the micro-expression recognition accuracy can be improved via adding the generated samples to the training data set. The experimental results on the two benchmark databases including CASME II and MMEW demonstrate the superiority of the proposed method over other state-of-the-art micro-expression recognition methods.