Combining Global Navigation Satellite System (GNSS) with visual and inertial sensors can give smooth pose estimation without drifting. The fusion system gradually degrades to Visual-Inertial Odometry (VIO) with the number of satellites decreasing, which guarantees robust global navigation in GNSS unfriendly environments. In this letter, we propose an open-sourced invariant filter-based platform, InGVIO, to tightly fuse monocular/stereo visual-inertial measurements, along with raw data from GNSS. InGVIO gives highly competitive results in terms of computational load compared to current graph-based algorithms, meanwhile possessing the same or even better level of accuracy. Thanks to our proposed marginalization strategies, the baseline for triangulation is large although only a few cloned poses are kept. Moreover, we define the infinitesimal symmetries of the system and exploit the various structures of its symmetry group, being different from the total symmetries of the VIO case, which elegantly gives results for the pattern of degenerate motions and the structure of unobservable subspaces. We prove that the properly-chosen invariant error is still compatible with all possible symmetry group structures of InGVIO and has intrinsic consistency properties. Besides, InGVIO has strictly linear error propagation without linearization error. InGVIO is tested on both open datasets and our proposed fixed-wing datasets with variable levels of difficulty and various numbers of satellites. The latter datasets, to the best of our knowledge, are the first datasets open-sourced to the community on a fixed-wing aircraft with raw GNSS.

Visual-Inertial Odometry (VIO) is an approach to give high-accuracy pose estimation in GNSS(Global Navigation Satellite System)-denied environments. However, visual-inertial navigation slowly drifts on its four unobservable directions, namely the translations and yaw in the world frame. The visual and inertial sensors are unable to provide information about how the world frame is aligned with global geographic coordinates. In this letter, g-MSCKF, a Multi-State Constraint Kalman Filter based approach, is developed to combine visual, inertial and GNSS raw measurements. With GNSS raw measurements, g-MSCKF has the ability to use GNSS information even when the number of satellites is below 4. An alignment filter between the local world frame and the global geographic frame is proposed and serves as the initializer for g-MSCKF. Furthermore, unobservable directions may exist and vary when the number of satellites, the satellite-receiver spatial geometry and the receiver motion follow specific patterns. Inconsistency of the estimator may happen under those variable-unobservable circumstances and a variable observability constrained method is provided to avoid inconsistency. Our algorithm is evaluated on real-world open datasets where the sensors traverse indoors and outdoors. The results show that our solution gives globally smooth trajectories in GNSS-intermittent situations with full exploitation of sensor potentials.

Due to the susceptibility of depth and thermal images to environmental interferences, researchers began to combine three modalities for salient object detection (SOD). In this letter, we propose an efficient transformer network (ETFormer) based on multimodal hybrid fusion and representation learning for RGB-D-T SOD. First, unlike most works, we design a backbone to extract three modal information, and propose a multi-modal multi-head attention module (MMAM) for feature fusion, which improves network performance while reducing compute redundancy. Secondly, we reassembled a three-modal dataset called R-D-T ImageNet-1K to pretrain the network to solve the problem that other modalities are still using RGB modality during pretraining. Finally, through extensive experiments, our proposed method can combine the advantages of different modalities and achieve better performance compared to other existing methods.

A positioning system that relies solely on observations based on a local frame, such as visual-inertial odometry (VIO), suffers from unobservable directions, leading to cumulative estimation errors over time. To address this issue, we propose a tightly-coupled Visual-Inertial-GNSS-Barometer odometry (GBVIO) based on an invariant filter. The integration of Global Navigation Satellite System (GNSS) and barometric data enables global convergence. Our system supports both tightly coupled updates (using pseudorange and Doppler shift measurements) and loosely coupled updates (using global position data). We prove that the invariant filter using barometric observations remains invariant under stochastic unobservable transformations, thus exhibiting improved consistency. Validation through computer-based Monte Carlo simulations and real-world dataset experiments demonstrates the superiority of GBVIO over standalone VIO. All source code and datasets are publicly available.

The detection of mirrors and glass, which possess unique optical surface properties, has garnered significant attention in recent years. Due to their reflective and transparent nature, these surfaces are often difficult to distinguish from their surrounding environments, posing substantial challenges even for advanced deep learning models tasked with performing such detection. Current research primarily relies on complex network models that learn and fuse different modalities of images, such as RGB, depth, and thermal, to achieve mirror and glass detection. However, these approaches often overlook the inherent limitations in the raw data caused by sensor deficiencies when facing mirrors and glass surfaces. To address this issue, we applied mathematical and physical methods, such as three-point plane determination and steady-state heat conduction in two-dimensional planes, along with an RGB enhancement module, to reconstruct RGB, depth, and thermal data for mirrors and glass in two publicly available datasets: an RGB-D mirror detection dataset and an RGB-T glass detection dataset. Additionally, we synthesized four enhanced and ideal datasets. Furthermore, we propose a double weight Mamba fusion network (DWMFNet) that strengthens the model’s global perception of image information by extracting low-level clue weights and high-level contextual weights from the input data using the prior fusion feature extraction module (PFFE) and the deep fusion feature guidance module (DFFG). This is complemented by the Mamba module, which efficiently captures long-range dependencies, facilitating information complementarity between multi-modal features. Extensive experiments demonstrate that our data enhancement method significantly improves the model’s capability in detecting mirrors and glass surfaces.