To enhance system reliability and mitigate the vulnerabilities of the Global Navigation Satellite Systems (GNSS), it is common to fuse the Inertial Measurement Unit (IMU) and visual sensors with the GNSS receiver in the navigation system design, effectively enabling compensations with absolute positions and reducing data gaps. To address the shortcomings of a traditional Kalman Filter (KF), such as sensor errors, an imperfect non-linear system model, and KF estimation errors, a GRU-aided ESKF architecture is proposed to enhance the positioning performance. This study conducts Failure Mode and Effect Analysis (FMEA) to prioritize and identify the potential faults in the urban environment, facilitating the design of improved fault-tolerant system architecture. The identified primary fault events are data association errors and navigation environment errors during fault conditions of feature mismatch, especially in the presence of multiple failure modes. A hybrid federated navigation system architecture is employed using a Gated Recurrent Unit (GRU) to predict state increments for updating the state vector in the Error Estate Kalman Filter (ESKF) measurement step. The proposed algorithm’s performance is evaluated in a simulation environment in MATLAB under multiple visually degraded conditions. Comparative results provide evidence that the GRU-aided ESKF outperforms standard ESKF and state-of-the-art solutions like VINS-Mono, End-to-End VIO, and Self-Supervised VIO, exhibiting accuracy improvement in complex environments in terms of root mean square errors (RMSEs) and maximum errors.

Visual Inertial Odometry (VIO) has been extensively studied for navigation in GNSS-denied environments, but its performance can be heavily impacted by the complexity of the navigation environments such as weather conditions, illumination variation, flight dynamics, and environmental structure. Hybrid fusion approaches integrating Neural Networks (NN), especially Gated Recurrent units (GRU) with the Kalman filters (KF), such as Error-State Kalman Filter (ESKF) have shown promising results mitigating system nonlinearities due to challenging environmental conditions data issues, there is a lack of systematic studies quantitively analysing and comparing performance differences unhand. To address this gap and enable robust navigation in complex conditions, this study proposes and systematically analyses the performance of three hybrid fusion schemes for VIO-based navigation of Unmanned Aerial Vehicles (UAV). These three hybrid VIO schemes include Visual Odometry (VO) error compensation using NN, KF error compensation using NN, and prediction of Kalman gain using NN. The comparative analysis is performed using data generated in MATLAB incorporating the Unreal Engine involving diverse challenging environmental conditions: fog, rain, illumination level variability and variability in the number of features available for extraction during the UAV flight in the urban environment. The results demonstrate the performance improvement achieved by hybrid VIO fusion schemes compared to ESKF-based traditional fusion methods in the presence of multiple visual failure modes. Comparative analysis reveals notable improvement achieved by method 1 with enhancements of 93% in sunny, 91% in foggy and 90% in rainy conditions than the other two hybrid VIO architectures.

To enhance system reliability and mitigate vulnerabilities of the Global Navigation Satellite Systems (GNSS), it is common to fuse the Inertial Measurement Unit (IMU) and visual sensors with the GNSS receiver in the navigation system design, effectively enabling compensations with absolute position and reducing data gaps. To address shortcomings of traditional Kalman Filter (KF) such as sensor error, imperfect nonlinear system model, and KF estimation error, a GRU-aided ESKF architecture is proposed to enhance positioning performance. This study conducts Failure Mode and Effect Analysis (FMEA) to priories and identify the potential faults in the urban environment facilitating the design of improved fault-tolerant system architecture. The identified primary fault events are data association error and navigation environment error during fault conditions of feature mismatch, especially in the presence of multiple failure modes. A hybrid federated navigation system architecture is employed using a Gated Recurrent Unit (GRU) to predict state increments for updating the state vector in the Error Estate Kalman Filter (ESKF) measurement step. The proposed algorithm’s performance is evaluated in a simulation environment in MATLAB under multiple visually degraded conditions. Comparative results provide evidence that the GRU-aided ESKF outperforms standard ESKF and state-of-the-art solutions like VINS-Mono, End-to-End VIO, and Self-Supervised VIO, exhibiting higher efficiency in complex environments.

Vision-based navigation systems (VINS) are increasingly utilised as an alternative to GNSS for UAVs operating in urban environments, but they suffer from performance degradation under visual fault conditions like illumination variation, rapid motion, texture-less environments, and weather effects. While hybrid architecture incorporating Kalman filters and machine learning (ML) improves performance, they often lack evidence of providing contingency for non-Gaussian error distributions, limiting operational safety. To address these shortcomings, an enhanced hybrid VINS architecture is proposed, featuring a Bayesian GRU-based error correction network (B-GRU) to provide a contingency while compensating model errors. To the best of the authors’ knowledge, this is the first attempt to estimate uncertainty using a B-GRU compensator while addressing data uncertainty for VINS applications. The system architecture integrates an Error-State Kalman Filter (ESKF) and the B-GRU, compensating for position errors with uncertainty prediction. The proposed approach is validated using datasets from MATLAB incorporated in an Unreal Engine simulated environment, replicating the complex fault conditions. The ML model is trained on various visual failure modes to adapt the variability in the signal patterns during flights with simulated datasets and tested across varied flight paths and lighting scenarios. The results demonstrate that the fusion strategy effectively corrects erroneous measurements arising from corrupted sensor data and imperfect models and achieves an improvement of 78.06% compared to SOTA hybrid VIO on the horizontal axis while capturing complex flight dynamics in an unseen environment. A comparative analysis demonstrates the effectiveness of B-GRU in mitigating failure modes with a predictive error boundary, achieving a 72% improvement in performance compared to the architecture that integrates GRU-based error compensation. This approach shows a step forward in enhancing positioning accuracy and contingency in challenging urban environments.