The performance of the UAVs while executing various mission profiles greatly depends on the selection of planning algorithms. Reinforcement learning (RL) algorithms can effectively be utilized for robot path planning. Due to random action selection in case of action ties, the traditional Q-learning algorithm and its other variants face the issues of slow convergence and suboptimal path planning in high-dimensional navigational environments. To solve these problems, we propose an improved deep Q-network (DQN), incorporating an efficient tie-breaking mechanism, prioritized experience replay (PER), and L2-regularization. The adopted tie-breaking mechanism improves the action selection and ultimately helps in generating an optimal trajectory for the UAV in a 3D cluttered environment. To improve the convergence speed of the traditional Q-algorithm, prioritized experience replay is used, which learns from experiences with high temporal difference (TD) error and avoids uniform sampling of stored transitions during training. This also allows the prioritization of high-reward experiences (e.g., reaching a goal), which helps the agent to rediscover these valuable states and improve learning. Moreover, L2-regularization is adopted that encourages smaller weights for more stable and smoother Q-values to reduce the erratic action selections and promote smoother UAV flight paths. Finally, the performance of the proposed method is presented and thoroughly compared against the traditional DQN, demonstrating its superior effectiveness.

This paper presents an interval-observer-based actuator fault detection and isolation (FDI) method for underactuated autonomous underwater vehicles (AUVs) under bounded hydrodynamic uncertainty and time-varying ocean currents. A locally frozen linear time-invariant (LTI) representation enables deterministic set-membership analysis, and the robust adaptive propagated interval observer (RAPIO) propagates admissible center–radius state bounds within a Lyapunov framework. Adaptivity is introduced through a reinforcement learning (RL)-augmented uncertainty-bound modulation mechanism, where an offline-trained agent scales a nonnegative channel-wise slack term without modifying the scheduled observer-gain rule or the nominal center predictor. Under the stated observer and disturbance-envelope conditions, positivity, stability, and diagnostic-channel inclusion hold for any bounded learning signal. Actuator loss-of-effectiveness (LoE) faults are represented through the actuator-effectiveness channel and detected through interval-consistency violations, enabling axis-wise isolation of surge, yaw-rate, and pitch-rate actuator faults. The same schedule-blind decision layer is additionally evaluated with structurally distinct additive-bias and stuck/jam actuator models. All stuck/jam events are detected, and bias-magnitude sweeps identify channel-wise 100%-detection boundaries with zero false alarms. A structured 72-case scenario sweep shows reliable detection, strong false-alarm rejection, and acceptable detection delays compared with benchmark observers.

Reliable fault detection and isolation (FDI) for actuators and sensors in underactuated autonomous underwater vehicles (AUVs) is challenging because nonlinear hydrodynamic coupling redistributes fault signatures, and persistent ocean currents can trigger false alarms. This paper presents a reinforcement learning (RL)-augmented dual-layer FDI framework for actuator and sensor faults in underactuated AUVs. The actuator layer uses a robust adaptive propagated interval observer (RAPIO) that evaluates thruster and control-surface residuals against a calibrated dynamics-consistency tube. The sensor layer forms estimator-consistency residuals for Doppler velocity log (DVL), depth, and inertial measurement unit (IMU) measurements against a reference-separated finite-time extended state observer (FTESO). An offline-trained soft actor–critic (SAC) policy schedules bounded actuator uncertainty margins and sensor alarm thresholds according to operating confidence. The scheduled actuator error dynamics remain Metzler and Hurwitz, preserving positive interval propagation and center-error input-to-state stability (ISS) independent of policy convergence. A Schmitt-trigger alarm and signal-space disambiguation rule classify healthy, actuator-only, sensor-only, and simultaneous-fault conditions under explicit residual-separation and persistence conditions. Across 72 simultaneous-fault episodes over a $4\times 6$ uncertainty–current grid, the proposed method achieved $100\%$ detection coverage for both actuator and sensor faults with only five false-alarm events, retaining full coverage in the severe-current, high-uncertainty subset where the selected actuator and sensor baselines achieved only 88.9% and 70.4% detection, respectively, with more false alarms. These results indicate that the proposed bounded RL scheduler can deliver reliable, certifiable actuator and sensor fault diagnosis under significant operational uncertainty.