High−precision and robust localization is critical for intelligent vehicle and transportation systems, while the sensor signal loss or variance could dramatically affect the localization performance. The vehicle localization problem in an environment with Global Navigation Satellite System (GNSS) signal errors is investigated in this study. The error state Kalman filtering (ESKF) and Rauch–Tung–Striebel (RTS) smoother are integrated using the data from Inertial Measurement Unit (IMU) and GNSS sensors. A segmented RTS smoothing algorithm is proposed in order to estimate the error state, which is typically close to zero and mostly linear, which allows more accurate linearization and improved state estimation accuracy. The proposed algorithm is evaluated using simulated GNSS signals with and without signal errors. The simulation results demonstrate its superior accuracy and stability for state estimation. The designed ESKF algorithm yielded an approximate 3% improvement in long straight line and turning scenarios compared to classical EKF algorithm. Additionally, the ESKF−RTS algorithm exhibited a 10% increase in the localization accuracy compared to the ESKF algorithm. In the double turning scenarios, the ESKF algorithm resulted in an improvement of about 50% in comparison to the EKF algorithm, while the ESKF−RTS algorithm improved by about 50% compared to the ESKF algorithm. These results indicated that the proposed ESKF−RTS algorithm is more robust and provides more accurate localization.

Safe reinforcement learning (RL) that solves constraint-satisfactory policies provides a promising way to the broader safety-critical applications of RL in real-world problems such as robotics. Among all safe RL approaches, model-based methods reduce training time violations further due to their high sample efficiency. However, lacking safety robustness against the model uncertainties remains an issue in safe model-based RL, especially in training time safety. In this paper, we propose a distributional reachability certificate (DRC) and its Bellman equation to address model uncertainties and characterize robust persistently safe states. Furthermore, we build a safe RL framework to resolve constraints required by the DRC and its corresponding shield policy. We also devise a line search method to maintain safety and reach higher returns simultaneously while leveraging the shield policy. Comprehensive experiments on classical benchmarks such as constrained tracking and navigation indicate that the proposed algorithm achieves comparable returns with much fewer constraint violations during training. Our code is available at https://github.com/ManUtdMoon/Distributional-Reachability-Policy-Optimization. Note to Practitioners —Although it has been proven that RL can be applied in complex robotics control tasks, the training process of an RL control policy induces frequent failures because the agent needs to learn safety through constraint violations. This issue hinders the promotion of RL because a large amount of failure of robots is too expensive to afford. This paper aims to reduce the training-time violations of RL-based control methods, enabling RL to be leveraged in a broader application area. To achieve the goal, we first introduce a safety quantity describing the distribution of potential constraint violations in the long term. By imposing constraints on the quantile of the safety distribution, we can realize safety robust to the model uncertainty, which is necessary for real-world robot learning with environment uncertainty. Second, we further devise a shield policy aiming to minimize the constraint violation. The policy will intervene when the agent is about to violate state constraints, further enhancing exploration safety. Third, we implement a line search method to find an action pursuing near-optimal performance when fulfilling safety requirements strictly. Our experimental results indicate that the proposed algorithm reduces training-time violations significantly while maintaining competitive task performance. We make a step towards applying RL safely in real-world tasks. Our future work includes conducting physical verification on real robots to evaluate the algorithm and improving safety further by starting from an initially safe control policy that comes from domain knowledge.

This study investigates the coupled ride and directional performance characteristics of an articulated frame-steered vehicle (AFSV). A three-dimensional multi-body dynamic model of the vehicle is formulated integrating the hydro-mechanical frame steering and hydro-pneumatic suspension (HPS) systems. The model parameters are obtained from field-measured data acquired for an unsuspended AFSV prototype and a validated scaled HPS model. The HPS is implemented only at the front axle, which supports the driver cabin. The main parameters of the HPS, including the piston area, and flow areas of bleed orifices and check valves, are selected through design sensitivity analyses and optimisation, considering ride vibration, and roll- and yaw-plane stability performance measures. These include the frequency-weighted vertical vibration of the front unit, root-mean-square lateral acceleration during the sustained lateral load transfer ratio period prior to absolute rollover of the rear unit, and yaw-mode oscillation frequency following a lateral perturbation of the vehicle. The results suggested that the implementation of the HPS to the front unit alone could help preserve the directional stability limits compared to the unsuspended prototype vehicle and reduce the ride vibration exposure by nearly 30%. The results of sensitivity analyses revealed that the directional stability performance limits are only slightly affected by the HPS parameters. Further reduction in the ride vibration exposure was attained with the optimal design, irrespective of the payload variations. The vehicle operation at relatively higher speeds, however, would yield greater vibration exposure.

Though policy evaluation error profoundly affects the direction of policy optimization and the convergence property, it is usually ignored in policy iteration methods. This work incorporates the practical inexact policy evaluation into a simultaneous policy update paradigm to reach the Nash equilibrium of the nonlinear zero-sum games. In the proposed algorithm, the restriction of precise policy evaluation is removed by bounded evaluation error characterized by Hamiltonian without sacrificing convergence guarantees. By exploiting Fréchet differential, the practical iterative process of value function with estimation error is converted into Newton's method with variable steps, which are inversely proportional to evaluation errors. Accordingly, we construct a monotone scalar sequence that shares the same Newton's method with the value sequence to bound the error of the value function, which enjoys an exponential convergence rate. Numerical results show its convergence in affine systems, and the potential to cope with general nonlinear plants.

Safety is a critical concern when applying reinforcement learning (RL) to real-world control problems. A widely used method for ensuring safety is to learn a control barrier function with heuristic feasibility labels that come from expert demonstrations [1] or constraint functions [2]. However, their forward invariant sets fall short of the maximum feasible region because of inaccurate labels. This paper proposes an algorithm called feasible region iteration (FRI) that learns the maximum feasible region to generate accurate feasibility labels. The core of FRI is a constraint decay function (CDF), which comes with a self-consistency condition and naturally leads to the constraint Bellman equation. The optimal CDF, which represents the maximum feasible region, is learned through the iteration of feasible region identification and feasible region expansion. Experiment results show that our algorithm achieves near-zero constraint violations and comparable or higher performance than the baselines.