Reinforcement learning-based techniques, empowered by deep-structured neural nets, have demonstrated superiority over rule-based methods in terms of making high-level behavioral decisions due to qualities related to handling large state spaces. Nonetheless, their training time, sample efficiency and the feasibility of the learnt behaviors remain key concerns. In this paper, we propose a novel hierarchical reinforcement learning-based decision-making architecture for learning left-turn policies at unsignalized intersections with feasibility guarantees. The proposed technique is comprised of two layers; a high-level learning-based behavioral planning layer which adopts soft actor-critic (SAC) principles to learn high-level, non-conservative yet safe, driving behaviors, and a low-level motion planning layer that uses Model Predictive Control (MPC) framework to ensure feasibility of the two-dimensional left-turn maneuver. The high-level layer generates reference signals of velocity and yaw angles for the ego vehicle taking into account safety and collision avoidance with the intersection vehicles, whereas the low-level motion planning layer solves an optimization problem to track these reference commands taking into account several vehicle dynamic constraints and ride comfort. While training the behavioral SAC-based planning layer, We develop an adaptive entropy regularization technique that results in faster convergence, higher mean rewards, and lower collision rates. We validate the proposed decision-making scheme in simulated environments and compare with other model free Reinforcement Learning (RL) baselines. The results demonstrate that the proposed integrated framework possesses better training and navigation capabilities.

In this paper, a new approach for lane change and double-lane change planning and following for autonomous driving is proposed. Herein, we introduce a novel technique, based on exponential functions, to generate online and feasible lane change maneuvers; these maneuvers satisfy the constraints on the maximum allowable curvature a given vehicle can handle. In addition, a simultaneous local path planning and path-following control framework is adapted. The framework utilizes a multi-threading architecture to run the local planner module concurrently with the control module. The planning module generates parametric reference paths based on the proposed lane change and double-lane change maneuvers. The control module is based on a Model Predictive Path-following Control (MPFC) scheme, which achieves the path following objective while satisfying vehicle’s state and control limits. To validate the proposed framework, several real-time simulation scenarios are designed and tested on CARLA soft real-time vehicle simulator. The results show the effectiveness of the proposed framework in generating and smoothly following lane change maneuvers.

In this paper, a generic multi-sensor fusion framework is developed for the localization of intelligent vehicles and mobile robots. The localization framework is based on moving horizon estimation (MHE). Unlike the commonly used probabilistic filtering algorithms – for example, extended Kalman filter (EKF) and unscented Kalman filter (UKF) – MHE relies on solving successive least squares optimization problems over the innovation of multiple sensors’ measurements and a specific estimation horizon. In this paper, we present an efficient and generic multi-sensor fusion scheme, based on MHE. The proposed multi-sensor fusion scheme is capable of operating with different sensors’ rates, missing measurements, and outliers. Moreover, the proposed scheme is based on a multi-threading architecture to reduce its computational cost, making it more feasible for practical applications. The MHE fusion method is tested using simulated data as well as real experimental data sequences from an intelligent vehicle and a mobile robot combining measurements from different sensors to get accurate localization results. The performance of MHE is compared against that of UKF, where the MHE estimation results show superior performance.

A few studies have been conducted recently in order to improve the aerodynamic performance of Darrieus vertical-axis wind turbines with straight blades (H-type VAWTs). The blade pitch angle control is proposed to enhance the performance of H-type VAWTs. This paper aims to investigate the performance of an H-type VAWT in terms of its power output and self-starting capability using an intelligent blade pitch control strategy based on a multi-layer perceptron artificial neural network (MLP-ANN) method. The performance of the proposed blade pitch controller is investigated by adding a conventional controller (PID) to the MLP-ANN controller (i.e., a hybrid controller). The dynamics of an H-type VAWT is mathematically modeled in a nonlinear state space for the stability analysis in the sense of Lyapunov. The effectiveness of the proposed pitch control system is validated by building an H-type VAWT prototype model that is extensively tested outdoors under different conditions for both fixed and variable pitch angle configurations. Results demonstrated that the blade-pitching technique enhanced the power output of an H-type VAWT by approximately $22\%$. The hybrid controller that used a high percentage of the MLP-ANN controller achieved a better control performance by reducing the overshoot of the control response at high rotor speeds.

Autonomous driving at unsignalized intersections is still considered a challenging application for machine learning due to the complications associated with handling complex multi-agent scenarios characterized by a high degree of uncertainty. Automating the decision-making process in these safety-critical environments involves comprehending multiple levels of abstraction associated with learning robust driving behaviors to enable the vehicle to navigate efficiently. In this survey, we aim at exploring the state-of-the-art techniques implemented for decision-making applications, with a focus on algorithms that combine Reinforcement Learning (RL) and deep learning for learning traversing policies at unsignalized intersections. The reviewed schemes vary in the proposed driving scenario, in the assumptions made for the used intersection model, in the tackled challenges, and in the learning algorithms that are used. We have presented comparisons for these techniques to highlight their limitations and strengths. Based on our in-depth investigation, it can be discerned that a robust decision-making scheme for navigating real-world unsignalized intersection has yet to be developed. Along with our analysis and discussion, we recommend potential research directions encouraging the interested players to tackle the highlighted challenges. By adhering to our recommendations, decision-making architectures that are both non-overcautious and safe, yet feasible, can be trained and validated in real-world unsignalized intersections environments.