Unmanned air vehicles operate in highly dynamic and unknown environments where they can encounter unexpected and unseen failures. In the presence of emergencies, autonomous unmanned air vehicles should be able to land at a minimum distance or minimum time. Impaired unmanned air vehicles define actuator failures and this impairment changes their unstable and uncertain dynamics; henceforth, path planning algorithms must be adaptive and model-free. In addition, path planning optimization problems must consider the unavoidable actuator saturations, kinematic and dynamic constraints for successful real-time applications. Therefore, this paper develops 3D path planning algorithms for quadrotors with parametric uncertainties and various constraints. In this respect, this paper constructs a multi-dimensional particle swarm optimization and a multi-dimensional genetic algorithm to plan paths for translational, rotational, and Euler angles and generates the corresponding control signals. The algorithms are assessed and compared both in the simulation and experimental environments. Results show that the multi-dimensional genetic algorithm produces shorter minimum distance and minimum time paths under the constraints. The real-time experiments prove that the quadrotor exactly follows the produced path utilizing the available maximum rotor speeds.

Developing fault detection and diagnoses algorithms for the unmanned air vehicles such as the quadrotors is challenging since they are intrinsically non-linear, time-varying, unstable, and uncertain. This paper develops a reduced order Thau observer by only considering the uncertain rotational dynamics, which are re-constructed as the dominant linear and non-linear for the design purpose. Therefore, the proposed Thau observer is just third order and can reveal a rotational state estimation error in the presence of the quadrotor faults. This paper also equips the proposed Thau observer with a simple online adaptive fault estimation law, which is able to recognize up to two faulty actuators instantly using the estimated rotational state error. Lyapunov analysis confirms the error convergence in both the Thau observer states and the adaptive fault estimates. In addition, this paper constructs a batch type least-squares projection approach to quantify the magnitude percentages of the actuator failures. Moreover, to show the feasibility of the proposed algorithm, this paper extensively analyses the fault detection and diagnosis results performed in the simulation and real-time environments. Finally, to demonstrate the superiority of the proposed algorithm, it is compared with a recent Kalman filter based quadrotor fault estimation research under the equal conditions.

Model-free control approaches require advanced exploration-exploitation policies to achieve practical tasks such as learning to bipedal robot walk in unstructured environments. In this article, we first construct a comprehensive exploration-exploitation policy that carries quality knowledge about the long-term predictor and the control policy, and the control signal of the model-free algorithms. Therefore, the developed model-free algorithm continues exploration by adjusting its unknown parameters until the desired learning and control are accomplished. Second, we provide an utterly model-free adaptive law enriched with the exploration-exploitation policy and derived step-by-step using the exact analogy of the model-based solution. The obtained adaptive control law considers the control signal saturation and the control signal (input) delay. Performed Lyapunov stability analysis ensures the convergence of the adaptive law that can also be used for intelligent control approaches. Third, we implement the adaptive algorithm in real time on a challenging benchmark system: a fourth-order, coupled dynamics, input saturated, and time-delayed underactuated manipulator. The results show that the proposed adaptive algorithm explores larger state-action spaces and treats the vanishing gradient problem in both learning and control. Also, we notice from the results that the learning and control properties of the adaptive algorithm are optimized as required.

An autonomous humanoid robot (HR) with learning and control algorithms is able to balance itself during sitting down, standing up, walking and running operations, as humans do. In this study, reinforcement learning (RL) with a complete symbolic inverse kinematic (IK) solution is developed to balance the full lower body of a three-dimensional (3D) NAO HR which has 12 degrees of freedom. The IK solution converts the lower body trajectories, which are learned by RL, into reference positions for the joints of the NAO robot. This reduces the dimensionality of the learning and control problems since the IK integrated with the RL eliminates the need to use whole HR states. The IK solution in 3D space takes into account not only the legs but also the full lower body; hence, it is possible to incorporate the effect of the foot and hip lengths on the IK solution. The accuracy and capability of following real joint states are evaluated in the simulation environment. MapleSim is used to model the full lower body, and the developed RL is combined with this model by utilizing Modelica and Maple software properties. The results of the simulation show that the value function is maximized, temporal difference error is reduced to zero, the lower body is stabilized at the upright, and the convergence speed of the RL is improved with use of the symbolic IK solution.