Target tracking using an unmanned aerial vehicle (UAV) is a challenging robotic problem. It requires handling a high level of nonlinearity and dynamics. Model-free control effectively handles the uncertain nature of the problem, and reinforcement learning (RL)-based approaches are a good candidate for solving this problem. In this article, the Twin Delayed Deep Deterministic Policy Gradient Algorithm (TD3), as recent and composite architecture of RL, was explored as a tracking agent for the UAV-based target tracking problem. Several improvements on the original TD3 were also performed. First, the proportional-differential controller was used to boost the exploration of the TD3 in training. Second, a novel reward formulation for the UAV-based target tracking enabled a careful combination of the various dynamic variables in the reward functions. This was accomplished by incorporating two exponential functions to limit the effect of velocity and acceleration to prevent the deformation in the policy function approximation. In addition, the concept of multistage training based on the dynamic variables was proposed as an opposing concept to one-stage combinatory training. Third, an enhancement of the rewarding function by including piecewise decomposition was used to enable more stable learning behaviour of the policy and move out from the linear reward to the achievement formula. The training was conducted based on fixed target tracking followed by moving target tracking. The flight testing was conducted based on three types of target trajectories: fixed, square, and blinking. The multistage training achieved the best performance with both exponential and achievement rewarding for the fixed trained agent with the fixed and square moving target and for the combined agent with both exponential and achievement rewarding for a fixed trained agent in the case of a blinking target. With respect to the traditional proportional differential controller, the maximum error reduction rate is 86%. The developed achievement rewarding and the multistage training opens the door to various applications of RL in target tracking.

Long-term wind speed data for thirteen meteorological stations, measured over a five-year period, were statistically analyzed using the two-parameter Weibull distribution function. The purpose of this study is to reveal for the first time the wind power potentials in Chad and to provide a comprehensive wind map of the country. The results show that the values of the shape and scale parameters varied over a wide range. Analysis of the seasonal variations showed that higher wind speed values occur when the weather condition is generally dry and they drop considerably when the weather condition is wet. It was also observed that the wind speed increases as one moves from the southern zone to the Saharan zone. Although the wind power at each site varies significantly, however, the potentials of most of the sites were encouraging. Nevertheless, according to the PNNL classification system, they are favorable for small-scale applications only. A few stations in the middle of Sudanian and Sahel regions are found to be not feasible for wind energy generation due to their poor mean wind speed. The prevailing wind direction for both Saharan and Sahel regions is dominated by northeastern wind, while it diverged to different directions in the Sudanian zone.

The autonomous landing of an unmanned aerial vehicle (UAV) on a moving platform is an essential functionality in various UAV-based applications. It can be added to a teleoperation UAV system or part of an autonomous UAV control system. Various robust and predictive control systems based on the traditional control theory are used for operating a UAV. Recently, some attempts were made to land a UAV on a moving target using reinforcement learning (RL). Vision is used as a typical way of sensing and detecting the moving target. Mainly, the related works have deployed a deep-neural network (DNN) for RL, which takes the image as input and provides the optimal navigation action as output. However, the delay of the multi-layer topology of the deep neural network affects the real-time aspect of such control. This paper proposes an adaptive multi-level quantization-based reinforcement learning (AMLQ) model. The AMLQ model quantizes the continuous actions and states to directly incorporate simple Q-learning to resolve the delay issue. This solution makes the training faster and enables simple knowledge representation without needing the DNN. For evaluation, the AMLQ model was compared with state-of-art approaches and was found to be superior in terms of root mean square error (RMSE), which was 8.7052 compared with the proportional–integral–derivative (PID) controller, which achieved an RMSE of 10.0592.

This study presents a simulation result of an evaluation of the aerodynamic performance of a moving car with a wind turbine system. Sedan type cars (approaching the size of Proton Wira car) were modeled using the SolidWork software and simulation was done by ANSYS FLUENT software. Three car models with different wind turbine system positions (in front of the front bumper, on top of the hood and on top of the roof) plus one model without the wind turbine system were simulated. The study proved that the position of the wind turbine system installation will change the characteristic of the air flow around the car body and affects the aerodynamic performance of the car. Extended front bumper of a car is not significantly affecting the aerodynamic performance of the car. This extended bumper seems to be the suitable area to install a wind turbine system and the investigation shows that the aerodynamic performance of the car improved due to lower drag coefficient, C_d..