Recently, autonomous driving has become one of the most popular topics for smart vehicles. However, traditional control strategies are mostly rule-based, which have poor adaptability to the time-varying traffic conditions. Similarly, they have difficulty coping with unexpected situations that may occur any time in the real-world environment. Hence, in this paper, we exploited Deep Reinforcement Learning (DRL) to enhance the quality and safety of autonomous driving control. Based on the road scenes and self-driving simulation modules provided by AirSim, we used the Deep Deterministic Policy Gradient (DDPG) and Recurrent Deterministic Policy Gradient (RDPG) algorithms, combined with the Convolutional Neural Network (CNN), to realize the autonomous driving control of self-driving cars. In particular, by using the real-time images of the road provided by AirSim as the training data, we carefully formulated an appropriate reward-generation method to improve the convergence speed of the adopted DDPG and RDPG models and the control performance of moving driverless cars.

Nowadays, drones are expected to be used in several engineering and safety applications both indoors and outdoors, e.g., exploration, rescue, sport, entertainment, and convenience. Among those applications, it is important to make a drone capable of flying autonomously to carry out an inspection patrol. In this paper, we present a novel method that uses ArUco markers as a reference to improve the accuracy of a drone on autonomous straight take-off, flying forward, and landing based on Deep Reinforcement Learning (DRL). More specifically, the drone first detects a specific marker with one of its onboard cameras. Then it calculates the position and orientation relative to the marker so as to adjust its actions for achieving better accuracy with a DRL method. We perform several simulation experiments with different settings, i.e., different sets of states, different sets of actions and even different DRL methods, by using the Robot Operating System (ROS) and its Gazebo simulator. Simulation results show that our proposed methods can efficiently improve the accuracy of the considered actions.

In this study, an unmanned aerial vehicle (UAV) with a camera and laser ranging module was developed to inspect bridge cracks. Four laser ranging units were installed adjacent to the camera to measure the distance from the camera to the object to calculate the object’s projection plane and overcome the limitation of vertical photography. The image processing method was adopted to extract crack information and calculate crack sizes. The developed UAV was used in outdoor bridge crack inspection tests; for images taken at a distance of 2.5 m, we measured the crack length, and the error between the result and the real length was less than 0.8%. The developed UAV has a dual-lens design, where one lens is used for bridge inspections and the other lens is used for flight control. The camera of the developed UAV can be rotated from the horizontal level to the zenith according to user requirements; thus, this UAV achieves high safety and efficiency in bridge inspections.

Due to its NP-Hard property, the Travelling Salesman Problem (TSP) has long been a prominent research topic in path planning. The goal is to design the algorithm with the fastest execution speed in order to find the path with the lowest travelling cost. In particular, new generative AI technology is continually emerging. The question of how to exploit algorithms from this realm to perform TSP path planning, especially in dynamic environments, is an important and interesting problem. The TSP application scenario investigated by this paper is that of an Unmanned Aerial Vehicle (UAV) that needs to patrol all specific ship-targets on the sea surface before returning to its origin. Hence, during the flight, we must consider real-time changes in wind velocity and direction, as well as the dynamic addition or removal of ship targets due to mission requirements. Specifically, we implement a Deep Reinforcement Learning (DRL) model based on the Transformer architecture, which is widely used in Generative AI, to solve the TSP path-planning problem in dynamic environments. Finally, we conduct numerous simulation experiments to compare the performance of our DRL model and the traditional heuristic algorithm, the Simulated Annealing (SA) method, in terms of operation time and path distance in solving the ordinary TSP, to verify the advantages of our model. Notably, traditional heuristic algorithms cannot be applied to dynamic environments, in which wind velocity and direction can change at any time.

Autonomous driving in urban areas is challenging because it requires understanding vehicle movements, traffic rules, map topologies and unknown environments in the highly complex driving environment, and thus typical urban traffic scenarios include various potentially hazardous situations. Therefore, training self-driving cars by using traditional deep learning models not only requires the labelling of numerous datasets but also takes a large amount of time. Because of this, it is important to find better alternatives for effectively training self-driving cars to handle vehicle behavior and complex road shapes in dynamic environments and to follow line guidance information. In this paper, we propose a method for training a self-driving car in simulated urban traffic scenarios to be able to judge the road conditions on its own for crossing an unsignalized intersection. In order to identify the behavior of traffic flow at the intersection, we use the CARLA (CAR Learning to Act) self-driving car simulator to build the intersection environment and simulate the process of traffic operation. Moreover, we attempt to use the DDPG (Deep Deterministic Policy Gradient) and RDPG (Recurrent Deterministic Policy Gradient) learning algorithms of the DRL (Deep Reinforcement Learning) technology to train models based on the CNN (Convolutional Neural Network) architecture. Specifically, the observation image of the semantic segmentation camera installed on the self-driving car and the vehicle speed are used as the model input. Moreover, we design an appropriate reward mechanism for performing training according to the current situation of the self-driving car judged from sensing data of the obstacle sensor, collision sensor and lane invasion detector. Doing so can improve the convergence speed of the model to achieve the purpose of the self-driving car autonomously judging the driving paths so as to accomplish accurate and stable autonomous driving control.

Advanced Driver Assistance Systems (ADAS) are only applied to relatively simple scenarios, such as highways. If there is an emergency while driving, the driver should take control of the car to deal properly with the situation at any time. Obviously, this incurs the uncertainty of safety. Recently, in the literature, several studies have been proposed for the above-mentioned issue via Artificial Intelligence (AI). The achievement is exactly the aim that we look forward to, i.e., the autonomous vehicle. In this paper, we realize the autonomous driving control via Deep Reinforcement Learning (DRL) based on the CARLA (Car Learning to Act) simulator. Specifically, we use the ordinary Red-Green-Blue (RGB) camera and semantic segmentation camera to observe the view in front of the vehicle while driving. Then, the captured information is utilized as the input for different DRL models so as to evaluate the performance, where the DRL models include DDPG (Deep Deterministic Policy Gradient) and RDPG (Recurrent Deterministic Policy Gradient). Moreover, we also design an appropriate reward mechanism for these DRL models to realize efficient autonomous driving control. According to the results, only the RDPG strategies can finish the driving mission with the scenario that does not appear/include in the training scenario, and with the help of the semantic segmentation camera, the RDPG control strategy can further improve its efficiency.