As self-driving cars have been developed targeting level 4 and 5 autonomous driving, the capability of the vehicle to handle environmental effects has been considered importantly. The sensors installed on autonomous vehicles can be easily affected by blockages (e.g., rain, snow, dust, fog, and others) covering the surface of them. In a virtual environment, we can safely observe the behavior of the vehicle and the degradation of the sensors by blockages. In this letter, the CARLA simulator-based evaluation framework has been developed and the assessment of lane detection performance under sensor blockage by heavy rain, which was analyzed by using the experimental data. Thus, we thoroughly note that the accuracy of lane detection for the autonomous vehicle has been decreased as the rainfall rate increases, and the impact of the blockage is more critical to curved lanes than straight lanes. Finally, we have suggested a critical rainfall rate causing safety failures of the autonomous vehicles, based on reasonably established rainfall equation based on experimental rain datasets.

This paper proposes two algorithms for adaptive driving in urban environments: the first uses vision deep learning, which is named the sparse spatial convolutional neural network (SSCNN); and the second uses a sensor integration algorithm, named the sensor-weighted integration field (SWIF). These algorithms utilize three kinds of sensors, namely vision, Light Detection and Range (LiDAR), and GPS sensors, and decide critical motions for autonomous vehicle, such as steering angles and vehicle speed. SSCNN, which is used for lane recognition, has 2.7 times faster processing speed than the existing spatial CNN method. Additionally, the dataset for SSCNN was constructed by considering both normal and abnormal driving in 7 classes. Thus, lanes can be recognized by extending lanes for special characteristics in urban settings, in which the lanes can be obscured or erased, or the vehicle can drive in any direction. SWIF generates a two-dimensional matrix, in which elements are weighted by integrating both the object data from LiDAR and waypoints from GPS based on detected lanes. These weights are the integers, indicating the degree of safety. Based on the field formed by SWIF, the safe trajectories for two vehicles’ motions, steering angles, and vehicle speed are generated by applying the cost field. Additionally, to flexibly follow the desired steering angle and vehicle speed, the Proportional-Integral-Differential (PID) control is moderated by an integral anti-windup scheme. Consequently, as the dataset considers characteristics of the urban environment, SSCNN is able to be adopted for lane recognition on urban roads. The SWIF algorithm is also useful for flexible driving owing to the high efficiency of its sensor integration, including having a resolution of 2 cm per pixel and speed of 24 fps. Thus, a vehicle can be successfully maneuvered with minimized steering angle change, without lane or route departure, and without obstacle collision in the presence of diverse disturbances in urban road conditions.

Path tracking system plays a key technology in autonomous driving. The system should be driven accurately along the lane and be careful not to cause any inconvenience to passengers. To address such tasks, this research proposes hybrid tracker based optimal path tracking system. By applying a deep learning based lane detection algorithm and a designated fast lane fitting algorithm, this research developed a lane processing algorithm that shows a match rate with actual lanes with minimal computational cost. In addition, three modified path tracking algorithms were designed using the GPS based path or the vision based path. In the driving system, a match rate for the correct ideal path does not necessarily represent driving stability. This research proposes hybrid tracker based optimal path tracking system by applying the concept of an observer that selects the optimal tracker appropriately in complex road environments. The driving stability has been studied in complex road environments such as straight road with multiple 3-way junctions, roundabouts, intersections, and tunnels. Consequently, the proposed system experimentally showed the high performance with consistent driving comfort by maintaining the vehicle within the lanes accurately even in the presence of high complexity of road conditions. Code will be available in https://github.com/DGIST-ARTIV.

In autonomous driving, using a variety of sensors to recognize preceding vehicles at middle and long distances is helpful for improving driving performance and developing various functions. However, if only LiDAR or cameras are used in the recognition stage, it is difficult to obtain the necessary data due to the limitations of each sensor. In this paper, we proposed a method of converting the vision-tracked data into bird’s eye-view (BEV) coordinates using an equation that projects LiDAR points onto an image and a method of fusion between LiDAR and vision-tracked data. Thus, the proposed method was effective through the results of detecting the closest in-path vehicle (CIPV) in various situations. In addition, even when experimenting with the EuroNCAP autonomous emergency braking (AEB) test protocol using the result of fusion, AEB performance was improved through improved cognitive performance than when using only LiDAR. In the experimental results, the performance of the proposed method was proven through actual vehicle tests in various scenarios. Consequently, it was convincing that the proposed sensor fusion method significantly improved the adaptive cruise control (ACC) function in autonomous maneuvering. We expect that this improvement in perception performance will contribute to improving the overall stability of ACC.

Lane segmentation and Lane Keeping Assist System (LKAS) play a vital role in autonomous driving. While deep learning technology has significantly improved the accuracy of lane segmentation, real-world driving scenarios present various challenges. In particular, heavy rainfall not only obscures the road with sheets of rain and fog but also creates water droplets on the windshield or lens of the camera that affects the lane segmentation performance. There may even be a false positive problem in which the algorithm incorrectly recognizes a raindrop as a road lane. Collecting heavy rain data is challenging in real-world settings, and manual annotation of such data is expensive. In this research, we propose a realistic raindrop conversion process that employs a contrastive learning-based Generative Adversarial Network (GAN) model to transform raindrops randomly generated using Python libraries. In addition, we utilize the attention mask of the lane segmentation model to guide the placement of raindrops in training images from the translation target domain (real Rainy-Images). By training the ENet-SAD model using the realistically Translated-Raindrop images and lane ground truth automatically extracted from the CARLA Simulator, we observe an improvement in lane segmentation accuracy in Rainy-Images. This method enables training and testing of the perception model while adjusting the number, size, shape, and direction of raindrops, thereby contributing to future research on autonomous driving in adverse weather conditions.

The emergence of convolutional neural networks (CNNs) has led to significant advancements in various computer vision tasks. Among them, stereo matching is one of the most popular research areas that enables the reconstruction of 3D information, which is difficult to obtain with only a monocular camera. However, CNNs have their limitations, particularly their susceptibility to domain shift. The CNN-based stereo matching networks suffered from performance degradation under domain changes. Moreover, obtaining a significant amount of real-world ground truth data is laborious and costly when compared to acquiring synthetic data. In this paper, we propose an end-to-end framework that utilizes image-to-image translation to overcome the domain gap in stereo matching. Specifically, we suggest a horizontal attentive generation (HAG) module that incorporates the epipolar constraints when generating target-stylized left-right views. By employing a horizontal attention mechanism during generation, our method can address the issues related to small receptive field by aggregating more information of each view without using the entire feature map. Therefore, our network can maintain consistencies between each view during image generation, making it more robust for different datasets.