In this study, we propose a novel traffic sign detection algorithm based on the deeplearning approach. The proposed algorithm, which we termed the feature-selection-based attentionaldeconvolution detector (FSADD), is used along with the “you look only once” (YOLO) v5 structure for feature selection. When applying feature selection inside a detection algorithm, the network divides the extracted feature maps after the convolution layer into similar and non similar feature maps. Generally, the feature maps obtained after the convolution layers are the outputs of filters with random weights. Owing to the randomness of the filter, the network obtains various kinds of feature maps with unnecessary components, which degrades the detection performance. However, grouping feature maps with high similarities can increase the relativeness of each feature map, thereby improving the network detection of specific targets from images. Furthermore, the proposed FSADD model has modified sizes of the receptive fields for improved traffic sign detection performance. Many of the available general detection algorithms are unsuitable for the German traffic sign detection benchmark (GTSDB) because of the small sizes of these signs in the images. Experimental comparisons were performed with respect to the GTSDB to show that the proposed FSADD is comparable to the state-of-the-art while detecting 29 kinds of traffic signs with 73.9% accuracy of classification performances.

Pedestrian behavior recognition in the driving environment is an important technology to prevent pedestrian accidents by predicting the next movement. It is necessary to recognize current pedestrian behavior to predict future pedestrian behavior. However, many studies have recognized human visible characteristics such as face, body parts or clothes, but few have recognized pedestrian behavior. It is challenging to recognize pedestrian behavior in the driving environment due to the changes in the camera field of view due to the illumination conditions in outdoor environments and vehicle movement. In this paper, to predict pedestrian behavior, we introduce a position-information added two-stream convolutional neural network (CNN) with multi task learning that is robust to the limited conditions of the outdoor driving environment. The conventional two-stream CNN is the most widely used model for human-action recognition. However, the conventional two-stream CNN based on optical flow has limitations regarding pedestrian behavior recognition in a moving vehicle because of the assumptions of brightness constancy and piecewise smoothness. To solve this problem for a moving vehicle, the binary descriptor dense scale-invariant feature transform (SIFT) flow, a feature-based matching algorithm, is robust in moving-pedestrian behavior recognition, such as walking and standing, in a moving vehicle. However, recognizing cross attributes, such as crossing or not crossing the street, is challenging using the binary descriptor dense SIFT flow because people who cross the road or not act the same walking action, but their location on the image is different. Therefore, pedestrian position information should be added to the conventional binary descriptor dense SIFT flow two-stream CNN. Thus, learning biased toward action attributes is evenly learned across action and cross attributes. In addition, YOLO detection and the Siamese tracker are used instead of the ground-truth boundary box to prove the robustness in the action- and cross-attribute recognition from a moving vehicle. The JAAD and PIE datasets were used for training, and only the JAAD dataset was used as a testing dataset for comparison with other state-of-the-art research on multitask and single-task learning.

Detecting failure cases is an essential element for ensuring the safety self-driving system. Any fault in the system directly leads to an accident. In this paper, we analyze the failure of semantic segmentation, which is crucial for autonomous driving system, and detect the failure cases of the predicted segmentation map by predicting mean intersection of union (mIoU). Furthermore, we design a deep neural network for predicting mIoU of segmentation map without the ground truth and introduce a new loss function for training imbalance data. The proposed method not only predicts the mIoU, but also detects failure cases using the predicted mIoU value. The experimental results on Cityscapes data show our network gives prediction accuracy of 93.21% and failure detection accuracy of 84.8%. It also performs well on a challenging dataset generated from the vertical vehicle camera of the Hyundai Motor Group with 90.51% mIoU prediction accuracy and 83.33% failure detection accuracy.