This paper presents a nonintrusive prototype computer vision system for monitoring a driver's vigilance in real time. It is based on a hardware system for the real-time acquisition of a driver's images using an active IR illuminator and the software implementation for monitoring some visual behaviors that characterize a driver's level of vigilance. Six parameters are calculated: Percent eye closure (PERCLOS), eye closure duration, blink frequency, nodding frequency, face position, and fixed gaze. These parameters are combined using a fuzzy classifier to infer the level of inattentiveness of the driver. The use of multiple visual parameters and the fusion of these parameters yield a more robust and accurate inattention characterization than by using a single parameter. The system has been tested with different sequences recorded in night and day driving conditions in a motorway and with different users. Some experimental results and conclusions about the performance of the system are presented

This paper presents a road distress detection system involving the phases needed to properly deal with fully automatic road distress assessment. A vehicle equipped with line scan cameras, laser illumination and acquisition HW-SW is used to storage the digital images that will be further processed to identify road cracks. Pre-processing is firstly carried out to both smooth the texture and enhance the linear features. Non-crack features detection is then applied to mask areas of the images with joints, sealed cracks and white painting, that usually generate false positive cracking. A seed-based approach is proposed to deal with road crack detection, combining Multiple Directional Non-Minimum Suppression (MDNMS) with a symmetry check. Seeds are linked by computing the paths with the lowest cost that meet the symmetry restrictions. The whole detection process involves the use of several parameters. A correct setting becomes essential to get optimal results without manual intervention. A fully automatic approach by means of a linear SVM-based classifier ensemble able to distinguish between up to 10 different types of pavement that appear in the Spanish roads is proposed. The optimal feature vector includes different texture-based features. The parameters are then tuned depending on the output provided by the classifier. Regarding non-crack features detection, results show that the introduction of such module reduces the impact of false positives due to non-crack features up to a factor of 2. In addition, the observed performance of the crack detection system is significantly boosted by adapting the parameters to the type of pavement.

The use of object detection algorithms has become extremely important in autonomous vehicles. Object detection at high accuracy and a fast inference speed is essential for safe autonomous driving. Therefore, the balance between the effectiveness and efficiency of the object detector must be considered. This article proposes a one-stage object detection framework for improving the detection accuracy while supporting a true real-time operation based on the YOLOv4. The backbone network in the proposed framework is the CSPDarknet53_dcn(P). The last output layer in the CSPDarknet53 is replaced with deformable convolution to improve the detection accuracy. In order to perform feature fusion, a new feature fusion module PAN++ is designed and five scales detection layers are used to improve the detection accuracy of small objects. In addition, this article proposes an optimized network pruning algorithm to solve the problem that the real-time performance of the algorithm cannot be satisfied due to the limited computing resources of the vehicle-mounted computing platform. The method of sparse scaling factor is used to improve the existing channel pruning algorithm. Compared to the YOLOv4, the YOLOV4-5D improves the mean average precision by 4.23% on the BDD data sets and 1.68% on the KITTI data sets. Finally, by pruning the model, the inference speed of YOLOV4-5D is increased 31.3% and the memory is only 98.1 MB when the detection accuracy is almost unchanged. Nevertheless, the proposed algorithm is capable of real-time detection at faster than 66 frames/s (fps) and shows higher accuracy than the previous approaches with a similar fps.

In recent years, considerable progress has been made in semantic segmentation of images with favorable environments. However, the environmental perception of autonomous driving under adverse weather conditions is still very challenging. In particular, the low visibility at nighttime greatly affects driving safety. In this paper, we aim to explore image segmentation in low-light scenarios, thereby expanding the application range of autonomous vehicles. The segmentation algorithms for road scenes based on deep learning are highly dependent on the volume of images with pixel-level annotations. Considering the scarcity of labeled large-scale nighttime data, we performed synthetic data collection and data style transfer using images acquired in daytime based on the autonomous driving simulation platform and generative adversarial network, respectively. In addition, we also proposed a novel nighttime segmentation framework (SFNET-N) to effectively recognize objects in dark environments, aiming at the boundary blurring caused by low semantic contrast in low-illumination images. Specifically, the framework comprises a light enhancement network which introduces semantic information for the first time and a segmentation network with strong feature extraction capability. Extensive experiments with Dark Zurich-test and Nighttime Driving-test datasets show the effectiveness of our method compared with existing state-of-the art approaches, with 56.9% and 57.4% mIoU (mean of category-wise intersection-over-union) respectively. Finally, we also performed real-vehicle verification of the proposed models in road scenes of Zhenjiang city with poor lighting. The datasets are available at https://github.com/pupu-chenyanyan/semantic-segmentation-on-nightime.

This paper aims to solve the path following problem for an underactuated unmanned-surface-vessel (USV) based on deep reinforcement learning (DRL). A smoothly-convergent DRL (SCDRL) method is proposed based on the deep Q network (DQN) and reinforcement learning. In this new method, an improved DQN structure was developed as a decision-making network to reduce the complexity of the control law for the path following of a three-degree of freedom USV model. An exploring function was proposed based on the adaptive gradient descent to extract the training knowledge for the DQN from the empirical data. In addition, a new reward function was designed to evaluate the output decisions of the DQN, and hence, to reinforce the decision-making network in controlling the USV path following. Numerical simulations were conducted to evaluate the performance of the proposed method. The analysis results demonstrate that the proposed SCDRL converges more smoothly than the traditional deep Q learning while the path following error of the SCDRL is comparable to existing methods. Thanks to good usability and generality of the proposed method for USV path following, it can be applied to practical applications.

This paper investigates the tracking control problem of marine surface vessels (MSVs) in the presence of uncertain dynamics and external disturbances. The facts that actuators are subject to undesirable faults and input saturation are taken into account. Benefiting from the smoothness of the Gaussian error function, a novel saturation function is introduced to replace each nonsmooth actuator saturation nonlinearity. Applying the hand position approach, the original motion dynamics of underactuated MSVs are transformed into a standard integral cascade form so that the vector design method can be used to solve the control problem for underactuated MSVs. By combining the neural network technique and virtual parameter learning algorithm with the vector design method, and introducing an event triggering mechanism, a novel event-triggered indirect neuroadaptive fault-tolerant control scheme is proposed, which has several notable characteristics compared with most existing strategies: 1) it is not only robust and adaptive to uncertain dynamics and external disturbances but is also tolerant to undesirable actuator faults and saturation; 2) it reduces the acting frequency of actuators, thereby decreasing the mechanical wear of the MSV actuators, via the event-triggered control (ETC) technique; 3) it guarantees stable tracking without the a priori knowledge of the dynamics of the MSVs, external disturbances or actuator faults; and 4) it only involves two parameter adaptations—a virtual parameter and a lower bound on the uncertain gains of the actuators—and is thus more affordable to implement. On the basis of the Lyapunov theorem, it is verified that all signals in the tracking control system of the underactuated MSVs are bounded. Finally, the effectiveness of the proposed control scheme is demonstrated by simulations and comparative results.