The detection and classification of pavement distress (PD) play a critical role in pavement maintenance and rehabilitation. Research on PD automation detection and measurement has been actively conducted. However, types of PD are more necessary for road managers to take effective actions. Also, lack of a unified PD dataset leads to absence of a benchmark on various methods. This study makes three contributions to address these issues. Firstly, a large-scale PD dataset is prepared. This dataset is composed of 45,788 images captured with a high-resolution industrial camera installed on vehicles, in a variety of weather and illuminance conditions. Each image is annotated with bounding box representing location and type of distress. Secondly, a deep learning-based object detection framework, the YOLO network, is adopted to predict possible distress location and category. Comprehensive detection accuracy reaches 73.64%. The processing speed reaches 0.0347s/pic, as 9 times faster than Faster R-CNN and only 70% of SSD. Finally, the applicability of model under various illumination conditions is also explored. The results reveal that the method significantly outperforms with appropriate illumination. We conclude that the proposed YOLO-based approach is able to detect PD with high accuracy, which requires no manual feature extraction and calculation during detecting.

Roadside sensors, such as camera and millimeter-wave (MMW) radar, provide traffic information beyond the visual range of intelligent vehicles in cooperative vehicle-infrastructure systems. Unlike onboard equipment, roadside sensors are affiliated with different systems and lack synchronization in both space and time. In this paper, we propose a novel spatio-temporal synchronization method of asynchronous roadside MMW radar-camera for sensor fusion, which utilizes features of the scenario to extract lane line corner points to pre-calibrate the camera. Based on the consistent time flow rate of the separate sensors, multiple virtual detection lines are set up to match the time headway of successive vehicles and conduct objective matching to track data. Finally, a synchronization optimization model is formulated and a constrained nonlinear minimization solver is applied to tune the parameters. Measure data from Donghai Bridge in Shanghai is applied to verify the feasibility and effectiveness of the method. The results determine that there are 33 frames (33*40 ms) of temporal deviation between the camera and the radar in this case. After the synchronization, the average spatial deviation is reduced from 2.47 m to 0.42 m in the X-direction and 64.06 m to 2.34 m in the Y-direction, respectively. This study provides an economical and effective way to solve the problem of spatio-temporal synchronization of roadside sensors.

Potential fields have been integrated with local path-planning algorithms for autonomous vehicles (AVs) to tackle challenging scenarios with dense and dynamic obstacles. Most existing potential fields are isotropic without considering the traffic agent’s geometric shape and could cause failures due to local minima. We propose a three-dimensional potential field (TriPField) model to overcome this drawback by integrating an ellipsoid potential field with a Gaussian velocity field (GVF). Specifically, we model the surrounding vehicles as ellipsoids in corresponding ellipsoidal coordinates, where the formulated Laplace equation is solved with boundary conditions. Meanwhile, we develop a nonparametric GVF to capture the multi-vehicle interactions and then plan the AV’s velocity profiles, reducing the path search space and improving computing efficiency. Finally, a local path-planning framework with our TriPField is developed by integrating model predictive control to consider the constraints of vehicle kinematics. Our proposed approach is verified in three typical scenarios, i.e., active lane change, on-ramp merging, and car following. Experimental results show that our TriPField-based planner obtains a shorter, smoother local path with a slight jerk during control, especially in the scenarios with dense traffic flow, compared with traditional potential field-based planners. Our proposed TriPField-based planner can perform emergent obstacle avoidance for AVs with a high success rate even when the surrounding vehicles behave abnormally.

Deep learning has achieved promising results in pavement distress detection. However, the training model's effectiveness varies according to the data and scenarios acquired by different camera types and their installation positions. It is time consuming and labor intensive to recollect labeled data and retrain a new model every time the scene changes. In this paper, we propose a transfer learning pipeline to address this problem, which enables a distress detection model to be applied to other untrained scenarios. The framework consists of two main components: data transfer and model transfer. The former trains a generative adversarial network to transfer existing image data into a new scene style. Then, attentive CutMix and image melding are applied to insert distress annotations to synthesize the new scene's labeled data. After data expansion, the latter step transfers the feature extracted by the existing model to the detection application of the new scene through domain adaptation. The effects of varying degrees of knowledge transfer are also discussed. The proposed method is evaluated on two data sets from two different scenes with more than 40,000 images totally. This method can reduce the demand for training data by at least 25% when the model is applied in a new scene. With the same number of training images, the proposed method can improve the model accuracy by 26.55%.

The failure to estimate friction on the road is a significant cause of roadway departure crashes. It is even more relevant to autonomous vehicles (AVs) as they are currently not designed to measure the skid resistance of roads. Most traditional measurements are too inefficient for real-time AV control. Thus, in this paper, we propose a dynamic method to estimate pavement friction level using computer vision. We collect 100 sets of high-quality road images and their resistance values to train the model, and design two major methods for analysis: 1) texture identification, consisting of grayscale-enhanced local binary pattern and the gray-level gradient co-occurrence matrix and 2) a deep neural network based on domain knowledge (TLDKNet). We introduce two standards of classification for model training and propose three indices--underestimation error, overestimation error, and accuracy--to verify the performance of our algorithms. The results showed the great correlation between pavement texture and skid resistance. The TLDKNet yielded the best performance with an accuracy of 90.67% and only 2.67% underestimation error, showing that it is sufficiently conservative with regard to safety. Based on the proposed method of estimation, a framework for anti-skid driving control is developed regarding the car-following behavior and turning movements. The anti-skid control framework provides new insights into enhancing the AV safety performance.

Trajectory planning is essential for self-driving vehicles and has stringent requirements for accuracy and efficiency. The existing trajectory planning methods have limitations in the feasibility of planned trajectories and computational efficiency. This paper proposes a life-long learning framework to achieve effective and high-accuracy direct trajectory planning (DTP) tasks. Based on generative adversarial networks (GANs), this study develops a lightweight GDTP model to map the initial/final states and the control action sequence. Additionally, by embedding the GDTP into the rapidly-exploring random tree (RRT), a GDTP-RRT algorithm is further designed for long-distance and multi-stage planning tasks. Taking the tractor-trailer as an application case, we test the proposed method in multiple scenarios with varying characteristics. The experimental results show that the method can plan highly feasible trajectories in a short time, compared with the most applied algorithm – the cubic curve RRT* (CCRRT*). It is found that the tracking errors of our method are 29.1% and 44.1% lower than the CCRRT* in terms of position and heading angle. This paper provides an effective and stable vehicle trajectory planning method for complex self-driving tasks.