With the rapid development of intelligent vehicles and some successes in Advanced Driving Assistance Systems (ADAS), a new trend is that mixed levels of human driver engagements will be involved in the transportation system. Therefore, necessary visual guidance for drivers is vitally important under this situation to prevent potential risks. To advance the development of visual guidance systems, we introduce a novel vision-cloud data fusion methodology, integrating camera image and Digital Twin information from the cloud to help intelligent vehicles make better decisions. Target vehicle bounding box is drawn and matched by combining results of object detector running on ego vehicle and position information from the cloud. The best matching result, a 79.2% accuracy under 0.7 intersection over union threshold, is obtained with depth images served as an additional feature source. A case study on lane change prediction is conducted to show the effectiveness of the proposed data fusion methodology, where a multi-layer perceptron algorithm is proposed with modified approaches. Human-in-the-loop simulation results conducted in Unity game engine reveal that the proposed model can improve highway driving performance significantly in terms of safety, comfort, and environmental sustainability.
Deep learning models have achieved significant progress in developing self-driving algorithms. Despite their advantages, these algorithms typically require substantial amounts of data for effective training. Critical driving data, in particular, is essential for enhancing training efficiency and ensuring driving safety. However, existing methods for identifying critical data often rely on human prior knowledge or are disconnected from the training of self-driving algorithms. In this paper, we introduce a novel data distillation technique designed to autonomously identify critical data for training self-driving algorithms. We conducted experiments with both numerical simulations and the NGSIM dataset, which consists of real-world car trajectories on highway US-101, to validate our approach. In the numerical experiments, the distillation method achieved a test root mean squared error of 1.933 using only 200 distilled training data samples, demonstrating a significant improvement in data efficiency compared to the 1.872 test error obtained with 20,000 randomly sampled training samples. The distilled critical data represents only 1% of the original dataset, optimizing data usage and significantly enhancing computational efficiency. For real-world NGSIM data, we demonstrate the performance of the proposed method in scenarios with extremely sparse data availability and show that our proposed data distillation method outperforms other sampling baselines, including Herding and K-centering. These experimental results highlight the capability of the proposed method to autonomously identify critical data without relying on human prior knowledge.