Existing monocular depth estimation driving datasets are limited in the number of images and the diversity of driving conditions. The images of datasets are commonly in a low resolution and the depth maps are sparse. To overcome these limitations, we produce a Synthetic Digital City Dataset (SDCD) which was collected under 6 different weather driving conditions, and 6 common adverse perturbations caused by the data transmission. SDCD provides a total of 930 K high-resolution RGB images and corresponding perfect observed depth maps. The evaluation shows that depth estimation models which are trained on SDCD provide a clearer, smoother, and more precise long-range depth estimation compared to those trained on one of the best-known driving datasets KITTI. Moreover, we provide a benchmark to investigate the performance of depth estimation models in different adverse driving conditions. Instead of collecting data from the real world, we generate the SDCD under severe driving conditions with perfect observed data in the digital world, enhancing depth estimation for autonomous driving.
9 December 2024 in IEEE Transactions on Intelligent Transportation Systems
Rain deviates the distribution of rainy images and the clean, rain-free data typically used during perception model training, this kind of out-of-distribution (OOD) issue making it difficult for models to generalize effectively in rainy scenarios, leading the performance degrade of autonomous perception systems in visual tasks such as lane detection and depth estimation, posing serious safety risks. To address this issue, we propose the Ultra-Fast Deraining Plugin (UFDP), a model-efficient deraining solution specifically designed to realign the distribution of rainy images and their rain-free counterparts. UFDP not only effectively removes rain from images but also seamlessly integrates into existing visual perception models, significantly enhancing their robustness and stability under rainy conditions. Through a detailed analysis of single-image color histograms and dataset-level distribution, we demonstrate how UFDP improves the similarity between rainy and non-rainy image distributions. Additionally, qualitative and quantitative results highlight UFDP’s superiority over state-of-the-art (SOTA) methods, showing a 5.4% improvement in SSIM and 8.1% in PSNR. UFDP also excels in terms of efficiency, achieving 7 times higher FPS than the slowest method, reducing FLOPs by 53.7 times, and using 28.8 times fewer MACs, with 6.2 times fewer parameters. This makes UFDP an ideal solution for ensuring reliable performance in autonomous driving visual perception systems, particularly in challenging rainy environments.
Autonomous Valet Parking (AVP) represents the final operational task in a full-chain autonomous driving process, where vehicles are expected to navigate in complex and dynamic parking environments autonomously without human intervention. However, the progress of AVP research is currently hindered by the lack of task-specific datasets, making it challenging to develop and validate AVP algorithms effectively. By formalizing the static object features of typical parking lots, including locations, obstacles and attributes, this paper introduces the Vision-and-Language Parking (VLP) dataset, featuring 1044 onboard panoramic images and 301 commands, marking it as the first dataset for AVP tasks. Additionally, we develop an Agent-oriented Benchmark AI toolkit with 14 baselines from Rule-Based (RB) scripts, Reinforcement Learning (RL), Deep Learning (DL) and Multimodal Large Language Model (MLLM). The results show reinforcement learning faces significant trajectory exploration challenges, deep learning struggles with out-of-distribution data generalization and MLLM shows good language understanding but poor analysis of environmental observation. The dataset and benchmark proposed in this paper provide a foundational basis for the development, sharing, and expansion of AVP algorithms.