The applications of Vision-Language Models (VLMs) in the field of Autonomous Driving (AD) have attracted widespread attention due to their outstanding performance and the ability to leverage Large Language Models (LLMs). By integrating language data, the driving systems can be able to deeply understand real-world environments, improving driving safety and efficiency. In this work, we present a comprehensive and systematic survey of the advances in language models in this domain, encompassing perception and understanding, navigation and planning, decision-making and control, end-to-end autonomous driving, and data generation. We introduce the mainstream VLM tasks and the commonly utilized metrics. Additionally, we review current studies and applications in various areas and summarize the existing language-enhanced autonomous driving dataset thoroughly. At last, we discuss the benefits and challenges of VLMs in AD, and provide researchers with the current research gaps and future trends. https://github.com/ge25nab/Awesome-VLM-AD-ITS

Autonomous driving has rapidly developed and shown promising performance due to recent advances in hardware and deep learning techniques. High-quality datasets are fundamental for developing reliable autonomous driving algorithms. Previous dataset surveys either focused on a limited number or lacked detailed investigation of dataset characteristics. To this end, we present an exhaustive study of 265 autonomous driving datasets from multiple perspectives, including sensor modalities, data size, tasks, and contextual conditions. We introduce a novel metric to evaluate the impact of datasets, which can also be a guide for creating new datasets. Besides, we analyze the annotation processes, existing labeling tools, and the annotation quality of datasets, showing the importance of establishing a standard annotation pipeline. On the other hand, we thoroughly analyze the impact of geographical and adversarial environmental conditions on the performance of autonomous driving systems. Moreover, we exhibit the data distribution of several vital datasets and discuss their pros and cons accordingly. Finally, we discuss the current challenges and the development trend of the future autonomous driving datasets.

Deformable Linear Objects (DLOs) such as cables, wires, ropes, and elastic tubes are numerously present both in domestic and industrial environments. Unfortunately, robotic systems handling DLOs are rare and have limited capabilities due to the challenging nature of perceiving them. Hence, we propose a novel approach named RT-DLO for real-time instance segmentation of DLOs. First, the DLOs are semantically segmented from the background. Afterward, a novel method to separate the DLO instances is applied. It employs the generation of a graph representation of the scene given the semantic mask where the graph nodes are sampled from the DLOs center-lines whereas the graph edges are selected based on topological reasoning. RT-DLO is experimentally evaluated against both DLO-specific and general-purpose instance segmentation deep learning approaches, achieving overall better performances in terms of accuracy and inference time.

Precise perception is one of the key enablers of autonomous robotic operations. The right selection of sensors significantly influences the overall performance of the system. This paper provides a systematic approach for evaluation of various sensors available on the market. The main focus is to assess the performance in use cases of short to medium distance operations, especially relevant for precise manipulation and/or quality control. The evaluation is based solely on depth data (point clouds). We use six metrics to evaluate the sensors and propose a novel approach for low-cost fabrication of benchmark targets. The evaluation experiments are conducted on different materials to simulate various industrial environments. Our results provide a qualitative and quantitative comparison of different characteristics of various sensors and can be used to select an appropriate device for specific conditions.

Point cloud registration is a core task in 3D perception, which aims to align two point clouds. Moreover, the registration of point clouds with low overlap represents a harder challenge, where previous methods tend to fail. Recent deep learning-based approaches attempt to overcome this issue by learning to find overlapping regions in the whole scene. However, they still lack robustness and accuracy, and thus might not be suitable for real-world applications. Therefore, we present a novel registration pipeline that focuses on object-level alignment to provide a robust and accurate alignment of point clouds. By extracting and completing the missing points of the object of interest, a rough alignment can be achieved even for point clouds with low overlap captured from widely apart viewpoints. We provide a quantitative and qualitative evaluation on synthetic and real-world data captured with a Kinect v2. The proposed approach outperforms the current the current state-of-the-art methods by more than 29% w.r.t. the registration recall on the introduced synthetic dataset. We show that the overall performance and robustness increases due to the object-level alignment, while the baselines perform poorly as they take the entire scene into account.

Autonomous driving has rapidly developed and shown promising performance due to recent advances in hardware and deep learning techniques. High-quality datasets are fundamental for developing reliable autonomous driving algorithms. Previous dataset surveys either focused on a limited number or lacked detailed investigation of dataset characteristics. To this end, we present an exhaustive study of 265 autonomous driving datasets from multiple perspectives, including sensor modalities, data size, tasks, and contextual conditions. We introduce a novel metric to evaluate the impact of datasets, which can also be a guide for creating new datasets. Besides, we analyze the annotation processes, existing labeling tools, and the annotation quality of datasets, showing the importance of establishing a standard annotation pipeline. On the other hand, we thoroughly analyze the impact of geographical and adversarial environmental conditions on the performance of autonomous driving systems. Moreover, we exhibit the data distribution of several vital datasets and discuss their pros and cons accordingly. Finally, we discuss the current challenges and the development trend of the future autonomous driving datasets.