The rapid development of autonomous driving technology has driven continuous innovation in perception systems, with 4D millimeter-wave (mmWave) radar being one of the key sensing devices. Leveraging its all-weather operational characteristics and robust perception capabilities in challenging environments, 4D mmWave radar plays a crucial role in achieving highly automated driving. This review systematically summarizes the latest advancements and key applications of 4D mmWave radar in the field of autonomous driving. To begin with, we introduce the fundamental principles and technical features of 4D mmWave radar, delving into its comprehensive perception capabilities across distance, speed, angle, and time dimensions. Subsequently, we provide a detailed analysis of the performance advantages of 4D mmWave radar compared to other sensors in complex environments. We then discuss the latest developments in target detection and tracking using 4D mmWave radar, along with existing datasets in this domain. Finally, we explore the current technological challenges and future directions. This review offers researchers and engineers a comprehensive understanding of the cutting-edge technology and future development directions of 4D mmWave radar in the context of autonomous driving perception. 4D mmWave radar, autonomous driving, perception, multi-modal fusion, 3D object detection.

Human drivers instinctively reason with commonsense knowledge to predict hazards in unfamiliar scenarios and to understand the intentions of other road users. However, this essential capability is entirely missing from traditional decision-making systems in autonomous driving. In response, this paper presents DriveLLM, a decision-making framework that integrates large language models (LLMs) with existing autonomous driving stacks. This integration allows for commonsense reasoning in decision-making. DriveLLM also features a unique cyber-physical feedback system, allowing it to learn and improve from its mistakes. In real-world case studies, the proposed framework outperforms traditional decision-making methods in complex scenarios, including difficult edge cases. Furthermore, we propose a novel approach that allows the decision-making system to interact with human inputs while guarding against adversarial attacks. Empirical evaluations demonstrate that this framework responds correctly to complex human instructions. The code and test data is available: https://github.com/DriveLLM/DriveLLM

Long-tail effect, characterized by highly frequent occurrence of normal scenarios and the scarce appearance of extreme “long-tail” scenarios, ubiquitously exists in the vision-related problems in the real-world applications. Though many computer vision methods to date have already achieved feasible performance for most of the normal scenarios, it is still challenging for existing vision systems to accurately perceive the long-tail scenarios. This deficiency largely hinders the practical application of computer vision systems, since long-tail problems may incur fatal consequences, such as traffic accidents, taking the vision systems of autonomous vehicles as an example. In this paper, we firstly propose a theoretical framework named Long-tail Regularization (LoTR), for analyzing and tackling the long-tail problems in the vision perception of autonomous driving. LoTR is able to regularize the scarcely occurred long-tail scenarios to be frequently encountered. Then we present a Parallel Vision Actualization System (PVAS), which consists of closed-loop optimization and virtual-real interaction, to search for challenging long-tail scenarios and produce large-scale long-tail driving scenarios for autonomous vehicles. In addition, we introduce how to perform PVAS in Intelligent Vehicle Future Challenge of China (IVFC), the most durable autonomous driving competition around the world. Results over the past decade demonstrate that PVAS can effectively guide the collection of long-tail data to diminish the cost in the real world, and thus promote the capability of vision systems to adapt to complex environments, alleviating the impact of long-tail effect.

One of the most challenging problems in human-machine co-work is the gap between human intention and the machine's understanding and execution. Large Language Models (LLMs) have been showing superior abilities in solving such issue. In this paper, we design a universal framework that embeds LLMs as a vehicle “Co-Pilot” of driving, which can accomplish specific driving tasks with human intention satisfied based on the information provided. Meanwhile, a utilization workflow is defined to handle the interaction between humans and vehicles, and memory mechanism is introduced to organize the information involved in the tasks. Expert Oriented Black-Box tuning is proposed to improve the performance of the Co-Pilot without finetuning or training the LLMs. In the experiment, the Co-Pilot is applied to two different tasks, i.e., path tracking control and trajectory planning. The Co-Pilot adjusts vehicle operating conditions by selecting a proper controller or planning a certain trajectory to fit human intentions. Simulation tests are conducted to evaluate the performance and generality of the proposed module. The results show that the Co-Pilot can accomplish most of the tasks based on only natural language processing, although it is not flawless. Finally, a discussion about human-machine hybrid intelligence and further applications of LLMs in autonomous driving is made. We believe that such a framework has promising potential in further applications in the field of automous vehicles.

Cooperative computing is promising to enhance the performance and safety of autonomous vehicles benefiting from the increase in the amount, diversity as well as scope of data resources. However, effective and privacy-preserving utilization of multi-modal and multi-source data remains an open challenge during the construction of cooperative mechanisms. Recently, Transformers have demonstrated their potential in the unified representation of multi-modal features, which provides a new perspective for effective representation and fusion of diverse inputs of intelligent vehicles. Federated learning proposes a distributed learning scheme and is hopeful to achieve privacy-secure sharing of data resources among different vehicles. Towards privacy-preserving computing and cooperation in autonomous driving, this paper reviews recent progress of Transformers, federated learning as well as cooperative perception, and proposes a hierarchical structure of Transformers for intelligent vehicles which is comprised of Vehicular Transformers, Federated Vehicular Transformers and the Federation of Vehicular Transformers to exploit their potential in privacy-preserving collaboration.

Pavement defect detection is of profound significance regarding road safety, so it has been a trending research topic. In the past years, deep learning based methods have turned into a key technology, with advantages of high accuracy, strong robustness, and adaptability to complex pavement environments. This paper first reviews the methods based on image processing and 3D imaging. As for image-based processing techniques, they can be classified into three types based on how to label the collected data: fully supervised learning, unsupervised learning, and other methods. Different methods are further classified and compared with each other. Second, the pavement detection methods based on 3D data are sorted out, thereby summarizing their benefits, drawbacks, and application scenarios. Third, the study proposed the major challenges in the field of pavement defect detection, introduced validated datasets and evaluation metrics. Finally, on the basis of reviewing the literature in pavement defect detection, the promising direction is put forward.