As an important component of intelligent transportation systems, collaborative perception has made great progress in recent years. In collaborative perception, multiple devices rely on communication to interact and transfer perception information and use complementary information to overcome occlusion and long-distance perception problems. Although there have been some overviews and analyses of the development status and architecture design in this field, there has not been a comprehensive review of the specific task division and goals in perception and communication. In this paper, to facilitate the quick location of specific directions or tasks in vehicle-road collaboration, we first analyze the timing and role of each task in the workflow of cooperative perception. Then, based on perception goals, we divide perception tasks into detection, segmentation, and tracking and discuss communication tasks in terms of open issues such as delay robustness, communication bandwidth, positioning errors, countermeasures against attacks, and privacy preservation computation and comprehensively review the research in each task. We summarize research in downstream tasks of collaborative perception, which are important for safe decision-making and efficient operation of intelligent transportation. We summarize the domain-relevant datasets and discuss future research trends and challenges.

Noise has always been nonnegligible trouble in object detection by creating confusion in model reasoning, thereby reducing the informativeness of the data. It can lead to inaccurate recognition due to the shift in the observed pattern, that requires a robust generalization of the models. To implement a general vision model, we need to develop deep learning models that can adaptively select valid information from multimodal data. This is mainly based on two reasons. Multimodal learning can break through the inherent defects of single-modal data, and adaptive information selection can reduce chaos in multimodal data. To tackle this problem, we propose a universal uncertainty-aware multimodal fusion model. It adopts a multipipeline loosely coupled architecture to combine the features and results from point clouds and images. To quantify the correlation in multimodal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation. In this way, our model reduces the randomness in fusion and generates reliable output. Moreover, we conducted a completed investigation on the KITTI 2-D object detection dataset and its derived dirty data. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation. The experiment results demonstrate the benefits of our adaptive fusion. Our analysis on the robustness of multimodal fusion will provide further insights for future research.

In intelligent driving systems, the multisensor fusion perception system comprising multiple cameras and LiDAR has become a crucial component. It is essential to have stable extrinsic parameters among devices in a multisensor fusion system to achieve all-weather sensing with no blind zones. However, prolonged vehicle usage can result in immeasurable sensor offsets that lead to perception deviations. To this end, we have studied multisensor unified calibration, rather than the calibration between a single pair of sensors as previously done. Benefiting from the mutually constrained pose between different sensor pairs, the method improves calibration accuracy by around 20% compared to calibration for a pair of sensors. The study can serve as a foundation for multisensor unified calibration, enabling the overall automatic optimization of all camera and LiDAR sensors onboard a vehicle within a single framework.

Cameras and LiDARs are fundamental sensors for autonomous driving, with wide applicability. This article comprehensively studies and analyzes the calibration of the multiterminal camera-LiDAR devices from the perspectives of vehicle, roadside, and road cooperation, and outlines its related applications and far-reaching significance. Different from previous studies that focused on sensor calibration of a single platform, ignoring the differences between vehicle-end and roadside, this article particularly emphasizes the difference between vehicle-end and roadside, as well as heterogeneous calibration challenges emerging in vehicle-road cooperation scenarios. These challenges include cross-sensor temporal synchronization and spatial alignment in dynamic environments. It reveals the advantages and differences of different methods in dealing with constraints, such as the fixed installation postures of vehicle-mounted and roadside devices and the large-scale observation domain. This article further points out key research gaps, such as the adaptability of online calibration and the maintenance of calibration consistency among multiagents, providing important references for the technological evolution from isolated calibration to a sensor-collaborative calibration framework in traffic scenarios.