Large-scale 3D scene reconstruction and novel view synthesis are vital for autonomous vehicles, especially utilizing temporally sparse LiDAR frames. However, conventional explicit representations remain a significant bottleneck towards representing the reconstructed and synthetic scenes at unlimited resolution. Although the recently developed neural radiance fields (NeRF) have shown compelling results in implicit representations, the problem of large-scale 3D scene reconstruction and novel view synthesis using sparse LiDAR frames remains unexplored. To bridge this gap, we propose a 3D scene reconstruction and novel view synthesis framework called parent-child neural radiance field (PC-NeRF). Based on its two modules, parent NeRF and child NeRF, the framework implements hierarchical spatial partitioning and multi-level scene representation, including scene, segment, and point levels. The multi-level scene representation enhances the efficient utilization of sparse LiDAR point cloud data and enables the rapid acquisition of an approximate volumetric scene representation. With extensive experiments, PC-NeRF is proven to achieve high-precision novel LiDAR view synthesis and 3D reconstruction in large-scale scenes. Moreover, PC-NeRF can effectively handle situations with sparse LiDAR frames and demonstrate high deployment efficiency with limited training epochs. Our approach implementation and the pre-trained models are available at https://github.com/biter0088/pc-nerf .

Motion planning in off-road scenarios is particularly challenging due to diverse terrain features, surface characteristics, and environmental factors. Consequently, rule-based or fixed-parameter motion planning methods often fail to maintain optimal performance, especially in cross-scenario applications. To address these issues, we propose an innovative method for end-to-end motion planning in off-road cross-scenario applications that leverages lifelong learning. We employ a multi-layer map to represent various terrain features and a Transformer network to emulate human motion planning in diverse off-road environments. Additionally, we constructed a structured scene memory library to support our lifelong learning algorithm, enabling effective knowledge retention and transfer across different scenarios. This ensures robust performance even in data-scarce environments. Experimental results demonstrate that our method significantly improves performance in data-scarce off-road scenarios while ensuring robust adaptability and scalability across diverse and new scenarios.

At present, the one-stage detector based on the lightweight model can achieve real-time speed, but the detection performance is challenging. To enhance the discriminability and robustness of the model extraction features and improve the detector’s detection performance for small objects, we propose two modules in this work. First, we propose a receptive field enhancement method, referred to as adaptive receptive field fusion (ARFF). It enhances the model’s feature representation ability by adaptively learning the fusion weights of different receptive field branches in the receptive field module. Then, we propose an enhanced up-sampling (EU) module to reduce the information loss caused by up-sampling on the feature map. Finally, we assemble ARFF and EU modules on top of YOLO v3 to build a real-time, high-precision and lightweight object detection system referred to as the ARFF-EU network. We achieve a state-of-the-art speed and accuracy trade-off on both the Pascal VOC and MS COCO data sets, reporting 83.6% AP at 37.5 FPS and 42.5% AP at 33.7 FPS, respectively. The experimental results show that our proposed ARFF and EU modules improve the detection performance of the ARFF-EU network and achieve the development of advanced, very deep detectors while maintaining real-time speed.

Autonomous driving requires a structured understanding of the surrounding road maps and networks to navigate. However, considering the flexibility of autonomous vehicles and the variations in lane curvature and shape, the online and accurate extraction of road maps with fine-grained boundaries and road networks with lane topology in a unified framework remains challenging. This paper proposes SRSU, an online road map detection and network estimation framework for structured bird's-eye view road scene understanding. Specifically, we introduce a hierarchical map representation, i.e. , representing the road map as a set of ordered point sets with equivalent permutations and the road network as a directed graph, accurately describing the fine-grained map boundaries and lane topology in a unified framework. Building upon the above representation, we propose an online hierarchical map construction framework. It utilizes two sets of learnable hierarchical query embeddings to extract road maps with fine-grained boundaries and road networks with lane topologies, achieving a comprehensive understanding of the road scene. Furthermore, we introduce three empirical modules to enhance the accuracy of hierarchical map construction. These modules are termed auxiliary task prediction, multi-modal distillation, and higher-order interaction, responsible for enhancing the model's representational capabilities and providing valuable auxiliary information for subsequent tasks, generating robust features for final tasks, and learning the association information between different tasks, respectively. Finally, experiments on the nuScenes dataset demonstrate the proposed framework's effectiveness while highlighting the empirical module's superiority. Code will be available at https://github.com/jiapeng789/SRSU .

Autonomous vehicles need to accurately predict the multimodal behaviors of surrounding agents while planning motion policies that ensure safety, comfort, and adaptability in dynamic environments. Existing behavior prediction methods primarily model interactions based on agents’ historical trajectories but often neglect potential interactions in their future trajectories. This limitation compromises the accuracy and consistency of joint predictions. Additionally, the inherent uncertainty of dynamic environments necessitates motion strategies that can adapt to evolving scenarios. To address these challenges, this paper proposes an interactive policy planning framework that integrates adaptive trajectory optimization and joint conditional prediction modules to improve the accuracy and adaptability of motion policies in dynamic scenarios. Specifically, the adaptive trajectory optimization module incorporates a scene attribute-based trajectory refinement strategy, facilitating effective interaction between the ego vehicle’s trajectory and its surrounding environment, thereby generating accurate and adaptable trajectories. The joint conditional prediction module models future interactions among agents as a directed acyclic graph, leveraging its partial ordering structure to decompose the joint prediction task into a series of marginal and conditional predictions, thereby producing more accurate and scene-consistent predictions. Extensive experimental evaluations on the nuPlan dataset and its simulator demonstrate the superior performance of the proposed framework and modules in both trajectory prediction and closed-loop planning tasks.