Reconstructing large-scale RGB point clouds is an important task in robotics, supporting perception, navigation, and scene understanding. Despite advances in LiDAR inertial visual odometry (LIVO), its performance remains highly sensitive to extrinsic calibration. Meanwhile, 3D vision foundation models, such as VGGT, suffer from limited scalability in large environments and inherently lack metric scale. To overcome these limitations, we propose LiDAR-VGGT, a novel framework that couples LiDAR inertial odometry with the state-of-the-art VGGT model through a two-stage coarse-to-fine fusion pipeline: First, a pre-fusion module robustly estimates per-session VGGT poses and RGB point clouds with a coarse metric scale. Then, a post-fusion module enhances cross-modal 3D similarity transformation, using bounding-box–based regularization to reduce scale distortions caused by inconsistent FOVs between LiDAR and camera sensors. Extensive experiments across multiple datasets demonstrate that LiDAR-VGGT achieves improved visual quality and density compared to VGGT-based methods and LIVO baselines. The implementation of our method and the proposed novel RGB point cloud evaluation toolkit are released as open source.
1 January 2026 in IEEE Transactions on Automation Science and Engineering