Advancements in 4D mmWave radar with multi-sensor fusion have significantly enhanced the robustness of autonomous driving systems. In the context of “Mining 5.0” based on parallel intelligence theory, autonomous haulage need to achieve full autonomy in open-pit mines. Current systems use 3D mmWave radar, LiDAR, and cameras but have limited automation progress. This perspective discusses the limitations of these systems and how integrating 4D mmWave radar can improve mining autonomy. This perspective results from discussions at several recent Distributed/Decentralized Hybrid Workshops on Autonomous Mining (DHW-AM) and aims at enhancing the intelligence of future mining operations.
19 September 2024 in IEEE Transactions on Intelligent Vehicles
‘‘Mining 5.0,” in response to “Industry 5.0,” requires autonomous haulage systems to operate fully autonomously in open-pit mines. Current autonomous mining transportation systems rely on wireless transmission for edge dumping operations, which is inefficient and poses risks of potential communication loss and cybersecurity issues. Sensors such as lidar, cameras, and 3D mmWave radar do not support autonomous haulage to complete automation of edge dumping, as they struggle to detect obscured static obstacles and operate in harsh mining environments. We propose an innovative approach to address this challenge: integrating 4D mmWave radar into autonomous haulage systems. We have collected the world's first 4D mmWave radar dataset in the open pit mine to evaluate this approach, including four haulage operating scenarios under various lighting conditions. To quantify the precision of the 4D mmWave radar, we induce three points cloud comparison methods, a static object tracking algorithm, and point cloud image comparison to assess the system's ability to detect obscured static obstacles. Based on our findings, we conclude that using 4D mmWave radar enhances the autonomous haulage system's ability to detect obscured static obstacles in open pit mines, particularly at dumping sites.
In open-pit mining environments, large equipment frequently causes severe occlusion, creating critical perception blind spots for LiDAR. Meanwhile, 4D millimeter-wave (mmWave) radar provides penetration capability but is highly susceptible to multipath interference generated by metallic structures and uneven terrain. To address these challenges, this paper proposes a Blind-Spot Complementary Fusion (BSCF) framework that integrates 3D LiDAR and 4D mmWave radar through explicit geometric constraints. The proposed framework first suppresses multipath artifacts and performs calibrated spatiotemporal alignment between heterogeneous sensors. It then introduces high-confidence radar observations into LiDAR blind spots through spatial consistency verification, providing existence-level hidden-target risk cues under extreme occlusion. In addition, a Volume Recovery Rate (VRR) proxy metric is proposed to quantitatively describe envelope-level spatial evidence in occluded regions. Experiments conducted on real-world mining datasets demonstrate that the proposed method effectively suppresses severe multipath interference and improves scene-level cross-modal proximity by approximately 16.5%, while the overlap-region RMSE remains approximately 0.18 m. Under complete occlusion where LiDAR observations are unavailable, the framework provides target-existence risk cues with a VRR of up to 15.6%, providing a useful preliminary indication of hidden target existence. Ultimately, the proposed approach enhances perception robustness and safety for autonomous transportation systems in open-pit mines.