The capabilities of autonomous mobile robotic systems have been steadily improving due to recent advancements in computer science, engineering, and related disciplines such as cognitive science. In controlled environments, robots have achieved relatively high levels of autonomy. In more unstructured environments, however, the development of fully autonomous mobile robots remains challenging due to the complexity of understanding these environments. Many autonomous mobile robots use classical, learning-based or hybrid approaches for navigation. More recent learning-based methods may replace the complete navigation pipeline or selected stages of the classical approach. For effective deployment, autonomous robots must understand their external environments at a sophisticated level according to their intended applications. Therefore, in addition to robot perception, scene analysis and higher-level scene understanding (e.g., traversable/non-traversable, rough or smooth terrain, etc.) are required for autonomous robot navigation in unstructured outdoor environments. This paper provides a comprehensive review and critical analysis of these methods in the context of their applications to the problems of robot perception and scene understanding in unstructured environments and the related problems of localisation, environment mapping and path planning. State-of-the-art sensor fusion methods and multimodal scene understanding approaches are also discussed and evaluated within this context. The paper concludes with an in-depth discussion regarding the current state of the autonomous ground robot navigation challenge in unstructured outdoor environments and the most promising future research directions to overcome these challenges.

The capabilities of autonomous mobile robotic systems have been steadily improving due to recent advancements in computer science, engineering, and related disciplines such as cognitive science. In controlled environments, autonomous robots have been able to achieve relatively high levels of autonomy. In more unstructured environments, however, the realisation of autonomous mobile robots remains challenging due to limitations in the robots’ external environment understanding. Many autonomous mobile robots use classical, learning-based or hybrid approaches for navigation. The classical navigation approach typically includes robot perception, localisation, environmental mapping, path planning and motion control stages. More recent learning-based methods may replace the complete navigation pipeline or selected stages of the classical approach. For effective deployment, autonomous robots need to be able to understand their external environments at a sophisticated level according to their intended applications. Therefore, in addition to robot perception, scene analysis and higher-level scene understanding (e.g., traversable/non-traversable, and rough or smooth terrain) are required for autonomous robot navigation in unstructured outdoor environments. A wide number of alternative approaches have been proposed in recent years to attempt to address these scene understanding requirements. This paper provides a comprehensive review and critical analysis of these methods in the context of their applications to the problems of robot perception and scene understanding in unstructured environments, and the related problems of localisation, environment mapping and path planning. State-of-the-art sensor fusion methods and multimodal scene understanding approaches are also discussed and evaluated within this context. The paper concludes with an in-depth discussion regarding the current state of the autonomous ground robot navigation challenge in unstructured outdoor environments and the most promising future research directions to overcome these challenges.

The success of deep learning methods in a wide range of application areas has inspired many recent developments in the urban and off-road autonomous navigation domain. In particular, techniques for semantic scene understanding, a key aspect of the navigation pipeline, have been researched extensively, resulting in many real-world and synthetic datasets. However, in comparison to urban semantic segmentation datasets, the availability of datasets for off-road environments remains sparse. In this paper, we aim to overcome this challenge by introducing a methodology capable of efficiently generating photorealistic synthetic datasets for off-road environments with support for multiple sensor modalities. The developed approach is evaluated by presenting a synthetic multimodal dataset of four Western Australian outdoor scenarios suitable for ground robot navigation testing and training. This dataset contains monocular and stereo RGB image and LiDAR point cloud data, with respective semantic annotations covering 16 classes, along with IMU sensor data. The simulation of the four environments was carried out using the NVIDIA Isaac Sim robotics simulator platform. Experiments were conducted using state-of-the-art off-road and universal semantic segmentation models trained on the generated dataset to highlight the Sim2Real capabilities of photorealistic synthetic datasets in real-world off-road robot navigation applications and the limitations of existing datasets for generalised use.