Estimating the traversability of terrain should be reliable and accurate in diverse conditions for autonomous driving in off-road environments. However, learning-based approaches often yield unreliable results when confronted with unfamiliar contexts, and it is challenging to obtain manual annotations frequently for new circumstances. In this letter, we introduce a method for learning traversability from images that utilizes only self-supervision and no manual labels, enabling it to easily learn traversability in new circumstances. To this end, we first generate self-supervised traversability labels from past driving trajectories by labeling regions traversed by the vehicle as highly traversable. Using the self-supervised labels, we then train a neural network that identifies terrains that are safe to traverse from an image using a one-class classification algorithm. Additionally, we supplement the limitations of self-supervised labels by incorporating methods of self-supervised learning of visual representations. To conduct a comprehensive evaluation, we collect data in a variety of driving environments and perceptual conditions and show that our method produces reliable estimations in various environments. In addition, the experimental results validate that our method outperforms other self-supervised traversability estimation methods and achieves comparable performances with supervised learning methods trained on manually labeled data.
6 January 2023 in IEEE Robotics and Automation Letters
For the safe and successful navigation of autonomous vehicles in unstructured environments, the traversability of terrain should vary based on the driving capabilities of the vehicles. Actual driving experience can be utilized in a self-supervised fashion to learn vehicle-specific traversability. However, existing methods for learning self-supervised traversability are not highly scalable for learning the traversability of various vehicles. In this work, we introduce a scalable framework for learning self-supervised traversability, which can learn the traversability directly from vehicle-terrain interaction without any human supervision. We train a neural network that predicts the proprioceptive experience that a vehicle would undergo from 3D point clouds. Using a novel PU learning method, the network simultaneously identifies non-traversable regions where estimations can be overconfident. With driving data of various vehicles gathered from simulation and the real world, we show that our framework is capable of learning the self-supervised traversability of various vehicles. By integrating our framework with a model predictive controller, we demonstrate that estimated traversability results in effective navigation that enables distinct maneuvers based on the driving characteristics of the vehicles. In addition, experimental results validate the ability of our method to identify and avoid non-traversable regions.
11 June 2018 in IEEE Transactions on Circuits and Systems for Video Technology
We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms. Because most image processing algorithms rely heavily on low-level image features that are based mainly on local gradient information, we consider that gradient quantity can determine the proper exposure level, allowing a camera to capture the important image features in a manner robust to illumination conditions. We then extend this concept to a multi-camera system and present a new control algorithm to achieve both brightness consistency between adjacent cameras and a proper exposure level for each camera. We implement our prototype system with off-theshelf machine-vision cameras and demonstrate the effectiveness of the proposed algorithms on practical applications, including pedestrian detection, visual odometry, surround-view imaging, panoramic imaging and stereo matching.
Traversability estimation for mobile robots in off-road environments requires more than conventional semantic segmentation used in constrained environments like on-road conditions. Recently, approaches to learning a traversability estimation from past driving experiences in a self-supervised manner are arising as they can significantly reduce human labeling costs and labeling errors. However, the self-supervised data only provide supervision for the actually traversed regions, resulting in epistemic uncertainty due to the lack of knowledge on non-traversable regions, also referred to as negative data. Negative data can rarely be collected as the system can be severely damaged while logging the data. To mitigate the uncertainty in the estimation, we introduce a deep metric learning-based method to incorporate unlabeled data with a few positive and negative prototypes. Our method jointly learns binary segmentation that reduces uncertainty in addition to the regression of traversability. To firmly evaluate the proposed framework, we introduce a new evaluation metric that comprehensively evaluates the segmentation and regression. Additionally, we construct a driving dataset ‘Dtrail’ in off-road environments with a mobile robot platform, which is composed of numerous complex and diverse representations of off-road environments. We examine our method on Dtrail as well as the publicly available SemanticKITTI dataset.
14 July 2021 in IEEE Robotics and Automation Letters
Point cloud registration is a well-known way to align two different point clouds via a rigid transform estimation in robotics and computer vision applications. In particular, deep learning-based methods have recently attempted to extract highly distinguishable point features for each point. However, it is still challenging to learn highly distinctive point features in different locations but in a similar shape, bringing performance limitations. To address this issue, we propose Geometry Guided Network, namely G$^2$ Net, for point cloud registration with spherical positional encoding method and unsupervised geometry consistency loss. Combined with self-attention, the positional encoding learns the globally unique features by assigning global geometric positional information into irregular 3D points. The uniqueness of each feature is further strengthened with geometric consistency loss across two different point cloud sets. We demonstrate that G$^2$ Net outperforms current state-of-the-arts models in point cloud registration tasks with both full and partial registration experiments for the ModelNet40 and Augmented ICL-NUIM datasets. Various visualizations on learned features are provided for demonstrating the global shape-awareness of our methodology.