Manhattan and Atlanta worlds hold for the structured scenes with only vertical and horizontal dominant directions (DDs). To describe the scenes with additional sloping DDs, a mixture of independent Manhattan worlds seems plausible, but may lead to unaligned and unrelated DDs. By contrast, we propose a novel structural model called Hong Kong world. It is more general than Manhattan and Atlanta worlds since it can represent the environments with slopes, e.g., a city with hilly terrain, a house with sloping roof, and a loft apartment with staircase. Moreover, it is more compact and accurate than a mixture of independent Manhattan worlds by enforcing the orthogonality constraints between not only vertical and horizontal DDs, but also horizontal and sloping DDs. We further leverage the structural regularity of Hong Kong world for the line-based SLAM. Our SLAM method is reliable thanks to three technical novelties. First, we estimate DDs/vanishing points in Hong Kong world in a semi-searching way. We use a new consensus voting strategy for search, instead of traditional branch and bound. This method is the first one that can simultaneously determine the number of DDs, and achieve quasi-global optimality in terms of the number of inliers. Second, we compute the camera pose by exploiting the spatial relations between DDs in Hong Kong world. This method generates concise polynomials, and thus is more accurate and efficient than existing approaches designed for unstructured scenes. Third, we refine the estimated DDs in Hong Kong world by a novel filter-based method. Then we use these refined DDs to optimize the camera poses and 3D lines, leading to higher accuracy and robustness than existing optimization algorithms. In addition, we establish the first dataset of sequential images in Hong Kong world. Experiments showed that our approach outperforms state-of-the-art methods in terms of accuracy and/or efficiency.

We propose a new linear RGB-D SLAM formulation by utilizing planar features of the structured environments. The key idea is to understand given a structured scene and exploit its structural regularities such as the Manhattan world. This understanding allows us to decouple the camera rotation by tracking structural regularities, which makes SLAM problems free from highly nonlinear. Also, it provides a simple yet effective cue to represent planar features, which leads to a linear SLAM formulation. Given the accurate camera rotation, we jointly estimate the camera translation and planar landmarks in the global planar map within a linear Kalman filter. Our linear SLAM method, called L-SLAM, can understand not only the Manhattan world but the more general scenario of the Atlanta world, which consists of a vertical direction and a set of horizontal directions orthogonal to the vertical direction. To this end, we introduce a novel tracking-by-detection scheme that infers the underlying scene structure by Atlanta representation. With efficient Atlanta representation, we formulate a unified linear SLAM framework for structured environments. We evaluate L-SLAM on a synthetic dataset and RGB-D benchmarks, demonstrating comparable performance to other state-of-the-art SLAM methods without using expensive nonlinear optimization. We assess the accuracy of L-SLAM on a practical application of augmented reality.

Commercial visual–inertial odometry (VIO) systems have been gaining attention as cost-effective, off-the-shelf, six-degree-of-freedom (6-DoF) ego-motion-tracking sensors for estimating accurate and consistent camera pose data, in addition to their ability to operate without external localization from motion capture or global positioning systems. It is unclear from existing results, however, which commercial VIO platforms are the most stable, consistent, and accurate in terms of state estimation for indoor and outdoor robotic applications. We assessed four popular proprietary VIO systems (Apple ARKit, Google ARCore, Intel RealSense T265, and Stereolabs ZED 2) through a series of both indoor and outdoor experiments in which we showed their positioning stability, consistency, and accuracy. After evaluating four popular VIO sensors in challenging real-world indoor and outdoor scenarios, Apple ARKit showed the most stable and high accuracy/consistency, and the relative pose error was a drift error of about 0.02 m per second. We present our complete results as a benchmark comparison for the research community.

We present the first annotated benchmark datasets for evaluating free-flyer visual-inertial localization and mapping algorithms in a zero-g spacecraft interior. The Astrobee free-flying robots that operate inside the International Space Station (ISS) collected the datasets. Space intra-vehicular free-flyers face unique localization challenges: their IMU does not provide a gravity vector, their attitude is fully arbitrary, and they operate in a dynamic, cluttered environment. We extensively evaluate state-of-the-art visual navigation algorithms on these challenging Astrobee datasets, showing superior performance of classical geometry-based methods over recent data-driven approaches. The datasets include monocular images and IMU measurements, with multiple sequences performing a variety of maneuvers and covering four ISS modules. The sensor data is spatio-temporally aligned, and extrinsic/intrinsic calibrations, ground-truth 6-DoF camera poses, and detailed 3D CAD models are included to support evaluation. The datasets are available at: https://astrobee-iss-dataset.github.io/ .

In this study, we designed an obstacle avoidance algorithm for a quadrotor unmanned aerial vehicle (UAV) equipped with a wide field-of-view (FOV) stereo camera, utilizing a learning-based depth estimation approach. Depth estimation using monocular cameras is gaining interest as a viable alternative to large and heavy sensors, such as light detection and ranging (LiDAR) sensors. However, deep learning-based depth estimation has low accuracy unless the depth estimation is done in an environment similar to that of the training data. Therefore, we first designed a depth estimation network for a wide-FOV stereo camera using two cameras. Then, we estimated the depth image using a convolutional neural network and improved the accuracy using stereomatching. We used the estimated depth images to develop a simple behavior-arbitration-based control algorithm that steers the quadrotor away from 3-D obstacles. We conducted simulations and experiments using a real drone in an indoor and outdoor environment to validate our proposed algorithm. An analysis of the experimental results showed that the proposed method could be employed for navigation in cluttered environments.