The demand for multimodal sensing systems for robotics is growing due to increase in robustness, reliability and accuracy offered by these systems. These systems need to be spatially and temporally co-registered to be effective. In this paper, we propose a targetless and structureless spa- tiotemporal LiDAR-camera calibration method. Our method combines a closed-form solution with a modified structureless bundle adjustment where the coarse-to-fine approach does not require an initial guess on the spatiotemporal parameters. Also, as 3D features (structure) are calculated from triangulation only, there is no need to have a calibration target or to match 2D features with the 3D point cloud which provides flexibility in the calibration process and sensor configuration. We demonstrate the accuracy and robustness of the proposed method through both simulation and real data experiments using multiple sensor payloads mounted to hand-held, aerial and legged robot systems. Also, qualitative results are given in form of a colorized point cloud visualization.

Map-centric SLAM utilizes elasticity as a means of loop closure. This approach reduces the cost of loop closure while still providing large-scale fusion-based dense maps, when compared to trajectory-centric SLAM approaches. In this article, we present a novel framework, named ElasticLiDAR++ , for multimodal map-centric SLAM. Having the advantages of a map-centric approach, our method exhibits new features to overcome the shortcomings of existing systems associated with multimodal (LiDAR-inertial-visual) sensor fusion and LiDAR motion distortion. This is accomplished through the use of a local continuous-time trajectory representation. Also, our surface resolution preserving matching algorithm and normal-inverse-Wishart-based surfel fusion model enables nonredundant yet dense mapping. Furthermore, we present a robust metric loop closure model to make the approach stable regardless of where the loop closure occurs. Finally, we demonstrate our approach through both simulation and real data experiments using multiple sensor payload configurations and environments to illustrate its utility and robustness.

A rigid body motion, which can be decomposed into rotation and translation, is essential for engineers and scientists who deal with moving systems in a space. While translation is as simple as vector addition, rotation is hard to understand because rotations are non-Euclidean, and there are many ways to represent them. Additionally, each representation comes with complex operations, and the conversions between different representations are not unique. Therefore, in this tutorial we review rotation representations which are widely used in industry and academia such as rotation matrices, Euler angles, rotation axis-angles, unit complex numbers, and unit quaternions. In particular, for better understanding we begin with rotations in a two dimensional space and extend them to a three dimensional space. In that context, we learn how to represent rotations in a two dimensional space with rotation angles and unit complex numbers, and extend them respectively to Euler angles and unit quaternions for rotations in a three dimensional space. The definitions and properties of mathematical entities used for representing rotations as well as the conversions between various rotation representations are summarized in tables for the reader’s later convenience.

We present a novel method for mapping general three-dimensional environments, where sufficient geometric or visual information is not everywhere guaranteed and where the device motion is unconstrained as with handheld systems. The continuous-time simultaneous localization and mapping algorithm integrates a lidar, camera, and inertial measurement unit in a complementary fashion whereby all sensors contribute constraints to the optimization. The proposed algorithm is designed to expand the domain of mappable environments and therefore increase the reliability and utility of general purpose mobile mapping. A key component of the proposed algorithm is the incorporation of depth uncertainty into visual features, which is effective for noisy surfaces and allows features with and without depth estimates to be modeled in a unified manner. Results demonstrate a wider mappable domain on challenging environments compared to the state-of-the-art lidar or vision-based localization and mapping algorithms.

This paper proposes a novel method for extrinsic calibration between LiDAR and camera sensors given a single observation of one calibration target board. Particularly, in order to improve the accuracy and robustness we focus on the characteristics of the LiDAR sensor data which are often ignored in the previous work. First, LiDAR points are the intersections between rays and the object surface. Hence, noise in the measurement of the time of flight produces errors in the distance along the ray. Therefore, we need to project LiDAR points to the target plane along the ray directions, not along the surface normal direction. Second, LiDAR scans are not continuous but discrete for each bearing angle. That is why sometimes it is called a range image. Thus, edge points extracted based on the range discontinuity along each channel are in fact not on the real edges but within the surface boundaries. Experimental results with simulated and real data show that we overcome the limitations of the previous work and obtain a better accuracy given a single target observation and higher robustness with noisy LiDAR data, thanks to incorporating the LiDAR characteristics in the LiDAR-camera calibration.