We present a workflow for seamless real-time navigation and 3D thermal mapping in combined indoor and outdoor environments in a global reference frame. The automated workflow and partly real-time capabilities are of special interest for inspection tasks and also for other time-critical applications. We use a hand-held integrated positioning system (IPS), which is a real-time capable visual-aided inertial navigation technology, and augment it with an additional passive thermal infrared camera and global referencing capabilities. The global reference is realized through surveyed optical markers (AprilTags). Due to the sensor data’s fusion of the stereo camera and the thermal images, the resulting georeferenced 3D point cloud is enriched with thermal intensity values. A challenging calibration approach is used to geometrically calibrate and pixel-co-register the trifocal camera system. By fusing the terrestrial dataset with additional geographic information from an unmanned aerial vehicle, we gain a complete building hull point cloud and automatically reconstruct a semantic 3D model. A single-family house with surroundings in the village of Morschenich near the city of Jülich (German federal state North Rhine-Westphalia) was used as a test site to demonstrate our workflow. The presented work is a step towards automated building information modeling.

Keywords: Aerial, Oblique, Building Reconstruction, Discontinuities, Roof Overhangs, Texture Mapping, CityGML This paper proposes a two-stage method for the reconstruction of city buildings with discontinuities and roof overhangs from oriented nadir and oblique aerial images. To model the structures the input data is transformed into a dense point cloud, segmented and filtered with a modified marching cubes algorithm to reduce the positional noise. Assuming a monolithic building the remaining vertices are initially projected onto a 2D grid and passed to RANSAC-based regression and topology analysis to geometrically determine finite wall, ground and roof planes. If this should fail due to the presence of discontinuities the regression will be repeated on a 3D level by traversing voxels within the regularly subdivided bounding box of the building point set. For each cube a planar piece of the current surface is approximated and expanded. The resulting segments get mutually intersected yielding both topological and geometrical nodes and edges. These entities will be eliminated if their distance-based affiliation to the defining point sets is violated leaving a consistent building hull including its structural breaks. To add the roof overhangs the computed polygonal meshes are projected onto the digital surface model derived from the point cloud. Their shapes are offset equally along the edge normals with subpixel accuracy by detecting the zero-crossings of the second-order directional derivative in the gradient direction of the height bitmap and translated back into world space to become a component of the building. As soon as the reconstructed objects are finished the aerial images are further used to generate a compact texture atlas for visualization purposes. An optimized atlas bitmap is generated that allows perspectivecorrect multi-source texture mapping without prior rectification involving a partially parallel placement algorithm. Moreover, the texture atlases undergo object-based image analysis (OBIA) to detect window areas which get reintegrated into the building models. To evaluate the performance of the proposed method a proof-of-concept test on sample structures obtained from real-world data of Heligoland/Germany has been conducted. It revealed good reconstruction accuracy in comparison to the cadastral map, a speed-up in texture atlas optimization and visually attractive render results.

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

For professional use cases like police or fire brigade operations, a reliable self-localization is advantageous for the coordination of first responders (FRs). Self-localization is especially difficult in indoor scenarios where neither global navigation satellite systems’ signals may be received nor additional signals of opportunity exist for enabling reliable positioning. In this article, we propose an overall system that combines self-localization, communication of the FRs’ locations, 3-D building reconstruction or floor plans (if available), and visualization. The indoor navigation technique is based solely on inertial sensors and builds on a simultaneous localization and mapping technique. It is capable of using any information about the building layout as prior information for enhancing indoor positioning, georeferencing the positions, and finally, visualizing the results in a suitable visualization tool.