The reconstruction of realistic large-scale 3D scene models using aerial images or videos has significant applications in smart cities, surveying and mapping, the military and other fields. In the current state-of-the-art 3D-reconstruction pipeline, the massive scale of the scene and the enormous amount of input data are still considerable obstacles to the rapid reconstruction of large-scale 3D scene models. In this paper, we develop a professional system for large-scale 3D reconstruction. First, in the sparse point-cloud reconstruction stage, the computed matching relationships are used as the initial camera graph and divided into multiple subgraphs by a clustering algorithm. Multiple computational nodes execute the local structure-from-motion (SFM) technique, and local cameras are registered. Global camera alignment is achieved by integrating and optimizing all local camera poses. Second, in the dense point-cloud reconstruction stage, the adjacency information is decoupled from the pixel level by red-and-black checkerboard grid sampling. The optimal depth value is obtained using normalized cross-correlation (NCC). Additionally, during the mesh-reconstruction stage, feature-preserving mesh simplification, Laplace mesh-smoothing and mesh-detail-recovery methods are used to improve the quality of the mesh model. Finally, the above algorithms are integrated into our large-scale 3D-reconstruction system. Experiments show that the system can effectively improve the reconstruction speed of large-scale 3D scenes.

Due to the constraints posed by the complexity and vast amounts of geospatial data, achieving fine-grained drought geospatial interpolation using sparse observation points has always been a challenging problem. Therefore, to solve the problem, this letter proposes a new model called Geographical Attention Conditional Neural Processes (GACNPs) that extends Spatial Multi-Attention Conditional Neural Processes (SMACNPs). GACNPs as a type of probabilistic model, inherit the flexibility of neural networks in parameterizing stochastic processes. We design a geographic information embedding module by measuring the geographic information correlation between known observation points and target points, thereby better capturing the complexities in spatial data. Considering that real-world data distributions often do not conform to Gaussian distributions, we propose a nonlinear mapping module to handle those more complex distributions that cannot be expressed by Gaussian assumptions, such as multimodal distributions. We utilize MODIS satellite to obtain remote sensing indices with 1 km spatial resolution and integrating these with meteorological drought indices derived from ground weather stations for multi-source data fusion, we conducted an experimental analysis of drought disaster situation in Yunnan Province. Our study demonstrates that the use of GACNPs outperforms traditional methods and other neural network approaches in terms of spatial interpolation accuracy. This improvement facilitates more precise meteorological drought monitoring tasks.

Indoor scene analysis is important for some applications, such as augmented reality. Specifically, we confine the indoor scene analysis problem to two aspects: reconstructing the geometry and understanding the observed element. Traditional analysis methods focusing on 2D RGB images are limited because of the lack of stereo measurements, and thus RGBD-based indoor scene analysis has received wide attention. However, geometric analysis and semantic analysis have been treated as two separate problems in most recent works, leaving scene analysis incomplete. In our work, we combine a deep network architecture with a 3D online reconstruction algorithm and propose a complete pipeline to simultaneously analyse the indoor scene from the geometric level and the semantic level. We take a live depth camera as input and consider the scene analysis as two steps. The first step estimates the camera pose and labels scene objects for a single view. The second step fuses the scene objects into an integrated map for a global view. Specifically, we first transfer the input frame to geometric maps and propose a structural constraint iterative closest point (SC-ICP) algorithm for camera tracking. Then, we propose a structural constraint recurrent neural network (SC-RNN) to generate a semantic map for each frame. Finally, the geometric maps and semantic maps from multiple viewpoints can be fused into a complete model according to the camera pose. Our method can improve the accuracy of camera pose estimation and significantly analyse 3D indoor scenes consisting of high-quality geometric details and rich semantic information. It is noteworthy that our method can meet real-time requirements with frame rates of ≈25 Hz.

Real-world understanding serves as a medium that bridges the information world and the physical world, enabling the realization of virtual–real mapping and interaction. However, scene understanding based solely on 2D images faces problems such as a lack of geometric information and limited robustness against occlusion. The depth sensor brings new opportunities, but there are still challenges in fusing depth with geometric and semantic priors. To address these concerns, our method considers the repeatability of video stream data and the sparsity of newly generated data. We introduce a sparsely correlated network architecture (SCN) designed explicitly for online RGBD instance segmentation. Additionally, we leverage the power of object-level RGB-D SLAM systems, thereby transcending the limitations of conventional approaches that solely emphasize geometry or semantics. We establish correlation over time and leverage this correlation to develop rules and generate sparse data. We thoroughly evaluate the system’s performance on the NYU Depth V2 and ScanNet V2 datasets, demonstrating that incorporating frame-to-frame correlation leads to significantly improved accuracy and consistency in instance segmentation compared to existing state-of-the-art alternatives. Moreover, using sparse data reduces data complexity while ensuring the real-time requirement of 18 fps. Furthermore, by utilizing prior knowledge of object layout understanding, we showcase a promising application of augmented reality, showcasing its potential and practicality.