Geometric building models are essential in BIM technology. The reconstruction results using current methods are usually represented using mesh, which is limited to visualization purposes and hard to directly import into BIM or modeling software for further application. In this paper, we propose a building model reconstruction method based on a transformer network (DeepBuilding). Instead of reconstructing the polyhedron model of buildings, we strive to recover the CAD modeling operation of constructing the building models from the building point cloud. By representing the building model with its modeling sequence, the reconstruction results can be imported into BIM software for further application. We first translate the procedure of constructing a building model into a command sequence that can be vectorized and processed by the transformer network. Then, we propose a transformer-based network that can convert input point clouds into the vectorized representation of the modeling sequences by decoding the geometry information encoded in the point features. A tool is developed to convert the vectorized modeling sequence into a 3D shape representation (such as mesh) or file format that other BIM software supports. Comprehensive experiments are conducted, and the evaluation results demonstrate that our method can produce competitive reconstruction results with high geometric fidelity while preserving more details of the building reconstruction.

Multi-view stereo plays an important role in 3D reconstruction but suffers from low reconstruction efficiency and has difficulties reconstructing areas with low or repeated textures. To address this, we propose MVP-Stereo, a novel multi-view parallel patchmatch stereo method. MVP-Stereo employs two key techniques. First, MVP-Stereo utilizes multi-view dilated ZNCC to handle low texture and repeated texture by dynamically adjusting the matching window size based on image variance and using a portion of pixels to calculate matching costs without increasing computational complexity. Second, MVP-Stereo leverages multi-scale parallel patchmatch to reconstruct the depth map for each image in a highly efficient manner, which is implemented by CUDA with random initialization, multi-scale parallel spatial propagation, random refinement, and the coarse-to-fine strategy. Experiments on the Strecha dataset, the ETH3D benchmark, and the UAV dataset demonstrate that MVP-Stereo can achieve competitive reconstruction quality compared to state-of-the-art methods with the highest reconstruction efficiency. For example, MVP-Stereo outperforms COLMAP in reconstruction quality by around $30\%$ of reconstruction time, and achieves around $90\%$ of the quality of ACMMP and SD-MVS in only around $20\%$ of the time. In summary, MVP-Stereo can efficiently reconstruct high-quality point clouds and meet the requirements of several photogrammetric applications, such as emergency relief, infrastructure inspection, and environmental monitoring.

Multimodal fusion unlocks the potential of point cloud semantic segmentation, thereby driving advancements in surface observation and visual perception tasks. Although light detection and ranging (LiDAR) systems capture precise 3D structural geometry and optical images provide rich semantic and textural information, existing fusion methods struggle with limited cross-modal perception and insufficient information complementarity. To address these limitations, we propose a multi-stage LiDAR-image collaborative perception fusion network (MCPFNet) for point cloud semantic segmentation of urban scenes. At the middle fusion stage, the network incorporates an elevation-guided geometric-aware fusion module and a semantic-aware cross-attention fusion module to enable bidirectional feature injection between LiDAR and image modalities. In the late fusion stage, a bidirectional adaptive fusion module further refines semantic representations through gated weighting and bidirectional cross-attention mechanisms. Extensive experiments on three multimodal datasets with different resolutions, i.e., ISPRS Vaihingen, N3C-California, and UAVScenes, demonstrate that MCPFNet outperforms existing fusion methods, achieving mIoUs of 74.51%, 95.15%, and 62.76%, respectively. Hence, our multi-stage fusion and bidirectional interaction strategy is more reliable and accurate than existing methods in performing segmentation across diverse and complex urban scenes.

The study of convolutional neural networks for 3D point clouds is becoming increasingly popular, and the difficulty lies mainly in the disorder and irregularity of point clouds. At present, it is straightforward to propose a convolution operation and perform experimental validation. Although good results are achieved, the principles behind them are not explained—i.e., why this can solve the disorder and irregularity of point clouds—and it is difficult for the researchers to design a point cloud convolution network suitable for their needs. For this phenomenon, we propose a point convolution network framework based on spatial location correspondence. Following the correspondence principle can guide us in designing convolution networks adapted to our needs. We analyzed the intrinsic mathematical nature of the convolution operation, and we argue that the convolution operation remains the same when the spatial location correspondence between the convolution kernel points and the convolution range elements remains unchanged. Guided by this principle, we formulated a general point convolution framework based on spatial location correspondence, which explains how to handle a disordered point cloud. Moreover, we discuss different kinds of correspondence based on spatial location, including M-to-M, M-to-N, and M-to-1 relationships, etc., which explain how to handle the irregularity of point clouds. Finally, we give the example of a point convolution network whose convolution kernel points are generated based on the sample’s covariance matrix distribution according to our framework. Our convolution operation can be applied to various point cloud processing networks. We demonstrated the effectiveness of our framework for point cloud classification and semantic segmentation tasks, achieving competitive results with state-of-the-art networks.