Superpoint Transformers use superpoints as the basic processing units, thereby significantly reducing the number of tokens processed by Transformers. However, they have been seldom employed in point cloud roof plane instance segmentation, and existing superpoint Transformers suffer from limited performance due to the use of low-quality superpoints. To address this challenge, we establish a set of criteria that high-quality superpoints for Transformers should satisfy and introduce a corresponding two-stage superpoint generation process. The superpoints generated by our method not only have accurate boundaries, but also exhibit consistent geometric sizes and shapes, which greatly benefit the feature learning of superpoint Transformers. To compensate for the limitations of deep learning features when the training set size is limited, we incorporate multidimensional handcrafted features into the model. Additionally, we design a decoder that combines a Kolmogorov–Arnold Network with a Transformer module to improve instance prediction and mask extraction. Finally, our network’s predictions are refined using traditional algorithm-based post-processing. For evaluation, we annotated a real-world dataset and corrected annotation errors in the existing RoofN3D dataset. Experimental results show that our method achieves state-of-the-art performance on our dataset, as well as both the original and corrected RoofN3D datasets. Our model also shows significant advantages over existing methods when handling data with low point density, large density variations, or low 3D point precision. Moreover, it is not sensitive to plane boundary annotations during training, significantly reducing the annotation burden. We will release our code, trained models, and datasets.

Single-image plane segmentation plays an important role in understanding 3D indoor scenes, including applications such as 3D indoor reconstruction. In recent years, PlaneTR, a transformer-based architecture, has achieved remarkable performance in single-image plane instance segmentation. It has garnered significant attention from researchers and remains one of the most advanced algorithms in this field. However, PlaneTR has the following two major limitations: its ineffective utilization of line segment information within images and the high number of parameters. In this study, we propose an improved version of PlaneTR, named Spatial Prompt Learning PlaneTR (SPL-PlaneTR), to address these issues. Our approach effectively balances model complexity and performance. Specifically, to more effectively leverage structural information provided by line segments, we replace the original line segment’s transformer branch with a lightweight line segment prompt module and line segment prompt adapter. Additionally, we introduce spatial queries to replace conventional position queries, enabling the network to accurately localize planes across diverse indoor scenes. The experimental results demonstrate that our model, with fewer parameters, outperforms PlaneTR on both the original and noise-corrupted ScanNet datasets. Furthermore, SPL-PlaneTR achieves superior zero-shot transfer performance on the Matterport3D, ICL-NUIM RGB-D, and 2D-3D-S datasets compared to PlaneTR. Notably, our lightweight SPL-PlaneTR also surpasses several state-of-the-art algorithms in this domain. Our code and model have been publicly available.

Transformers have been seldom employed in point cloud roof plane instance segmentation, which is the focus of this study, and existing superpoint Transformers suffer from limited performance due to the use of low-quality superpoints. To address this challenge, we establish two criteria that high-quality superpoints for Transformers should satisfy and introduce a corresponding two-stage superpoint generation process. The superpoints generated by our method not only have accurate boundaries, but also exhibit consistent geometric sizes and shapes, both of which greatly benefit the feature learning of superpoint Transformers. To compensate for the limitations of deep learning features when the training set size is limited, we incorporate multidimensional handcrafted features into the model. Additionally, we design a decoder that combines a Kolmogorov-Arnold Network with a Transformer module to improve instance prediction and mask extraction. Finally, our network's predictions are refined using traditional algorithm-based postprocessing. For evaluation, we annotated a real-world dataset and corrected annotation errors in the existing RoofN3D dataset. Experimental results show that our method achieves state-of-the-art performance on our dataset, as well as both the original and reannotated RoofN3D datasets. Moreover, our model is not sensitive to plane boundary annotations during training, significantly reducing the annotation burden. Through comprehensive experiments, we also identified key factors influencing roof plane segmentation performance: in addition to roof types, variations in point cloud density, density uniformity, and 3D point precision have a considerable impact. These findings underscore the importance of incorporating data augmentation strategies that account for point cloud quality to enhance model robustness under diverse and challenging conditions.