Due to the inherent inter-class similarity and class imbalance of remote sensing images, it is difficult to obtain effective results in single-source semantic segmentation. We consider applying multi-modal data to the task of the semantic segmentation of HSR (high spatial resolution) remote sensing images, and obtain richer semantic information by data fusion to improve the accuracy and efficiency of segmentation. However, it is still a great challenge to discover how to achieve efficient and useful information complementarity based on multi-modal remote sensing image semantic segmentation, so we have to seriously examine the numerous models. Transformer has made remarkable progress in decreasing model complexity and improving scalability and training efficiency in computer vision tasks. Therefore, we introduce Transformer into multi-modal semantic segmentation. In order to cope with the issue that the Transformer model requires a large amount of computing resources, we propose a model, MFTransNet, which combines a CNN (convolutional neural network) and Transformer to realize a lightweight multi-modal semantic segmentation structure. To do this, a small convolutional network is first used for performing preliminary feature extraction. Subsequently, these features are sent to the multi-head feature fusion module to achieve adaptive feature fusion. Finally, the features of different scales are integrated together through a multi-scale decoder. The experimental results demonstrate that MFTransNet achieves the best balance among segmentation accuracy, memory-usage efficiency and inference speed.

With the goal of addressing the challenges of small, densely packed targets in remote sensing images, we propose a high-resolution instance segmentation model named QuadTransPointRend Net (QTPR-Net). This model significantly enhances instance segmentation performance in remote sensing images. The model consists of two main modules: preliminary edge feature extraction (PEFE) and edge point feature refinement (EPFR). We also created a specific approach and strategy named TransQTA for edge uncertainty point selection and feature processing in high-resolution remote sensing images. Multi-scale feature fusion and transformer technologies are used in QTPR-Net to refine rough masks and fine-grained features for selected edge uncertainty points while balancing model size and accuracy. Based on experiments performed on three public datasets: NWPU VHR-10, SSDD, and iSAID, we demonstrate the superiority of QTPR-Net over existing approaches.

A significant amount of research has been conducted on the segmentation of large-scale 3D point clouds. However, efficient point cloud feature identification from segmentation results is an essential capability for computer vision and surveying tasks. Feature description methods are algorithms that convert the point set of the point cloud feature into vectors or matrices that can be used for identification. While the point feature histogram (PFH) is an efficient descriptor method, it does not work well with objects that have smooth surfaces, such as planar, spherical, or cylindrical objects. This paper proposes a 3D point cloud feature identification method based on an improved PFH descriptor with a feature-level normal that can efficiently distinguish objects with smooth surfaces. Firstly, a feature-level normal is established, and then the relationship between each point’s normal and feature-level normal is calculated. Finally, the unknown feature is identified by comparing the similarity of the type-labeled feature and the unknown feature. The proposed method obtains an overall identification accuracy ranging from 71.9% to 81.9% for the identification of street lamps, trees, and buildings.