The identification of urban functional zones (UFZs) is crucial for urban planning and optimizing industrial layout. Fusing remote sensing images and social perception data is an effective way to identify UFZs. Previous studies on UFZs recognition often ignored band information outside the red–green–blue (RGB), especially three-dimensional (3D) urban morphology information. In addition, the probabilistic methods ignore the potential semantic information of Point of Interest (POI) data. Therefore, we propose an “Image + Text” multimodal data fusion framework for UFZs recognition. To effectively utilize the information of Gaofen-7(GF-7) stereo images, we designed a semi-transfer UFZs recognition model. The transferred model uses the pre-trained model to extract the deep features from RGB images, and a small self-built convolutional network is designed to extract the features from RGB bands, near-infrared (NIR) band, and normalized digital surface model (nDSM) generated by GF-7. Latent Dirichlet allocation (LDA) is employed to extract POI semantic features. The fusion features of the deep features of the GF-7 image and the semantic features of POI are fed into a classifier to identify UFZs. The experimental results show that: (1) The highest overall accuracy of 88.17% and the highest kappa coefficient of 83.91% are obtained in the Beijing Fourth Ring District. (2) nDSM and NIR data improve the overall accuracy of UFZs identification. (3) POI data significantly enhance the recognition accuracy of UFZs, except for shantytowns. This UFZs identification is simple and easy to implement, which can provide a reference for related research. However, considering the availability of POI data distribution, other data with socioeconomic attributes should be considered, and other multimodal fusion strategies are worth exploring in the future.

Automatically extracting urban buildings from remote sensing images has essential application value, such as urban planning and management. Gaofen-7 (GF-7) provides multi-perspective and multispectral satellite images, which can obtain three-dimensional spatial information. Previous studies on building extraction often ignored information outside the red–green–blue (RGB) bands. To utilize the multi-dimensional spatial information of GF-7, we propose a dual-stream multi-scale network (DMU-Net) for urban building extraction. DMU-Net is based on U-Net, and the encoder is designed as the dual-stream CNN structure, which inputs RGB images, near-infrared (NIR), and normalized digital surface model (nDSM) fusion images, respectively. In addition, the improved FPN (IFPN) structure is integrated into the decoder. It enables DMU-Net to fuse different band features and multi-scale features of images effectively. This new method is tested with the study area within the Fourth Ring Road in Beijing, and the conclusions are as follows: (1) Our network achieves an overall accuracy (OA) of 96.16% and an intersection-over-union (IoU) of 84.49% for the GF-7 self-annotated building dataset, outperforms other state-of-the-art (SOTA) models. (2) Three-dimensional information significantly improved the accuracy of building extraction. Compared with RGB and RGB + NIR, the IoU increased by 7.61% and 3.19% after using nDSM data, respectively. (3) DMU-Net is superior to SMU-Net, DU-Net, and IEU-Net. The IoU is improved by 0.74%, 0.55%, and 1.65%, respectively, indicating the superiority of the dual-stream CNN structure and the IFPN structure.