Urban Functional Zones (UFZs) are spatial units of the city divided according to specific functional activities. Detailed identification of UFZs is vital for optimizing urban management, guiding planning and design, and promoting sustainable development. However, existing UFZ recognition methods face significant challenges, such as difficulties in effectively integrating multi-source heterogeneous data, capturing dynamic spatiotemporal patterns, and addressing the complex interrelationships among various data types. These issues significantly limit the applicability of UFZ mapping in complex urban scenarios. To address these challenges, this paper proposes a tripartite neural network (TriNet) for multimodal data processing, including Remote Sensing (RS) images, Point of Interest (POI) data, and Origin–Destination (OD) data, fully utilizing the complementarity of different data types. TriNet comprises three specialized branches: ImgNet for spatial features extraction from images, POINet for functional density distribution features extraction from POI data, and TrajNet for spatiotemporal pattern features extraction from OD data. Finally, the method deeply fuses these features through a feature fusion module, which utilizes a two-layer fully connected network for deep fusion, allowing the model to fully utilize the interdependencies among the data types, significantly improving the UFZ classification accuracy. The experimental data are generated by mapping OpenStreetMap (OSM) vector into conceptual representations, integrating images with social sensing data to create a comprehensive UFZ classification benchmark. The method achieved an overall accuracy of 84.13% on the test set of Chongqing’s main urban area, demonstrating high accuracy and robustness in UFZ classification tasks. The experimental results show that the TriNet model performs effectively in UFZ classification.