Traditional Unmanned Aerial Vehicle (UAV) oblique photogrammetry for 3D real-scene modeling of historic cultural districts suffers from data gaps, insufficient texture, and poor accuracy in complex alleyway environments, hindering the widespread adoption of UAV technology. To address these challenges, this paper establishes a distortion region identification algorithm based on image grayscale variation range parameters. Then, through fusing UAV oblique photogrammetry, close-range smartphone photogrammetry, and Real-Time Kinematic (RTK) positioning technology, it ultimately constructs a 3D real-scene reconstruction technical framework. To validate the method’s effectiveness and reliability, a field experiment was conducted in the Zaoerxiang Historic Cultural District of Zhanggong District, Ganzhou City, Jiangxi Province, China. The experimental results demonstrate that the proposed algorithm can effectively identify distortions in the modeling results from UAV images. After fusing smartphone images from distorted regions and RTK measurements from ground control points (GCPs), the discrepancies in X, Y, and Z coordinates between the results and verification points mostly fall within 10 to 25 mm, while the differences from the measured lengths using a steel tape measure and a leveling rod were within the range of 10 to 20 mm. Furthermore, compared to approaches that rely solely on UAV images or on the fusion of UAV and all ground-based images for modeling, the method proposed in this paper restores building texture information in occluded areas and improves the accuracy of 3D real-scene modeling while simultaneously reducing data-processing and storage requirements and enhancing operational efficiency. It provides a referenceable technical framework for digital preservation, restoration planning, and smart cultural tourism of historic districts.