Close-range photogrammetry as a technique to acquire reality-based 3D data has, in recent times, seen a renewed interest due to developments in sensor technologies. Furthermore, the strong democratization of UAVs (Unmanned Aerial Vehicles) or drones means that close-range photogrammetry can now be used as a viable low-cost method for 3D mapping. In terms of software development, this led to the creation of many commercial black-box solutions (PhotoScan, Pix4D, etc.). This paper aims to demonstrate how the open source toolbox DBAT (Damped Bundle Adjustment Toolbox) can be used to generate detailed photogrammetric network diagnostics to help assess the quality of surveys processed by the commercial software, PhotoScan. In addition, the Apero module from the MicMac software suite was also used to provide an independent assessment of the results. The assessment is performed by the careful examination of some of the bundle adjustment metrics generated by both open source solutions. A UAV project was conducted on a historical church in the city center of Strasbourg, France, in order to provide a dataset with a millimetric level of precision. Results showed that DBAT can be used to reprocess PhotoScan projects under similar conditions and derive useful metrics from them, while Apero provides a completely independent verification of the results of commercial solutions. Overall, this paper shows that an objective assessment of photogrammetric results is important. In cases where problems are encountered in the project, this assessment method can be useful to detect errors that may not be explicitly presented by PhotoScan.