Structural displacement monitoring faces significant challenges under complex environmental conditions due to the loss or degradation of target features, making it difficult for traditional methods to ensure high accuracy and robustness. Therefore, this study proposes a structural displacement identification and quantification method that integrates YOLOv8n with an improved edge-orientation gradient-based template matching algorithm. By combining deep learning techniques with traditional template matching methods, the accuracy and robustness of monitoring are enhanced under adverse conditions such as noise and extremely low illumination. Specifically, in the edge-orientation gradient matching stage, the Canny-Devernay sub-pixel edge detection technique and an improved ellipse-fitting method are employed for sub-pixel edge extraction, and a five-level Gaussian pyramid structure is introduced to accelerate the matching speed. Experimental results show that the proposed method achieves high-precision displacement monitoring under sufficient illumination, and it maintains stable target localization and displacement quantification performance under conditions of noise interference and extremely low illumination. Notably, under salt-and-pepper noise interference, although YOLOv8n maintains a high level of localization confidence, the accuracy of gradient matching deteriorates, resulting in a root-mean-square error (RMSE) of 0.035 mm. This finding reveals the differential impact of various noise types on different stages of the algorithm. The proposed method offers a novel technological approach for precise structural displacement monitoring in complex environments.