To further improve the precision and efficiency of structural health monitoring technology and the theory of large-scale structures, full-field non-contact structural geometry morphology monitoring is expected to be a breakthrough technology in structural safety state monitoring and digital twins, owing to its economic, credible, high frequency, and holographic advantages. This study validates a proposed holographic visual sensor and algorithms in a computer-vision-based full-field non-contact displacement and vibration measurement. Using an automatic camera patrol experimental device, original segmental dynamic and static video monitoring data of a model bridge under various damage/activities were collected. According to the temporal and spatial characteristics of the series data, the holographic geometric morphology tracking algorithm was introduced. Additionally, the feature points set of the structural holography geometry and the holography feature contours were established. Experimental results show that the holographic visual sensor and the proposed algorithms can extract an accurate holographic full-field displacement signal, and factually and sensitively accomplish vibration measurement, while accurately reflecting the real change in structural properties under various damage/action conditions. The proposed method can serve as a foundation for further research on digital twins for large-scale structures, structural condition assessment, and intelligent damage identification.