Bolted connections have become the most widely used connection method in steel structures. Over the long-term service of the bolts, loosening damage and other defects will inevitably occur due to various factors. To ensure the stability of bolted connections, an efficient and precise method for identifying loosened bolts in a given structure is proposed based on computer vision technology. The main idea of this method is to combine deep learning with image processing techniques to recognize and label the loosening angle from bolt connection images. A rectangular steel plate was taken as the test research object, and three grade 4.8 ordinary bolts were selected for study. The analysis was conducted under two conditions: manual loosening and simulated loosening. The results showed that the method proposed in this article could accurately locate the position of the bolts and identify the loosening angle, with an error value of about ±0.1°, which proves the accuracy and feasibility of this method, meeting the needs of structural health monitoring.