Workers on construction sites face numerous health and safety risks. Authorities have made numerous attempts to enhance safety management; yet incidents continue to occur, impacting both worker health and the project’s forward momentum. To that end, developing strategies to improve construction site safety management is crucial. The goal of this project is to employ computer vision and deep learning methods to create a model that can recognize construction workers, their PPE and the surrounding heavy equipment from CCTV footage. Then, the hazards can be discovered and identified based on an analysis of the imagery data and other criteria including weather conditions, and the on-site safety officer can be contacted. Our own dataset was used to train the You Only Look Once model, version 5 (YOLO-v5), which was put to use as an object detection model. The detection model’s performance in tests showed promise for fast and accurate object recognition in the field.