The utilization of Internet-of-Things (IoT)-based technologies in the construction industry has recently grabbed the attention of numerous researchers and practitioners. Despite the improvements made to automate this industry using IoT-based technologies, there are several barriers to the further utilization of these leading-edge technologies. A review of the literature revealed that it lacks research focusing on the obstacles to the application of these technologies in Construction Site Safety Management (CSSM). Accordingly, the aim of this research was to identify and analyze the barriers impeding the use of such technologies in the CSSM context. To this end, initially, the extant literature was reviewed extensively and nine experts were interviewed, which led to the identification of 18 barriers. Then, the fuzzy Delphi method (FDM) was used to calculate the importance weights of the identified barriers and prioritize them through the lenses of competent experts in Hong Kong. Following this, the findings were validated using semi-structured interviews. The findings showed that the barriers related to “productivity reduction due to wearable sensors”, “the need for technical training”, and “the need for continuous monitoring” were the most significant, while “limitations on hardware and software and lack of standardization in efforts,” “the need for proper light for smooth functionality”, and “safety hazards” were the least important barriers. The obtained findings not only give new insight to academics, but also provide practical guidelines for the stakeholders at the forefront by enabling them to focus on the key barriers to the implementation of IoT-based technologies in CSSM.