Prefabricated building construction is a crucial approach to achieving “green” building goals, yet it differs significantly from traditional cast-in-place construction methods. Due to complex cross-operations, high mechanization requirements, insufficient worker experience, and inadequate safety measures, prefabricated construction faces greater safety management challenges than traditional methods. This study identifies 17 key indicators influencing unsafe behaviors among prefabricated building construction workers across four dimensions: physical environment, individual factors, team factors, and safety management. A combined weighting method, integrating the entropy weight method and the coefficient of variation method, was employed to determine the weight of each factor. Based on the weight analysis results, a system dynamics intervention model for unsafe behaviors of prefabricated building construction workers was developed and simulated using the Vensim platform. The results indicate that, among single intervention measures, improving individual comprehensive skills has the most significant effect, reducing unsafe behaviors by 37.78%, though this still falls short of the desired target. In contrast, combined interventions yield superior outcomes, with reductions of 68.87% for individual factor interventions, 59.93% for safety management interventions, 46.32% for team factor interventions, and 20.71% for physical environment interventions. These findings highlight the significant advantages of combined interventions. Based on the study results and the characteristics of prefabricated construction, this paper proposes specific intervention strategies, emphasizing the importance of training and institutional development to systematically enhance workers’ safety behaviors.