Previous literature has recognized that workers’ unsafe behavior is the combined result of both isolated individual cognitive processes and their interaction with others. Based on the consideration of both individual cognitive factors and social organizational factors, this paper aims to develop an Agent-Based Modeling (ABM) approach to explore construction workers’ sociocognitive processes under the interaction with managers, coworkers, and foremen. The developed model is applied to explore the causes of cognitive failure of construction workers and the influence of social groups and social organizational factors on the workers’ unsafe behavior. The results indicate that (1) workers’ unsafe behaviors are gradually reduced with the interaction with managers, foremen, and workers; (2) the foreman is most influential in reducing workers’ unsafe behaviors, and their demonstration role can hardly be ignored; (3) the failure of sociocognitive process of construction workers is affected by many factors, and cognitive process errors could be corrected under social norms; and (4) among various social organizational factors, social identity has the most obvious effect on reducing workers’ unsafe behaviors, and preventive measures are more effective than reactive measures in reducing workers’ unsafe behaviors.