At present, the low degree of informatization in the construction industry chain is one of the prominent problems impeding the application efficiency of prefabrication. To break through the bottleneck of information integration and interaction in the construction process of prefabricated buildings, this paper proposes a conceptual framework of an Intelligent Construction System for Prefabricated Buildings based on the Internet of Things (ICSPB-IoT) according to the needs of government supervision departments, contractors and owners. Then this paper describes the formation of the “information network” from “information flow”, based on which the operating mechanism of the ICSPB-IoT is discussed using the method of system engineering analysis. Finally, the proposed ICSPB-IoT is graded into different implementation levels according to the degree of intelligence of the Internet of Things information, and the implementation path is proposed for construction enterprises with different levels of digitalization. The results show that the ICSPB-IoT provides cross-phase, cross-organizational information interaction for government supervision departments, contractors and owners to improve the efficiency of communication and collaboration between them. This paper also provides decision-making support for the design, layout and implementation of the Internet of Things for prefabrication construction enterprises.