The current building industry is facing challenges of labor shortages and labor-intensive practices. Effectively collaborating with robots will be crucial for industry upgrading. This research introduces a MR iterative design and robot-assisted construction mode based on human–robot collaboration, facilitating an integrated process innovation from design to construction. The development of the ROCOS (Robot Collaboration System) comprises three key aspects: (1) Layout Stage: using MR technology to layout the site, forming a full-scale integrated virtual and physical digital twin design environment. (2) Design Stage: conducting virtual iterative design in the digital twin environment and automatically simulating assembly processes. (3) Assembly Stage: translating simulated results into assembly path commands and driving a robotic arm to perform actual assembly. In the end, this research setup two experiments to examine the feasibility of this iterative design–construction loop script. The results showed that although the presence of obstacles reduced the designer’s freedom and increased the number of steps, the designer could still finish both tasks. This means that the ROCOS has value in the prototype of human–robot collaboration. In addition, some valuable findings from users’ feedback showed that potential improvements can be addressed in operability, customization, and real construction scenarios.

Studies on room monitoring have only focused on objects in a singular and uniform posture or low-density groups. Considering the wide use of convolutional neural networks for object detection, especially person detection, we use deep learning and perspective correction techniques to propose a room monitoring system that can detect persons with different motion states, high-density groups, and small-sized persons owing to the distance from the camera. This system uses consecutive frames from the monitoring camera as input images. Two approaches are used: perspective correction and person detection. First, perspective correction is used to transform an input image into a 2D top-view image. This allows users to observe the system more easily with different views (2D and 3D views). Second, the proposed person detection scheme combines the Mask region-based convolutional neural network (R-CNN) scheme and the tile technique for person detection, especially for detecting small-sized persons. All results are stored in a cloud database. Moreover, new person coordinates in 2D images are generated from the final bounding boxes and heat maps are created according to the 2D images; these enable users to examine the system quickly in different views. Additionally, a system prototype is developed to demonstrate the feasibility of the proposed system. Experimental results prove that our proposed system outperforms existing schemes in terms of accuracy, mean absolute error (MAE), and root mean squared error (RMSE).