Human–computer interaction (HCI) with three-dimensional (3D) Building Information Modelling/Model (BIM) is the crucial ingredient to enhancing the user experience and fostering the value of BIM. Current BIMs mostly use keyboard, mouse, or touchscreen as media for HCI. Using these hardware devices for HCI with BIM may lead to space constraints and a lack of visual intuitiveness. Somatosensory interaction represents an emergent modality of interaction, e.g., gesture interaction, which requires no equipment or direct touch, presents a potential approach to solving these problems. This paper proposes a computer-vision-based gesture interaction system for BIM. Firstly, a set of gestures for BIM model manipulation was designed, grounded in human ergonomics. These gestures include selection, translation, scaling, rotation, and restoration of the 3D model. Secondly, a gesture understanding algorithm dedicated to 3D model manipulation is introduced in this paper. Then, an interaction system for 3D models based on machine vision and gesture recognition was developed. A series of systematic experiments are conducted to confirm the effectiveness of the proposed system. In various environments, including pure white backgrounds, offices, and conference rooms, even when wearing gloves, the system has an accuracy rate of over 97% and a frame rate maintained between 26 and 30 frames. The final experimental results show that the method has good performance, confirming its feasibility, accuracy, and fluidity. Somatosensory interaction with 3D models enhances the interaction experience and operation efficiency between the user and the model, further expanding the application scene of BIM.

Building information modeling (BIM)-based interior design automation remains constrained by a semantic mismatch: engineering constraints are explicit and categorical, whereas aesthetic style is implicit, contextual, and difficult to formalize. As a result, existing systems often overfit local visual similarity or rely on rigid rules, producing recommendations that drift stylistically at the scene level or conflict with professional design logic. This paper proposes KsDesign, a neuro-symbolic framework for interpretable, retrieval-based BIM scene completion that unifies visual style perception with explicit design knowledge. Offline, KsDesign mines category-level co-occurrence and compatibility patterns from curated designer-quality interiors and encodes them as a weighted Furniture-Matching Knowledge Graph (FMKG). Online, it learns style representations exclusively from BIM-derived 2D renderings/projections of 3D family models and BIM scenes, and applies a knowledge-guided attention mechanism to weight contextual furniture cues, synthesizing a global scene-style representation for candidate ranking and retrieval. In a Top-3 (K = 3) evaluation on 10 BIM test scenes with a 20-expert consensus ground truth, KsDesign consistently outperforms single-modal baselines, achieving 86.7% precision in complex scenes and improving average precision by 23.5% (up to 40%), with a 15.5% average recall increase. These results suggest that global semantic constraints can serve as a logical regularizer, mitigating the local biases of purely visual matching and yielding configurations that are both aesthetically coherent and logically valid. We further implement in-authoring explainability within Revit, exposing KG-derived influence weights and evidence paths to support rationale inspection and immediate family insertion. Finally, the knowledge priors and traceable intermediate representations provide a robust substrate for integration with LLM-driven conversational design agents, enabling constraint-aware, verifiable generation and interactive iteration.