While a popular representation of 3D data, point clouds may contain noise and need filtering before use. Existing point cloud filtering methods either cannot preserve sharp features or result in uneven point distributions in the filtered output. To address this problem, this paper introduces a point cloud filtering method that considers both point distribution and feature preservation during filtering. The key idea is to incorporate a repulsion term with a data term in energy minimization. The repulsion term is responsible for the point distribution, while the data term aims to approximate the noisy surfaces while preserving geometric features. This method is capable of handling models with fine-scale features and sharp features. Extensive experiments show that our method quickly yields good results with relatively uniform point distribution.

Cone-beam computed tomography (CBCT) is one of the most widely used digital models in dental practices. A critical step in virtual dental treatment planning is to accurately delineate all tooth-bone structures from CBCT with high fidelity. Previous studies have established several methods for CBCT segmentation using deep learning. However, the inherent resolution discrepancy and the loss of occlusal information in CBCT largely limited its clinical applicability. Here, we present a Deep Dental Multimodal Analysis (DDMA) framework consisting of a CBCT segmentation model, an intraoral scan (IOS) segmentation model (the most accurate digital dental model), and a fusion model to generate 3D fused tooth-bone structures with high fidelity and accurate occlusal information. Our model was trained with a large-scale dataset with 503 CBCT and 28,559 IOS meshes manually annotated by human experts. For CBCT segmentation, our framework achieves an average Dice coefficient of 93.99%, significantly outperforming the baselines. For IOS segmentation, our model achieves a mIoU of 93.07% and 95.70% on the maxillary and mandible on a test dataset of 200 IOS meshes, 1.77% and 3.52% higher than the state-of-art method. In addition, our DDMA framework takes 20 to 25 minutes to generate the fused model, compared to over 5 hours by human experts. Notably, real-world clinical cases demonstrate that our model can visualize tooth-bone structures and relationships during the entire orthodontic treatment and predict clinical risks like dehiscence and fenestration. These findings demonstrate the potential of deep multimodal learning to improve the quality of digital dental models and guide clinical treatment and diagnostic decisions.

Phase shifting profilometry is an important technique for reconstructing the three-dimensional (3D) geometry of objects with purely diffuse surfaces. However, it is challenging to measure the transparent objects due to the pattern aliasing caused by light refraction and multiple reflections inside the object. In this work, we analyze the aliasing fringe pattern formation for transparent objects and then, propose to learn the front surface light intensity distribution based on the formation principle by using the diffusion models for generating the non-aliased fringe patterns reflected from the front surface only. With the generated fringe patterns, the 3D shape of the transparent objects can be reconstructed via the conventional structured light. We show the feasibility and performance of the proposed method on the data of purely transparent objects that are not seen in the training stage. Moreover, we found it could be generalized to other cases with local-transparent and translucent objects, showing the potential capability of the diffusion based learnable framework in tackling the problems of transparent object reconstruction.

Reconstruction of moving target surfaces based on active image sensing techniques, such as phase-shifting profilometry, has attracted intensive research in recent years. The measurement error caused by object motion can be addressed successfully by tracking the object movement. However, it either requires high-cost color imaging equipment or is limited by the assumption of 2D translation movement. Therefore, this paper proposes what we believe to be a new method to reconstruct the kinematic object surfaces with any 2D movement sensed by affordable monochrome camera. An improved RAFT optical flow algorithm is proposed to track the object based on the object fringe pattern image directly. The feature points on the object are retrieved immune to the fringe pattern illumination. Then, the RANSAC algorithm and an iteration selection process are employed to select feature points with high quality optical flow. At last, the motion is described mathematically, and the dynamic object is reconstructed successfully. Experiments are presented to verify the effectiveness of the proposed method.

Dynamic three-dimensional (3D) shape measurement techniques based on phase-shifting profilometry (PSP) have achieved considerable progress in previous studies. However, for dynamic scenes with arbitrary directional motion, accurately measuring the 3D shape and/or motion of objects remains an unsolved challenge. When an object moves in any direction within 3D space, both its spatial positions and the height values of individual surface points change. Moreover, the height variations at each pixel are typically inconsistent, resulting in different unknown phase-shift errors across various locations. To address the challenge, this paper proposes a PSP dynamic 3D shape measurement method that combines transformer-based motion tracking and motion compensation. We propose to use the COTR network to achieve motion tracking of objects under coded fringe illumination. By integrating this tracking information into the motion error estimation process of PSP, the proposed method enables pixel-wise error compensation, allowing for high-precision recovery of phase information for objects with 3D motion. Experimental results validate the effectiveness of the proposed method, demonstrating a significant reduction in motion-induced errors and improved recovery of true surface geometry.