Structured-light 3D reconstruction is an active measurement technique that extracts spatial geometric information of objects by projecting fringe patterns and analyzing their distortions. It has been widely applied in industrial inspection, cultural heritage digitization, virtual reality, and other related fields. This review presents a comprehensive analysis of mainstream fringe-based reconstruction methods, including Fringe Projection Profilometry (FPP) for diffuse surfaces and Phase Measuring Deflectometry (PMD) for specular surfaces. While existing reviews typically focus on individual techniques or specific applications, they often lack a systematic comparison between these two major approaches. In particular, the influence of different projection schemes such as Digital Light Processing (DLP) and MEMS scanning mirror–based laser scanning on system performance has not yet been fully clarified. To fill this gap, the review analyzes and compares FPP and PMD with respect to measurement principles, system implementation, calibration and modeling strategies, error control mechanisms, and integration with deep learning methods. Special focus is placed on the potential of MEMS projection technology in achieving lightweight and high-dynamic-range measurement scenarios, as well as the emerging role of deep learning in enhancing phase retrieval and 3D reconstruction accuracy. This review concludes by identifying key technical challenges and offering insights into future research directions in system modeling, intelligent reconstruction, and comprehensive performance evaluation.
25 December 2023 in IEEE Transactions on Instrumentation and Measurement
Fringe projection profilometry (FPP) is an extensively used active three-dimensional (3D) measurement technique. However, it faces challenges in achieving synchronous improvement of measurement range, speed, accuracy and shadow issues. To meet the demand for rapid 3D reconstruction of two projection views only using a single-shot phase-shifting grating, we initially propose a fast and large-scale 3D reconstruction system based on deep learning with a dual-projector and single-camera configuration, named DL_DPSL. The key feature is that the entire measurement process only requires two projectors to project one image simultaneously. To validate the effect, we constructed a simulated and a real system and corresponding simulation ( ˜3300 sets) and real dataset ( ˜1000 sets) respectively. The experimental results show that based ResUNet the DL_DPSL system can recover the phases of two projection views from one-shot superimposed phase-shifting grating within 0.018 s inclusive projection time when using two NVIDIA GeForce RTX 3090 graphics cards. Additionally, despite reducing the projection time by half, in the simulation dataset, exists the mean average error was reduced by a maximum of 20%. And similar performance improvements in the continuous area of the measured objects in the real dataset. The DL_DPSL system has better accuracy and stability compared to the current fastest deep learning system. DL_DPSL breaks the existing dual-projector structured light 3D measurement system paradigm, providing a new solution to promote the development and industrial application of FPP technology, with broad application prospects. (Simulation and real datasets are publicly available on the following GitHub repository: https://github.com/LiYiMingM/DPSL3D-measurement).
1 December 2023 in IEEE Transactions on Instrumentation and Measurement
A new type of fringe projection profilometry (FPP) system can be constructed by using a MEMS micro-vibration mirror. However, the MEMS scanning speed generally surpasses the capture speed of an ordinary camera, making it challenging to take advantage of high-speed projection. In this paper, we propose a solution called the phase-shifting superimposition projection framework, which projects patterns with different phase shifts in one exposure. Using the phase-shifting superimposition projection framework as the algorithm core, we propose an internal-external layered phase-shifting method to extract phases accurately. The internal phase-shifting method superimposes multiple projection patterns with different phase shifts into a high-quality camera image. Subsequently, the external phase-shifting method uses camera images to extract wrapped phases. Furthermore, we establish a mathematical model to demonstrate the anti-harmonics ability of the proposed method. Finally, experimental results demonstrate that the proposed method can significantly improve the accuracy of phase retrieval without adding extra camera images. Due to the ability to generate anti-harmonics images, the phase-shifting superimposition projection framework also holds significant application potential for the DLP binary defocusing technique, the Fourier transform profilometry, and the construction of high-quality datasets.
19 May 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Deep-learning-based structured light 3D reconstruction technology (SL3D) provides excellent solutions for intelligent manufacturing. However, the scarcity of real-world datasets covering full-process data and diverse objects hampers the validation of new ideas. Moreover, limited research on dataset construction strategies, such as scene backgrounds and sample distribution, reduces network performance. We investigate the impact of background stability on foreground accuracy (BS-FA) and find a whiteboard background improved foreground prediction accuracy by up to 82% over a black background. Guided by BS-FA, we develop the SL3D-BF, a background-effective SL3D dataset for industrial use, featuring approximately 2,100 scenes with diverse objects like metal/plastic workpieces, plaster sculptures, and standard parts for precise evaluation. It uniquely includes shadow and foreground masks, absent in prior datasets, and offers full-process data from gratings to 3D point clouds, totaling 100,800 gratings. We also establish an initial benchmark for future research by conducting evaluation experiments with advanced methods. Furthermore, we investigate the relationship between the spatial frequency of sample occurrence and the model predictive ability to minimize the time and resource demands of dataset construction. Most importantly, SL3D-BF is also a valuable resource for tasks like depth estimation, defect detection, and semantic segmentation. The dataset is available at: https://github.com/LiYiMingM/Dataset_SL3D_BF.
5 June 2025 in IEEE Transactions on Instrumentation and Measurement
Precise and rapid three-dimensional (3D) measurement of metal parts on production lines is essential. Modern artificial intelligence (AI) supports sensing devices to see the world in 3D vision. Digital fringe 3D profilometry reconstruction (DF-3D) based on deep learning has the potential to revolutionize society. Nevertheless, attaining the high precision and robust DF-3D through a single-shot end-to-end (SS-E2E) AI network remains an unresolved challenge. For fast and precise 3D reconstruction, this work proposes an SS-E2E absolute phase prediction network strategy, composed of a multi-path branch auxiliary supervision network and guided by the fringe-phase physical model (MPS_XNet). Specifically, MPS_XNet is based on the three key stages of the phase retrieval model, including numerator, denominator, and wrapped phase supervision, as auxiliary branches to assist the main path of absolute phase for long-distance computation in interpretable features. Besides, we designed a physics-based supervisory loss function to correctly guide the network training along the path defined by the physical model. Experiments have demonstrated that MPS_XNet outperforms the state-of-the-art (SOTA) regression network paradigm on five datasets (metal workpieces and complex contours) and our measurement system, which not only compensates for insufficient input information but also robustly and interpretably guides the network training through converting the long-distance challenge to multiple short-distance sub-tasks. Our strategy achieves a groundbreaking 90% MAE reduction in phase retrieval through physical model design and task decomposition in 20 ms. It remains promising for future SOTA networks, enabling real-time, high-precision 3D measurement in industrial and scientific applications. The source code is publicly available at https://github.com/LiYiMingM/Physical-Model-Supervised-Machine-Learning-3D-reconstruction.
14 October 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Accurate estimation of burden surface depth plays a crucial role in constructing the temperature field and optimizing reaction control in volatile kilns. However, most image-based depth estimation techniques require high-quality input images and achieve limited accuracy, which restrict their applications in actual harsh working conditions such as high temperature, heavy dust and dense smoke. In this study, a deep learning-based monocular depth estimation model is proposed to measure the burden surface depth in the volatile kiln head zone. The proposed model integrates an encoder-decoder network with an attention module. The encoder-decoder network outputs a set of deep semantic features, while the attention module intelligently fuses multi-level features to predict a probability distribution over depth intervals for each pixel. A volatile kiln prototype is designed and constructed to generate image datasets of the kiln head zone which approximate real data collected from industrial production sites. Results demonstrate that the proposed model has a depth prediction error of RMSE = 11.008 mm for the burden surface region, outperforming state-of-the-art neural networks and the traditional depth-from-defocus method. Code and datasets are available at https://github.com/LLLcong/Attention-MonoDepth.