Shijie Feng

Dr. Shijie Feng

Smart Computational Imaging (SCI) Laboratory, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 October 2018 in Optics and Lasers in Engineering
Verified
Journal Article
Closed access
CZChao Zuo
Chao Zuo
Shijie FengShijie Feng
Shijie Feng
LHLei Huang
Lei Huang
TTTianyang Tao
Tianyang Tao
WYWei Yin
Wei Yin
QCQian Chen
Qian Chen

28 February 2019 in Advanced Photonics
Verified
Journal Article
Open access
PDF file
Shijie FengShijie Feng
Shijie Feng
QCQian Chen
Qian Chen
GGGuohua Gu
Guohua Gu
TTTianyang Tao
Tianyang Tao
LZLiang Zhang
Liang Zhang
CZChao Zuo
Chao Zuo
In many optical metrology techniques, fringe pattern analysis is the central algorithm for recovering the underlying phase distribution from the recorded fringe patterns. Despite extensive research efforts for decades, how to extract the desired phase information, with the highest possible accuracy, from the minimum number of fringe patterns remains one of the most challenging open problems. Inspired by recent successes of deep learning techniques for computer vision and other applications, we demonstrate for the first time, to our knowledge, that the deep neural networks can be trained to perform fringe analysis, which substantially enhances the accuracy of phase demodulation from a single fringe pattern. The effectiveness of the proposed method is experimentally verified using carrier fringe patterns under the scenario of fringe projection profilometry. Experimental results demonstrate its superior performance, in terms of high accuracy and edge-preserving, over two representative single-frame techniques: Fourier transform profilometry and windowed Fourier transform profilometry. Optical measurement techniques such as holographic interferometry,1 electronic speckle pattern interferometry,2 and fringe projection profilometry3 are quite popular for noncontact measurements in many areas of science and engineering, and have been extensively applied for measuring various physical quantities, such as displacement, strain, surface profile, and refractive index. In all these techniques, the information about the measured physical quantity is stored in the phase of a two-dimensional fringe pattern. The accuracy of measurements carried out by these optical techniques is thus fundamentally dependent on the accuracy with which the underlying phase distribution of the recorded fringe patterns is demodulated. Over the past few decades, tremendous efforts have been devoted to developing various techniques for fringe analysis. The techniques can be broadly classified into two categories: (1) phase-shifting (PS) methods that require multiple fringe patterns to extract phase information,4 and (2) spatial phase-demodulation methods that allow phase retrieval from a single fringe pattern, such as the Fourier transform (FT),5 windowed Fourier transform (WFT),6 and wavelet transform (WT) methods.7 Compared with spatial phase demodulation methods, multiple-shot PS techniques are generally more robust and can achieve pixel-wise phase measurement with higher resolution and accuracy. Furthermore, the PS measurements are quite insensitive to nonuniform background intensity and fringe modulation. Nevertheless, due to their multishot nature, these methods are difficult to apply to dynamic measurements and are more susceptible to external disturbance and vibration. Thus, for many applications, phase extraction from a single fringe pattern is desired, which falls under the purview of spatial fringe analysis. In contrast to PS techniques where the phase map is demodulated on a pixel-by-pixel basis, phase estimation at a pixel according to spatial methods is influenced by the pixel’s neighborhood, or even all pixels in the fringe pattern, which provides better tolerance to noise, yet at the expense of poor performance around discontinuities and isolated regions in the phase map.8,9 Deep learning is a powerful machine learning technique that employs artificial neural networks with multiple layers of increasingly richer functionality and has shown great success in numerous applications for which data are abundant.10,11 In this letter, we demonstrate experimentally for the first time, to our knowledge, that the use of a deep neural network can substantially enhance the accuracy of phase demodulation from a single fringe pattern. To be concrete, the networks are trained to predict several intermediate results that are useful for the calculation of the phase of an input fringe pattern. During the training of the networks, we capture PS fringe images of various scenes to generate the training data. The training label (ground truth) of each training datum is a pair of intermediate results calculated from the PS algorithm. After appropriate training, the neural network can blindly take only one input fringe pattern and output the corresponding estimates of these intermediate results with high fidelity. Finally, a high-accuracy phase map can be retrieved through the arctangent function with the intermediate results estimated through deep learning. Experimental results on fringe projection profilometry confirm that this deep-learning-based method is able to substantially improve the quality of the retrieved phase from only a single fringe pattern, compared to state-of-the-art methods. Here, the network configuration is inspired by the basic process of most phase demodulation techniques, which is briefly recalled as follows. The mathematical form of a typical fringe pattern can be represented as <!-- named anchor --> (1)I(x,y)=A(x,y)+B(x,y)cos ϕ(x,y),where I(x,y) is the intensity of the fringe pattern, A(x,y) is the background intensity, B(x,y) is the fringe amplitude, and ϕ(x,y) is the desired phase distribution. Here, x and y refer to the pixel coordinates. In most phase demodulation techniques, the background intensity A(x,y) is regarded as a disturbance term and should be removed from the total intensity. Then a wrapped phase map is recovered from an inverse trigonometric function whose argument is a ratio for which the numerator characterizes the phase sine [sin ϕ(x,y)] and the denominator characterizes the phase cosine [cos ϕ(x,y)]: <!-- named anchor --> (2)ϕ(x,y)=arctanM(x,y)D(x,y)=arctancB(x,y)sin ϕ(x,y)cB(x,y)cos ϕ(x,y),where c is a constant dependent on the phase demodulation algorithm (e.g., in FT c=0.5, in N-step PS c=N/2), and M(x,y) and D(x,y) represent the shorthand for the numerator and denominator terms, respectively. Note that the signs of M(x,y) and D(x,y) can be further used to uniquely define a quadrant for each calculation of ϕ(x,y). With the four-quadrant phasor space, the phase values at each point can be determined modulo 2π.In order to emulate the process above, two different convolutional neural networks (CNN) are constructed, which are connected cascadedly according to Fig. 1. The first convolutional neural network (CNN1) uses the raw fringe pattern I(x,y) as input and estimates the background intensity A(x,y) of the fringe pattern. With the estimated background image A(x,y) and the original fringe image I(x,y), the second convolutional neural network (CNN2) is trained to predict the numerator M(x,y) and the denominator D(x,y) of the arctangent function, which are fed into the subsequent arctangent function [Eq. (2)] to obtain the final phase distribution ϕ(x,y). <!-- named anchor --> Fig. 1 Flowchart of the proposed method where two convolutional networks (CNN1 and CNN2) and the arctangent function are used together to determine the phase distribution. For CNN1 (in red), the input is the fringe image I(x,y), and the output is the estimated background image A(x,y). For CNN2 (in green), the inputs are the fringe image I(x,y) and the background image A(x,y) predicted by CNN1, and the outputs are the numerator M(x,y) and the denominator D(x,y). The numerator and denominator are then fed into the arctangent function to calculate the phase ϕ(x,y). To generate the ground truth data used as the label to train the two convolutional neural networks, the phase retrieval is achieved by using the N-step PS method. The corresponding N PS fringe patterns acquired can be represented as <!-- named anchor --> (3)In(x,y)=A(x,y)+B(x,y)cos[ϕ(x,y)−δn],where the index n=0,1,…,N−1, and δn is the phase shift that equals 2πnN. With the orthogonality of trigonometric functions, the background intensity can be obtained as <!-- named anchor --> (4)A(x,y)=1N∑n=0N−1In(x,y). With the least square method, the phase can be calculated as <!-- named anchor --> (5)ϕ(x,y)=arctan∑n=0N−1In(x,y)sin δn∑n=0N−1In(x,y)cos δn. Thus, the numerator and the denominator of the arctangent function in Eq. (2) can be expressed as <!-- named anchor --> (6)M(x,y)=∑n=1N−1In(x,y)sin δn=N2B(x,y)sin ϕ(x,y), <!-- named anchor --> (7)D(x,y)=∑n=0N−1In(x,y)cos δn=N2B(x,y)cos ϕ(x,y). The expressions above show that the numerator M(x,y) and the denominator D(x,y) are closely related to the original fringe pattern in Eq. (1) through a quasilinear relationship with the background image

1 August 2013 in Optics and Lasers in Engineering
Verified
Journal Article
Closed access
CZChao Zuo
Chao Zuo
QCQian Chen
Qian Chen
GGGuohua Gu
Guohua Gu
Shijie FengShijie Feng
Shijie Feng
FFFangxiaoyu Feng
Fangxiaoyu Feng
GSGuochen Shen
Guochen Shen

1 March 2018 in Optics and Lasers in Engineering
Verified
Journal Article
Closed access
CZChao Zuo
Chao Zuo
TTTianyang Tao
Tianyang Tao
Shijie FengShijie Feng
Shijie Feng
LHLei Huang
Lei Huang
AAAnand Asundi
Anand Asundi
QCQian Chen
Qian Chen

10 August 2012 in Optics Express
Verified
Journal Article
Open access
PDF file
CZChao Zuo
Chao Zuo
QCQian Chen
Qian Chen
GGGuohua Gu
Guohua Gu
Shijie FengShijie Feng
Shijie Feng
FFFangxiaoyu Feng
Fangxiaoyu Feng
This paper describes an easy-to-implement three-dimensional (3-D) real-time shape measurement technique using our newly developed high-speed 3-D vision system. It employs only four projection fringes to realize full-field phase unwrapping in the presence of discontinuous or isolated objects. With our self-designed pattern generation hardware and a modified low-cost DLP projector, the four designed patterns can be generated and projected at a switching speed of 360 Hz. Using a properly synchronized high-speed camera, the high-speed fringe patterns distorted by measured objects can be acquired and processed in real-time. The resulting system can capture and display high-quality textured 3-D data at a speed of 120 frames per second, with the resolution of 640 × 480 points. The speed can be trebled if a camera with a higher frame rate is employed. We detail our shape measurement technique, including the four-pattern decoding algorithm as well as the hardware design. Some evaluation experiments have been carried out to demonstrate the validity and practicability of the proposed technique.

6 November 2018 in Measurement Science and Technology
Verified
Journal Article
Closed access
PDF file
Shijie FengShijie Feng
Shijie Feng
LZLiang Zhang
Liang Zhang
CZChao Zuo
Chao Zuo
TTTianyang Tao
Tianyang Tao
QCQian Chen
Qian Chen
GGGuohua Gu
Guohua Gu
In optical metrology, fringe projection is considered to be one of the most reliable techniques for recovering the shape of objects. For this technique, however, it is challenging to measure objects with a large variation in surface reflectivity, e.g. a scenario containing both dark and bright objects. Researchers have thus developed various approaches to high dynamic range (HDR) three-dimensional (3D) measurements over the years. In this paper, we present an overview of these techniques, as well as a new and definitive classification for them. We implement a set of representative techniques to measure objects with different characteristics of reflectance and discuss the advantages and constraints of the techniques according to the comparative results. Moreover, challenging problems and future research directions are discussed to advance HDR 3D measurement techniques.