In this paper, we propose a measurement method of micro-optical elements with parallel phase-shifting digital holographic phase imaging. This method can record four phase-shifting holograms with a phase difference of π/2 in a single shot and correct the pixel mismatch error of the polarization camera using a bilinear interpolation algorithm, thereby producing high-resolution four-step phase-shifting holograms. This method reconstructs the real phase information of the object to be measured through a four-step phase-shifting algorithm. The reproduced image eliminates the interference of zero-order images and conjugate images, overcoming the problem that traditional phase-shifting digital holography cannot be measured in real time. A simulation analysis showed that the relative error of this measurement method could reach 0.0051%. The accurate surface topography information of the object was reconstructed from an experimental measurement through a microlens array. Multiple measurements yielded a mean absolute error and a mean relative error for the vertical height of the microlens array down to 5.9500 nm and 0.0461%, respectively.