Spectral confocal displacement sensors are non-contact optoelectronic sensors widely utilized for their high accuracy, speed, and ability to measure diverse surfaces. However, challenges including vibration, angular deflection, and surface quality variations can reduce sensor stability and accuracy when performing measurements such as lithium battery wafer thickness, wafer warpage, and optical component surface topography. This study proposes a line-spot-based measurement method using a binary diffractive lens and cylindrical lens with a 20× objective, and then the overall structure is simulated and optimized by using ZEMAX, which realizes a confocal measurement system with a measurement range of 800 μm, line spot length of 3.8 mm, and width of 0.2 mm. The system, calibrated with a nanometer displacement stage, achieved 30 nm resolution and significantly improved dynamic stability (standard deviation (SD) of 0.013 μm) compared to a point spectral confocal sensor (SD of 0.064 μm). The results indicate the proposed sensor exhibits improved stability during scanning measurements.