Three-dimensional (3D) cameras are expensive because they employ additional charged coupled device sensors and optical elements, e.g., lasers or complicated scanning mirror systems. One passive optical method, shape from focus (SFF), provides an efficient low cost solution for 3D cameras. However, mechanical vibration of the SFF imaging system causes jitter noise along the optical axis, which makes it difficult to obtain accurate shape information of objects. In traditional methods, this error cannot be removed and increases as the estimation of the shape recovery progresses. Therefore, the final 3D shape may be inaccurate. We introduce an accurate depth estimation method using an adaptive neural network (ANN) filter to remove the jitter noise effects. Jitter noise is modeled by both Gaussian distribution and non-Gaussian distribution. Then, focus curves are modeled by quadratic functions. The ANN filter is designed as an optimal estimator restoring the original position of each frame of the input image sequence in the modeled jitter noise, as a pre-processing step before the initial depth map is obtained. The proposed method was evaluated using image sequences of both synthetic and real objects. Experimental results demonstrate that it is reasonably efficient and that its accuracy is comparable with that of existing systems.