The lens design is challenging and time-consuming, requiring tedious human trial and error. Recently, joint design of lens and image processing networks based on differentiable ray tracing techniques has emerged, which provides the possibility to reduce the difficulty of traditional lens design. However, existing joint design pipelines cannot optimize all parameters, including materials and high-order aspheric terms, nor do they use diffraction theory to calculate point spread functions (PSFs) accurately. In this work, we propose a fully automated joint design framework, especially for smartphone telephoto lenses, which starts from optical design indicators, uses Delano diagrams to calculate reasonable optical initial structures, and jointly optimizes the lens system and the image processing network. Considering the diffraction effect, a differentiable PSF calculation method based on the Fresnel-Kirchhoff diffraction model is used for end-to-end joint optimization. This work can reduce the difficulty of the lens design and provide an accurate PSF calculation method considering the diffraction effect for end-to-end joint optimization.
Differentiable optics integrates optical design with image processing, enabling joint optimization of optical systems and neural networks through task-specific evaluation metrics. However, controlling lens tolerance sensitivity during optimization remains an unresolved challenge, often resulting in significant performance degradation due to manufacturing and assembly tolerances. In this work, we propose a fast sensitivity control method tailored for different optical models. Our approach addresses sensitivity from three key aspects: tolerance sensitivity, uniformity of tolerance degradation, and symmetry of tolerance boundaries. Using the wavefront-focusing method, we enhance optimization stability by compensating for defocus and focusing the evaluation on the optimal image plane. Additionally, the optical path differential method eliminates the need for repeated ray tracing of perturbed systems, significantly reducing computational costs. Experimental results with two advanced smartphone lenses, a wide-angle and a telephoto lens, validate the effectiveness of our method. This work provides a practical solution for desensitization optimization in differentiable optics, offering new possibilities for optical desensitization and joint design in computational imaging tasks.