This paper presents a phase retrieval algorithm that incorporates sparsity priors into total variation and framelet regularization. The proposed algorithm exploits the sparsity priors in both the gradient domain and the spatial distribution domain to impose desirable characteristics on the reconstructed image. We utilize structured illuminated patterns in holography, consisting of three light fields. The theoretical and numerical analyses demonstrate that when the illumination pattern parameters are non-integers, the three diffracted data sets are sufficient for image restoration. The proposed model is solved using the alternating direction multiplier method. The numerical experiments confirm the theoretical findings of the lighting mode settings, and the algorithm effectively recovers the object from Gaussian and salt–pepper noise.