Deep neural network models generally enhance super-resolution reconstruction of remote sensing images but may distort feature edge details. Recovering low-resolution remote sensing images with clear texture and high-fidelity edge details is challenging. Recent approaches improve feature fusion in the frequency domain, yielding promising results. We propose a Dual-Domain Optimization Network (DDOM) based on Discrete Fourier Transform (DFT) and frequency domain complex-valued neural networks. Unlike end-to-end approaches in the image domain, DDOM incorporates frequency-domain information via DFT transformation operators, preserving high-level semantics (phase) and low-level statistical information (magnitude). Using lightweight complex neural networks and SwinTransformer architecture, the frequency-domain and image-domain subnetworks are designed. The dual-domain data consistency constraints ensure the positivity of model optimization. Extensive experiments show superior performance over existing methods in quantitative and qualitative evaluations. The proposed scheme’s robustness is verified on additional datasets. Code and model configurations are available at http://github.com/YB-Cheng/DDOM.