Aiming to address the problems of the high bit error rate (BER) of demodulation or low classification accuracy of modulation signals with a low signal-to-noise ratio (SNR), we propose a double-residual denoising autoencoder method with a channel attention mechanism, referred to as DRdA-CA, to improve the SNR of modulation signals. The proposed DRdA-CA consists of an encoding module and a decoding module. A squeeze-and-excitation (SE) ResNet module containing one residual connection is modified and then introduced into the autoencoder as the channel attention mechanism, to better extract the characteristics of the modulation signals and reduce the computational complexity of the model. Moreover, the other residual connection is further added inside the encoding and decoding modules to optimize the network degradation problem, which is beneficial for fully exploiting the multi-level features of modulation signals and improving the reconstruction quality of the signal. The ablation experiments prove that both the improved SE module and dual residual connections in the proposed method play an important role in improving the denoising performance. The subsequent experimental results show that the proposed DRdA-CA significantly improves the SNR values of eight modulation types in the range of −12 dB to 8 dB. Especially for 16QAM and 64QAM, the SNR is improved by 8.38 dB and 8.27 dB on average, respectively. Compared to the DnCNN denoising method, the proposed DRdA-CA makes the average classification accuracy increase by 67.59∼74.94% over the entire SNR range. When it comes to the demodulation, compared with the RLS and the DnCNN denoising algorithms, the proposed denoising method reduces the BER of 16QAM by an average of 63.5% and 40.5%, and reduces the BER of 64QAM by an average of 46.7% and 18.6%. The above results show that the proposed DRdA-CA achieves the optimal noise reduction effect.