Because swell noises are very common in marine seismic data, it is extremely important to attenuate them to improve the signal-to-noise ratio (S/N). Compared to process noises in the time domain, we have built a frequency-domain convolutional neural network (CNN) based on the short-time Fourier transform to address swell noises. In the numerical experiments, we quantitatively evaluate the denoising performances of the time- and frequency-domain CNNs, compare the impacts of network structures on attenuating swell noises, and study how network parameter choices impact the quality of the denoised signal based on peak S/N, structural similarity, and root-mean-square-error indices. These results help us to build an optimal CNN model. Furthermore, to illustrate the superiority of our proposed method, we compare the conventional and proposed CNN methods. To address the generalization capability of CNN, we adopt transfer learning by using fine tuning to adjust the weights of the pretrained model with a small amount of target data. The application of transfer learning improves the quality of the denoised images, which further proves that our proposed method with transfer learning has the potential to be deployed in actual seismic data acquisition.