Ground-penetrating radar (GPR) is extensively used for evaluating the condition of ballast layers. However, the detection accuracy is often compromised by the significant interference caused by concrete sleepers, which can obscure the characteristics of the underlying signals. To address this issue, this paper proposed a novel variant of the Cycle-Consistent Generative Adversarial Network (CycleGAN) model utilizing unsupervised learning methods. This model is specifically designed for handling unpaired GPR A-scan data and integrates spectral normalization to enhance its efficacy. Through numerical simulations and field experiments, datasets both with and without sleeper interference were established to train and validate the model. The performance of the proposed method is evaluated through both quantitative and qualitative analysis across various test sets, demonstrating its superior ability to suppress sleeper interference. Furthermore, the model’s robust generalization capabilities were further validated in field applications on a test line. This research advances the precision of GPR data interpretation for railway ballasted track.