Coherent noise in digital holographic microscopy (DHM) seriously degrades the accuracy of quantitative phase imaging, limiting its applications in fields such as nondestructive testing. However, traditional numerical denoising methods struggle to achieve an ideal balance between noise suppression, detail preservation, and computational efficiency. To address this challenge, we propose a multi-scale attention efficient network (MAENet). This network employs a dual-encoder architecture to achieve complementary extraction of multi-scale features. To efficiently integrate the features from these two branches, a dual-branch dense attention fusion (DDAF) module is designed. It performs a weighted fusion of features from the dual branches via an adaptive attention mechanism and enhances feature representation via dense residual connections, significantly boosting the model’s denoising performance. Furthermore, a hierarchical fusion strategy is adopted to preserve high-frequency details in the shallow layers of the network while performing feature fusion in the deeper layers, thereby maximizing protection of image textures while effectively suppressing noise. To address the lack of paired training data in real-world scenarios, a DHM simulation system capable of simulating the key physical characteristics of coherent noise was constructed. Extensive experiments on the simulated dataset show that MAENet achieves a PSNR of 33.25 dB and an SSIM of 0.93042, outperforming various mainstream denoising algorithms and demonstrating its excellent performance in suppressing coherent noise, providing an effective solution for denoising in coherent imaging systems.