In recent years, with the advancement of medical imaging technology, medical image segmentation has played a key role in assisting diagnosis and treatment planning. Current deep learning-based medical image segmentation methods mainly adopt encoder-decoder architecture design and have received wide attention. However, these methods still have some limitations, including: (1) Existing methods are often influenced by the significant semantic information gap when supplementing features for the decoder. (2) Existing methods do not simultaneously consider global and local information interaction during decoding, resulting in ineffective semantic recovery. Therefore, this paper proposes a novel Enhanced Decoder Semantic Recovery Network to address these challenges. Firstly, the Multi-Level Semantic Fusion (MLSF) module is introduced, which effectively fuses low-level features of the original image, encoder features, high-level features of the deepest network layer, and decoder features, and assigns weights based on semantic gaps. Secondly, the Multiscale Spatial Attention (MSSA) and Cross Convolution Channel Attention (CCCA) modules are employed to obtain richer feature information. Finally, the Global-Local Semantic Recovery (GLSR) module is designed to achieve better semantic recovery. Experiments on public datasets such as BUSI, CVC-ClinicDB, and Kvasir-SEG demonstrate that the proposed model improves IoU compared to suboptimal algorithms by 0.81%, 0.85% and 1.98%, respectively, significantly enhancing the performance of 2D medical image segmentation. This method provides effective technical support for further development in the field of medical imaging.