Recently, frequent water shortages and reductions in water flow have been observed in the northern segment of the Yellow River within the Beijing–Hangzhou Grand Canal. In response, a water replenishment program has been initiated. This study is focused on the section of the Grand Canal north of the Yellow River and uses the GSFLOW model to examine interaction between surface water and groundwater, as well as the effect of water replenishment. The results indicate that, after the water replenishment, the efficiency of water replenishment was highest in the Xiao Canal (64.30%), followed by the Wei Canal (39.09%), the South Canal (12.11%), and the North Canal, which exhibited the lowest efficiency (5.75%). This variation can be attributed to greater water loss with increasing distance from the replenishment source, leading to lower replenishment efficiency. Surface water recharge to groundwater was extended by 32 days, with replenishment effects persisting even after the water supply ceased. The maximum influence distance on either side of the canal reached 5.73 km, with an average impact distance of 1.48 km, resulting in a total affected area of 974.7 km², accounting for 2.2% of the study area. Water replenishment positively influenced the recovery of groundwater levels along the Grand Canal.