The geomagnetic reference map is an important component of underwater geomagnetic matching navigation, and it directly affects navigation and positioning accuracy. However, representative global geomagnetic models still cannot fully meet the requirements of high-precision matching navigation. To address this, this study proposes a local marine geomagnetic anomaly model reconstruction method via multi-source data fusion based on a fully connected neural network. The proposed MAG-DFN method uses the EMAG2 model as the background field and learns model residuals from shipborne measurements to reconstruct the geomagnetic model. Two sub-areas in the South China Sea and the western Pacific were selected for this experiment, each containing two withheld 1° × 1° validation blocks, and four baseline methods were introduced for comparison. The results show that MAG-DFN reduces MAE by 14.71%, 23.35%, 22.09%, and 8.96%, and RMSE by 12.41%, 18.87%, 16.73%, and 6.65%, in the four validation blocks, respectively. In addition, MAG-DFN outperforms all baseline methods. These results indicate that the MAG-DFN method, by integrating high-precision shipborne data, can effectively reduce the local deviations of the EMAG2 and better support matching navigation.