Quantifying the spatial variation in soil organic carbon (SOC) accumulation and its determining factors is vital for improving soil fertility and quality. The Fenhe River Basin is a major grain-producing area in Shanxi province in northern China, whereas the spatial heterogeneity of SOC accumulation is poorly known. Therefore, this study investigated SOC accumulation and its driving factors in arable soils across three representative counties (JL, FY, and XF) in the upper, middle, and lower reaches of the Fenhe River, respectively, which differ in climatic conditions and soil characteristics. SOC was determined using the potassium dichromate oxidation method. Results showed that both SOC content and stock were significantly lower in JL County (27–32% and 27–29% lower, respectively) compared to FY and XF. Within each county, the coefficient of variation was the highest in JL County (37.01%), indicating the highest spatial heterogeneity. Kriging interpolation indicated apparent spatial variation in SOC accumulation across the three counties. In JL, SOC was relatively higher in the northwest and southeast, while in FY, higher values occurred in the east. In XF, SOC generally decreased from north to south. The Spearman rank correlation and Random Forest analysis showed that SOC content and stock were significantly influenced by pH and cation exchange capacity (CEC), and pH had negative influences in JL County. Notably, in XF, available phosphorus and potassium, CEC, and clay proportion had positive influences on the SOC content and stock, while sand proportion had a negative relationship. These findings highlight the divergent, spatially heterogeneous nature of SOC accumulation across the Fenhe River Basin, driven by different factors. This study emphasizes the importance of specific soil management strategies in different locations to enhance carbon sequestration at the basin scale.