China introduced the county medical community (CMC) reform, aimed to provide high-quality medical resources to rural citizens, in 2017. This study examines the impact of the reform on the medical service efficiency of county-level public general hospitals in Shanxi Province, China. In total, 92 county-level public general hospitals from Shanxi Province were taken as the research objective, and the super-efficiency SBM-DEA model was applied to measure medical service efficiency. Further, a two-way fixed-effect model was used to evaluate the impact of CMC reform on the medical service efficiency of county-level public general hospitals by using health statistics data from 2014 to 2018. The study reveals that the CMC reform improved the medical service efficiency of county-level public general hospitals by 15.6%. Moreover, the CMC reform had regional heterogeneity in its impact on the medical service efficiency of county-level public general hospitals. The CMC reform improved the medical service efficiency of hospitals in the southern region more than in the northern region of the province. The medical service efficiency of hospitals in the central region was also improved by CMC reform, but the causal relationship was not found significant. Further, hospital-level factors (e.g., fixed assets, hospital stay, and regional health center) and environmental factors (e.g., GDP, population, urbanization rate, and government subsidies) affected the medical service efficiency of county-level public hospitals during the process of promoting the CMC reform.