Agricultural machinery is effective in improving food production and labor productivity, but it also raises environmental concerns. The impacts of agricultural machinery on the green total factor productivity (GTFP) of grain in China are still under debate in the scientific literature. This study proposed an integrated framework for confronting this issue. The findings suggest that both agricultural mechanization and the GTFP of grain demonstrate a consistent upward trend with moderate fluctuations between 2001 and 2019. By expanding the spatial pattern, there is a positive spatial correlation between them. In addition, we compared the results in three grain functional areas by using the spatial Durbin model (SDM). There were significantly positive spatial spillover effects in major grain-producing areas, which were attributed to the trans-regional operation of agricultural machinery and its carbon reduction effects on neighboring provinces. Notably, the direct effects in major grain-marketing areas and producing-marketing balance areas were significantly positive because agricultural machinery has played a critical role in filling the gap in local labor shortages in grain production. Accordingly, adaptive strategies including building the “Internet + agricultural machinery operation” platform, implementing the land consolidation suitable for machinery, and developing low-carbon agricultural machinery should be fully considered by Chinese policy-makers to promote mechanized agriculture and a low-carbon economy. The findings of this study can help us better understand the role of agricultural machinery in improving green grain productivity in China and thus have significance for the modern and green transformation of agricultural production systems.