We aimed to clarify the effect of the hemoglobin glycation index (HGI)–diabetes mellitus (DM) system in evaluating the risk of incident stroke. We followed up on 2934 subjects in rural regions of Northeast China, established Cox proportional hazards models to evaluate the effects of the HGI–DM system in describing stroke risk, and further conducted a discrimination analysis to confirm the improvement in HGI based on the traditional stroke risk model. After a median of 4.23 years of follow-up, 79 subjects developed stroke or related death. DM-high HGI condition significantly elevated the risk of incident stroke (hazard ratio (HR): 2.655, 95% confidence interval (CI): 1.251–5.636). In addition, higher HGI levels elevated the risk of stroke, even if the patients did not have DM (HR: 1.701, 95% CI: 1.136–2.792), but DM failed to bring an extra risk of incident stroke to patients with lower HGI levels (HR: 1.138, 95% CI: 0.337–3.847). The discrimination analysis indicated that the integrated discrimination index (IDI) of the HGI model was 0.012 (95% CI: 0.007–0.015) and that the net reclassification index (NRI) was 0.036 (95% CI: 0.0198–0.0522). These results indicated HGI was associated with the onset of stroke, and high HGI indicated an aggravated trend in glycemic status and increased risk of incident stroke. The HGI–DM system enabled us to identify the different glucose statuses of patients, to conduct suitable treatment strategies, as well as to improve the predictability of incident stroke based on the traditional model.