Fertilization is a critical measure for vegetation restoration and ecological reconstruction in degraded grasslands. However, little is known about the long-term effects of different combinations of nitrogen (N), phosphorus (P), potassium (K) on plant and microbial communities in degraded grasslands. This study conducted a four-year (2017–2020) N, P, K addition experiment in the Khorchin Grassland, a degraded typical grassland located in Zhalute Banner, Tongliao City, Inner Mongolia, to investigate the effects of fertilization treatment on plant functional groups and microbial communities after grazing exclusion. Our results showed that the addition of P, NP, and NPK compound fertilizers significantly increased aboveground biomass of the plant community, which is mainly related to the improvement of nutrient availability to promote the growth of specific plant functional groups, especially annual and biennial plants and perennial bunchgrasses. However, the addition of N, P, and NP fertilizers significantly reduced the species diversity of the plant community. At the same time, the addition of N, P, and NP fertilizers and the application of N and NP significantly reduced fungal species diversity but had no significant effect on soil bacteria. Our study provides new insights into the relationships between different types of fertilization and plant community productivity and biodiversity in degraded grasslands over four years of fertilization, which is critical for evaluating the effect of fertilization on the restoration of degraded grassland.