Climate change has exacerbated the frequency and severity of droughts worldwide. Evaluating the response of gross primary productivity (GPP) to drought is thus beneficial to improving our understanding of the impact of drought on the carbon cycle balance. Although many studies have investigated the relationship between vegetation productivity and dry/wet conditions, the capability of different drought indices of assessing the influence of water deficit is not well understood. Moreover, few studies consider the effects of drought on vegetation with a focus on periods of drought. Here, we investigated the spatial-temporal patterns of GPP, the standardized precipitation evapotranspiration index (SPEI), and the vapor pressure deficit (VPD) in China from 2001 to 2020 and examined the relationship between GPP and water deficit/drought for different vegetation types. The results revealed that SPEI and GPP were positively correlated over approximately 70.7% of the total area, and VPD was negatively correlated with GPP over about 66.2% of the domain. Furthermore, vegetation productivity was more negatively affected by water deficit in summer and autumn. During periods of drought, the greatest negative impact was on deciduous forests and croplands, and woody savannas were the least impacted. This research provides a scientific reference for developing mitigation and adaptation measures to lessen the impact of drought disasters under a changing climate.