Soil moisture in deep soil layers is a relatively stable water resource for vegetation growth in the semi-arid Loess Plateau of China. Characterizing the variations in deep soil moisture and its influencing factors at a moderate watershed scale is important to ensure the sustainability of vegetation restoration efforts. In this study, we focus on analyzing the variations and factors that influence the deep soil moisture (DSM) in 80–500 cm soil layers based on a soil moisture survey of the Ansai watershed in Yan'an in Shanxi Province. Our results can be divided into four main findings. (1) At the watershed scale, higher variations in the DSM occurred at 120–140 and 480–500 cm in the vertical direction. At the comparable depths, the variation in the DSM under native vegetation was much lower than that in human-managed vegetation and introduced vegetation. (2) The DSM in native vegetation and human-managed vegetation was significantly higher than that in introduced vegetation, and different degrees of soil desiccation occurred under all the introduced vegetation types. Caragana korshinskii and black locust caused the most serious desiccation. (3) Taking the DSM conditions of native vegetation as a reference, the DSM in this watershed could be divided into three layers: (i) a rainfall transpiration layer (80–220 cm); (ii) a transition layer (220–400 cm); and (iii) a stable layer (400–500 cm). (4) The factors influencing DSM at the watershed scale varied with vegetation types. The main local controls of the DSM variations were the soil particle composition and mean annual rainfall; human agricultural management measures can alter the soil bulk density, which contributes to higher DSM in farmland and apple orchards. The plant growth conditions, planting density, and litter water holding capacity of introduced vegetation showed significant relationships with the DSM. The results of this study are of practical significance for vegetation restoration strategies, especially for the choice of vegetation types, planting zones, and proper human management measures.