Basins located in loess hilly–gully regions often suffer flood disasters during the flood season. Meanwhile, the underlying surface of the region can increase the rainfall losses, thereby reducing the flood volume. Therefore, the prediction of rainfall losses on the underlying surface is necessary for scientifically and reasonably forecasting the flood volume. The relationship between the rainfall losses and underlying characteristics was investigated and a method for predicting the rainfall losses using HEC-HMS was presented in this paper with a case study in the Gedong basin, a typical loess hilly region of western Shanxi Province in northern China. Results showed that HEC-HMS could be applied to loess hilly–gully regions. The loss computation results suggested that the losses of sub-basins varied with the density of rainfall. The analysis of influences of rainfall losses, including forestland percentage and slope, indicated that the former had a positive impact, while the latter had a negative influence. The impact of forestland percentage is larger than that of slope. Furthermore, with the increase of forestland percentage, its correlation with rainfall losses was enhanced, and the correlation coefficient ranged between 0.64 and 0.84 from the 1970s to the 2010s.