Simple hydrological models, such as the Seasonal Water Yield Model developed by the Natural Capital Project (InVEST SWYM), are attractive as data requirements are relatively easy to satisfy. However, simple models may produce unrealistic results when the underlying hydrological processes are inadequately described. We used the variation in performance of the InVEST SWYM across watersheds to identify correlates of poorly modeled outcomes of InVEST SWYM. We grouped 749 watersheds from across North America into five bioclimatic regions using nine environmental variables. For each region, we compared the predicted flow patterns to actual flow conditions over a 15-year period. The correlation between the modeled and actual flows was highly dispersed and relatively poor, with 92% of r² values less than 0.5 and 42% less than 0.1. We linked cryospheric variables to model performance in the bioclimatic region with the poorest model performance (the Low elevation Boreal Sub-humid region—LeBSh). After incorporating cryospheric conditions into the InVEST SWYM, predictions improved significantly in 30% of the LeBSh watersheds. We provide a relatively straightforward approach for identifying processes that simple hydrological models may not consider or which need further attention or refinement.