Water-related soil erosion is a major environmental concern for catchments with barren topography in arid and semi-arid regions. With the growing interest in irrigation infrastructure development in arid regions, the current study investigates the runoff and sediment yield for the Gomal River catchment, Pakistan. Data from a precipitation gauge and gridded products (i.e., GPCC, CFSR, and TRMM) were used as input for the SWAT model to simulate runoff and sediment yield. TRMM shows a good agreement with the data of the precipitation gauge (≈1%) during the study period, i.e., 2004–2009. However, model simulations show that the GPCC data predicts runoff better than the other gridded precipitation datasets. Similarly, sediment yield predicted with the GPCC precipitation data was in good agreement with the computed one at the gauging site (only 3% overestimated) for the study period. Moreover, GPCC overestimated the sediment yield during some years despite the underestimation of flows from the catchment. The relationship of sediment yields predicted at the sub-basin level using the gauge and GPCC precipitation datasets revealed a good correlation (R² = 0.65) and helped identify locations for precipitation gauging sites in the catchment area. The results at the sub-basin level showed that the sub-basin located downstream of the dam site contributes three (3) times more sediment yield (i.e., 4.1%) at the barrage than its corresponding area. The findings of the study show the potential usefulness of the GPCC precipitation data for the computation of sediment yield and its spatial distribution over data-scarce catchments. The computations of sediment yield at a spatial scale provide valuable information for deciding watershed management strategies at the sub-basin level.