Advancements in radiotherapy (RT) techniques such as intensity modulation, image guidance, and hypofractionation have facilitated a satisfactory survival outcome in prostate cancer (PCa) patients. However, virtually all PCa patients suffer from various types and extents of radiation toxicities, which are mainly gastrointestinal (GI) and genitourinary (GU) in nature, eroding their quality of life. Thus, early mitigation and preventative measures should be offered, enabled by accurate toxicity prediction. This scoping review provides a comprehensive summary of reported acute and late GI and GU toxicity predictors of conventional fractionation (CFRT), moderate hypofractionation (MHRT), and ultra-hypofractionation (UHRT). A total of 169 studies published between the years 2000 and 2024 (inclusive) were identified from four databases, with 127 and 78 studies investigating GI and GU toxicities, respectively. Univariate analysis was employed in 139 studies to identify predictors, while 94 studies involved multivariate analysis, 40 involved internal model validation, and 5 performed external model validation. Among all studies, dosimetric predictors are the most reported factors, followed by patient, clinical, treatment, disease, genetic, and radiomic features. However, their applicability and performance have not yet been extensively proven in external validation involving multicenter studies. Future predictive studies should also focus on deeper multimodality information, such as radiomics, in addition to the categories of factors consolidated in this study, for an all-rounded investigation. A multicenter study is highly encouraged for prospective external validation. Further investigations into delivered doses and sub-volumes of various regions of interest are necessary. Comprehensive reporting items suggested in this work shall facilitate the reproducibility and comparability of the results.