(1) Background: Social isolation, which has numerous adverse effects on health status, is prevalent among cancer patients. This review proposes to identify the influencing factors of social isolation among cancer patients. (2) Methods: Articles published in English or Chinese from six electronic databases before December 2023 were identified via a systematic search. A manual search was also performed. (3) Results: Twenty-eight studies were identified in this systematic review. The factors associated with social isolation can be summarized into the following categories: demographic characteristics, having cancer, health status, coping, social support and social interaction. Despite the heterogeneity, 20 factors were significantly associated with social isolation, including age, gender, comorbidity burden, education level, residence, medical insurance, occupation status, personality, race, smoking status, having children, not living alone, household income level, marital status, the role of primary caregiver, physical health status, mental health status, social health status, coping styles, and the level of social support and social interaction. (4) Conclusions: The systematic review showed that cancer patients’ social isolation was influenced by their demographic characteristics, cancer-related factors, physical condition, psychological status, social health status, coping styles, and level of social support and social interaction. In addition, future group intervention could be considered to improve social isolation.