Obesity, a known risk factor for various types of cancer, reduces the number and function of cytotoxic immune cells in the tumor immune microenvironment (TIME). However, the impact of obesity on CD4+ T cells remains unclear. Therefore, this study aimed to clarify the impact of obesity on CD4+ T cells in the TIME. A tumor-bearing obese mouse model was established by feeding with 45% high-fat diet (HFD), followed by inoculation with a colon cancer cell line MC38. Tumor growth was significantly accelerated compared to that in mice fed a control diet. Tumor CD4+ T cells showed a significant reduction in number and an increased expression of programmed death-1 (PD-1), and decreased CD107a expression and cytokine such as IFN-γ and TNF-α production, indicating dysfunction. We further established CD4+ T cell-depleted HFD-fed model mice, which showed reduced tumor infiltration, increased PD-1 expression in CD8+ T cells, and obesity-induced acceleration of tumor growth in a CD4+ T cell-dependent manner. These findings suggest that the reduced number and dysfunction of CD4+ T cells due to obesity led to a decreased anti-tumor response of both CD4+ and CD8+ T cells to ultimately accelerate the progression of colorectal cancer. Our findings may elucidate the pathogenesis for poor outcomes of colorectal cancer associated with obesity.