Health-related quality of life (HRQOL) is an important indicator of treatment effectiveness. An unhealthy lifestyle can have a negative impact on quality of life. This study aimed to investigate changes in health-related lifestyle over time after surgery for colorectal cancer and their impact on HRQOL. Healthy lifestyle habits examined in this study included physical activity, smoking, alcohol consumption, fruit and vegetable intake, sleep, and obesity levels. An observational study design was used. A total of 75 post-operative colorectal cancer patients were recruited from two medical centers in Taiwan. Data were collected through structured questionnaires. Mean HRQOL scores at 1, 3, and 5 months after discharge were 102.5 (SD = 18.8), 102.9 (SD = 20.1), and 103.0 (SD = 18.9), respectively. A generalized estimating equation analysis showed that alcohol consumption (p = 0.009), fruit and vegetable intake (p = 0.020), physical activity (p = 0.023), sleep quality (p < 0.001), and obesity (p = 0.035) were important predictors of post-operative quality of life in patients with colorectal cancer. The impact of smoking on HRQOL did not reach statistical significance. Colorectal cancer patients tend to have better HRQOL after surgery if they stay physically active, eat enough fruits and vegetables, and sleep well.