Recently, endoplasmic reticulum (ER) stress has been shown to influence tumor progression and immune cell function in the tumor microenvironment (TME). However, the underlying role of ER stress-related gene patterns in colorectal cancer (CRC) development remains unclear. We analyzed the ER stress-related gene patterns in 884 patients with CRC from the Gene Expression Omnibus database and evaluated the cell-infiltrating patterns in the TME. Two ER stress-related patterns were identified in patients with CRC that had distinct cell-infiltrating patterns in the TME and clinical characteristics. A risk score and nomogram based on 14 screened prognosis-correlated genes was built and validated to predict patient survival. Patients with a higher risk score were shown to have an unfavorable prognosis, and the risk score was associated with cell infiltration and drug sensitivity. Furthermore, spatial transcriptomics data were utilized to explore ER stress-related gene patterns in CRC tissues, and it was shown that ER stress phenotype involves in the formation of the immunosuppressive TME. This study demonstrated that ER stress-related gene patterns play a role in influencing the TME and predicting prognosis. These analyses of ER stress in the TME of CRC might deepen our understanding of CRC progression and immune escape and provide novel insights into therapeutic strategies.