Tumor immunotherapy, modulating innate and adaptive immunity, has become an important therapeutic strategy. However, the tumor immune microenvironment’s (TIME) complexity and heterogeneity challenge tumor immunotherapy. Hydrogel is a hydrophilic three-dimensional (3D) mesh structure with good biocompatibility and drug release control, which is widely used in drug delivery, agriculture, industry, etc. Hydrogels loaded with immune cells, cytokines, immune checkpoint inhibitors, and anti-tumor drugs can achieve targeted delivery and ultimately activate the immune response in the TIME. In this review, we will summarize the components of the TIME and their immune effects, the emerging immunomodulatory agents, the characteristics and functions of hydrogels, and how hydrogels regulate innate and adaptive immune cells in the TIME.