As immune checkpoint inhibitors have shown good clinical efficacy, immune checkpoint blockade has become a vital strategy in cancer therapy. However, approximately only 12.5% patients experience benefits from immunotherapy. Herein, we identified the cancer differentiation inducer chlorogenic acid (CGA, now in the phase II clinical trial in China for glioma treatment) to be a small-molecular immune checkpoint inhibitor that boosted the antitumor effects of the anti-PD-1 antibody. CGA suppressed the expression of PD-L1 induced by interferon-γ in tumor cell culture through inhibition of the p-STAT1-IRF1 pathway and enhanced activity of activated T-cells. In two murine tumor xenografts, combination therapy of CGA with anti-PD-1 antibody decreased the expression of PD-L1 and IRF1 and increased the inhibitory effect of the anti-PD-1 antibody on tumor growth. Particularly, the activity of tumor infiltrated T cells was enhanced by CGA. CGA improved the gene expression of granzymes in tumor-infiltrated immune cells. In conclusion, through induction of differentiation, CGA appeared to suppress the expression of PD-L1 on cancer cells, effectively promoting infiltrated T cells in the tumor and boosting the antitumor effect of the anti-PD-1 antibody. Thus, CGA might serve as a promising agent to enhance anticancer immunotherapy if combined with anti-PD-1 antibodies.