Pancreatic cancer is a digestive tract malignancy that poses a serious threat to human health. Compounds derived from traditional Chinese medicines have been an important source of anticancer drugs and adjuvant agents to regulate the tumor immune microenvironment in patients with pancreatic cancer. In this study, icariin was purified from Herba Epimedii using macropores, and its bioactivity against pancreatic cancer was also investigated. We found that icariin has direct inhibitory and immunomodulatory effects on tumor cells. In vitro experiments showed that icariin can inhibit the migration and proliferation of Panc02 pancreatic cancer cells and induce apoptosis. Our in vivo experiments show that icariin inhibits the development of mouse pancreatic cancer by inhibiting tumor-infiltrating M2 macrophages and polymorphonuclear myeloid-derived suppressor cells (MDSCs) (PMN-MDSCs). In addition, icariin inhibits the polarization of RAW 264.7 cells into M2 macrophages by inhibiting the expression of ARG1 and MRC1 and downregulating the IL4-STAT6 signaling pathway. In conclusion, the inhibitory effect of icariin on pancreatic cancer can not only directly affect tumor cells but also inhibit tumor development by regulating the tumor immune microenvironment.