Pancreatic ductal adenocarcinoma (PDAC) is an exceptionally aggressive tumor, characterized by a low survival rate. A successful treatment strategy should not be limited to targeting the cancer cells alone, but should adopt a more comprehensive approach, taking into account other influential factors. These include the extracellular matrix (ECM) and the immune microenvironment, both of which are integral components of the tumor microenvironment. This review delineates the roles of pancreatic stellate cells, differentiated cancer-associated fibroblasts, and the interleukin family, either independently or in conjunction, in the progression of the precursor lesion pancreatic intraepithelial neoplasia (PanIN) and PDAC. These elements contribute to ECM deposition and immunosuppression in PDAC. Therapeutic strategies integrating interleukin and/or stromal blockade for PDAC immunomodulation and fibrogenesis have yielded inconsistent results. A deeper comprehension of the intricate interplay between inflammation, fibrosis, and immune responses could pave the way for more effective treatment targets, by elucidating the mechanisms and causes of ECM fibrosis during PDAC evolution.