Infiltration of diverse cell types into tumor microenvironment plays a critical role in cancer progression including metastasis. We previously reported that SFMBT2 (Scm-like with four mbt domains 2) regulates the expression of matrix metalloproteinases (MMPs) and migration and invasion of cancer cells in prostate cancer. Here we investigated whether the down-regulation of SFMBT2 regulates the infiltration of preadipocytes and tumor-associated macrophages (TAMs) in prostate cancer. We found that the down-regulation of SFMBT2 promotes the infiltration of preadipocytes and TAMs through up-regulation of CXCL8, CCL2, CXCL10, and CCL20 expression in prostate cancer. Expression of CXCL8, CCL2, CXCL10, and CCL20 was also elevated in prostate cancer patients having a higher Gleason score (≥8), which had substantially lower SFMBT2 expression. We also found that the up-regulation of CXCL8, CCL2, CXCL10, and CCL20 expression is dependent on NF-κB activation in prostate cancer cells expressing a low level of SFMBT2. Moreover, increased IL-6 from infiltrated preadipocytes and TAMs promoted migration and invasion of prostate cancer cells expressing a low level of SFMBT2. Our study may suggest that SFMBT2 a critical regulator for the infiltration of preadipocytes and TAMs into the prostate tumor microenvironment. Thus, the regulation of SFMBT2 may provide a new therapeutic strategy to inhibit prostate cancer metastasis, and SFMBT2 could be used as a potential biomarker in prostate cancer metastasis.