Purpose: Antiendocrine therapy remains the most effective treatment for estrogen receptor–positive (ER⁺) breast cancer, but development of resistance is a major clinical complication. Effective targeting of mechanisms that control the loss of ER dependency in breast cancer remains elusive. We analyzed breast cancer–associated fibroblasts (CAF), the largest component of the tumor microenvironment, as a factor contributing to ER expression levels and antiendocrine resistance. Experimental Design: Tissues from patients with ER⁺ breast cancer were analyzed for the presence of CD146-positive (CD146^pos) and CD146-negative (CD146^neg) fibroblasts. ER-dependent proliferation and tamoxifen sensitivity were evaluated in ER⁺ tumor cells cocultured with CD146^pos or CD146^neg fibroblasts. RNA sequencing was used to develop a high-confidence gene signature that predicts for disease recurrence in tamoxifen-treated patients with ER⁺ breast cancer. Results: We demonstrate that ER⁺ breast cancers contain two CAF subtypes defined by CD146 expression. CD146^neg CAFs suppress ER expression in ER⁺ breast cancer cells, decrease tumor cell sensitivity to estrogen, and increase tumor cell resistance to tamoxifen therapy. Conversely, the presence of CD146^pos CAFs maintains ER expression in ER⁺ breast cancer cells and sustains estrogen-dependent proliferation and sensitivity to tamoxifen. Conditioned media from CD146^pos CAFs with tamoxifen-resistant breast cancer cells are sufficient to restore tamoxifen sensitivity. Gene expression profiles of patient breast tumors with predominantly CD146^neg CAFs correlate with inferior clinical response to tamoxifen and worse patient outcomes. Conclusions: Our data suggest that CAF composition contributes to treatment response and patient outcomes in ER⁺ breast cancer and should be considered a target for drug development. Clin Cancer Res; 23(7); 1710–21. ©2016 AACR.