Remote sensing Change Detection (RSCD) is a critical research direction in the field. Recently, Vision foundation models (VFMs) (such as Fast Segment Anything Model, FastSAM) have demonstrated remarkable effectiveness in image segmentation tasks. SAM-CD employs a FastSAM-based encoder (pre-trained on large-scale vision datasets) to extract multi-scale features from bi-temporal images independently, followed by feature fusion prior to final interpretation, generating high-precision change detection results. However, current research in change detection has not adequately explored the utilization of features extracted from VFMs. Therefore, this study proposes the DDSAM-CD model, which primarily consists of a dense feature aggregation network (DFAN) and a differential enhancement network (DEN). The DFAN comprehensively fuses multi-scale features derived from FastSAM. In the DEN, bi-temporal features are first differentially fused to capture temporal change information. Subsequently, low-scale features extracted by FastSAM are incorporated and fused with the differential features through multi-scale feature integration. The optimized differential enhancement module significantly improves edge-region accuracy while resolving scale discrepancies, thereby delivering more reliable results for change detection tasks. The quantitative and qualitative analyses demonstrate that the proposed DDSAM model outperforms state-of-the-art RSCD methods on both the LEVIR-CD and WHU-CD datasets. Specifically, DDSAM achieved Precision, Recall, F1-score, and mIoU values of 93.79%, 90.97%, 92.38%, and 92.37% on LEVIR-CD, respectively; corresponding metrics on WHU-CD were 92.93%, 88.72%, 90.79%, and 91.24%. Additionally, the DDSAM model has achieved significant improvements in the accurate identification of edge regions.