In federated cross-domain recommendation, the existing research has widely focused on scenarios where at least some users or items overlap. How to enable cross-domain recommendation over non-overlapping users and items has emerged as a challenging research direction. However, current privacy-preserving cross-domain recommendation methods struggle to achieve effective semantic alignment across domains in such scenarios, hindering their ability to capture latent correlations between interaction graphs. Moreover, when refining noisy node representations and interaction sequences, these methods often overlook temporal noise implicitly embedded in semantic information, as well as the potential of jointly leveraging textual and temporal features for collaborative denoising. To address these challenges, this paper proposes a Semantic Federated Clustering-driven Cross-Domain Recommendation model (SFC-CDR). Specifically, we first design a clustering-anchored contrastive learning framework that performs clustering using weighted multi-layer node semantic representations. The weighted aggregated centroids serve as semantic anchors to dynamically update the pool of positive and negative samples in contrastive learning, thereby achieving geometric alignment in the shallow feature space and capturing fine-grained semantic correlations in the implicit concept space. Furthermore, we design a Joint Temporal–Frequency Filtering (JTFF) module. In the frequency domain, features are processed via an MoE adapter and FFT-based filtering, while in the temporal domain, a temporal convolutional network (TCN) is employed to model temporal patterns in user–item interaction sequences. The resulting temporal and frequency-domain representations are integrated through Temporal–Frequency Refinement Gating (TFRG), enabling complementary refinement of node semantic representations and user behavior sequences. Extensive experiments conducted on four real-world datasets demonstrate that SFC-CDR outperforms state-of-the-art baselines.