Training Mixture-of-Experts (MoE) models introduces sparse and highly imbalanced all-to-all communication that dominates iteration time. Conventional load-balancing methods fail to exploit the deterministic topology of Rail architectures, leaving multi-NIC bandwidth underutilized. We present RailS, a distributed load-balancing framework that minimizes all-to-all completion time in MoE training. RailS leverages the Rail topology’s symmetry to prove that uniform sending ensures uniform receiving, transforming global coordination into local scheduling. Each node independently executes a Longest Processing Time First (LPT) spraying scheduler to proactively balance traffic using local information. RailS activates N parallel rails for fine-grained, topology-aware multipath transmission. Across synthetic and real-world MoE workloads, RailS improves bus bandwidth by 20%–78% and reduces completion time by 17%–78%. For Mixtral workloads, it shortens iteration time by 18%–40% and achieves near-optimal load balance, fully exploiting architectural parallelism in distributed training.

Clustered federated learning has garnered significant attention as an effective strategy for enhancing model performance in non-independent and identically distributed (non-IID) data scenarios. This approach improves model performance in such environments by calculating the similarity between users and clustering them into multiple groups. However, several challenges arise when implementing this method, particularly in balancing flexibility, communication costs, and model performance. To address these issues, this paper proposes a novel hierarchical federated learning framework that balances both network and model performance. The framework performs principal component analysis (PCA) on device-side image datasets to assess the similarity of private data across devices and, in conjunction with network performance measurements, dynamically adjusts communication strategies to minimize latency while ensuring stable model performance. By weighting similarity and communication metrics, the framework optimizes communication efficiency without significantly compromising model performance. To validate the proposed method’s effectiveness, we employed three publicly available datasets and compared it against four baseline methods. The experimental results demonstrate that SC-Fed (segmented clustering-federated learning) achieves a maximum accuracy improvement of 7.56% over baseline methods, while also reducing the average waiting time by 54.6%. These results indicate that the proposed algorithm significantly enhances the applicability and efficiency of clustered federated learning in practical training scenarios.

Distributed deep neural network (DNN) training in multi-tenant cloud environments generates long-lived, bandwidth-intensive, and interdependent flows that challenge traditional fairness-based network scheduling. We present Gsched, a framework that orchestrates centralized planning with hardware priority enforcement to realize communication interleaving, thereby eliminating contention and maximizing pipeline efficiency. Unlike methods relying on complex rate control or custom switch hardware, Gsched leverages standard commodity switch features like DSCP and SPQ to enforce strict priority scheduling. It employs a cold-start profiling mechanism to capture task characteristics without user intervention, and a centralized solver to compute the optimal interleaving order that maximizes Average Transmission Efficiency (ATE). Our analysis proves that this prioritization strategy deterministically drives the system to a contention-free state, while a local perturbation theorem guarantees robustness against distributed execution conflicts via network backpressure. Across workloads, Gsched achieves performance close to the state-of-the-art Crux under heterogeneous workloads and remains competitive in highly diverse cases. More importantly, Gsched provides a theoretical foundation that justifies scalable per-bottleneck scheduling in multi-bottleneck networks.

Federated learning (FL) is being increasingly adopted in consumer-electronics (CE)-enabled vehicular networks to support privacy-preserving model training. However, the performance of existing FL frameworks significantly degrades under practical heterogeneous conditions, where devices differ in data distribution, computing capabilities, and wireless link quality. To address these challenges, we develop a delay-aware system model that quantifies per-CE computation and communication costs, and formulates a resource-constrained integer programming strategy for CE selection. Building on this model, we propose FedSA-CE, a selection-aware clustering framework tailored for personalized FL in dynamic vehicular environments. FedSA-CE extracts compact CE representations using truncated Singular Value Decomposition (SVD), groups CEs based on cosine similarity, and adaptively selects participants via a contribution– latency objective with time-decaying weighting. The framework prioritizes CEs with rapidly changing gradients in early training phases to accelerate convergence and gradually incorporates more diverse CEs in later stages to enhance generalization while maintaining acceptable waiting time. A similarity-weighted personalized aggregation scheme further improves local model adaptation. Experiments on 4 benchmark datasets demonstrate that FedSA-CE achieves an average accuracy improvement of 24.7% and reduces average waiting time by 83.0% compared with 7 state-of-the-art baselines. Moreover, under all extreme non-independent and identically distributed (Non-IID) settings, FedSA-CE yields an average minimum accuracy gain of 2.92% over the best-performing competing method across all datasets.