Edge devices employing federated learning encounter several obstacles, including (1) the non-independent and identically distributed (Non-IID) nature of client data, (2) limitations due to communication bottlenecks, and (3) constraints on computational resources. To surmount the Non-IID data challenge, personalized federated learning has been introduced, which involves training tailored networks at the edge; nevertheless, these methods often exhibit inconsistency in performance. In response to these concerns, a novel framework for personalized federated learning that incorporates adaptive pruning of edge-side data is proposed in this paper. This approach, through a two-staged pruning process, creates customized models while ensuring strong generalization capabilities. Concurrently, by utilizing sparse models, it significantly condenses the model parameters, markedly diminishing both the computational burden and communication overhead on edge nodes. This method achieves a remarkable compression ratio of 3.7% on the Non-IID dataset FEMNIST, with the training accuracy remaining nearly unaffected. Furthermore, the total training duration is reduced by 46.4% when compared with the standard baseline method.
As the size of deep learning models continues to expand, the elongation of inference time has gradually evolved into a significant challenge to efficiency and practicality for autoregressive models. This work introduces a hybrid model acceleration strategy based on branch prediction, which accelerates autoregressive model inference without requiring retraining and ensures output consistency with the original model. Specifically, the algorithm employs two models with different parameter sizes aimed at the same task. The smaller model generates a series of potential tokens that are then parallelly validated by the larger model to determine their acceptability. By orchestrating the workflow of the large and small models through a branch-prediction strategy, the algorithm conceals the validation time of the larger model when predictions are successful, thereby accelerating inference. We propose a binomial distribution-based prediction function that blends theoretical principles with empirical evidence, specifically designed for the nuanced requirements of accelerating inference within a hybrid model framework. The entire algorithm was designed and implemented on the llama model for text generation and translation tasks. The experimental results indicate significant improvements. The proposed algorithm achieves a 1.2× to 3.4× increase in inference speed compared to the original model, consistently outperforming the speculative sampling inference acceleration algorithm.
Federated Learning serves as a distributed framework for machine learning. Traditional approaches to federated learning often assume the independence and identical distribution (IID) of client data. However, real-world scenarios frequently feature personalized characteristics in client data, deviating from the IID assumption. Additionally, challenges such as substantial communication overhead and limited resources at edge nodes hinder the practical implementation of federated learning. In response to the challenges in deploying federated learning, including uneven data distribution, communication bottlenecks, and resource limitations at edge nodes, this paper introduces an individualized federated learning framework based on model pruning. This framework effectively adapts the client's local model to the personalized distribution of local data while meeting the model aggregation requirements on the server. Utilizing sparse operations, the framework achieves personalized model pruning, efficiently compresses model parameters, and reduces computational load on edge nodes. Presently, our approach demonstrates a compression ratio of 3.8% on the non-IID dataset Feminist without compromising final training accuracy, resulting in a 12.3% acceleration in training speed.