The issue of potential privacy leakage during centralized AIs model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally without uploading their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The selection policy is critical to an FL process in terms of round efficiency, the final models quality as well as fairness. In this paper, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C2MAB-based method is proposed for estimating the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. In addition to the theoretical results, more empirical ones can be derived from our real training experiments on public datasets.

Federated learning (FL), arising as a privacy-preserving machine learning paradigm, has received notable attention from the public. In each round of synchronous FL training, only a fraction of available clients are chosen to participate, and the selection decision might have a significant effect on the training efficiency, as well as the final model performance. In this article, we investigate the client selection problem under a volatile context, in which the local training of heterogeneous clients is likely to fail due to various kinds of reasons and in different levels of frequency. Intuitively, too much training failure might potentially reduce the training efficiency, while too much selection on clients with greater stability might introduce bias, thereby resulting in degradation of the training effectiveness. To tackle this tradeoff, we, in this article, formulate the client selection problem under joint consideration of effective participation and fairness. Furthermore, we propose E3CS, a stochastic client selection scheme as a solution. According to our experimental results over a public data set, the proposed selection scheme is able to achieve up to $2\times $ faster convergence to a fixed model accuracy while maintaining the same level of final model accuracy, compared with the state-of-the-art selection schemes.

Federated learning (FL) has shown great potential as a privacy-preserving solution to training a centralized model based on local data from available clients. However, we argue that, over the course of training, the available clients may exhibit some volatility in terms of the client population, client data, and training status. Considering these volatilities, we propose a new learning scenario termed volatile federated learning (volatile FL) featuring set volatility, statistical volatility, and training volatility. The volatile client set along with the dynamic of clients’ data and the unreliable nature of clients (e.g., unintentional shutdown and network instability) greatly increase the difficulty of client selection. In this article, we formulate and decompose the global problem into two subproblems based on alternating minimization. For an efficient settlement for the proposed selection problem, we quantify the impact of clients’ data and resource heterogeneity for volatile FL and introduce the cumulative effective participation data (CEPD) as an optimization objective. Based on this, we propose upper confidence bound-based greedy selection, dubbed UCB-GS, to address the client selection problem in volatile FL. Theoretically, we prove that the regret of UCB-GS is strictly bounded by a finite constant, justifying its theoretical feasibility. Furthermore, experimental results show that our method significantly reduces the number of training rounds (by up to 62%) while increasing the global model’s accuracy by 7.51%.

Language models (LMs) have demonstrated superior performance in detecting fraudulent activities on Blockchains. Nonetheless, the sheer volume of Blockchain data results in excessive memory and computational costs when training LMs from scratch, limiting their capabilities to large-scale applications. In this paper, we present ZipZap, a framework tailored to achieve both parameter and computational efficiency when training LMs on large-scale transaction data. First, with the frequency-aware compression, an LM can be compressed down to a mere 7.5% of its initial size with an imperceptible performance dip. This technique correlates the embedding dimension of an address with its occurrence frequency in the dataset, motivated by the observation that embeddings of low-frequency addresses are insufficiently trained and thus negating the need for a uniformly large dimension for knowledge representation. Second, ZipZap accelerates the speed through the asymmetric training paradigm: It performs transaction dropping and cross-layer parameter-sharing to expedite the pre-training process, while revert to the standard training paradigm for fine-tuning to strike a balance between efficiency and efficacy, motivated by the observation that the optimization goals of pre-training and fine-tuning are inconsistent. Evaluations on real-world, large-scale datasets demonstrate that ZipZap delivers notable parameter and computational efficiency improvements for training LMs. Our implementation is available at: https://github.com/git-disl/ZipZap.

As a new computing paradigm, mobile edge computing (MEC) pushes the centralized cloud resources close to the edge network, which significantly reduces the pressure of the backbone network and meets the requirements of emerging mobile applications. To achieve high performance of the MEC system, it is essential to design efficient task offloading and service placement schemes, which are responsible for offloading tasks to the edge servers while considering the heterogeneity and diversity of computation services. Our MEC system aims to maximize the long-term average network utility while maintaining the stability of the edge network. Considering that synchronous manner overlooks the scenarios endowed with asymmetric update frequencies for service placement and task offloading, we propose an online algorithm based on the two-timescale Lyapunov optimization in a stochastic network environment without requiring the future information. By making asynchronous decisions on service placement and task offloading with different control parameters $V$ , we can achieve a time-average sub-optimal solution that is close to the offline optimum. In addition, we introduce the varying control parameter $V(t)$ and $\Omega$ -additive approximation to enhance the robustness of the proposed algorithm within an error $\Omega$ . Finally, rigorous theoretical analysis and extensive trace-driven experimental results show that the proposed algorithm achieves the $[O(1/V), O(V)]$ performance-backlog tradeoff and is more competitive than benchmarks.