23 April 2021 in IEEE Transactions on Mobile Computing
The emerging Federated Learning (FL) enables IoT devices to collaboratively learn a shared model based on their local datasets. However, due to end devices’ heterogeneity, it will magnify the inherent synchronization barrier issue of FL and result in non-negligible waiting time when local models are trained with the identical batch size. Moreover, the useless waiting time will further lead to a great strain on devices’ limited battery life. Herein, we aim to alleviate the negative impact of synchronization barrier through adaptive batch size during model training. When using different batch sizes, stability and convergence of the global model should be enforced by assigning appropriate learning rates on different devices. Therefore, we first study the relationship between batch size and learning rate, and formulate a scaling rule to guide the setting of learning rate in terms of batch size. Then we theoretically analyze the convergence rate of global model and obtain a convergence upper bound. On these bases, we propose an efficient algorithm that adaptively adjusts batch size with scaled learning rate for heterogeneous devices to reduce the waiting time and save battery life. We conduct extensive simulations and testbed experiments, and the experimental results demonstrate the effectiveness of our method.
19 October 2021 in IEEE Journal on Selected Areas in Communications
Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing. To fully utilize the widely distributed data, we concentrate on a wireless edge computing system that conducts model training using decentralized peer-to-peer (P2P) methods. However, there are two major challenges on the way towards efficient P2P model training: limited resources (e.g., network bandwidth and battery life of mobile devices) and time-varying network connectivity due to device mobility or wireless channel dynamics, which receives less attention in recent years. To address these two challenges, this paper studies the impact of topology construction on the P2P training performance. Specifically, we dynamically construct an efficient P2P topology, where model aggregation occurs at the edge. In a nutshell, we first formulate the topology construction for P2P learning (TCPL) problem with resource constraints as an integer programming problem. Then a learning-driven method is proposed to adaptively construct a topology at each training epoch. We evaluate the performance of our proposed algorithm through extensive simulations and physical platform. Evaluation results show that our method can improve the model training efficiency by about 11% with resource constraints, reduce the communication cost by 30% and the network traffic consumption by about 60% under the same accuracy requirement compared to the benchmarks.
17 December 2020 in IEEE Transactions on Mobile Computing
Under the paradigm of edge computing, the enormous data generated at the network edge can be processed locally. To make full utilization of these widely distributed data, we focus on an edge computing system that conducts distributed machine learning using gradient-descent based approaches. To ensure the system’s performance, there are two major challenges: how to collect data from multiple data source nodes for training jobs and how to allocate the limited resources on each edge server among these jobs. In this paper, we jointly consider the two challenges for distributed training (without service requirement), aiming to maximize the system throughput while ensuring the system’s quality of service (QoS). Specifically, we formulate the joint problem as a mixed-integer non-linear program, which is NP-hard, and propose an efficient approximation algorithm. Furthermore, we take service placement into consideration for diverse training jobs and propose an approximation algorithm. We also analyze that our proposed algorithm can achieve the constant bipartite approximation under many practical situations. We build a test-bed to evaluate the effectiveness of our proposed algorithm in a practical scenario. Extensive simulation results and testing results show that the proposed algorithms can improve the system throughput 56-69 percent compared with the conventional algorithms.
24 September 2018 in IEEE Journal on Selected Areas in Communications
In [1] , reference [16] should be replaced as follows: [16] P. Chao, Y. Tan, and L. T. Yang, “New algorithms for the minimum-cost single-source unsplittable flow problem,” in Proc. 21st Int. Conf. Adv. Inf. Netw. Appl. Workshops (AINAW) , Niagara Falls, ON, Canada, May 2007.