17 November 2020 in IEEE/ACM Transactions on Networking
There is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL), in which the model training is distributed over mobile user equipment (UEs), exploiting UEs' local computation and training data. Despite its advantages such as preserving data privacy, FL still has challenges of heterogeneity across UEs' data and physical resources. To address these challenges, we first propose FEDL, a FL algorithm which can handle heterogeneous UE data without further assumptions except strongly convex and smooth loss functions. We provide a convergence rate characterizing the trade-off between local computation rounds of each UE to update its local model and global communication rounds to update the FL global model. We then employ FEDL in wireless networks as a resource allocation optimization problem that captures the trade-off between FEDL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FEDL is non-convex, we exploit this problem's structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights into problem design. Finally, we empirically evaluate the convergence of FEDL with PyTorch experiments, and provide extensive numerical results for the wireless resource allocation sub-problems. Experimental results show that FEDL outperforms the vanilla FedAvg algorithm in terms of convergence rate and test accuracy in various settings.
17 July 2017 in IEEE Journal on Selected Areas in Communications
This paper investigates the optimal resource allocation of a downlink non-orthogonal multiple access (NOMA) system consisting of one base station and multiple users. Unlike existing short-term NOMA designs that focused on the resource allocation for only the current transmission timeslot, we aim to maximize a long-term network utility by jointly optimizing the data rate control at the network layer and the power allocation among multiple users at the physical layer, subject to practical constraints on both the short-term and long-term power consumptions. To solve this problem, we leverage the recently developed Lyapunov optimization framework to convert the original long-term optimization problem into a series of online rate control and power allocation problems in each timeslot. The power allocation problem, however, is shown to be non-convex in nature and thus cannot be solved with a standard method. However, we explore two structures of the optimal solution and develop a dynamic programming-based power allocation algorithm, which can derive a globally optimal solution, with a polynomial computational complexity. Extensive simulation results are provided to evaluate the performance of the proposed joint rate control and power allocation framework for NOMA systems, which demonstrate that the proposed NOMA design can significantly outperform multiple benchmark schemes, including orthogonal multiple access schemes with optimal power allocation and NOMA schemes with non-optimal power allocation, in terms of average throughput and data delay.
25 October 2018 in IEEE Transactions on Communications
A promising solution to address the spectrum shortage in 5G cellular systems is the deployment of millimeter wave (mmWave) heterogeneous networks (HetNets). However, a key challenge for mmWave HetNets is to manage the user mobility and handovers among mmWave small cells exploiting highly directional antennas and conventional microwave macro cells. In this paper, we propose a new efficient handover decision algorithm based on a Markov Decision Process (MDP) to optimize the overall service experience of users in mmWave HetNets. By utilizing user’s mobility information (velocity and location), the proposed algorithm avoids excessive handovers and effectively tackles beamforming misalignments and signal blockages in mmWave small cells. To improve the computational efficiency of the MDP, we apply the action elimination method by exploiting unique handover properties of mmWave HetNets. While maintaining optimality, theoretical analysis shows that our proposed handover decision algorithm can reduce the computational complexity by 0.25M|Bs| + 0.25M=(M – 1) times where M is the number of base stations and |Bs| is the number of beamwidth options for mmWave beamforming. Mobility-trace driven numerical results demonstrate the optimality of our proposed algorithm compared with other benchmark schemes and its low computational complexity over the traditional approach.
14 September 2022 in IEEE Transactions on Parallel and Distributed Systems
Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized and FL environments. It is well-known that Nesterov Accelerated Gradient (NAG) is a more advantageous form of momentum, but it is not clear how to quantify the benefits of NAG in FL so far. This motives us to propose FedNAG, which employs NAG in each worker as well as NAG momentum and model aggregation in the aggregator. We provide a detailed convergence analysis of FedNAG and compare it with FedAvg. Extensive experiments based on real-world datasets and trace-driven simulation are conducted, demonstrating that FedNAG increases the learning accuracy by 3–24% and decreases the total training time by 11–70% compared with the benchmarks under a wide range of settings.
11 July 2024 in IEEE/ACM Transactions on Networking
With the proliferation of the Internet of Things (IoT) and the rising interconnectedness of devices, network security faces significant challenges, especially from anomalous activities. While traditional machine learning-based intrusion detection systems (ML-IDS) effectively employ supervised learning methods, they possess limitations such as the requirement for labeled data and challenges with high dimensionality. Recent unsupervised ML-IDS approaches such as AutoEncoders and Generative Adversarial Networks (GAN) offer alternative solutions but pose challenges in deployment onto resource-constrained IoT devices and in interpretability. To address these concerns, this paper proposes a novel federated unsupervised anomaly detection framework – FedPCA – that leverages Principal Component Analysis (PCA) and the Alternating Directions Method Multipliers (ADMM) to learn common representations of distributed non-i.i.d. datasets. Building on the FedPCA framework, we propose two algorithms, FedPE in Euclidean space and FedPG on Grassmann manifolds. Our approach enables real-time threat detection and mitigation at the device level, enhancing network resilience while ensuring privacy. Moreover, the proposed algorithms are accompanied by theoretical convergence rates even under a sub-sampling scheme, a novel result. Experimental results on the UNSW-NB15 and TON-IoT datasets show that our proposed methods offer performance in anomaly detection comparable to non-linear baselines, while providing significant improvements in communication and memory efficiency, underscoring their potential for securing IoT networks.
3 September 2021 in IEEE Transactions on Multimedia
To deal with noises in partial label learning (PLL), existing approaches try to perform disambiguation either by identifying the ground-truth label or by averaging the candidate labels. However, these methods can be easily misled by the false-positive noisy labels in the candidate set, and fail to generalize well in testing. When labeling information is ambiguous, learning paradigms should depend more on the underlying data structure. Large margin nearest neighbour (LMNN) is a popular strategy to consider instance and class correlations in supervised learning, but can not be directly used in weakly-supervised PLL due to the ambiguity of labeling information. In this paper, we propose a novel Generalized Large Margin kNN for Partial Label Learning (GLMNN-PLL), which employs the principle of LMNN to adapt the framework to PLL by generalizing the constraint from the same class to similarly-labeled. Generally, GLMNNPLL aims to learn a new metric and perform disambiguation by reorganizing the underlying data structure, that is, making similarly labeled instances closer to each other while making differently labeled instances separated by a large margin. As two close instances with shared labels do not necessarily belong to the same class, we put a weight on each instance pair. An efficient algorithm is designed to optimize the proposed method and the convergence is analyzed in this paper. Moreover, we present a theoretical analysis of the generalization error bound for GLMNN-PLL. Comprehensive experiments on controlled UCI datasets as well as real-world partial label datasets from various domains demonstrate the superiorities of the proposed method.