6 July 2023 in IEEE Transactions on Artificial Intelligence
Though successful, federated learning (FL) presents new challenges for machine learning, especially when the issue of data heterogeneity, also known as Non-IID data, arises. To cope with the statistical heterogeneity, previous works incorporated a proximal term in local optimization or modified the model aggregation scheme at the server side or advocated clustered federated learning approaches where the central server groups agent population into clusters with jointly trainable data distributions to take the advantage of a certain level of personalization. While effective, they lack a deep elaboration on what kind of data heterogeneity and how the data heterogeneity impacts the accuracy performance of the participating clients. In contrast to many of the prior FL approaches, we demonstrate not only the issue of data heterogeneity in current setups is not necessarily a problem but also in fact it can be beneficial for the FL participants. Our observations are intuitive: (1) Dissimilar labels of clients (label skew) are not necessarily considered data heterogeneity, and (2) the principal angle between the clients' data subspaces spanned by their corresponding principal vectors of data is a better estimate of the data heterogeneity. Our code is available at https://github.com/MMorafah/FL-SC-NIID .
30 September 2019 in IEEE Access
Physical layer security offers an efficient means to decrease the risk of confidential information leakage through wiretap links. In this paper, we address the physical-layer security in a cooperative wireless subnetwork that includes a source-destination pair and multiple relays, exchanging information in the presence of a malevolent eavesdropper. Specifically, the eavesdropper is active in the network and transmits artificial noise (AN) with a multiple-antenna transmitter to confound both the relays and the destination. We first analyse the secrecy capacity of the direct source-to-destination transmission in terms of intercept probability (IP) and secrecy outage probability (SOP). A decode-andforward incremental relaying (IR) protocol is then introduced to improve security of communications in the presence of the active eavesdropper. Within this context, and depending on the availability of channel state information, three different schemes (one optimal and two sub-optimal) are proposed to select a trusted relay and improve the achievable secrecy rate. For each one of these schemes, and for both selection and maximum ratio combining at the destination and eavesdropper, we derive new and exact closed-form expressions for the IP and SOP. Our analysis and simulation results demonstrate the superior performance of the proposed IR-based selection schemes for secure communication. They also confirm the existence of a floor phenomenon for the SOP in the absence of AN.
11 August 2022 in IEEE Access
We propose a novel technique for dataset summarization by selecting representatives from a large, unsupervised dataset. The approach is based on the concept of self-rank , defined as the minimum number of samples needed to express all dataset samples with an accuracy proportional to the rank- $K$ approximation. As the exact computation of self-rank requires a computationally expensive combinatorial search, we propose an efficient algorithm that jointly estimates self-rank and selects the most informative samples in a linear order of complexity w.r.t the data size. We derive a new upper bound for the approximation ratio (AR), the ratio of obtained projection error using selected samples to the best rank- $K$ approximation error. The best previously known AR for self-representative low-rank approximation was presented in ICML 2017, which was further improved by the bound $\sqrt {1+K}$ reported in NeurIPS 2019. Both of these bounds are obtained by brute force search, which is not practical, and these bounds depend solely on $K$ , the number of selected samples. In contrast, we describe an adaptive AR that takes into consideration the spectral properties and spikiness measure of the original dataset, ${\boldsymbol {A}\in \mathbb {R}^{N\times M}}$ . In particular, our performance bound is proportional to the condition number $\kappa (\boldsymbol {A})$ . Our derived AR is expressed as $1+(\kappa (\boldsymbol {A})^{2}-1)/(N-K)$ , which approaches 1 and is optimal in two extreme spectral distribution instances. In the worst case, AR is shown not to exceed 1.25 for the proposed method. Our proposed algorithm enjoys linear complexity w.r.t. size of original dataset, which results in filling a historical gap between practical and theoretical methods in finding representatives. In addition to evaluating the proposed algorithm on a synthetic dataset, we show that it can be utilized in real-world applications such as graph node sampling for optimizing the shortest path criterion, learning a classifier with representative data, and open-set identification.