26 August 2019 in IEEE Journal of Selected Topics in Signal Processing
2 December 2021 in IEEE Transactions on Wireless Communications
Federated learning (FL) is an attractive paradigm for making use of rich distributed data while protecting data privacy. Nonetheless, non-ideal communication links and limited transmission resources may hinder the implementation of fast and accurate FL. In this paper, we study joint optimization of communications and FL based on analog aggregation transmission in realistic wireless networks. We first derive closed-form expressions for the expected convergence rate of FL over the air, which theoretically quantify the impact of analog aggregation on FL. Based on the analytical results, we develop a joint optimization model for accurate FL implementation, which allows a parameter server to select a subset of workers and determine an appropriate power scaling factor. Since the practical setting of FL over the air encounters unobservable parameters, we reformulate the joint optimization of worker selection and power allocation using controlled approximation. Finally, we efficiently solve the resulting mixed-integer programming problem via a simple yet optimal finite-set search method by reducing the search space. Simulation results show that the proposed solutions developed for realistic wireless analog channels outperform a benchmark method, and achieve comparable performance of the ideal case where FL is implemented over error-free wireless channels.
5 October 2022 in IEEE Transactions on Wireless Communications
For distributed learning among collaborative users, this paper develops and analyzes a communication-efficient scheme for federated learning (FL) over the air, which incorporates 1-bit compressive sensing (CS) into analog-aggregation transmissions. To facilitate design parameter optimization, we analyze the efficacy of the proposed scheme by deriving a closed-form expression for the expected convergence rate. Our theoretical results unveil the tradeoff between convergence performance and communication efficiency as a result of the aggregation errors caused by sparsification, dimension reduction, quantization, signal reconstruction and noise. Then, we formulate a joint optimization problem to mitigate the impact of these aggregation errors through joint optimal design of worker scheduling and power scaling policy. An enumeration-based method is proposed to solve this non-convex problem, which is optimal but becomes computationally infeasible as the number of devices increases. For scalable computing, we resort to the alternating direction method of multipliers (ADMM) technique to develop an efficient implementation that is suitable for large-scale networks. Simulation results show that our proposed 1-bit CS based FL over the air achieves comparable performance to the ideal case where conventional FL without compression and quantification is applied over error-free aggregation, at much reduced communication overhead and transmission latency.
26 April 2024 in IEEE Transactions on Wireless Communications
Data-driven machine learning techniques have been advocated for signal detection in complex wireless environments. However, when applied to wideband spectrum sensing scenarios, they face practical challenges including very large data dimensionality, insufficient training data, and implicit inter-band dependencies. Current literature focuses on deep convolutional models, whose inherent model structure is not well suited for representing the diverse spectrum occupancy patterns of practical wideband networks, causing inefficient performance-complexity tradeoff and excessive sensing time. To address these issues, this paper develops a novel Spectrum Transformer with multi-task learning for wideband spectrum sensing at high sample efficiency. Empowered by the multi-head self-attention mechanism, the transformer architecture is designed to effectively learn both the inner-band spectral features and the inter-band spectrum occupancy correlations in the wideband regime. Simulations show that the proposed Spectrum Transformer outperforms the existing methods based on convolutional neural networks especially in the small-data case, by achieving higher sensing accuracy with an 89% reduction in model complexity.
1 April 2022 in IEEE Internet of Things Journal
As a promising distributed learning technology, analog aggregation-based federated learning over the air (FLOA) provides high communication efficiency and privacy provisioning under the edge computing paradigm. When all edge devices (workers) simultaneously upload their local updates to the parameter server (PS) through commonly shared time-frequency resources, the PS obtains the averaged update only rather than the individual local ones. While such a concurrent transmission and aggregation scheme reduces the latency and communication costs, it unfortunately renders FLOA vulnerable to Byzantine attacks. Aiming at Byzantine-resilient FLOA, this article starts from analyzing the channel inversion (CI) mechanism that is widely used for power control in FLOA. Our theoretical analysis indicates that although CI can achieve good learning performance in the benign scenarios, it fails to work well with limited defensive capability against Byzantine attacks. Then, we propose a novel scheme called the best effort voting (BEV) power control policy that is integrated with stochastic gradient descent (SGD). Our BEV-SGD enhances the robustness of FLOA to Byzantine attacks, by allowing all the workers to send their local updates at their maximum transmit power. Under worst-case attacks, we derive the expected convergence rates of FLOA with CI and BEV power control policies, respectively. The rate comparison reveals that our BEV-SGD outperforms its counterpart with CI in terms of better convergence behavior, which is verified by experimental simulations.
17 September 2018 in IEEE Signal Processing Letters
This letter presents a fast algorithm named iterative Vandermonde decomposition and shrinkage-thresholding (IVDST), which offers a low-complexity solution to atomic norm minimization (ANM) in off-grid compressed sensing for line spectral estimation from few measurements. It implements the ANM principle via the accelerated proximal gradient (APG) technique, without invoking computationally expensive semidefinite programming (SDP). To approximate the proximal operator in each APG iteration, Vandermonde decomposition is applied to utilize the Toeplitz structure inherent in the line spectral model, and the low-rank property of the Toeplitz-structured matrix is enforced via a simple shrinkage-thresholding operation. The IVDST algorithm effectively reduces the order of computational complexity compared to SDP-based solutions. It also offers an explicit way to bridge the ANM principle with classic super-resolution line spectral estimation algorithms, such as MUSIC.