10 June 2022 in IEEE Journal on Selected Areas in Communications
This paper studies the joint device selection and power control scheme for wireless federated learning (FL), considering both the downlink and uplink communications between the parameter server (PS) and the terminal devices. In each round of model training, the PS first broadcasts the global model to the terminal devices in an analog fashion, and then the terminal devices perform local training and upload the updated model parameters to the PS via over-the-air computation (AirComp). First, we propose an AirComp-based adaptive reweighing scheme for the aggregation of local updated models, where the model aggregation weights are directly determined by the uplink transmit power values of the selected devices and which enables the joint learning and communication optimization simply by the device selection and power control. Furthermore, we provide a convergence analysis for the proposed wireless FL algorithm and the upper bound on the expected optimality gap between the expected and optimal global loss values is derived. With instantaneous channel state information (CSI), we formulate the optimality gap minimization problems under both the individual and sum uplink transmit power constraints, respectively, which are shown to be solved by the semidefinite programming (SDR) technique. Numerical results reveal that our proposed wireless FL algorithm achieves close to the best performance by using the ideal FedAvg scheme with error-free model exchange and full device participation.
7 June 2023 in IEEE Internet of Things Journal
This article considers a smart-duplex (SD) powered user-centric ultra dense networks (UC-UDNs), where each user is served cooperatively by multiple access points (APs) adopting the de-cellular concept to achieve desired Quality-of-Service (QoS). The average QoS satisfaction ratio maximization problem for the considered SD UC-UDN is formulated as a Markov decision process (MDP) with large discrete action space by designing the user association and power allocation. To reduce the action space, user association and power allocation are modeled as a two-layer tree, and selecting an action for each user is equivalent to finding the path from the root to one leaf of the constructed tree. Then, a multiagent tree-structured policy gradient (MATSPG)-based deep reinforcement learning (DRL) algorithm is proposed to solve the MDP problem, whose training process is shown to be equivalent to that of the two-layer neural networks. Next, the time and space complexity of searching one action in the proposed MATSPG are also proved to be lower than the conventional DRL algorithms. Finally, simulations show that the proposed MATSPG algorithm significantly improves the average QoS satisfaction ratio than the conventional multiagent deep deterministic policy gradient and multiagent deep Q-network methods in typical scenarios.
13 July 2023 in IEEE Internet of Things Journal
We investigate the coexistence of task-oriented and data-oriented communications in an Internet of Things system that shares a group of channels, and study the scheduling problem to jointly optimize the weighted Age of Incorrect Information (AoII) and throughput, which are the performance metrics of the two types of communications, respectively. This problem is formulated as a Markov decision problem, which is difficult to solve due to the large discrete action space and the time-varying action constraints induced by the stochastic availability of channels. By exploiting the intrinsic properties of this problem and reformulating the reward function based on channel statistics, we first simplify the solution space, state space, and optimality criteria, and convert it to an equivalent Markov game, for which the large discrete action space issue is greatly relieved. Then, we propose Whittle’s index guided multiagent proximal policy optimization (WI-MAPPO) algorithm to solve the considered game, where the embedded Whittle’s index module further shrinks the action space, and the proposed offline training algorithm extends the training kernel of conventional multiagent proximal policy optimization to address the issue of time-varying constraints. Finally, numerical results validate that the proposed algorithm significantly outperforms state-of-the-art Age of Information (AoI)-based algorithms under scenarios with insufficient channel resources.
13 February 2025 in IEEE Internet of Things Journal
Mobile edge caching (MEC) is a promising technique to improve the quality of service (QoS) for mobile users (MU) by bringing data to the network edge. However, optimizing the crucial QoS aspects of message freshness and service promptness, measured by age of information (AoI) and service delay, respectively, entails a tradeoff due to their competition for shared edge resources. This article investigates this tradeoff by formulating their weighted sum minimization as a sequential decision-making problem, incorporating high-dimensional, discrete-valued, and linearly constrained design variables. First, to assess the feasibility of the considered problem, we characterize the corresponding achievable region by deriving its superset with the rate stability theorem and its subset with a novel stochastic policy, and develop a sufficient condition for the existence of solutions. Next, to efficiently solve this problem, we propose a mixed-order drift-plus-penalty algorithm by jointly considering the linear and quadratic Lyapunov drifts and then optimizing them with dynamic programming (DP). Finally, by leveraging the Lyapunov optimization technique, we demonstrate that the proposed algorithm achieves an $O(1/V)$ versus $O(V)$ tradeoff for the average AoI and average service delay.
9 December 2024 in IEEE Transactions on Mobile Computing
Multicasting is an efficient technique for simultaneously transmitting common messages from the base station (BS) to multiple mobile users (MUs). Multicast scheduling over multiple channels, which aims to jointly minimize the energy consumption of the BS and the latency of serving asynchronized requests from the MUs, is formulated as an infinite-horizon Markov decision process (MDP) problem with a large discrete action space, multiple time-varying constraints, and multiple time-invariant constraints. To address these challenges, this paper proposes a novel distribution-embedding multi-agent proximal policy optimization (DE-MAPPO) algorithm, which consists of one modified MAPPO and one distribution-embedding module: The former one handles the large discrete action space and timevarying constraints by modifying the structure of the actor networks and the training kernel of the conventional MAPPO; and the latter one iteratively adjusts the action distribution to satisfy the time-invariant constraints. Moreover, a performance upper bound of the considered MDP is derived by solving a two-step optimization problem. Finally, numerical results demonstrate that our proposed algorithm outperforms the existing ones in terms of applicability, effectiveness, and robustness, and achieves comparable performance to the derived upper bound
9 October 2023 in IEEE Internet of Things Journal
This paper considers the problem of distributed estimation in wireless sensor networks (WSN), which is anticipated to support a wide range of applications such as the environmental monitoring, weather forecasting, and location estimation. To this end, we propose a joint model and data driven distributed estimation method by designing the optimal quantizers and fusion center (FC) based on the Bayesian and minimum mean square error (MMSE) criterions. First, universal mean square error (MSE) lower bound for the quantization-based distributed estimation is derived and adopted as the design metric for the quantizers. Then, the optimality of the mean-fusion operation for the FC with MMSE criterion is proved. Next, by exploiting different levels of the statistic information of the desired parameter and observation noise, a joint model and data driven method is proposed to train parts of the quantizer and FC modules as deep neural networks (DNNs), and two loss functions derived from the MMSE criterion are adopted for the sequential training scheme. Furthermore, we extend the above results to the case with multi-bit quantizers, considering both the parallel and one-hot quantization schemes. Finally, simulation results reveal that the proposed method outperforms the state-of-the-art schemes in typical scenarios.