28 May 2025 in IEEE Journal on Selected Areas in Communications
Industrial Cyber-Physical Systems (ICPS) have emerged as a critical component in the industrial domain. To facilitate seamless collaboration among massive devices, cloud-edge computing architectures have emerged as a key enabler for ICPS, leveraging distributed intelligence to orchestrate devices and computational tasks. In cloud-edge computing, efficient task offloading and resource management are essential for optimizing task performance and reducing energy costs. However, conventional centralized resource management strategies struggle to satisfy the real-time, adaptability, and performance demands of dynamic ICPS systems. Industrial Cyber-Physical Systems (ICPS) have emerged as a critical component in the industrial domain. To facilitate seamless collaboration among massive devices, cloud-edge computing architectures have emerged as a key enabler for ICPS, leveraging distributed intelligence to orchestrate devices and computational tasks. In cloud-edge computing, efficient task offloading and resource management are essential for optimizing task performance and reducing energy costs. However, conventional centralized resource management strategies struggle to satisfy the real-time, adaptability, and performance demands of dynamic ICPS systems. In this paper, we propose the Distributed Transformer-based Actor-Critic (DTAC) algorithm to jointly determine task offloading and resource management decisions in cloud-edge computing networks, particularly for delay-sensitive applications in ICPS. The DTAC algorithm integrates the powerful transformer model with the popular actor-critic architecture to address the challenge of a hybrid high-dimensional action space. We first train a centralized model to learn coordination among user equipments (UEs) and then introduce a decentralized transfer learning (TL) approach to efficiently adapt the centralized model into the DTAC framework. Using the DTAC model, each UE can independently manage its local resources based solely on local information, avoiding the significant signaling overhead inherent in centralized approaches. Simulation results demonstrate that DTAC not only outperforms other MARL and TL schemes in both small-and large-scale scenarios, but also exhibits strong generalization capabilities in inexperienced settings. Furthermore, DTAC and decentralized TL approaches significantly reduce training costs by 73% compared to other methods, making them more practical for ICPS deployment.
24 January 2024 in IEEE Transactions on Wireless Communications
In the IEEE 802.11ax Wireless Local Area Networks (WLANs), Orthogonal Frequency Division Multiple Access (OFDMA) has been applied to enable the high-throughput WLAN amendment. However, with the growth of the number of devices, it is difficult for the Access Point (AP) to schedule uplink transmissions, which calls for an efficient access mechanism in the OFDMA uplink system. Based on Multi-Agent Proximal Policy Optimization (MAPPO), we propose a Mean-Field Multi-Agent Proximal Policy Optimization (MFMAPPO) algorithm to improve the throughput and guarantee the fairness. Motivated by the Mean-Field games (MFGs) theory, a novel global state and action design are proposed to ensure the convergence of MFMAPPO in the massive access scenario. The Multi-Critic Single-Policy (MCSP) architecture is deployed in the proposed MFMAPPO so that each agent can learn the optimal channel access strategy to improve the throughput while satisfying fairness requirement. Extensive simulation experiments are performed to show that the MFMAPPO algorithm 1) has low computational complexity that increases linearly with respect to the number of stations 2) achieves nearly optimal throughput and fairness performance in the massive access scenario, 3) can adapt to various diverse and dynamic traffic conditions without retraining, as well as the traffic condition different from training traffic.
22 July 2024 in IEEE Internet of Things Journal
With the development of wireless communication systems, the large-scale deployment of Internet of Things (IoT) devices becomes popular. Due to limited energy, the multiple access approaches without carrier sensing requirement are widely deployed in IoT devices, including Aloha and time-division multiple access (TDMA). However, these approaches encounter the transmission inefficiency issue, especially in dynamic heterogeneous networks comprising nodes with diverse protocols and varying numbers and transmission configurations over time. In this article, combining curriculum learning (CL) and multitask reinforcement learning (MTRL), we propose the curriculum multitask heterogeneous-agent proximal policy optimization (CMHA) algorithm to improve the throughput performance while guaranteeing fairness in dynamic heterogeneous networks. We introduce the elastic weight consolidation (EWC) in the CMHA to further enhance generalization capacity, which can better address the challenging MTRL problem in dynamic heterogeneous networks. Combining the monotonic improvement feature of heterogeneous-agent proximal policy optimization (HAPPO) and the generalization capacity of EWC, the proposed CMHA can achieve a nearly monotonic improvement in all possible scenarios. The simulations show that the CMHA algorithm 1 ) has sufficient generalization capacity for massive scenarios in dynamic heterogeneous networks; 2) can significantly enhance the network throughput; and 3) can guarantee the fairness of both agents and heterogeneous nodes.
28 January 2025 in IEEE Open Journal of the Communications Society
The Multi-Link Operation (MLO) is a key technique in the upcoming IEEE 802.11be Extremely High Throughput standard, also known as Wi-Fi 7. MLO enables Multi-Link Devices (MLDs) to utilize multiple communication links simultaneously, thereby enhancing throughput. However, research has shown that MLO performance deteriorates in heterogeneous networks coexisting with Single-Link Devices (SLDs), especially in asymmetric scenarios. In this paper, we address the multiple access problem for MLDs in both homogeneous and heterogeneous dynamic uplink networks. We propose a Multi-Player Combinatorial Upper Confidence Bound (MP-CUCB) algorithm designed for Simultaneous Transmit and Receive (STR) MLDs to better exploit the potential of MLO. The MP-CUCB algorithm enables each MLD to independently make access decisions to improve throughput while guaranteeing the fairness requirements of SLDs. Simulation results demonstrate that the MP-CUCB algorithm can: 1) better utilize the potential of MLO, achieving a 20% improvement compared to the standard CSMA scheme while ensuring fairness requirements for both MLDs and SLDs; 2) address the severe performance degradation issue in asymmetric heterogeneous scenarios, achieving a 33% improvement compared to the standard CSMA; 3) effectively adapt to dynamic networks with new accessing nodes, achieving short convergence times.
8 April 2025 in IEEE Transactions on Mobile Computing
Next-generation multiple access techniques are crucial for providing low-latency and highly efficient data transmission services. Recently, Deep Reinforcement Learning (DRL) has emerged as a prevalent approach in the multiple access domain, aiming to facilitate user coordination and enhance transmission efficiency. However, current DRL approaches face challenges, including limited generalization ability, low sample efficiency, and the complexities associated with Partially Observable Markov Decision Processes (POMDP), which hinder their application in heterogeneous networks with varying numbers of nodes and configurations. In this paper, we propose a foundation model-based multiple access (FMA) algorithm. To address severe POMDP and sample inefficiency issues, we decompose the multiple access problem into two parts: a transmission decision part and a configuration estimation part. We leverage the strong generalization and inference capabilities of the foundation model, utilizing a Deep Learning (DL) approach instead of DRL for training, and adopt the Low-Rank Adaptation (LoRA) technique to fine-tune the foundation model for downstream multiple access tasks. Simulation results demonstrate that: 1) through the decomposition, the FMA approach exhibits sufficient generalization and inference abilities to adapt to various scenarios with various protocols, configurations, and numbers of heterogeneous nodes; 2) by incorporating expert knowledge, the FMA approach can significantly enhance network performance while ensuring certain fairness requirement for heterogeneous nodes.
20 January 2026 in IEEE Wireless Communications Letters
The exponential growth of heterogeneous mobile applications has raised challenges in current Radio Access Networks (RAN). While the network slicing technique offers virtualized networks with independent resources to satisfy diverse Quality of Service (QoS) requirements, persistent challenges remain in dynamic resource allocation and intent-driven slice configuration. For resource allocation, conventional Deep Reinforcement Learning (DRL)-based approaches encounter challenges in addressing the high-dimensional state-action space in large-scale networks with massive Base Stations (BS) and users. For slice configuration, current static intent-driven slice management frameworks suffer from inflexible architectures that fail to adaptively optimize network slicing configurations. In this letter, we propose a multi-Large Language Model (LLM) cooperation-based Joint Network Slicing and Resource Allocation (JNSRA) algorithm to jointly optimize the configurations of intent-driven network slices and resource allocation in large-scale multi-BS networks. In JNSRA, we propose to train Agent LLM through LLM alignment, which can enhance cooperation among slices and BS by regarding joint resource allocation actions as sequences. Moreover, we propose a multi-LLM joint optimization framework to jointly optimize Agent, leading to enhanced network slicing and resource allocation. Simulation results illustrate that JNSRA outperforms other DRL and heuristic approaches, and the proposed multi-LLM collaboration can further enhance the reward, satisfaction ratio and throughput.