Automatically learning medium access control (MAC) communication protocols via multi-agent reinforcement learning (MARL) has received huge attention to cater to the extremely diverse real-world scenarios expected in 6G wireless networks. Several state-of-the-art solutions adopt the centralized training with decentralized execution (CTDE) learning method, where agents learn optimal MAC protocols by exploiting the information exchanged with a central unit. Despite the promising results achieved in these works, two notable challenges are neglected. First, these works were designed to be trained in computer simulations assuming an omniscient environment and neglecting communication overhead issues, thus making the implementation impractical in real-world scenarios. Second, the learned protocols fail to generalize outside of the scenario they were trained on. In this paper, we propose a new feasible learning framework that enables practical implementations of training procedures, thus allowing learned MAC protocols to be tailor-made for the scenario where they will be executed. Moreover, to address the second challenge, we leverage the concept of state abstraction and imbue it into the MARL framework for better generalization. As a result, the policies are learned in an abstracted observation space that contains only useful information extracted from the original high-dimensional and redundant observation space. Simulation results show that our feasible learning framework exhibits performance comparable to that of the infeasible solutions. In addition, the learning frameworks adopting observation abstraction offer better generalization capabilities, in terms of the number of UEs, number of data packets to transmit, and channel conditions.
18 February 2020 in IEEE Internet of Things Journal
The massive machine-type communication (mMTC) usage scenario, also known as the massive Internet of Things (mIoT), involves a huge number of MTC devices having high requirements on increased battery lifetime, which autonomously transfer small amounts of data with relaxed delay requirements, without human intervention. The current cellular network is unsuitable for this scenario, due to the limited uplink resources allocated to the physical random access channel (PRACH) and to the physical uplink shared channel (PUSCH). With this in mind, in this article, we propose a new framework, customized for massive MTC services, that includes a joint control of the dynamic resource allocation between the PRACH and the PUSCH, and a new random access (RA) procedure based on an adaptive access class barring (ACB) scheme that appropriately spreads RA reattempts in time. In addition, to further increase the transmission efficiency, we adopt the sparse code multiple access (SCMA) technique for PUSCH resources, because SCMA results as the most promising nonorthogonal multiple access (NOMA) technique to support massive MTC connectivity with small-size data. The simulation results show that the proposed control framework significantly improves the number of succeeded communications in comparison with the other proposals available in the literature. Furthermore, an energy consumption model is introduced to show the considerable energy saving achieved by the MTC devices which adopt our joint control scheme.
20 October 2022 in IEEE Transactions on Vehicular Technology
5 G networks may not be competent in fully supporting the high data rates, system capacity, and spectral efficiency expected in the upcoming 6 G networks. An enabling factor to achieve these requirements is the adoption of the Sparse Code Multiple Access (SCMA), which is one of the most promising code-domain Non-Orthogonal Multiple Access (NOMA) techniques. Nowadays, there are two major challenges for the implementation of SCMA in practical networks. First, the design of a near-optimal codebook set at the transmitter, that heavily impacts the network performance. Second, an efficient decoding algorithm at the receiver, that dominates the computational complexity of the system. The two aspects are not independent from each other; therefore the optimization of an SCMA network should address jointly both the Encoder and the Decoder side. Moreover, such solution is far from optimal when designed for general purpose use, since it does not exploit the channel statistical information known at the transmitter and the receiver side. However, most of the works available in the literature address only one aspect and also are not tailored for a given radio channel. In this paper, we design an end-to-end SCMA en/deconding structure based on the integration between a state-of-the-art AutoEncoder (AE) architecture and a novel Wasserstein Generative Adversarial Network (WGAN). The aim is to jointly create a near-optimal codebook design and a non-computationally heavy Decoder that are robust to the given channel statistics. Finally, we trained our model to be flexible to support different channel conditions and compared its performance against the state-of-the-art AE and the best conventional Decoders. The simulation results show that our innovative design exhibits decoding performance very similar to conventional Decoders with a reduced decoding latency.
26 August 2025 in IEEE Open Journal of the Communications Society
In this work, we propose an automated wireless medium access control (MAC) protocol design based on multi-agent reinforcement learning (MARL), in which multiple wireless devices (WDs) and a base station (BS) exchange control messages without prior knowledge of their meanings to coordinate data delivery across the network. In current approaches, the BS acts as a MAC expert, while the WDs are the only learning entities. As a result, the learned MAC protocols are constrained by the BS’s predefined knowledge. In contrast, our approach enables both WDs and the BS to act as learning agents, fostering the collaborative development of new ad hoc communication protocols. However, this approach introduces additional challenges in managing heterogeneous agents that learn simultaneously, alongside conventional issues such as the learned protocols being overfitted to training parameters, including the number of transmitted packets and the number of WDs managed by the BS. To overcome these limitations, we propose a novel learning framework that facilitates the emergence of MAC protocols with robust generalization capabilities. This framework incorporates innovative features such as a novel BS policy model, two distinct reward functions for the WDs and the BS, and different update routines for the agents’ policies. Moreover, we leverage the concepts of parameter sharing and state abstraction to enhance generalization in the WDs’ policies. The MAC protocols generated using the proposed framework are evaluated against state-of-the-art approaches. Simulation results show that the proposed solution significantly outperforms the benchmarks in terms of generalization capabilities.
21 December 2022 in IEEE Internet of Things Journal
In view of the challenges foreseen in futuristic massive IoT (mIoT) scenarios, characterized by a huge deployment of energy-constrained IoT devices, we propose an efficient next-generation multiple access (NGMA) scheme where several innovative solutions combine and integrate. First, we aim to maximize the spectral efficiency of the physical uplink-shared channel (PUSCH) by adopting the most appropriate nonorthogonal multiple access (NOMA) technique and by introducing an efficient contention-based approach for data transmission in the unused PUSCH resources. Second, on the basis of the traffic load, the proposed scheme dynamically sets the UpLink (UL) radio resources dimensioning by searching the optimal tradeoff between lowering the collision probability in the physical random access channel (PRACH) and providing enough radio resources in the PUSCH to permit the data transmission to all the succeeded access requests. Third, it is enriched by a series of strategic procedures to make feasible the traffic load estimation, even when the PRACH is in overload condition. Fourth, we exploit artificial intelligence technologies to further optimize the performances obtained. The proposed scheme is compared with other benchmark schemes available in the literature. It outperforms them in the most important metrics for the mIoT scenario (i.e., spectral efficiency and energy consumption), especially in high load conditions, which is the expected requirement for Beyond 5G and 6G networks.
To tackle the heterogeneous requirements of beyond 5G (B5G) and future 6G wireless networks, conventional medium access control (MAC) procedures need to evolve to enable base stations (BSs) and user equipments (UEs) to automatically learn innovative MAC protocols catering to extremely diverse services. This topic has received significant attention, and several reinforcement learning (RL) algorithms, in which BSs and UEs are cast as agents, are available with the aim of learning a communication policy based on agents' local observations. However, current approaches are typically overfitted to the environment they are trained in, and lack robustness against unseen conditions, failing to generalize in different environments. To overcome this problem, in this work, instead of learning a policy in the high dimensional and redundant observation space, we leverage the concept of observation abstraction (OA) rooted in extracting useful information from the environment. This in turn allows learning communication protocols that are more robust and with much better generalization capabilities than current baselines. To learn the abstracted information from observations, we propose an architecture based on autoencoder (AE) and imbue it into a multi-agent proximal policy optimization (MAPPO) framework. Simulation results corroborate the effectiveness of leveraging abstraction when learning protocols by generalizing across environments, in terms of number of UEs, number of data packets to transmit, and channel conditions.