In the era of 5G and beyond wireless networks, the simultaneous support of enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low Latency Communications (URLLC) poses significant challenges in managing radio resources efficiently. By leveraging the puncturing technique, we propose an intelligent resource management framework for meeting the strict latency and reliability requirement of URLLC services and the high data rate for eMBB services. In particular, a semi-supervised learning and deep reinforcement learning (DRL) based architecture is proposed to manage the resources intelligently. We decompose the optimization problem into two subproblems: 1) resource block allocation (RBA) strategy for eMBB slice, and 2) URLLC scheduling. Through extensive simulations and performance evaluations, we demonstrate the effectiveness of the proposed technique in optimizing resource utilization, minimizing latency for URLLC users, and maximizing the throughput for eMBB services. Simulation findings demonstrate that the proposed methodology can ensure the URLLC reliability requirements while maintaining higher average sum rate for eMBB and higher convergence rate. The proposed framework paves the way for the efficient coexistence of diverse services, enabling wireless network operators to optimize resource allocation, improve user experience, and meet the specific requirements of eMBB and URLLC applications.

As the idea of a new wireless communication standard (5G) started to circulate around the world, there was much speculation regarding its performance, making it necessary to carry out further research by keeping in view the challenges presented by it. 5G is considered a multi-system support network due to its ability to provide benefits to vertical industries. Due to the wide range of devices and applications, it is essential to provide support for massively interconnected devices. Network slicing has emerged as the key technology to meet the requirements of the communications network. In this paper, we present a review of the latest achievements of 5G network slicing by comparing the architecture of The Next Generation Mobile Network Alliance’s (NGMN’s) and 5G-PPP, using the enabling technologies software-defined networking (SDN) and network function virtualization (NFV). We then review and discuss machine learning (ML) techniques and their integration with network slicing for beyond 5G networks and elaborate on how ML techniques can be useful for mobility prediction and resource management. Lastly, we propose the use case of network slicing based on ML techniques in a smart seaport environment, which will help to manage the resources more efficiently.

The demands of ultra-reliable low-latency communication (URLLC) in “NextG" cellular networks necessitate innovative approaches for efficient resource utilization. The current literature on 6G O-RAN primarily addresses improved mobile broadband (eMBB) performance or URLLC latency optimization individually, often neglecting the intricate balance required to optimize both simultaneously under practical constraints. This paper addresses this gap by proposing a DRL-based resource allocation framework integrated with meta-learning to manage eMBB and URLLC services adaptively. Our approach efficiently allocates heterogeneous network resources, aiming to maximize energy efficiency (EE) while minimizing URLLC latency, even under varying environmental conditions. We highlight the critical importance of accurately estimating the traffic distribution flow in the multi-connectivity (MC) scenario, as its uncertainty can significantly degrade EE. The proposed framework demonstrates superior adaptability across different path loss models, outperforming traditional methods and paving the way for more resilient and efficient 6G networks.

Most existing studies consider the deep reinforcement learning (DRL) based Q-learning approach due to its ability to quickly converge to a near-optimal solution, resulting in effective allocation of resources and power. DRL-based Q-network discretizes the continuous power values which results in poor performance. It is challenging to allocate resources effectively in fast varying channel conditions in dynamic vehicular environments. In this work, we propose two approaches to overcome these challenges. First, we present a DRL-based energy-efficient resource allocation approach where we use a twin delayed deep deterministic policy gradient (TD3) scheme based on Thompson sampling to solve the power and resource allocation problem. Second, we present a dynamic meta-transfer learning framework to enhance the policy’s ability to adjust to new channel conditions. Simulation results shows that the proposed TD3 approach based on Thompson sampling enhances the system performance. Moreover, the proposed DRL-based dynamic meta-transfer learning framework takes 80% less samples to adapt to a new environment.

The rapid growth of vehicle-to-network (V2N) communication demands efficient handover decision-making strategies to ensure seamless connectivity and maximum throughput. However, the dynamic nature of V2N scenarios poses challenges for traditional handover algorithms. To address this, we propose a deep reinforcement learning (DRL)-based approach for optimizing handover decisions in dynamic V2N communication. We leverages the advantages of transfer learning and meta-learning to generalize across time-evolving source and target tasks. In this paper, we derive generalization bounds for our DRLbased approach, specifically focusing on optimizing the handover process in V2N communication. The derived bounds provide theoretical guarantees on the expected generalization error of the learned handover time function for the target task. To implement our framework, we propose a meta-learning framework, Adaptto-evolve (A2E), based on the double deep Q-networks (DDQN) with Thompson sampling approach. The A2E framework enables quick adaptation to new tasks by minimizing the error upper bounds with divergence measures. Through transfer learning, the meta-learner dynamically evolves its handover decision-making strategy to maximize average throughput while reducing the number of handovers. We use Thompson sampling with the DDQN to balance exploration and exploitation. The DDQN with The Thompson sampling approach, ensuring efficient and effective learning, forms the foundation for optimizing the metatraining process, resulting in improvement in cumulated packet lossby48.02%in highway settings and 46.32%in rural settings

The advent of Ultra-Reliable Low Latency Communication (URLLC) along with the emergence of Open RAN (ORAN) architectures presents unprecedented challenges and opportunities in Radio Resource Management (RRM) for next-generation communication systems. This paper presents a comprehensive trade-off analysis of Deep Reinforcement Learning (DRL) approaches designed to enhance URLLC performance within ORAN’s flexible and dynamic framework. By investigating various DRL strategies for optimizing RRM parameters, we explore the intricate balance between reliability, latency, and the newfound adaptability afforded by the ORAN principles. Through extensive simulation results, our study compares the efficacy of different DRL models in achieving URLLC objectives in an ORAN context, highlighting the potential of DRL to navigate the complexities introduced by ORAN. The proposed study provides valuable information on the practical implementation of DRL-based RRM solutions in ORAN-enabled wireless networks. It sheds light on the benefits and challenges of integrating DRL and ORAN for URLLC enhancements. Our findings demonstrate that the proposed twin-delayed deep-deterministic policy gradient (TD3) integrated with Thompson Sampling (TS) achieves reliability levels above 99% in more than 80% of instances, outperforming baseline DRL methods in maintaining stringent URLLC reliability requirements, offering a roadmap for future research to pursue efficient, reliable, and flexible communication systems.