The advancing applications based on machine learning and deep learning in communication networks have been exponentially increasing in the system architectures of enabled software-defined networking, network functions virtualization, and other wired/wireless networks. With data exposure capabilities of graph-structured network topologies and underlying data plane information, the state-of-the-art deep learning approach, graph neural networks (GNN), has been applied to understand multi-scale deep correlations, offer generalization capability, improve the accuracy metrics of prediction modelling, and empower state representation for deep reinforcement learning (DRL) agents in future intelligent network management and orchestration. This paper contributes a taxonomy of recent studies using GNN-based approaches to optimize the control policies, including offloading strategies, routing optimization, virtual network function orchestration, and resource allocation. The algorithm designs of converged DRL and GNN are reviewed throughout the selected studies by presenting the state generalization, GNN-assisted action selection, and reward valuation cooperating with GNN outputs. We also survey the GNN-empowered application deployment in the autonomous control of optical networks, Internet of Healthcare Things, Internet of Vehicles, Industrial Internet of Things, and other smart city applications. Finally, we provide a potential discussion on research challenges and future directions.

This paper provides a comprehensive survey of the integration of graph neural networks (GNN) and deep reinforcement learning (DRL) in end-to-end (E2E) networking solutions. We delve into the fundamentals of GNN, its variants, and the state-of-the-art applications in communication networking, which reveal the potential to revolutionize access, transport, and core network management policies. This paper further explores DRL capabilities, its variants, and the trending applications in E2E networking, particularly in enhancing dynamic network (re)configurations and resource management. By fusing GNN with DRL, we spotlight novel approaches, ranging from radio access networks to core management and orchestration, across E2E network layers. Deployment scenarios in smart transportation, smart factory, and smart grids demonstrate the practical implications of our survey topic. Lastly, we point out potential challenges and future research directions, including the critical aspects for modelling explainability, the reduction in overhead consumption, interoperability with existing schemes, and the importance of reproducibility. Our survey aims to serve as a roadmap for future developments in E2E networking, guiding through the current landscape, challenges, and prospective breakthroughs in the algorithm modelling toward network automation using GNN and DRL.

Internet of Things (IoT) devices can integrate with applications requiring intensive contextual data processing, intelligent vehicle control, healthcare remote sensing, VR, data mining, traffic management, and interactive applications. However, there are computationally intensive tasks that need to be completed quickly within the time constraints of IoT devices. To address this challenge, researchers have proposed computation offloading, where computing tasks are sent to edge servers instead of being executed locally on user devices. This approach involves using edge servers located near users in cellular network base stations, and also known as Mobile Edge Computing (MEC). The goal is to offload tasks to edge servers, optimizing both latency and energy consumption. The main objective of this paper mentioned in the summary is to design an algorithm for time- and energy-optimized task offloading decision-making in MEC environments. Therefore, we developed a Lagrange Duality Resource Optimization Algorithm (LDROA) to optimize for both decision offloading and resource allocation for tasks, whether to locally execute or offload to an edge server. The LDROA technique produces superior simulation outcomes in terms of task offloading, with improved performance in computation latency and cost usage compared to conventional methods like Random Offloading, Load Balancing, and the Greedy Latency Offloading scheme.

The limited coverage extension of mobile edge computing (MEC) necessitates exploring cooperation with unmanned aerial vehicles (UAV) to leverage advanced features for future computation-intensive and mission-critical applications. Moreover, the workflow for task offloading in software-defined networking (SDN)-enabled 5G is significant to tackle in UAV-MEC networks. In this paper, deep reinforcement learning (DRL) SDN control methods for improving computing resources are proposed. DRL-based SDN controller, termed DRL-SDNC, allocates computational resources, bandwidth, and storage based on task requirements, upper-bound tolerable delays, and network conditions, using the UAV system architecture for task exchange between MECs. DRL-SDNC configures rule installation based on state observations and agent evaluation indicators, such as network congestion, user equipment computational capabilities, and energy efficiency. This paper also proposes the training deep network architecture for the DRL-SDNC, enabling interactive and autonomous policy enforcement. The agent learns from the UAV-MEC environment through experience gathering and updates its parameters using optimization methods. DRL-SDNC collaboratively adjusts hyperparameters and network architecture to enhance learning efficiency. Compared with baseline schemes, simulation results demonstrate the effectiveness of the proposed approach in optimizing resource efficiency and achieving satisfied quality of service for efficient utilization of computing and communication resources in UAV-assisted networking environments.

Network functions virtualization (NFV) has become the platform for decomposing the sequence of virtual network functions (VNFs), which can be grouped as a forwarding graph of service function chaining (SFC) to serve multi-service slice requirements. NFV-enabled SFC consists of several challenges in reaching the reliability and efficiency of key performance indicators (KPIs) in management and orchestration (MANO) decision-making control. The problem of SFC fault tolerance is one of the most critical challenges for provisioning service requests, and it needs resource availability. In this article, we proposed graph neural network (GNN)-based deep reinforcement learning (DRL) to enhance SFC fault tolerance (GRL-SFT), which targets the chain graph representation, long-term approximation, and self-organizing service orchestration for future massive Internet of Everything applications. We formulate the problem as the Markov decision process (MDP). DRL seeks to maximize the cumulative rewards by maximizing the service request acceptance ratios and minimizing the average completion delays. The proposed model solves the VNF management problem in a short time and configures the node allocation reliably for real-time restoration. Our simulation result demonstrates the effectiveness of the proposed scheme and indicates better performance in terms of total rewards, delays, acceptances, failures, and restoration ratios in different network topologies compared to reference schemes.

Network slicing is introduced for elastically instantiating logical network infrastructure isolation to support different application types with diversified quality of service (QoS) class indicators. In particular, vehicular communications are a trending area that consists of massive mission-critical applications in the range of safety-critical, intelligent transport systems, and on-board infotainment. Slicing management can be achieved if the network infrastructure has computing sufficiency, a dynamic control policy, elastic resource virtualization, and cross-tier orchestration. To support the functionality of slicing management, incorporating core network infrastructure with deep learning and reinforcement learning has become a hot topic for researchers and practitioners in analyzing vehicular traffic/resource patterns before orchestrating the steering policies. In this paper, we propose QoS-driven management by considering (edge) resource block utilization, scheduling, and slice instantiation in a three-tier resource placement, namely, small base stations/access points, macro base stations, and core networks. The proposed scheme integrates recurrent neural networks to trigger hidden states of resource availability and predict the output of QoS. The intelligent agent and slice controller, namely, RDQ3N, gathers the resource states from three-tier observations and optimizes the action on allocation and scheduling algorithms. Experiments are conducted on both physical and virtual representational vehicle-to-everything (V2X) environments; furthermore, service requests are set to massive thresholds for rendering V2X congestion flow entries.