With the exponential growth of the Internet of things (IoT) in remote sensing image applications, network resource orchestration and data privacy are significant aspects to handle in bigdata cellular networks. The image data sharing procedure toward central cloud servers in order to perform real-time classifications has leaked client personalization and heavily burdened the communication networks. Thus, the deployment of IoT image sensors in privacy-constrained sectors requires an optimized federated learning (FL) scheme to efficiently consider both aspects of securing data privacy and maximizing the model accuracy with sufficient communication and computation resources. In this article, an adaptive model communication scheme with virtual resource optimization for edge FL is proposed by converging a deep q-learning algorithm to enforce a self-learning agent interacting with network functions virtualization orchestrator and software-defined networking based architecture. The agent targets to optimize the resource control policy of virtual multi-access edge computing entities in virtualized infrastructure manager. The proposed scheme trains the learning model and weighs the optimal actions for particular network states by using an epsilon-greedy strategy. In the exploitation phase, the scheme considers multiple spatial-resolution sensing conditions and allocates computation offloading resources for global multiconvolutional neural networks model aggregation based on the congestion states. In the simulation results, the quality of service and global collaborative model performance metrics were evaluated in terms of delay, packet drop ratios, packet delivery ratios, loss values, and overall accuracy.

To build intelligent model learning in conventional architecture, the local data are required to be transmitted toward the cloud server, which causes heavy backhaul congestion, leakage of personalization, and insufficient use of network resources. To address these issues, federated learning (FL) is introduced by offering a systematical framework that converges the distributed modeling process between local participants and the parameter server. However, the challenging issues of insufficient participant scheduling, aggregation policies, model offloading, and resource management still remain within conventional FL architecture. In this survey article, the state-of-the-art solutions for optimizing the orchestration in FL communications are presented, primarily querying the deep reinforcement learning (DRL)-based autonomy approaches. The correlations between the DRL and FL mechanisms are described within the optimized system architectures of selected literature approaches. The observable states, configurable actions, and target rewards are inquired into to illustrate the applicability of DRL-assisted control toward self-organizing FL systems. Various deployment strategies for Internet of Things applications are discussed. Furthermore, this article offers a review of the challenges and future research perspectives for advancing practical performances. Advanced solutions in these aspects will drive the applicability of converged DRL and FL for future autonomous communication-efficient and privacy-aware learning.

The advancing applications based on machine learning and deep learning in communication networks have been exponentially increasing in the system architectures of enabled software-defined networking, network functions virtualization, and other wired/wireless networks. With data exposure capabilities of graph-structured network topologies and underlying data plane information, the state-of-the-art deep learning approach, graph neural networks (GNN), has been applied to understand multi-scale deep correlations, offer generalization capability, improve the accuracy metrics of prediction modelling, and empower state representation for deep reinforcement learning (DRL) agents in future intelligent network management and orchestration. This paper contributes a taxonomy of recent studies using GNN-based approaches to optimize the control policies, including offloading strategies, routing optimization, virtual network function orchestration, and resource allocation. The algorithm designs of converged DRL and GNN are reviewed throughout the selected studies by presenting the state generalization, GNN-assisted action selection, and reward valuation cooperating with GNN outputs. We also survey the GNN-empowered application deployment in the autonomous control of optical networks, Internet of Healthcare Things, Internet of Vehicles, Industrial Internet of Things, and other smart city applications. Finally, we provide a potential discussion on research challenges and future directions.

This paper provides a comprehensive survey of the integration of graph neural networks (GNN) and deep reinforcement learning (DRL) in end-to-end (E2E) networking solutions. We delve into the fundamentals of GNN, its variants, and the state-of-the-art applications in communication networking, which reveal the potential to revolutionize access, transport, and core network management policies. This paper further explores DRL capabilities, its variants, and the trending applications in E2E networking, particularly in enhancing dynamic network (re)configurations and resource management. By fusing GNN with DRL, we spotlight novel approaches, ranging from radio access networks to core management and orchestration, across E2E network layers. Deployment scenarios in smart transportation, smart factory, and smart grids demonstrate the practical implications of our survey topic. Lastly, we point out potential challenges and future research directions, including the critical aspects for modelling explainability, the reduction in overhead consumption, interoperability with existing schemes, and the importance of reproducibility. Our survey aims to serve as a roadmap for future developments in E2E networking, guiding through the current landscape, challenges, and prospective breakthroughs in the algorithm modelling toward network automation using GNN and DRL.

A vehicular network is composed of an in-vehicle network (IVN) and Internet of Vehicles (IoV). IVN exchanges information among in-vehicle devices. IoV constructs Vehicle-to-X (V2X) networks outside vehicles and exchanges information among V2X elements. These days, in-vehicle devices that require high bandwidth is increased for autonomous driving services. Thus, the spread of data for vehicles is exploding. This kind of data is exchanged through IoV. Even if the Ethernet backbone of IVN carries a lot of data in the vehicle, the explosive increase in data from outside the vehicle can affect the backbone. That is, the transmission efficiency of the IVN backbone will be reduced due to excessive data traffic. In addition, when IVN data traffic is transmitted to IoV without considering IoV network conditions, the transmission efficiency of IoV is also reduced. Therefore, in this paper, we propose an IoV access gateway to controls the incoming data traffic to the IVN backbone and the outgoing data traffic to the IoV in the network environment where IVN and IoV are integrated. Computer simulations are used to evaluate the performance of the proposed system, and the proposed system shows better performance in the accumulated average transmission delay.