Software-defined network (SDN) and network function virtualization (NFV) are acknowledged as the most promising technologies to effectively allocate resource for network service. A service function chain (SFC), which can deploy virtualized network functions (VNFs) and chain them with associated flows allocation, can be used to represent each network service owing to the introduction of the SDN/NFV technology. Co-hosted applications on multiple Internet of Things terminals have dynamic and time-varying service requirements, in order to allocate network resources optimally and meet the end-to-end delay requirements of services, sufficient strategies are required to satisfy the continuously changing service demands. In this article, a deep learning model that combines the multitask regression layer above the graph neural networks is first presented to predict the future resource requirements of each VNF instance. The SFC deployment problem is then solved using the integer nonlinear programming (INLP) approach, and a novel prediction-assisted Viterbi algorithm is presented to overcome the scalability problem of the INLP approach. According to the simulation findings, the proposed deep model provides a minimum of a 6.2% improvement in prediction accuracy over baseline prediction models, and the proposed SFC deployment strategy has been demonstrated to deliver better performance in terms of acceptance ratio and revenue, compared to the current passive deployment algorithms.

Service function chain (SFC) has been recognized as one of the most important technologies that can satisfy dynamic service demands in the edge clouds. However, how to efficiently embed SFCs in the dynamic edge-cloud scenarios remains as a challenging problem. Considering different network topologies, we devise two deep reinforcement learning (DRL)-based methods for two network sizes: a deep deterministic policy gradient (DDPG) based method for the small-scale networks and an asynchronous advantage actor-critic (A3C) based approach for the large-scale networks. Simulation results demonstrate that our proposals can efficiently deal with the SFC-DMP in edge clouds and outperform the state-of-the-art methods in terms of the delay.

The unmanned aerial vehicles (UAVs) networks are very vulnerable to smart jammers that can choose their jamming strategy based on the ongoing channel state accordingly. Although reinforcement learning (RL) algorithms can give UAV networks the ability to make intelligent decisions, the high-dimensional state space makes it difficult for algorithms to converge quickly. This article proposes a knowledge-based RL method, which uses domain knowledge to compress the state space that the agent needs to explore and then improve the algorithm convergence speed. Specifically, we use the inertial law of the aircraft and the law of signal attenuation in free space to guide the highly efficient exploration of the UAVs in the state space. We incorporate the performance indicators of the receiver and the subjective value of the task into the design of the reward function, and build a virtual environment for pretraining to accelerate the convergence of anti-jamming decisions. In addition, the algorithm proposed is completely based on observable data, which is more realistic than those studies that assume the position or the channel strategy of the jammer. The simulation shows that the proposed algorithm can outperform the benchmarks of model-free RL algorithm in terms of converge speed and averaged reward.

Unmanned aerial vehicle (UAV) networks have a wide range of applications, such as in the Internet of Things (IoT), 5G communications, and so forth. However, the communications between UAVs and UAVs to ground control stations mainly use radio channels, and therefore these communications are vulnerable to cyberattacks. With the advent of software-defined radio (SDR), smart attacks that can flexibly select attack strategies according to the defender’s state information are gradually attracting the attention of researchers and potential attackers of UAV networks. The smart attack can even induce the defender to take a specific defense strategy, causing even greater damage. Inspired by symmetrical thinking, a solution using a software-defined network (SDN) to combat software-defined radio was proposed. We propose a network architecture which uses dual controllers, including a UAV flight controller and SDN controller, to achieve collaborative decision-making. Built on the top of the SDN, the state information of the whole network converges quickly and is fitted to an environment model used to develop an improved Dyna-Q-based reinforcement learning algorithm. The improved algorithm integrates the power allocation and track planning of UAVs into a unified action space. The simulation data showed that the proposed communication solution can effectively avoid smart jamming attacks and has faster learning efficiency and higher convergence performance than the compared algorithms.

Network function virtualization and multiaccess edge computing have been introduced by Internet service providers to deal with various challenges, which hinder them from satisfying the increasing demand of the low-latency network applications for the Internet of Things (IoT). In edge clouds, any network application for the IoT can be expressed as a service function chain (SFC) consisting of several strictly ordered virtual network functions (VNFs), which can be geographically placed onto edge clouds close to the terminals. However, regarding the large number of terminals and constantly dynamics of edge clouds, determining the placement of VNFs and routing service paths that optimizes the end-to-end delays is a challenging problem. This problem can be also called SFC dynamic orchestration problem. To exactly solve the problem, an integer linear programming (ILP) model is formulated. Then, it is difficult to apply the exact method to deal with the SFC dynamic orchestration problem in the large-scale networks, and this article presents a Lagrangian relaxation heuristic-based algorithm for the optimization, thus reducing the computational complexity. It is demonstrated that the proposed algorithm can efficiently achieve a near-optimal solution with a theoretical analysis. The obtained simulation results show that the proposed algorithm can approximate the performance of ILP’s solution and outperform the current benchmarks in terms of the end-to-end delay and service acceptance ratio.

The service function chain (SFC) routing has been increasingly associated with multi-domain networks. However, several multi-domain SFC routing issues, linked to cross-domain interactions and emphasized by the heterogeneity of substrate topologies and administrative authorities, present a challenge. We propose the Cooperative Multi-agent Reinforcement Learning (CMRL) based algorithm for multi-domain SFC routing problem. Simulations on the real-world network topology demonstrate that our proposal can provide more than two and four times reduction in terms of the run time and network resource cost, respectively.