Service function chain (SFC) has been recognized as one of the most important technologies that can satisfy dynamic service demands in the edge clouds. However, how to efficiently embed SFCs in the dynamic edge-cloud scenarios remains as a challenging problem. Considering different network topologies, we devise two deep reinforcement learning (DRL)-based methods for two network sizes: a deep deterministic policy gradient (DDPG) based method for the small-scale networks and an asynchronous advantage actor-critic (A3C) based approach for the large-scale networks. Simulation results demonstrate that our proposals can efficiently deal with the SFC-DMP in edge clouds and outperform the state-of-the-art methods in terms of the delay.

The unmanned aerial vehicles (UAVs) networks are very vulnerable to smart jammers that can choose their jamming strategy based on the ongoing channel state accordingly. Although reinforcement learning (RL) algorithms can give UAV networks the ability to make intelligent decisions, the high-dimensional state space makes it difficult for algorithms to converge quickly. This article proposes a knowledge-based RL method, which uses domain knowledge to compress the state space that the agent needs to explore and then improve the algorithm convergence speed. Specifically, we use the inertial law of the aircraft and the law of signal attenuation in free space to guide the highly efficient exploration of the UAVs in the state space. We incorporate the performance indicators of the receiver and the subjective value of the task into the design of the reward function, and build a virtual environment for pretraining to accelerate the convergence of anti-jamming decisions. In addition, the algorithm proposed is completely based on observable data, which is more realistic than those studies that assume the position or the channel strategy of the jammer. The simulation shows that the proposed algorithm can outperform the benchmarks of model-free RL algorithm in terms of converge speed and averaged reward.

With the large-scale commercial application of data center networks, the security problem of data center network is attracting more and more attention. However, security functions are usually placed on proprietary hardware, which makes the delivery of security service inflexible and of high cost. The emerging software-defined networking and network function virtualization in a joint manner are promising technology that can solve these outlined problems through a process named security service chaining (SSC) composition. Determining the composition of SSC that optimizes the resource allocation is a challenging problem, particularly without violating security and resource requirements. This problem is called the dynamic SSC composition problem, and an integer linear programming formulation with implementation in GLPK is provided. A novel heuristic solution is provided based on the breadth first search algorithm. The obtained experimental results show that the proposed algorithm can perform better than the compared ones, and the advantage of the proposed mechanism is also demonstrated via NetFPGA-10G prototype.

Unmanned aerial vehicle (UAV) networks have a wide range of applications, such as in the Internet of Things (IoT), 5G communications, and so forth. However, the communications between UAVs and UAVs to ground control stations mainly use radio channels, and therefore these communications are vulnerable to cyberattacks. With the advent of software-defined radio (SDR), smart attacks that can flexibly select attack strategies according to the defender’s state information are gradually attracting the attention of researchers and potential attackers of UAV networks. The smart attack can even induce the defender to take a specific defense strategy, causing even greater damage. Inspired by symmetrical thinking, a solution using a software-defined network (SDN) to combat software-defined radio was proposed. We propose a network architecture which uses dual controllers, including a UAV flight controller and SDN controller, to achieve collaborative decision-making. Built on the top of the SDN, the state information of the whole network converges quickly and is fitted to an environment model used to develop an improved Dyna-Q-based reinforcement learning algorithm. The improved algorithm integrates the power allocation and track planning of UAVs into a unified action space. The simulation data showed that the proposed communication solution can effectively avoid smart jamming attacks and has faster learning efficiency and higher convergence performance than the compared algorithms.

Network function virtualization and multiaccess edge computing have been introduced by Internet service providers to deal with various challenges, which hinder them from satisfying the increasing demand of the low-latency network applications for the Internet of Things (IoT). In edge clouds, any network application for the IoT can be expressed as a service function chain (SFC) consisting of several strictly ordered virtual network functions (VNFs), which can be geographically placed onto edge clouds close to the terminals. However, regarding the large number of terminals and constantly dynamics of edge clouds, determining the placement of VNFs and routing service paths that optimizes the end-to-end delays is a challenging problem. This problem can be also called SFC dynamic orchestration problem. To exactly solve the problem, an integer linear programming (ILP) model is formulated. Then, it is difficult to apply the exact method to deal with the SFC dynamic orchestration problem in the large-scale networks, and this article presents a Lagrangian relaxation heuristic-based algorithm for the optimization, thus reducing the computational complexity. It is demonstrated that the proposed algorithm can efficiently achieve a near-optimal solution with a theoretical analysis. The obtained simulation results show that the proposed algorithm can approximate the performance of ILP’s solution and outperform the current benchmarks in terms of the end-to-end delay and service acceptance ratio.

Network function virtualization (NFV) and software defined networking (SDN) have recently introduced together, which show great potential in openness, innovation, flexibility and scalability of the networks. Embedding SDN/NFV-enabled service function chain (SFC) has been considered as an efficient way to achieve the end-to-end communication. Considering the multiple users’ delay diversities oriented to future networks, from the perspective of network operators, determining the placement of virtual network functions (VNFs) and routing the service paths is a challenging problem. However, the traditional solutions (e.g., exact, heuristic, meta-heuristic, game, etc.) based on the SFC dynamic embedding problem (SFC-DMP) follow the strong assumptions as usual, which might limit their applications in dynamic and complex network scenarios. To address this narrow, we are motivated to present a novel approach for handling SFC-DMP in the cloud datacenter networks that simultaneously combines the quantum genetic algorithm (QGA) and fuzzy logic. Simulation results show the efficient performance of our proposal.