Faced with fluctuating network traffic and unknown underlying network traffic dynamics, developing an effective orchestration model with low network cost is still a critical issue in Network Functions Virtualization (NFV)-enabled networks. Thus we propose a Deep Reinforcement Learning based Quality of Service (QoS)/Quality of Experience (QoE)-Aware Adaptive Online Orchestration (DRL-QOR) approach to adapt to the real- time network variations. We formulate the stochastic resource optimization as a Parameterized Action Markov Decision Process (PAMDP), with QoE and specific QoS requirements as key factors in formulating the reward function, aiming to maximize QoE while satisfying QoS constraints. Then we propose DRL-QOR to solve the Non-deterministic Polynomial hard (NP-hard) problem with consideration of improving the long-term profits, where deep neural network combinatorial optimization theory is extended under the constraints of the binary integer programming model. Extensive experimental results in real USANET topology demonstrate that our proposed DRL-QOR converges fast during the training process. Compared with other benchmarks that only consider the current system performance, it shows good performance in QoE provisioning and QoS requirements maintenance for orchestrating SFCs.

In software defined network/network function virtualization (SDN/NFV)-enabled cloud environment, cloud services can be implemented as service function chains (SFCs), which consist of a series of ordered virtual network functions. However, due to fluctuations of cloud traffic and without knowledge of cloud computing network configuration, designing SFC optimization approach to obtain flexible cloud services in dynamic cloud environment is a pivotal challenge. In this paper, we propose a fault tolerance oriented SFC optimization approach based on deep reinforcement learning. We model fault tolerance oriented SFC elastic optimization problem as a Markov decision process, in which the reward is modeled as a weighted function, including minimizing energy consumption and migration cost, maximizing revenue benefit and load balancing. Then, taking binary integer programming model as constraints of quality of cloud services, we design optimization approaches for single-agent double deep Q-network (SADDQN) and multi-agent DDQN (MADDQN). Among them, MADDQN decentralizes training tasks from control plane to data plane to reduce the probability of single point of failure for the centralized controller. Experimental results show that the designed approaches have better performance. MADDQN can almost reach the upper bound of theoretical solution obtained by assuming a prior knowledge of the dynamics of cloud traffic.

In autonomous driving research area, vehicular positioning has become an critical issue. For vehicular positioning, Global Positioning System (GPS), Inertial Measurement Unit (IMU) are commonly used. However, when GPS signal is lost or vehicular networks are interrupted, the positioning accuracy degrades. In order to improve the positioning accuracy, this paper proposes a multiple data fusion framework of GPS/IMU Aided by vehicular networks, which can optimize vehicle localization systems for accuracy, robustness. Specifically, at the first phase, Attention based Long Short Term Memory (ALSTM) network is utilized to fuse GPS and IMU information. Comparing with traditional GPS and IMU fusion methods, this model avoids requiring any prior information about state transfer model. It aims to enhance the exploitation of past state information by integrating an attention layer above the LSTM layer to operate more optimally and comprehensively. At the second phase, the Vehicular to Everything (V2X) network is developed to further improve localization accuracy. A collaborative graph model based on Graph Convolutional Network (GCN) for vehicular networks is proposed. Each communication node is modeled as an undirected graph node, and can exchange its GPS information with other nodes. At the final phase, a nonlinear regression layer is designed to fuse the information from the prior two phases. Numerical results show that the proposed ALSTM-GCN networks can fuse GPS, IMU, and V2X observation information better, which can achieve high localization accuracy in comparison with the conventional approach.

Network function virtualization (NFV) is a ground-breaking technology that decouples network functions (NFs) from customized hardware to support more flexible network services and network resource allocation. However, these improvements also lead to an increase in the possibility of service function chain (SFC) failure due to hardware failures, software bugs, or resource contention. This could lead to minor problems or even serious consequences. Unfortunately, the existing failure detection methods have multiple issues, such as small detection range, single detection function, heavy overhead, and low accuracy. Consequently, we propose FullSight, a feasible framework based on deep learning (DL) models that can efficiently integrate both the control plane and programmable data plane for fault detection and classification. This framework obtains the status and indicators of components and network that cause service quality performance degradation through two planes. These indicators are ultimately sent to the knowledge plane for preprocessing, dimensionality reduction, and fault analysis. In addition, we propose two algorithms based on text convolutional neural network (textCNN) and bidirectional encoder representations from transformers (BERT) to classify SFC faults. We implement and evaluate the proposed FullSight prototype extensively on a prototype with thirteen programmable switches and twenty end-hosts. Our experimental results show that FullSight can rapidly and accurately detect and identify eight categories of fine-grained SFC failures, compared with other state-of-the-art methods. Besides, compared with SFC Path Tracer and Pingmesh, our framework can reduce the average bandwidth overhead of the data plane by 57% and 84%, respectively, and achieve detection accuracy of more than 98%.

Network function virtualization (NFV) is considered to be a promising paradigm for reducing network infrastructure cost. However, the mapping from network physical resources to service function chain (SFC), which is an ordered sequence of virtual network functions, is challenging, especially when a dynamic wireless network is involved. In this article, we propose a novel lightweight SFC embedding framework based on reinforcement learning in the NFV-enabled wireless network for reducing the end-to-end wired and wireless delay and leveraging the SFC acceptance ratio. We model the embedding problem as a Markov decision process, in which the reward is modeled as an adjustable weighted function, including combined delay (wired and wireless delay) and remaining resource. Accordingly, a lightweight and adjustable $Q$ -learning-based SFC embedding strategy is proposed. To validate the effectiveness of our proposed framework, extensive simulations are performed by using real data traces and compared with a classical genetic algorithm (GA) and a cluster-based method MAPLE. The results of the simulation show that the proposed framework is efficient and performs better than both GA and MAPLE in terms of delay and service acceptance probability. The prototype of the framework is also built for demonstrating the feasibility of the proposed framework.

Software-Defined Network is an emerging networking paradigm that enables intelligent and flexible network management. Specifically, the design of the control plane is crucial. Therefore, in order to avoid a single point of failure, multiple controllers are deployed constantly in a distributed manner on the control plane. In this paper, we propose a controller placement approach based on multiple objectives (MODECP), including network delay, network security, load-balancing rate, and link occupancy. In the controller placement stage, an improved multi-objective differential evolution algorithm is proposed to search for controllers’ positions and assign switches to controllers reasonably. Furthermore, an improved affinity propagation algorithm is proposed to obtain the number of controllers placed in the network partition stage, comprehensively considering the delay, node security, and load. Simulations are performed based on several topologies from Internet Topology Zoo. Extensive results show that the proposed algorithm can realize trade-offs among multiple objectives and improve network performance in delay, security, controller load, and link occupancy compared to the single-objective based approach. Moreover, compared with the genetic algorithm and random placement algorithm, the proposed algorithm performs better with low latency, high security, low load rate, and low link overhead.