Network Function Virtualization (NFV) technology utilizes software to implement network function as virtual instances, which reduces the cost on various middlebox hardware. A Virtual Network Function (VNF) instance requires multiple resource types in the network (e.g., CPU, memory). Therefore, an efficient VNF placement policy should consider both the resource utilization problem and the Quality of Service (QoS) of flows, which is proved NP-hard. Recent studies employ Deep Reinforcement Learning (DRL) to solve the VNF placement problem, but existing DRL-based solutions cannot generalize well to different topologies. In this paper, we propose to combine the advantage of DRL and Graph Neural Network (GNN) to design our VNF placement scheme DeepOpt. Simulation results show that DeepOpt outperforms the state-of-the-art VNF placement schemes and shows a much better generalization ability in different network topologies.

Traditional routing schemes usually use fixed models for routing policies and thus are not good at handling complicated and dynamic traffic, leading to performance degradation (e.g., poor quality of service). Emerging Deep Reinforcement Learning (DRL) coupled with Software-Defined Networking (SDN) provides new opportunities to improve network performance with automatic traffic analysis and policy generation. However, existing DRL-based routing solutions usually rely on all node information to make routing decisions for the network and hence are both hard to converge in large networks and vulnerable to topology changes. In this paper, we propose ScaleDeep, a scalable DRL-based routing scheme for SDN, which improves the routing performance and is resilient to topology changes. Essentially, ScaleDeep takes advantage of partial control on network nodes and DRL. We select a set of critical nodes from a network as driver nodes, which can simulate the entire network operation, based on the control theory. By observing the traffic variation on the driver nodes, DRL dynamically adjusts some link weights for a weighted shortest path algorithm to change the routing paths and improve the routing performance. Limiting the control on driver nodes improves the convergence ability of DRL and reduces the dependency of the DRL agent on the fixed network topology. To validate the performance of ScaleDeep, we conduct packet-level simulations on different topologies. The results show that ScaleDeep outperforms existing DRL-based schemes by reducing the average flow completion time by up to 36% and exhibiting better robustness against minor topology changes.

Burdened with the specialized bearing requirements brought by the deep fusion of the Internet and the economic society, the current Internet is faced with a series of great challenges including intelligence, diversification, personalization, robustness and efficiency because of some defects such as rigid structure, simple bearing structure based on IP, and disability in suppressing unknown threats. This paper proposes to improve the function, performance, efficiency and security of the Internet from the perspective of network architecture, introduces the “structure definability” to all layers of the network, and provides a network architecture with a multi-modal presentation of network functions in all layers called Polymorphic Smart Network (PINet). PINet supports full-dimensional definition and multi-modal presentation of addressing and routing, switching mode, interconnection mode, network element, transmission protocol, service properties and so on. Experimental results have verified that PINet supports the incremental evolution of the Internet and fundamentally meets the business requirement of network intelligence, diversification, personalization, high robustness and high efficiency.

As modern communication networks are growing more complicated and dynamic, designing a good Traffic Engineering (TE) policy becomes difficult due to the complexity of solving the optimal traffic scheduling problem. Deep Reinforcement Learning (DRL) provides us with a chance to design a model-free TE scheme through machine learning. However, existing DRL-based TE solutions cannot be applied to large networks. In this paper, we propose to combine the control theory and DRL to design a TE scheme. Our proposed scheme ScaleDRL employs the idea from the pinning control theory to select a subset of links in the network and name them critical links. Based on the traffic distribution information, we use a DRL algorithm to dynamically adjust the link weights for the critical links. Through a weighted shortest path algorithm, the forwarding paths of the flows can be dynamically adjusted. The packet-level simulation shows that ScaleDRL reduces the average end-to-end transmission delay by up to 39% compared to the state-of-the-art in different network topologies.

The ever-growing carbon emission of information infrastructure accounts for a significant proportion of the global carbon emissions. Existing studies reduce carbon consumption mainly by improving power efficiency on specific facilities or energy source structures. However, these methods do not jointly consider the impact of computation and network resource distribution on carbon emission. In this paper, we propose a data-driven scheme named EcoNet using reinforcement learning to reduce carbon emissions by jointly scheduling computation and network resources. We dynamically monitor the status of the computation and network facilities using cloud-edge collaboration and software-defined networking. Based on the collected status information, we formulate the resource scheduling problem as an optimization problem, which comprehensively considers the carbon emission, electricity price, and quality of service. The problem has high computation complexity, and we solve the problem with the proposed EcoNet to achieve efficient scheduling and near-optimal performance based on the collected network status information. The evaluation results show that EcoNet can maintain good Quality of Service and save at least 17% of the overall cost considering the electricity bills and carbon emissions.