Deep reinforcement learning (DRL) has been introduced to the routing and spectrum assignment (RSA) of elastic optical networks (EONs) where the RSA policies are learnt during the interaction of a DRL agent with the EON environment. Upon each new traffic arrival, the DRL agent senses the EON state, and makes the RSA decision accordingly. Therefore, the EON state feature extraction is essential for the performance. However, current approaches, mainly based on fully connected neural networks or convolutional neural networks, are unsatisfactory in EON state feature extraction due to the following two reasons. (1) The optical network topology information is not well considered. (2) The path-level features are highly related to the RSA decision, due to the spectrum continuity constraint, while path-level feature extraction has not been well exploited. To overcome the above shortcomings, in this paper, we propose to utilize the Graph Convolutional Neural Network (GCN) and the Recurrent Neural Network (RNN) for the feature extraction of the network topology, and the aggregation of link-level features to path-level features. By doing this, critical RSA-related information can be sensed by the DRL agent to make better actions. Simulation results demonstrate that our proposed method outperforms previous approaches.

We review our work on an intra-data center (DC) network based on co-deployment of optical packet switching (OPS) and optical circuit switching (OCS), conducted within the framework of a five-year-long national R&D program in Japan (∼March 2016). For the starter, preceding works relevant to optical switching technologies in intra-DC networks are briefly reviewed. Next, we present the architecture of our torus-topology OPS and agile OCS intra-DC network, together with a new flow management concept, where instantaneous optical path on-demand, so-called Express Path is established. Then, our hybrid optoelectronic packet router (HOPR), which handles 100 Gbps (25 Gbps × 4-wavelength) optical packets and its enabling device and sub-system technologies are presented. The HOPR aims at a high energy-efficiency of 0.09 [W/Gbps] and low-latency of 100 ns regime. Next, we provide the contention resolution strategies in the OPS and agile OCS network and present the performance analysis with the simulation results. It is followed by the discussions on the power consumption of intra-DC networks. We compare the power consumption and the throughput of a conventional fat-tree topology with the N -dimensional torus topology. Finally, for further power saving, we propose a new scheme, which shuts off HOPR buffers according to the server operation status.

In this letter, we study the deep reinforcement learning (DRL)-based routing, modulation and spectrum assignment problem in the elastic optical networks. We emphasize the importance of proper reward design of the DRL framework and we propose to include some heuristic information to the reward design. This introduction of human knowledge to the machine learning, is to reduce the exploration blindness of the latter and lead to more efficient learning of better policies. We make it clear that what kind of heuristic information should be included in the reward design is an open question. Specifically, we propose to consider the spectrum fragmentation level of each candidate path as the heuristic information. As a result, the DRL agent is more inclined to choose the candidate path that leads to lower spectrum fragmentation level, which is more friendly for future traffic requests. Simulation results show that the proposed heuristic reward design scheme outperforms both the simple-reward DRL based approaches and the heuristic rule-based approaches.

To improve the network scalability, a large elastic optical network is typically segmented into multiple autonomous domains, where each domain possesses high autonomy and privacy. This architecture is referred to as the multi-domain elastic optical network (MDEON). In the MDEON, the routing, modulation, and spectrum allocation (RMSA) for the inter-domain service requests are challenging. As a result, deep reinforcement learning (DRL) has been introduced recently where the RMSA policies are learned during the interaction of the DRL agents with the MDEON environment. Due to the autonomy of each domain in the MDEON, joint RMSA is essential to improve the overall performance. To realize the joint RMSA, we propose a hierarchical reinforcement learning (HRL) framework which consists of a high-level DRL module and multiple low-level DRL modules (one for each domain), with the collaboration of DRL modules. More specifically, for the inter-domain service requests, the high-level module obtains some abstracted information from the low-level DRL modules and generates the inter-domain RMSA decision for the low-level modules. Then the low-level DRL module gives the intra-domain RMSA decision and feeds back to the high-level module. The proposed HRL framework preserves the autonomy of each single domain while delivering effective overall network performance through the cooperation of the high-level and low-level DRL modules. Simulation results demonstrate that our proposed method outperforms previous approaches.

A torus-topology photonic data center network with smart flow control is presented, where optical packet switching (OPS), optical circuit switching (OCS), and virtual OCS (VOCS) schemes are all enabled on a unified hardware platform via the combination of hybrid optoelectronic routers (HOPR) and a centralized network controller. An upgraded HOPR is being developed with the goal of handling 100 Gbps optical packets with a high energy efficiency of 90 mW/Gbps and low latency of less than 100 ns. The architecture of HOPR and its enabling technologies are reviewed, including a label processor, optical switch, and optoelectronic shared buffer. We also explain the concept and operation mechanisms of both the data and control planes in the hybrid OPS/OCS/VOCS torus network. The performance of these three transmission modes is evaluated by numerical simulations for a network composed of 4096 HOPRs.

The reinforcement learning-based routing, modulation, and spectrum assignment has been regarded as an emerging paradigm for resource allocation in the elastic optical networks. One limitation is that the learning process is highly dependent on the training environment, such as the traffic pattern or the optical network topology. Therefore, re-training is required in case of network topology or traffic pattern variations, which consumes a great amount of computation power and time. To ease the requirement of re-training, we propose a policy distillation scheme, which distills knowledge from a well-trained teacher model and then transfers the knowledge to the to-be-trained student model, so that the training of the latter can be accelerated. Specifically, the teacher model is trained for one training environment (e.g., the topology and traffic pattern) and the student model is for another training environment. The simulation results indicate that our proposed method can effectively speed up the training process of the student model, and it even leads to a lower blocking probability, compared with the case that the student model is trained without knowledge distillation.