An improved neural network-based equalization method is proposed and experimentally demonstrated. The up-to-137 Gb/s transmission of four level pulse amplitude modulation (PAM-4) signals with 25 G class 850 nm optical devices is achieved over an in-house fabricated 40 cm optical backplane. An in-depth investigation is conducted regarding the impact of delayed taps and spans on equalization performance. A performance comparison of the proposed method with the traditional maximum likelihood sequence estimation (MLSE) and decision feedback equalization (DFE) is also undertaken. For the bit rate from 80 to 100 Gb/s, the proposed method achieves an adopted hard-decision forward error correction (HD-FEC) requirement at a received optical power (RoP) of −9 and −8 dBm, while DFE and MLSE cannot meet the HD-FEC requirement. When the bit rate increases from 120 to 137 Gb/s, the proposed equalization method still successfully maintains the acceptable system performance at an RoP of −4 and −2.5 dBm. Furthermore, the specific bit error rate (BER) performances for varied maximum achievable bit rate under different RoPs by applying MLSE and the proposed method are also analyzed. This provides an important potential solution to realize the future data centers.

In this paper, an improved end-to-end autoencoder based on reinforcement learning by using Decision Tree for optical transceivers is proposed and experimentally demonstrated. Transmitters and receivers are considered as an asymmetrical autoencoder combining a deep neural network and the Adaboost algorithm. Experimental results show that 48 Gb/s with 7% hard-decision forward error correction (HD-FEC) threshold under 65 km standard single mode fiber (SSMF) is achieved with proposed scheme. Moreover, we further experimentally study the Tree depth and the number of Decision Tree, which are the two main factors affecting the bit error rate performance. Experimental research afterwards showed that the effect from the number of Decision Tree as 30 on bit error rate (BER) flattens out under 48 Gb/s for the fiber range from 25 km and 75 km SSMF, and the influence of Tree depth on BER appears to be a gentle point when Tree Depth is 5, which is defined as the optimal depth point for aforementioned fiber range. Compared to the autoencoder based on a Fully-Connected Neural Network, our algorithm uses addition operations instead of multiplication operations, which can reduce computational complexity from 10⁸ to 10⁷ in multiplication and 10⁶ to 10⁸ in addition on the training phase.