The advent of silica-based low-cost standard single-mode fibers revolutionized the whole communication industry. The deployment of optical fibers in the networks induces a paradigm shift in the communication technologies used for long-haul information transfer. However, the communication using the optical fibers is affected by several linear and nonlinear effects. The most common linear effects are attenuation and chromatic dispersion, whereas the dominant nonlinear effect is the Kerr effect. The Kerr effect induces a power-dependent nonlinear distortion for the signal propagating in the optical fiber. The detrimental effects of the Kerr nonlinearity limit the capacity of long-haul optical communication systems. Fiber Kerr nonlinearity compensation using digital signal processing (DSP) techniques has been well investigated over several years. In this paper, we provide a comprehensive tutorial, including the fundamental mathematical analysis, on the characteristics of the optical fiber channel, the origin of the Kerr nonlinearity effect, the theory of the pulse propagation in the optical fiber, and the numerical and analytical tools for solving the pulse propagation equation. In addition, we provide a concise review of various DSP techniques for fiber nonlinearity compensation, such as digital back-propagation, Volterra series-based nonlinearity equalization, perturbation theory-based nonlinearity compensation, and phase conjugation. We also carry out numerical simulation and the complexity evaluation of the selected nonlinearity compensation techniques.

Derived from the regular perturbation treatment of the nonlinear Schrödinger equation, a machine learning-based scheme to mitigate the intra-channel optical fiber nonlinearity is proposed. Referred to as the perturbation theory-aided (PA) learned digital back propagation (LDBP), the proposed scheme constructs a deep neural network (DNN) in a way similar to the split-step Fourier method: linear and nonlinear operations alternate. Inspired by the perturbation analysis, the intra-channel cross phase modulation term is conveniently represented by matrix operations in the DNN. The introduction of this term in each nonlinear operation considerably improves the performance, as well as enables the flexibility of PA-LDBP by adjusting the numbers of spans per step. The proposed scheme is evaluated by numerical simulations of a single-carrier optical fiber communication system operating at 32 Gbaud with 64-quadrature amplitude modulation and 20 × 80 km transmission distance. The results show that the proposed scheme achieves approximately 1 dB, 1.2 dB, 1.2 dB, and 0.5 dB performance gain in terms of ${\it Q}$ factor over LDBP, when the numbers of spans per step are 1, 2, 4, and 10, respectively. Two methods are proposed to reduce the complexity of PA-LDBP, i.e., pruning the number of perturbation coefficients and chromatic dispersion compensation in the frequency domain for multi-span per step cases. Investigation of the performance and complexity suggests that PA-LDBP attains improved performance gains with reduced complexity when compared to LDBP in the cases of 4 and 10 spans per step.

Fiber nonlinearity effects cap achievable rates and ranges in long-haul optical fiber communication links. Conventional nonlinearity compensation methods, such as perturbation theory-based nonlinearity compensation (PB-NLC), attempt to compensate for the nonlinearity by approximating analytical solutions to the signal propagation over optical fibers. However, their practical usability is limited by model mismatch and the immense computational complexity associated with the analytical computation of perturbation triplets and the nonlinearity distortion field. Recently, machine learning techniques have been used to optimise parameters of PB-based approaches, which traditionally have been determined analytically from physical models. It has been claimed in the literature that the learned PB-NLC approaches have improved performance and/or reduced computational complexity over their non-learned counterparts. In this paper, we first revisit the acclaimed benefits of the learned PB-NLC approaches by carefully carrying out a comprehensive performance-complexity analysis utilizing state-of-the-art complexity reduction methods. Interestingly, our results show that least squares-based PB-NLC with clustering quantization has the best performance-complexity trade-off among the learned PB-NLC approaches. Second, we advance the state-of-the-art of learned PB-NLC by proposing and designing a fully learned structure by adopting the noiseless Manakov equation as the channel propagation model. We apply a bi-directional recurrent neural network for learning perturbation triplets that are alike those obtained from the analytical computation and are used as input features for the neural network to estimate the nonlinearity distortion field. Finally, we demonstrate through numerical simulations that our proposed fully learned approach achieves an improved performance-complexity trade-off compared to the existing learned and non-learned PB-NLC techniques.

The first-order (FO) perturbation theory-based nonlinearity compensation (PB-NLC) technique has been widely investigated to combat the detrimental effects of the intra-channel Kerr nonlinearity in polarization-multiplexed (Pol-Mux) optical fiber communication systems. However, the NLC performance of the FO-PB-NLC technique is significantly limited in highly nonlinear regimes of the Pol-Mux long-haul optical transmission systems. In this paper, we extend the FO theory to second-order (SO) to improve the NLC performance. This technique is referred to as the SO-PB-NLC. A detailed theoretical analysis is performed to derive the SO perturbative field for a Pol-Mux optical transmission system. Following that, we investigate a few simplifying assumptions to reduce the implementation complexity of the SO-PB-NLC technique. The numerical simulations for a single-channel system show that the SO-PB-NLC technique provides an improved bit-error-rate performance and increases the transmission reach, in comparison with the FO-PB-NLC technique. The complexity analysis demonstrates that the proposed SO-PB-NLC technique has a reduced computational complexity when compared to the digital back-propagation with one step per span.

The regular perturbation (RP) series used to analytically approximate the solution of the nonlinear Schrodinger equation has a serious energy divergence problem when truncated to the first-order. The enhanced RP (ERP) method can improve the accuracy of the first-order RP approximation by solving the energy divergence problem. In this paper, we propose an ERP-based nonlinearity compensation technique, referred to as ERP-NLC, to compensate for the fiber nonlinearity in a polarization-division multiplexed dispersion unmanaged optical communication system. We also propose a modified perturbation-based NLC (PB-NLC) technique by simple phase-rotation (PR) of the nonlinear coefficient matrix, referred to as the PR-PB-NLC. The PR-PB-NLC can be considered as a by-product of the ERP-NLC technique. We show through numerical simulation that, for a 256 Gb/s single-channel system, the proposed ERP-NLC technique improves the Q-factor performance by ~1.2 dB and ~0.6 dB when compared to the electronic dispersion compensation (EDC) and the PBNLC techniques, respectively, at a transmission distance of 2800 km. Also, the result for a 1.28 Tb/s wavelength-division multiplexed five-channel transmission system at the same transmission distance shows that the Q-factor performance of the ERP-NLC technique is improved by ~0.6 dB and ~0.4 dB when compared to the EDC and the PB-NLC techniques, respectively. The simulation results for the PR-PB-NLC technique for a single or five-channel transmission system shows an improved Q-factor performance when compared to the EDC and PB-NLC techniques. Finally, we show that the proposed performance enhancement comes with a negligible increase in the computational complexity for the ERP-NLC and PR-PB-NLC techniques when compared to the PB-NLC technique.

Fiber nonlinearity imposes limitations on the transmission distances in optical fiber networks. Fiber nonlinearity compensation (NLC) becomes essential for extending the transmission reach; however, conventional methods like digital backpropagation (DBP) experience challenges related to the intricacies of computational demands. To mitigate the fiber nonlinearity cost-effectively, subcarrier multiplexing (SCM) emerges as a promising solution compared to single-carrier systems. However, the SCM performance is limited by nonlinear effects such as self-subcarrier nonlinearity (SSN) and cross-subcarrier nonlinearity (CSN). In previous studies, a combination of SCM with DBP, named SCM-DBP, has been employed to address these issues. Concurrently, deep learning-assisted NLC, for example, learned DBP (LDBP), has shown promise in enhancing performance and reducing complexity. In this paper, we aim to apply learning to the SCM-DBP by holistically combining the principles of the SCM-DBP and LDBP approaches, denoted as SCM-LDBP, to mitigate SSN and CSN cost-effectively. To investigate the efficacy of our proposed SCM-LDBP technique, we carry out numerical simulations for both a contemporary 32 Gbaud and a strategic 120 Gbaud SCM transmission system over a 1600km optical fiber link. With only two interfering subcarriers in the back-propagation routine, our proposed SCM-LDBP demonstrates a 0.3dB Q factor improvement and a 31.7% complexity reduction in the 32 Gbaud system when compared to the SCM-DBP. Similarly, in the 120 Gbaud system, the proposed SCM-LDBP demonstrates a 0.2dB Q factor improvement and a 37.8% reduction in complexity over the SCM-DBP technique.