In this paper, we provide an overview of recent progress on advanced digital signal processing (DSP) techniques for high-capacity long-haul coherent optical fiber transmission systems. Not only the linear impairments existing in optical transmission links need to be compensated, but also, the nonlinear impairments require proper algorithms for mitigation because they become major limiting factors for long-haul large-capacity optical transmission systems. Besides the time domain equalization (TDE), the frequency domain equalization (FDE) DSP also provides a similar performance, with a much-reduced computational complexity. Advanced DSP also plays an important role for the realization of space division multiplexing (SDM). SDM techniques have been developed recently to enhance the system capacity by at least one order of magnitude. Some impressive results have been reported and have outperformed the nonlinear Shannon limit of the single-mode fiber (SMF). SDM introduces the space dimension to the optical fiber communication. The few-mode fiber (FMF) and multi-core fiber (MCF) have been manufactured for novel multiplexing techniques such as mode-division multiplexing (MDM) and multi-core multiplexing (MCM). Each mode or core can be considered as an independent degree of freedom, but unfortunately, signals will suffer serious coupling during the propagation. Multi-input–multi-output (MIMO) DSP can equalize the signal coupling and makes SDM transmission feasible. The machine learning (ML) technique has attracted worldwide attention and has been explored for advanced DSP. In this paper, we firstly introduce the principle and scheme of coherent detection to explain why the DSP techniques can compensate for transmission impairments. Then corresponding technologies related to the DSP, such as nonlinearity compensation, FDE, SDM and ML will be discussed. Relevant techniques will be analyzed, and representational results and experimental verifications will be demonstrated. In the end, a brief conclusion and perspective will be provided.

This paper provides an overview of latest progress on the novel advanced digital signal processing (DSP) techniques for long-haul mode division multiplexing (MDM) systems with high capacity. Space-division multiplexing (SDM) techniques have been developed for a period to increase the capacity of optical communication system by at least one order of magnitude through MDM techniques using few-mode fibers (FMFs) or multi-core multiplexing (MCM) using multi-core fibers (MCFs). The signals in MDM links are mainly impaired by the linear and nonlinear effects in FMFs, making DSP techniques become necessary to undo these impairments. In this paper, we not only review the advanced multiple-input multiple-output (MIMO) DSP techniques for compensating linear impairments in FMFs, but also enclose the state of the art of novel DSP techniques to deal with nonlinear effects. Firstly, we introduce the MIMO schemes for equalizing modal crosstalk and modal dispersion. Then, we focus on the fast tracking of time-varying (TV) channels in FMF links through frequency-domain (FD) recursive least square (RLS) algorithm. Besides, we also cover the mainstream DSP solutions for mode-dependent loss (MDL) and several possible methods to compensate nonlinearity in FMF. Moreover, artificial intelligence (AI) technologies are also discussed for its high nonlinearity tolerance and may bring a revolution in MDM systems on the process of channel equalization, link monitoring, etc. In the end, a brief conclusion and perspective will be provided.

Few-mode erbium-doped fiber amplifiers (FM-EDFAs) are one of the most important optical subsystems for successful space division multiplexed transmission systems. In this paper, we propose a new FM-EDFA designed to achieve significantly reduced differential modal gain (DMG) via strong mode coupling. Using a new numerical model based on a fiber transfer matrix, the DMGs of FM-EDFAs are systematically investigated and two different types of six-mode fiber amplifiers are analyzed, as exemplar demonstrations. In a uniformly doped step-index fiber, the DMG can be reduced from 9.3 to 1.1 dB (i.e., 8.2 dB reduction) and further reduced to 0.5 dB in a dual-layer doping structure.

We present simple, low loss and broadband mode scramblers for mode division multiplexed (MDM) transmission based on few-mode fibers. By simply shortening the length of the long-period fiber grating (LPFG), the optical bandwidth is significantly enhanced and >260 nm bandwidth is predicted in our simulations with a grating length of 0.613 cm. In an experimental demonstration we fabricate a mechanically induced LPFGs using a commercially available 3-dimentional (3D) printing technique and wideband operation is confirmed over the C band with a low loss (0.2 dB) and low mode dependent loss (0.1 dB).

Using principal modes (PMs) can avoid the inter-mode crosstalk in fiber-optic communication and reduce the complexity of digital signal processing. In this paper, a new detection method for PMs based on spatially and spectrally resolved imaging (S ² ) is used to recover their intensity distributions and inter-mode dispersion parameters. This method collects the optical interference information on a two-dimensional plane at different frequencies. The collected data can be used to characterize the principal modes, including their patterns and differential mode group delays. Due to the frequency invariance of PMs even in mode coupling state, which is studied carefully, this proposed method is shown to be robust. Analyses based on a four-mode fiber show that the distributed fluctuation of the coupling coefficients break the frequency invariance of PMs. We experimentally measure the mode characteristics of a four-mode fiber with S ² method. The results show that the mode group delays of degenerate modes can be separated in mode coupling state, which is exactly appropriate for the modified measurement method.