Digital coherent transceivers have revolutionized optical fiber communications due to their superior performance offered compared to intensity modulation and direct detection based alternatives. As systems employing digital coherent transceivers seek to approach their information theoretic capacity, the use of multilevel modulation formats combined with appropriate forward error correction becomes essential. Given this context, in this tutorial paper, we, therefore, explore the digital signal processing (DSP) utilized in a coherent transceiver with a focus on multilevel modulation formats. By way of an introduction, we open by discussing the photonic technology required to realize a coherent transceiver. After discussing this interface between the analog optical channel and the digital domain, the rest of the paper is focused on DSP. We begin by discussing algorithms that correct for imperfections in the optical to digital conversion, including IQ imbalance and timing skew. Next, we discuss channel equalization including means for their realization for both quasi-static and dynamic channel impairments. Synchronization algorithms that correct for the difference between the transmitter and receiver oscillators both optical and electrical are then discussed and issues associated with symbol decoding highlighted. For most of the cases, we start with polarization division multiplexed quadrature phase-shift keying format as a basis and then discuss the extension to allow for high-order multilevel formats. Finally, we conclude by discussing some of the open research challenges in the field.

We propose a novel frequency-domain adaptive equalizer in digital coherent optical receivers, which can reduce computational complexity of the conventional time-domain adaptive equalizer based on finite-impulse-response (FIR) filters. The proposed equalizer can operate on the input sequence sampled by free-running analog-to-digital converters (ADCs) at the rate of two samples per symbol; therefore, the arbitrary initial sampling phase of ADCs can be adjusted so that the best symbol-spaced sequence is produced. The equalizer can also be configured in the butterfly structure, which enables demultiplexing of polarization tributaries apart from equalization of linear transmission impairments. The performance of the proposed equalization scheme is verified by 40-Gbits/s dual-polarization quadrature phase-shift keying (QPSK) transmission experiments.

We propose a novel method of compensation for imbalance between in-phase (I) and quadrature (Q) channels in the front-end circuit of digital coherent optical receivers. Adaptive finite-impulse-response (FIR) filters in the butterfly configuration, which are commonly used for signal equalization and polarization demultiplexing, are modified so as to allow for adjustment of any imbalance between the IQ channels. IQ imbalances under consideration include the gain mismatch, the phase mismatch, and the timing-delay skew. Computer simulations for the dual-polarization quadrature-amplitude-modulation (QAM) format up to an order of 256 show that such IQ imbalances can severely degrade the system performance, especially for higher order QAM; however, using the proposed scheme, we can compensate for them without any significant penalty over a wide range of imbalances.

The recent evolution of passive optical network standards and related research activities for physical layer solutions that achieve bit rates well above 10 Gbps per wavelength ( λ ) is discussed. We show that the advancement toward 50, 100, and 200 G b p s / λ will certainly require a strong introduction of advanced digital signal processing (DSP) technologies for linear, and maybe nonlinear, equalization and for forward error correction. We start by reviewing in detail the current standardization activities in the International Telecommunication Union and the Institute of Electrical and Electronics Engineers, and then we present a comparison of the DSP approaches for traditional direct detection solutions and for future coherent detection approaches.

Future passive optical networks (PONs) for 200Gb/s/λ, and beyond, pose a significant technological challenge. The use of coherent technology in access networks provides a great solution based on concepts from mature technologies to achieve the speeds needed for very-high-speed PON (VHSP). In this paper, we provide an overview of the currently demonstrated technologies and propose a possible simplified optical network unit (ONU) for time division multiplex PON (TDM-PON). The proposed ONU uses a single polarization heterodyne receiver, using either a balanced photodiode or a single-ended one and an electro-absorption modulated laser (EML)-based transmitter. The experimental demonstration using the proposed ONU in a bidirectional, symmetrical transmission over fiber distances of 20 km and 40 km shows the viability of the technology. The downstream direction achieves a power budget of 34.3/29.3 dB for 20 km and 33.9/29 dB for 40 km, for a balanced/single-ended receiver, whereas the upstream transmission achieves 29.3 dB for both scenarios.

We propose a novel method of in-band estimation of optical signal-to-noise ratio (OSNR) using a digital coherent receiver, where OSNR is determined from second- and fourth-order statistical moments of equalized signals in any modulation format. Our proposed method is especially important in recently-developed Nyquist wavelength-division multiplexed (WDM) systems and/or reconfigurable optical-add/drop-multiplexed (ROADM) networks, because in these systems and networks, we cannot apply the conventional OSNR estimation method based on optical-spectrum measurements of the in-band signal and the out-of-band noise. Effectiveness of the proposed method is validated with computer simulations of Nyquist-WDM systems and ROADM networks using 25-Gbaud quadrature phase-shift keying (QPSK) and 16 quadrature-amplitude modulation (16-QAM) formats.