Nonlinear distortion has always been a challenge for optical communication due to the nonlinear transfer characteristics of the fiber itself. The next frontier for optical communication is a second type of nonlinearities, which results from optical and electrical components. They become the dominant nonlinearity for shorter reaches. The highest data rates cannot be achieved without effective compensation. A classical countermeasure is receiver-side equalization of nonlinear impairments and memory effects using Volterra series. However, such Volterra equalizers are architecturally complex and their parametrization can be numerical unstable. This contribution proposes an alternative nonlinear equalizer architecture based on machine learning. Its performance is evaluated experimentally on coherent 88 Gbaud dual polarization 16QAM 600 Gb/s back-to-back measurements. The proposed equalizers outperform Volterra and memory polynomial Volterra equalizers up to 6th orders at a target bit-error rate (BER) of ${10}^{-2}$ by 0.5 dB and 0.8 dB in optical signal-to-noise ratio (OSNR), respectively.

Neuromorphic computing implementing spiking neural networks (SNN) is a promising technology for reducing the footprint of optical transceivers, as required by the fast-paced growth of data center traffic. In this work, an SNN nonlinear demapper is designed and evaluated on a simulated intensity-modulation direct-detection link with chromatic dispersion. The SNN demapper is implemented in software and on the analog neuromorphic hardware system BrainScaleS-2 (BSS-2). For comparison, linear equalization (LE), Volterra nonlinear equalization (VNLE), and nonlinear demapping by an artificial neural network (ANN) implemented in software are considered. At a pre-forward error correction bit error rate of exponent-product2e-3, the software SNN outperforms LE by 1.5 dB, VNLE by 0.3 dB and the ANN by 0.5 dB. The hardware penalty of the SNN on BSS-2 is only 0.2 dB, i.e., also on hardware, the SNN performs better than all software implementations of the reference approaches. Hence, this work demonstrates that SNN demappers implemented on electrical analog hardware can realize powerful and accurate signal processing fulfilling the strict requirements of optical communications.

We successfully demonstrate 400Gbit/s transmission at 1577nm over an unamplified 40km standard single-mode fiber (SSMF) link with 11dB loss target. Commercially available high-bandwidth coherent driver modulator (HB-CDM) and high-bandwidth micro integrated coherent receiver (HB-μICR) designed for Cband are used in combination with two TO-can packaged and thermally tuned low-cost L-band laser sources. With the aid of a memory-less 1 sample/symbol nonlinear offline digital-signal processing (DSP) correcting for the nonlinear transfer curve of the driver and modulator we can maximize the transmitter optical power, which is critical for unamplified fiber applications. In addition, multistage carrier recovery enables stable performance also with the large phase noise of the used coherent laser source, originally part of an electro-absorption modulated laser module (EML) and designed for extremely price-sensitive passive optical network (PON) applications.

In this article, we discuss challenges and options for scaling IM/DD transceivers towards 800Gbps. Our focus is CWDM4 PAM4 transmission and our target distance is 10km in O-band, which is a most urgent use case for next generation optical short reach systems like data centre interconnects and networks. At this reach and rate, chromatic dispersion (CD) becomes the main challenge. Its mitigation is essential and primarily done with digital signal processing. State of the art techniques, however, make transceivers quickly too complex. We show upon measurement results how neural network equalization can meet Volterra equalization performance with 30% less hardware multiplier complexity. When also applying magnitude weight pruning, an additional 43% reduction is possible without performance loss across all CWDM4 lanes. If needed, an added MLSE stage can further push performance in both cases. In any of these configurations, a key enabler against strong CD penalties is duobinary training, which is applicable to all feed-forward equalization architectures.

Coherent systems based on QPSK rely on differential encoding to avoid catastrophic error propagation. A simple solution for soft differential decoding is presented that limits the penalty after FEC to 0.75dB, similar to pre-FEC binary differential decoding.

We analyze the advances of deep learning in optical coherent modems on the physical layer with respect to modulation design, equalization and signal detection and give an outlook on a combined control and physical layer optimization using neural networks.