Nonlinear impairments caused by devices and fiber transmission links in a coherent optical communication system can severely limit its transmission distance and achievable capacity. In this paper, we propose a low-complexity pruned-convolutional-neural-network-(CNN)-based nonlinear equalizer, to compensate nonlinear signal impairments for coherent optical communication systems. By increasing the size of the effective receptive field with an 11 × 11 large convolutional kernel, the performance of feature extraction for CNNs is enhanced and the structure of the CNN is simplified. And by performing the channel-level pruning algorithm, to prune the insignificant channels, the complexity of the CNN model is dramatically reduced. These operations could save the important component of the CNN model and reduce the model width and computation amount. The performance of the proposed CNN-based nonlinear equalizer was experimentally evaluated in a 120 Gbit/s 64-quadrature-amplitude-modulation (64-QAM) coherent optical communication system over 375 km of standard single-mode fiber (SSMF). The experimental results showed that, compared to a CNN-based nonlinear equalizer with a 6 × 6 normal convolutional kernel, the proposed CNN-based nonlinear equalizer with an 11 × 11 large convolutional kernel, after channel-level pruning, saved approximately 15.5% space complexity and 43.1% time complexity, without degrading the equalization performance. The proposed low-complexity pruned-CNN-based nonlinear equalizer has great potential for application in realistic devices and holds promising prospects for coherent optical communication systems.

In this paper, an improved complex-valued convolutional neural network (CvCNN) structure to be placed at the received side is proposed for nonlinearity compensation in a coherent optical system. This complex-valued global convolutional kernel-assisted convolutional neural network equalizer (CvGNN) has been verified in terms of Q-factor performance and complexity compared to seven other related nonlinear equalizers based on both the 64 QAM experimental platform and the QPSK numerical platform. The global convolution operation of the proposed CvGNN is more suitable for the calculation process of perturbation coefficients, and the global receptive field can also be more effective at extracting effective information from perturbation feature maps. The introduction of CvCNN can directly focus on the complex-valued perturbation feature maps themselves without separately processing the real and imaginary parts, which is more in line with the waveform-dependent physical characteristics of optical signals. Based on the experimental platform, compared with the real-valued neural network with small convolutional kernel (RvCNNC), the proposed CvGNNC improves the Q-factor by ∼2.95 dB at the optimal transmission power, while reducing the time complexity by ∼44.7%.

In this paper, Gaussian smoothing (GS), non-local means (NLM), and Quaternion Wavelet Transform (QWT) have been described in detail. Furthermore, a Brillouin optical time domain analysis (BOTDA) experimental system was built to verify the denoising algorithms. The principal and experimental analyses show that the QWT algorithm is a more efficient image denoising method. The results indicate that the GS algorithm can obtain the highest signal-to-noise ratio (SNR), frequency uncertainty, and Brillouin frequency shift (BFS) accuracy, and can be executed in an imperceptible time, but the GS algorithm has the lowest spatial resolution. After being denoised by using NLM algorithm, although SNR, frequency uncertainty, BFS accuracy, and spatial resolution significantly improved, it takes 40 min to implement the NLM denoising algorithm for a BGS image with 200 × 100,000 points. Processed by the QWT denoising algorithm, although SNR increases to 17.26 dB and frequency uncertainty decreases to 0.24 MHz, a BFS accuracy of only 0.2 MHz can be achieved. Moreover, the spatial resolution is 3 m, which is not affected by the QWT denoising algorithm. It takes less than 32 s to denoise the same raw BGS data. The QWT image denoising technique is suitable for BGS data processing in the BOTDA sensor system.