High demand for 6G wireless has made photonics-aided D-band (110–170 GHz) communication a research priority. Photonics-aided technology integrates optical and wireless communications to boost spectral efficiency and transmission distance. This study presents a Radio-over-Fiber (RoF) communication system utilizing photonics-aided technology for 4600 m long-distance D-band transmission. We successfully show the transmission of a 50 Gbps (25 Gbaud) QPSK signal utilizing a 128.75 GHz carrier frequency. Notwithstanding these encouraging outcomes, RoF systems encounter considerable obstacles, including pronounced nonlinear distortions and phase noise related to laser linewidth. Numerous factors can induce nonlinear impairments, including high-power amplifiers (PAs) in wireless channels, the operational mechanisms of optoelectronic devices (such as electrical amplifiers, modulators, and photodiodes), and elevated optical power levels during fiber transmission. Phase noise (PN) is generated by laser linewidth. Despite the notable advantages of classical Volterra series and deep neural network (DNN) methods in alleviating nonlinear distortion, they display considerable performance limitations in adjusting for phase noise. To address these problems, we propose a novel post-processing approach utilizing a two-dimensional convolutional neural network (2D-CNN). This methodology allows for the extraction of intricate features from data preprocessed using traditional Digital Signal Processing (DSP) techniques, enabling concurrent compensation for phase noise and nonlinear distortions. The 4600 m long-distance D-band transmission experiment demonstrated that the proposed 2D-CNN post-processing method achieved a Bit Error Rate (BER) of 5.3 × 10⁻³ at 8 dBm optical power, satisfying the soft-decision forward error correction (SD-FEC) criterion of 1.56 × 10⁻² with a 15% overhead. The 2D-CNN outperformed Volterra series and deep neural network approaches in long-haul D-band RoF systems by compensating for phase noise and nonlinear distortions via spatiotemporal feature integration, hierarchical feature extraction, and nonlinear modelling.

In this paper, we explore the enhancement of a 4.6 km dual-polarization 2 × 2 MIMO D-band photonic-assisted terahertz communication system using iterative pruning-based deep neural network (DNN) nonlinear equalization techniques. The system employs advanced digital signal processing (DSP) methods, including down-conversion, resampling, matched filtering, and various equalization algorithms to combat signal distortions. We demonstrate the effectiveness of DNN and iterative pruning techniques in significantly reducing bit error rates (BERs) across a range of symbol rates (10 Gbaud to 30 Gbaud) and polarization states (vertical and horizontal). Before pruning, at 10 GBaud transmission, the lowest BER was 0.0362, and at 30 GBaud transmission, the lowest BER was 0.1826, both of which did not meet the 20% soft-decision forward error correction (SD-FEC) threshold. After pruning, the BER at different transmission rates was reduced to below the hard decision forward error correction (HD-FEC) threshold, indicating a substantial improvement in signal quality. Additionally, the pruning process contributed to a decrease in network complexity, with a maximum reduction of 85.9% for 10 GBaud signals and 63.0% for 30 GBaud signals. These findings indicate the potential of DNN and pruning techniques to enhance the performance and efficiency of terahertz communication systems, providing valuable insights for future high-capacity, long-distance wireless networks.