For W-band long-range mm-wave wireless transmission systems, nonlinearity issues resulting from photoelectric devices, optical fibers, and wireless power amplifiers can be handled by deep learning equalization algorithms. In addition, the PS technique is considered an effective measure to further increase the capacity of the modulation-constraint channel. However, since the probabilistic distribution of m-QAM varies with the amplitude, there have been difficulties in learning valuable information from the minority class. This limits the benefit of nonlinear equalization. To overcome the imbalanced machine learning problem, we propose a novel two-lane DNN (TLD) equalizer using the random oversampling (ROS) technique in this paper. The combination of PS at the transmitter and ROS at the receiver improved the overall performance of the W-band wireless transmission system, and our 4.6-km ROF delivery experiment verified its effectiveness for the W-band mm-wave PS-16QAM system. Based on our proposed equalization scheme, we achieved single-channel 10-Gbaud W-band PS-16QAM wireless transmission over a 100 m optical fiber link and a 4.6 km wireless air-free distance. The results show that compared with the typical TLD without ROS, the TLD-ROS can improve the receiver‘s sensitivity by 1 dB. Furthermore, a reduction of 45.6% in complexity was achieved, and we were able to reduce training samples by 15.5%. Considering the actual wireless physical layer and its requirements, there is much to be gained from the joint use of deep learning and balanced data pre-processing techniques.

In this paper, we explore the enhancement of a 4.6 km dual-polarization 2 × 2 MIMO D-band photonic-assisted terahertz communication system using iterative pruning-based deep neural network (DNN) nonlinear equalization techniques. The system employs advanced digital signal processing (DSP) methods, including down-conversion, resampling, matched filtering, and various equalization algorithms to combat signal distortions. We demonstrate the effectiveness of DNN and iterative pruning techniques in significantly reducing bit error rates (BERs) across a range of symbol rates (10 Gbaud to 30 Gbaud) and polarization states (vertical and horizontal). Before pruning, at 10 GBaud transmission, the lowest BER was 0.0362, and at 30 GBaud transmission, the lowest BER was 0.1826, both of which did not meet the 20% soft-decision forward error correction (SD-FEC) threshold. After pruning, the BER at different transmission rates was reduced to below the hard decision forward error correction (HD-FEC) threshold, indicating a substantial improvement in signal quality. Additionally, the pruning process contributed to a decrease in network complexity, with a maximum reduction of 85.9% for 10 GBaud signals and 63.0% for 30 GBaud signals. These findings indicate the potential of DNN and pruning techniques to enhance the performance and efficiency of terahertz communication systems, providing valuable insights for future high-capacity, long-distance wireless networks.

With the rise of 6G, the exponential growth of data traffic, the proliferation of emerging applications, and the ubiquity of smart devices, the demand for spectral resources is unprecedented. Terahertz communication (100 GHz–3 THz) plays a key role in alleviating spectrum scarcity through ultra-broadband transmission. In this study, terahertz optical carrier-based systems are employed, where fiber-optic components are used to generate the optical signals, and the signal is transmitted via direct detection in the receiver side, without relying on fiber-optic transmission. In these systems, deep learning-based equalization effectively compensates for nonlinear distortions, while probability shaping (PS) enhances system capacity under modulation constraints. However, the probability distribution of signals processed by PS varies with amplitude, making it challenging to extract useful information from the minority class, which in turn limits the effectiveness of nonlinear equalization. Furthermore, in IM-DD systems, optical multipath interference (MPI) noise introduces signal-dependent amplitude jitter after direct detection, degrading system performance. To address these challenges, we propose a lightweight neural network equalizer assisted by the Synthetic Minority Oversampling Technique (SMOTE) and a clustering method. Applying SMOTE prior to the equalizer mitigates training difficulties arising from class imbalance, while the low-complexity clustering algorithm after the equalizer identifies edge jitter levels for decision-making. This joint approach compensates for both nonlinear distortion and jitter-related decision errors. Based on this algorithm, we conducted a 3.75 Gbaud W-band PAM4 wireless transmission experiment over 300 m at Fudan University’s Handan campus, achieving a bit error rate of 1.32 × 10⁻³, which corresponds to a 70.7% improvement over conventional schemes. Compared to traditional equalizers, the proposed new equalizer reduces algorithm complexity by 70.6% and training sequence length by 33%, while achieving the same performance. These advantages highlight its significant potential for future optical carrier-based wireless communication systems.

For W-band long-range mm-wave wireless transmission systems, nonlinearity issues resulting from photoelectric devices, optical fiber, and wireless power amplifiers can be handled by deep learning equalization algorithms. Besides, the PS technique is considered an effective measure to further increase the capacity of the modulation-constraint channel. However, since the proba-bilistic distribution of m-QAM varies with the amplitude, there have been difficulties in learning valuable information from the minority class. This limits the gain of nonlinear equalization. To overcome the imbalanced machine learning problem, we propose a novel two-lane DNN (TLD) equalizer with the random over-sampling (ROS) technique in this paper. The combination of PS at the transmitter and ROS at the receiver improves the overall performance of the W-band wireless transmission system, and our 4.6-km ROF delivery experiment verifies its effectiveness for the W-band mm-wave PS-16QAM system. Based on our proposed equalization scheme, we realize single-channel 10-Gbaud W-band PS-16QAM wireless transmission over 100 m optical fiber link and 4.6-km wireless air-free distance. The results show that compared with the typical TLD without ROS, the TLD-ROS can improve the receiver sensitivity by 1dB. Furthermore, a reduction of 45.6% in complexity is realized, and training samples can be reduced by 15.5%. Considering the actual wireless physical layer with its requirements, there is much to be gained from the joint use of deep learning and balanced data pre-processing techniques.