This paper proposes a novel method to improve the anti-dispersion ability of the all-optical orthogonal frequency division multiplexing (AO-OFDM) system. By replacing the Sinc-shaped filter with a Gauss-shaped filter for sub-carrier generation and inserting a cyclic prefix (CP), the impact of dispersion on the system can be significantly mitigated. Formula derivation and numerical analysis of the pulse-shaping function of the AO-OFDM system in the time domain for each cycle indicated that the pulse-shaping function generated by the Gauss-shaped filter was less affected by the dispersion effect than that of the Sinc-shaped filter. Meanwhile, less inter-carrier crosstalk between carriers was also observed. After carrying out system transmission simulations employing these two different filters, we found that the AO-OFDM system based on the Gauss-shaped filter could greatly improve the anti-dispersion ability compared with the system based on a Sinc-shaped filter. When the parameter settings in both schemes were identical, that is, the number of subcarriers was 32 and the power of a single subcarrier was −13 dBm, the bit error rate (BER) of the system based on the proposed Gauss-shaped filter after 60 km SMF transmission was only 1.596 × 10⁻³, while the BER of the traditional Sinc-shaped filter based system scheme was as high as 8.545 × 10⁻².

For high-speed optical communication systems, laser phase noise (LPN) stands as a pivotal factor influencing the quality of the received signal. Therefore, the employment of a highly accurate carrier phase recovery (CPR) algorithm at the receiving end is indispensable to ensure the reliability of transmission. While a CPR algorithm called principal component-based phase estimation (PCPE) has been proven to be capable of achieving low-complexity and high-performance phase recovery for even-bit quadrature amplitude modulation (QAM) (i.e., square QAM) signals, it is not compatible with traditional cross-shaped odd-bit QAM signals. To circumvent this problem, a signal constellation design scheme based on geometric shaping (GS) is proposed. The pair-wise optimization (PO) algorithm is used to optimize the constellation structure of 32QAM and 128QAM signals in order to obtain results that are compatible with the PCPE algorithm. Monte Carlo simulation results reveal that for odd-bit QAM signals utilizing PCPE for phase recovery, the proposed GS constellations enhance the mutual information (MI) performance across the entire measured signal-to-noise (SNR) range. Moreover, compared with regular 32QAM and 128QAM constellations using the well-known blind phase search (BPS) algorithm, the proposed GS and PCPE scheme can achieve SNR gains of 1.10 dB and 2.59 dB, respectively, when considering the 20% soft-decision forward error correction (SD-FEC) overhead. Verification through commercial simulation software corroborates these findings, demonstrating that the proposed GS constellations are particularly suitable for the PCPE algorithm, especially under conditions of high optical signal-to-noise ratio (OSNR). To the best of our knowledge, this is the first time that the incompatibility between the PCPE algorithm and odd-bit QAM signals has been investigated, and the proposed GS scheme has broadened the application scope of the low-complexity CPR algorithm.