The concept and significance of the so called nonlinear Shannon limit are reviewed and their relation to the channel capacity is analyzed from an information theory point of view. It is shown that this is a limit (if at all) holding only for conventional detection strategies. Indeed, it should only be considered as a limit to the information rate that can be achieved with a given modulation/detection scheme. By virtue of some simple examples and theoretical results, it is also shown that, using the same approximated models commonly adopted for deriving the nonlinear Shannon limit, the information rate can be arbitrarily increased by increasing the input power. To this aim, the validity of some popular approximations to the output distribution is also examined to show that their application outside the scope for which they were devised can lead to pitfalls. To the best of our belief, the existence of a true nonlinear Shannon limit has still not been demonstrated, and the problem of determining the channel capacity of a fiber-optic system in the presence of Kerr nonlinearities is still an open issue.

The performance of optical fiber systems based on nonlinear frequency-division multiplexing (NFDM) or on more conventional transmission techniques is compared through numerical simulations. Some critical issues affecting NFDM systems-namely, the strict requirements needed to avoid burst interaction due to signal dispersion and the unfavorable dependence of performance on burst length-are investigated, highlighting their potentially disruptive effect in terms of spectral efficiency. Two digital processing techniques are finally proposed to halve the guard time between NFDM symbol bursts and reduce the size of the processing window at the receiver, increasing spectral efficiency and reducing computational complexity.

After reviewing models and mitigation strategies for interchannel nonlinear interference (NLI), we study its characteristics and coherence properties. Based on this study, we devise an NLI mitigation strategy which exploits the synergic effect of phase and polarization noise compensation (PPN) and subcarrier multiplexing with symbol-rate optimization. This synergy persists even for high-order modulation alphabets and Gaussian symbols. A particle method for the computation of the resulting achievable information rate and spectral efficiency (SE) is presented and employed to lower-bound the channel capacity. The dependence of the SE on the link length, amplifier spacing, and presence or absence of in-line dispersion compensation is studied. Single-polarization and dual-polarization scenarios with either independent or joint processing of the two polarizations are considered. Numerical results show that, in links with ideal distributed amplification, an SE gain of about 1 bit/s/Hz/polarization can be obtained (or, in alternative, the system reach can be doubled at a given SE) with respect to single-carrier systems without PPN mitigation. The gain is lower with lumped amplification, increases with the number of spans, decreases with the span length, and is further reduced by in-line dispersion compensation. For instance, considering a dispersion-unmanaged link with lumped amplification and an amplifier spacing of 60 km, the SE after 80 spans can be be increased from 4.5 to 4.8 bit/s/Hz/polarization, or the reach raised up to 100 spans (+25%) for a fixed SE.

Constellation shaping is a practical and effective technique to improve the performance and the rate adaptivity of optical communication systems. In principle, it could also be used to mitigate the impact of nonlinear effects, possibly increasing the information rate beyond the current limit dictated by fiber nonlinearity. However, this appealing idea is frustrated by the difficulty of designing an effective shaping strategy that takes into account the nonlinearity and long memory of the fiber channel, as well as the possible interplay with other nonlinearity mitigation strategies. As a result, only little progress has been made so far, while the optimal shaping distribution and the ultimate channel capacity remain unknown. In this work, we describe a novel technique to optimize the shaping distribution in a very general setting and high-dimensional space. For a simplified block-memoryless nonlinear optical channel, the capacity lower bound obtained by the proposed technique can be expressed analytically, establishing the conditions for an unbounded growth of capacity with power. In a more realistic scenario, the technique can be implemented by a rejection sampling algorithm driven by a suitable cost function, and the corresponding achievable information rate estimated numerically. The combination of the proposed technique with an improved (non-Gaussian) decoding metric yields a new capacity lower bound for the dual-polarization WDM channel.

The performance of different probabilistic amplitude shaping (PAS)techniques in the nonlinear regime is investigated, highlighting its dependence on the PAS block length and the interaction with carrier phase recovery (CPR). Different PAS implementations are considered, based on different distribution matching (DM) techniques—namely, sphere shaping, shell mapping with different number of shells, and constant composition DM—and amplitude-to-symbol maps. When CPR is not included, PAS with optimal block length provides a nonlinear shaping gain with respect to a linearly optimized PAS (with infinite block length); among the considered DM techniques, the largest gain is obtained with sphere shaping. On the other hand, the nonlinear shaping gain becomes smaller, or completely vanishes, when CPR is included, meaning that in this case all the considered implementations achieve a similar performance for a sufficiently long block length. Similar results are obtained in different link configurations ( $1\times 180$ km, $15\times 80$ km, and $27\times 80$ km single-mode-fiber links), and also including laser phase noise, except when in-line dispersion compensation is used. Furthermore, we define a new metric, the nonlinear phase noise (NPN) metric, which is based on the frequency resolved logarithmic perturbation models and explains the interaction of CPR and PAS. We show that the NPN metric is highly correlated with the performance of the system. Our results suggest that, in general, the optimization of PAS in the nonlinear regime should always account for the presence of a CPR algorithm. In this case, the reduction of the rate loss (obtained by using sphere shaping and increasing the DM block length) turns out to be more important than the mitigation of the nonlinear phase noise (obtained by using constant-energy DMs and reducing the block length), the latter being already granted by the CPR algorithm.

To achieve the maximum information transfer and face a possible eavesdropper, the samples transmitted in continuous-variable quantum key distribution (CV-QKD) protocols are to be drawn from a continuous Gaussian distribution. As a matter of fact, in practical implementations the transmitter has a finite (power) dynamics and the Gaussian sampling can be only approximated. This requires the quantum protocols to operate at small powers. In this paper, we show that a suitable probabilistic amplitude shaping of a finite set of symbols allows to approximate at will the optimal channel capacity also for increasing average powers. We investigate the feasibility of this approach in the framework of CV-QKD, propose a protocol employing discrete quadrature amplitude modulation assisted with probabilistic amplitude shaping, and we perform the key generation rate analysis assuming a wiretap channel and lossless homodyne detection.