Generalized Maximum Likelihood for Cross-Polarization Modulation Effects Compensation

We investigate the mitigation of nonlinearities with advanced signal processing with a particular focus on cross-polarization effects. Based on relaxations of a previously introduced model for cross-polarization effects, this paper proposes a Generalized Maximum Likelihood algorithm. It performs a joint blind channel estimation and symbol detection which accounts for the statistical prior distributions of the XPolM crosstalk coefficients. This avoids an overestimation of the XPolM crosstalk coefficients. A practical method for both fast computations and optimal performance is presented. Monte-Carlo simulations show that the proposed algorithm performs close to the theoretical limits. Large performance improvement is obtained, which is particularly emphasized with higher order modulation such as 16QAM. Finally, using Nyquist pulse shaping and polarization division multiplexed QPSK modulation, the experiments are shown to be in accordance with the simulations and show up to 0.7dB improvement in Q-factor for the worst case samples.