We investigate the potential of adaptive blind equalizers based on
variational inference for carrier recovery in optical communications. These
equalizers are based on a low-complexity approximation of maximum likelihood
channel estimation. We generalize the concept of variational autoencoder (VAE)
equalizers to higher order modulation formats encompassing probabilistic
constellation shaping (PCS), ubiquitous in optical communications, oversampling
at the receiver, and dual-polarization transmission. Besides black-box
equalizers based on convolutional neural networks, we propose a model-based
equalizer based on a linear butterfly filter and train the filter coefficients
using the variational inference paradigm. As a byproduct, the VAE also provides
a reliable channel estimation. We analyze the VAE in terms of performance and
flexibility over a classical additive white Gaussian noise (AWGN) channel with
inter-symbol interference (ISI) and over a dispersive linear optical
dual-polarization channel. We show that it can extend the application range of
blind adaptive equalizers by outperforming the state-of-the-art
constant-modulus algorithm (CMA) for PCS for both fixed but also time-varying
channels. The evaluation is accompanied with a hyperparameter analysis.Comment: Published (Open Access) in IEEE Journal on Selected Areas in
Communications, Sep 202
