End-to-End Deep Learning in Optical Fibre Communication Systems

Conventional communication systems consist of several signal processing blocks, each performing an individual task at the transmitter or receiver, e.g. coding, modulation, or equalisation. However, there is a lack of optimal, computationally feasible algorithms for nonlinear fibre communications as most techniques are based upon classical communication theory, assuming a linear or perturbed by a small nonlinearity channel. Consequently, the optimal end-to-end system performance cannot be achieved using transceivers with sub-optimum modules. Carefully chosen approximations are required to exploit the data transmission potential of optical fibres. In this thesis, novel transceiver designs tailored to the nonlinear dispersive fibre channel using the universal function approximator properties of artificial neural networks (ANNs) are proposed and experimentally verified. The fibre-optic system is implemented as an end-to-end ANN to allow transceiver optimisation over all channel constraints in a single deep learning process. While the work concentrates on highly nonlinear short-reach intensity modulation/direct detection (IM/DD) fibre links, the developed concepts are general and applicable to different models and systems. Found in many data centre, metro and access networks, the IM/DD links are severely impaired by the dispersion-induced inter-symbol interference and square-law photodetection, rendering the communication channel nonlinear with memory. First, a transceiver based on a simple feedforward ANN (FFNN) is investigated and a training method for robustness to link variations is proposed. An improved recurrent ANN-based design is developed next, addressing the FFNN limitations in handling the channel memory. The systems' performance is verified in first-in-field experiments, showing substantial increase in transmission distances and data rates compared to classical signal processing schemes. A novel algorithm for end-to-end optimisation using experimentally-collected data and generative adversarial networks is also developed, tailoring the transceiver to the specific properties of the transmission link. The research is a key milestone towards end-to-end optimised data transmission over nonlinear fibre systems

Digital Back Propagation via Sub-band Processing in Spatial Multiplexing Systems

An advanced digital backward-propagation (DBP) method using a separate-channels approach (SCA) is investigated for the compensation of inter-channel nonlinearities in spatial- and wavelength-multiplexed systems. Compared to the conventional DBP, intra- and inter-mode cross-phase modulation can be efficiently compensated by including the effect of the inter-channel walk-off in the nonlinear step of the split-step Fourier method. We found that the SCA-DBP relaxes the step size requirements by a factor of 10, while improving performance by 0.8 dB for large walk-off and strong linear coupling. For the first time, it is shown that in spatial multiplexed systems transmission performance can be improved by sub-band processing of back propagated channels

Optical Fiber Communication Systems Based on End-to-End Deep Learning: (Invited Paper)

We investigate end-to-end optimized optical transmission systems based on feedforward or bidirectional recurrent neural networks (BRNN) and deep learning. In particular, we report the first experimental demonstration of a BRNN auto-encoder, highlighting the performance improvement achieved with recurrent processing for communication over dispersive nonlinear channels

Experimental Investigation of Deep Learning for Digital Signal Processing in Short Reach Optical Fiber Communications

We investigate methods for experimental performance enhancement of auto-encoders based on a recurrent neural network (RNN) for communication over dispersive nonlinear channels. In particular, our focus is on the recently proposed sliding window bidirectional RNN (SBRNN) optical fiber autoencoder. We show that adjusting the processing window in the sequence estimation algorithm at the receiver improves the reach of simple systems trained on a channel model and applied "as is" to the transmission link. Moreover, the collected experimental data was used to optimize the receiver neural network parameters, allowing to transmit 42 Gb/s with bit-error rate (BER) below the 6.7% hard-decision forward error correction threshold at distances up to 70km as well as 84 Gb/s at 20 km. The investigation of digital signal processing (DSP) optimized on experimental data is extended to pulse amplitude modulation with receivers performing sliding window sequence estimation using a feed-forward or a recurrent neural network as well as classical nonlinear Volterra equalization. Our results show that, for fixed algorithm memory, the DSP based on deep learning achieves an improved BER performance, allowing to increase the reach of the system.Comment: Invited paper at the IEEE International Workshop on Signal Processing Systems (SiPS) 202

Digital nonlinearity compensation in high-capacity optical communication systems considering signal spectral broadening effect

Nyquist-spaced transmission and digital signal processing have proved effective in maximising the spectral efficiency and reach of optical communication systems. In these systems, Kerr nonlinearity determines the performance limits, and leads to spectral broadening of the signals propagating in the fibre. Although digital nonlinearity compensation was validated to be promising for mitigating Kerr nonlinearities, the impact of spectral broadening on nonlinearity compensation has never been quantified. In this paper, the performance of multi-channel digital back-propagation (MC-DBP) for compensating fibre nonlinearities in Nyquist-spaced optical communication systems is investigated, when the effect of signal spectral broadening is considered. It is found that accounting for the spectral broadening effect is crucial for achieving the best performance of DBP in both single-channel and multi-channel communication systems, independent of modulation formats used. For multi-channel systems, the degradation of DBP performance due to neglecting the spectral broadening effect in the compensation is more significant for outer channels. Our work also quantified the minimum bandwidths of optical receivers and signal processing devices to ensure the optimal compensation of deterministic nonlinear distortions

High-Cardinality Hybrid Shaping for 4D Modulation Formats in Optical Communications Optimized via End-to-End Learning

In this paper we carry out a joint optimization of probabilistic (PS) and geometric shaping (GS) for four-dimensional (4D) modulation formats in long-haul coherent wavelength division multiplexed (WDM) optical fiber communications using an auto-encoder framework. We propose a 4D 10 bits/symbol constellation which we obtained via end-to-end deep learning over the split-step Fourier model of the fiber channel. The constellation achieved 13.6% reach increase at a data rate of approximately 400 Gbits/second in comparison to the ubiquitously employed polarization multiplexed 32-QAM format at a forward error correction overhead of 20%.Comment: 5 pages, 3 figure

Experimental demonstration of dispersion tolerant end-to-end deep learning-based IM-DD transmission system

We experimentally demonstrate an IM-DD system relying on deep neural networks from transmitter to receiver delivering 42 Gb/s over 20 and 40 km at 1550 nm below 3.8×10−3 . The ANN is trained to tolerate deviations in dispersion by as much as ±170ps/n

End-to-end optimized transmission over dispersive intensity-modulated channels using bidirectional recurrent neural networks

We propose an autoencoding sequence-based transceiver for communication over dispersive channels with intensity modulation and direct detection (IM/DD), designed as a bidirectional deep recurrent neural network (BRNN). The receiver uses a sliding window technique to allow for efficient data stream estimation. We find that this sliding window BRNN (SBRNN), based on end-to-end deep learning of the communication system, achieves a significant bit-error-rate reduction at all examined distances in comparison to previous block-based autoencoders implemented as feed-forward neural networks (FFNNs), leading to an increase of the transmission distance. We also compare the end-to-end SBRNN with a state-of-the-art IM/DD solution based on two level pulse amplitude modulation with an FFNN receiver, simultaneously processing multiple received symbols and approximating nonlinear Volterra equalization. Our results show that the SBRNN outperforms such systems at both 42 and 84 Gb/s, while training fewer parameters. Our novel SBRNN design aims at tailoring the end-to-end deep learning-based systems for communication over nonlinear channels with memory, such as the optical IM/DD fiber channel