Channel estimation and beam training with machine learning applications for millimetre-wave communication systems

The fifth generation (5G) wireless system will extend the capabilities of the fourth generation (4G) standards to serve more users and provide timely communication. To this end, the carriers of 5G systems will be able to operate at higher frequency bands, such as the millimetre-wave (mmWave) bands that span from 30 GHz to 300 GHz, to obtain greater bandwidths and higher data rates. As a result, the deployment of 5G networks is required to accommodate more antennas and offer pervasive coverage with controlled power consumption. The complexity of 5G systems introduces new challenges to traditional signal processing techniques. To address these challenges, a major step is to integrate machine learning (ML) algorithms into wireless communication systems. ML can learn patterns from datasets to achieve control and optimisation of complex radio frequency (RF) networks. This PhD thesis focuses on developing efficient channel estimation methods and beam training strategies with the application of ML algorithms for mmWave wireless systems. Firstly, the channel estimation and signal detection problem is investigated for orthogonal frequency-division multiplexing (OFDM) systems that operate at mmWave bands. A deep neural network (DNN)-based joint channel estimation and signal detection approach is proposed to achieve multi-user detection in a one-shot process for non-orthogonal multiple access (NOMA) systems. The DNN acts as the receiver, which can recover the transmitted data by learning the channel implicitly from suitable training. The proposed approach can be adapted to work for both single-input and single-output (SISO) systems and multiple-output and multipleoutput (MIMO) systems. This DNN-based approach is shown to provide good performance for OFDM systems that suffer from severe inter-symbol interference or where small numbers of pilot symbols are used. Secondly, the beam training and tracking problem is studied for mmWave channels with receiver mobility. To reduce the signalling overhead caused by frequent beam training, a lowcomplexity beam training strategy is proposed for mobile mmWave channels, which searches a set of selected beams obtained based on the recent beam search results. By searching only the adjacent beams to the one recently used, the proposed beam training strategy can reduce the beam training delay significantly while maintaining high transmission rates. The proposed strategy works effectively for channel datasets generated using either the stochastic or the raytracing channel model. This strategy is shown to approach the performance for an exhaustive beam search while saving up to 92% on the required beam training overhead. Thirdly, the proposed low-complexity beam training strategy is enhanced with the use of deep reinforcement learning (DRL) for mobile mmWave channels. A DRL-based beam training algorithm is proposed, which can intelligently switch between different beam training methods such that the average beam training overhead is minimised while achieving good spectral efficiency or energy efficiency performance. Given the desired performance requirement in the reward function for the DRL model, the spectral efficiency or energy efficiency can be maximised for the current channel condition by controlling the number of activated RF chains. The DRL-based approach can adjust the amount of beam training overhead required according to the dynamics of the environment. This approach can provide a good overhead-performance trade-off and achieve higher data rates in channels with significant levels of signal blockage