Convolutional neural networks analysed via inverse problem theory and sparse representations

Inverse problems in imaging such as denoising, deblurring, superresolution have been addressed for many decades. In recent years, convolutional neural networks (CNNs) have been widely used for many inverse problem areas. Although their indisputable success, CNNs are not mathematically validated as to how and what they learn. In this study, the authors prove that during training, CNN elements solve for inverse problems which are optimum solutions stored as CNN neuron filters. They discuss the necessity of mutual coherence between CNN layer elements in order for a network to converge to the optimum solution. They prove that required mutual coherence can be provided by the usage of residual learning and skip connections. They have set rules over training sets and depth of networks for better convergence, i.e. performance. They have experimentally validated theoretical assertions