Orthogonally Regularized Deep Networks For Image Super-resolution

Deep learning methods, in particular trained Convolutional Neural Networks (CNNs) have recently been shown to produce compelling state-of-the-art results for single image Super-Resolution (SR). Invariably, a CNN is learned to map the low resolution (LR) image to its corresponding high resolution (HR) version in the spatial domain. Aiming for faster inference and more efficient solutions than solving the SR problem in the spatial domain, we propose a novel network structure for learning the SR mapping function in an image transform domain, specifically the Discrete Cosine Transform (DCT). As a first contribution, we show that DCT can be integrated into the network structure as a Convolutional DCT (CDCT) layer. We further extend the network to allow the CDCT layer to become trainable (i.e. optimizable). Because this layer represents an image transform, we enforce pairwise orthogonality constraints on the individual basis functions/filters. This Orthogonally Regularized Deep SR network (ORDSR) simplifies the SR task by taking advantage of image transform domain while adapting the design of transform basis to the training image set

Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors

High resolution Magnetic Resonance (MR) images are desired for accurate diagnostics. In practice, image resolution is restricted by factors like hardware and processing constraints. Recently, deep learning methods have been shown to produce compelling state-of-the-art results for image enhancement/super-resolution. Paying particular attention to desired hi-resolution MR image structure, we propose a new regularized network that exploits image priors, namely a low-rank structure and a sharpness prior to enhance deep MR image super-resolution (SR). Our contributions are then incorporating these priors in an analytically tractable fashion \color{black} as well as towards a novel prior guided network architecture that accomplishes the super-resolution task. This is particularly challenging for the low rank prior since the rank is not a differentiable function of the image matrix(and hence the network parameters), an issue we address by pursuing differentiable approximations of the rank. Sharpness is emphasized by the variance of the Laplacian which we show can be implemented by a fixed feedback layer at the output of the network. As a key extension, we modify the fixed feedback (Laplacian) layer by learning a new set of training data driven filters that are optimized for enhanced sharpness. Experiments performed on publicly available MR brain image databases and comparisons against existing state-of-the-art methods show that the proposed prior guided network offers significant practical gains in terms of improved SNR/image quality measures. Because our priors are on output images, the proposed method is versatile and can be combined with a wide variety of existing network architectures to further enhance their performance.Comment: Accepted to IEEE transactions on Image Processin

Predicting Slice-to-Volume Transformation in Presence of Arbitrary Subject Motion

This paper aims to solve a fundamental problem in intensity-based 2D/3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotation and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our approach is attractive in challenging imaging scenarios, where significant subject motion complicates reconstruction performance of 3D volumes from 2D slice data. We extensively evaluate the effectiveness of our approach quantitatively on simulated MRI brain data with extreme random motion. We further demonstrate qualitative results on fetal MRI where our method is integrated into a full reconstruction and motion compensation pipeline. With our CNN regression approach we obtain an average prediction error of 7mm on simulated data, and convincing reconstruction quality of images of very young fetuses where previous methods fail. We further discuss applications to Computed Tomography and X-ray projections. Our approach is a general solution to the 2D/3D initialization problem. It is computationally efficient, with prediction times per slice of a few milliseconds, making it suitable for real-time scenarios.Comment: 8 pages, 4 figures, 6 pages supplemental material, currently under review for MICCAI 201

Optimizasyon yöntemlerinin süper çözünürlük üzerine katkı analizi

In this study, the benefits of choosing a robust optimization function with super resolution are analyzed. For this purpose, the different optimizers are included in the simple Convolutional Neural Network (CNN) architecture SRNET, to reveal the performance of the each method. Findings of this research provides that Adam and Nadam optimizers are robust when compared to (Stochastic Gradient Descent) SGD, Adagrad, Adamax and RMSprop. After experimental simulations, we have achieved the 35.91 (dB)/0.9960 and 35.97 (dB)/0.9961 accuracy rates on Set5 images from Adam and Nadam optimizers, respectively (9-1-5 network structure and filter sizes 128 and 64). These results show that selected optimization function for the CNN model plays an important role in increasing the accuracy rate in the super-resolution problem.Bu çalışmada, süper çözünürlükte sağlam bir optimizasyon fonksiyonu seçmenin yararları analiz edilmiştir. Bu amaçla her yöntemin performansını ortaya çıkarmak için farklı optimize ediciler, basit Evrişimsel Sinir Ağı (CNN) mimarisi SRNET' e dahil edilmiştir. Bu araştırmanın bulguları, Adam ve Nadam optimize edicilerin Stokastik Gradyan İnişi (SGD), Adagrad, Adamax ve RMSprop ile karşılaştırıldığında daha kararlı olduğunu göstermektedir. Deneysel simülasyonlardan sonra, Adam ve Nadam optimize edicilerinden Set5 görüntülerinde sırasıyla 35.91 (dB)/0.9960 ve 35.97 (dB)/0.9961 doğruluk oranlarına ulaştık (9-1-5 ağ yapısı ve filtre boyutları 128 ve 64). Bu sonuçlar, CNN modeli için seçilen optimizasyon fonksiyonunun süper çözünürlük probleminde doğruluk oranını arttırmada önemli bir rol oynadığını göstermektedir

Decomposition Ascribed Synergistic Learning for Unified Image Restoration

Learning to restore multiple image degradations within a single model is quite beneficial for real-world applications. Nevertheless, existing works typically concentrate on regarding each degradation independently, while their relationship has been less exploited to ensure the synergistic learning. To this end, we revisit the diverse degradations through the lens of singular value decomposition, with the observation that the decomposed singular vectors and singular values naturally undertake the different types of degradation information, dividing various restoration tasks into two groups,\ie, singular vector dominated and singular value dominated. The above analysis renders a more unified perspective to ascribe the diverse degradations, compared to previous task-level independent learning. The dedicated optimization of degraded singular vectors and singular values inherently utilizes the potential relationship among diverse restoration tasks, attributing to the Decomposition Ascribed Synergistic Learning (DASL). Specifically, DASL comprises two effective operators, namely, Singular VEctor Operator (SVEO) and Singular VAlue Operator (SVAO), to favor the decomposed optimization, which can be lightly integrated into existing convolutional image restoration backbone. Moreover, the congruous decomposition loss has been devised for auxiliary. Extensive experiments on blended five image restoration tasks demonstrate the effectiveness of our method, including image deraining, image dehazing, image denoising, image deblurring, and low-light image enhancement.Comment: 13 page

Test like you Train in Implicit Deep Learning

Implicit deep learning has recently gained popularity with applications ranging from meta-learning to Deep Equilibrium Networks (DEQs). In its general formulation, it relies on expressing some components of deep learning pipelines implicitly, typically via a root equation called the inner problem. In practice, the solution of the inner problem is approximated during training with an iterative procedure, usually with a fixed number of inner iterations. During inference, the inner problem needs to be solved with new data. A popular belief is that increasing the number of inner iterations compared to the one used during training yields better performance. In this paper, we question such an assumption and provide a detailed theoretical analysis in a simple setting. We demonstrate that overparametrization plays a key role: increasing the number of iterations at test time cannot improve performance for overparametrized networks. We validate our theory on an array of implicit deep-learning problems. DEQs, which are typically overparametrized, do not benefit from increasing the number of iterations at inference while meta-learning, which is typically not overparametrized, benefits from it