Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods

Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

Recently, several models based on deep neural networks have achieved great success in terms of both reconstruction accuracy and computational performance for single image super-resolution. In these methods, the low resolution (LR) input image is upscaled to the high resolution (HR) space using a single filter, commonly bicubic interpolation, before reconstruction. This means that the super-resolution (SR) operation is performed in HR space. We demonstrate that this is sub-optimal and adds computational complexity. In this paper, we present the first convolutional neural network (CNN) capable of real-time SR of 1080p videos on a single K2 GPU. To achieve this, we propose a novel CNN architecture where the feature maps are extracted in the LR space. In addition, we introduce an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output. By doing so, we effectively replace the handcrafted bicubic filter in the SR pipeline with more complex upscaling filters specifically trained for each feature map, whilst also reducing the computational complexity of the overall SR operation. We evaluate the proposed approach using images and videos from publicly available datasets and show that it performs significantly better (+0.15dB on Images and +0.39dB on Videos) and is an order of magnitude faster than previous CNN-based methods

Phase Coexistence Near a Morphotropic Phase Boundary in Sm-doped BiFeO3 Films

We have investigated heteroepitaxial films of Sm-doped BiFeO3 with a Sm-concentration near a morphotropic phase boundary. Our high-resolution synchrotron X-ray diffraction, carried out in a temperature range of 25C to 700C, reveals substantial phase coexistence as one changes temperature to crossover from a low-temperature PbZrO3-like phase to a high-temperature orthorhombic phase. We also examine changes due to strain for films greater or less than the critical thickness for misfit dislocation formation. Particularly, we note that thicker films exhibit a substantial volume collapse associated with the structural transition that is suppressed in strained thin films

Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction

In the recent years, convolutional neural networks have transformed the field of medical image analysis due to their capacity to learn discriminative image features for a variety of classification and regression tasks. However, successfully learning these features requires a large amount of manually annotated data, which is expensive to acquire and limited by the available resources of expert image analysts. Therefore, unsupervised, weakly-supervised and self-supervised feature learning techniques receive a lot of attention, which aim to utilise the vast amount of available data, while at the same time avoid or substantially reduce the effort of manual annotation. In this paper, we propose a novel way for training a cardiac MR image segmentation network, in which features are learnt in a self-supervised manner by predicting anatomical positions. The anatomical positions serve as a supervisory signal and do not require extra manual annotation. We demonstrate that this seemingly simple task provides a strong signal for feature learning and with self-supervised learning, we achieve a high segmentation accuracy that is better than or comparable to a U-net trained from scratch, especially at a small data setting. When only five annotated subjects are available, the proposed method improves the mean Dice metric from 0.811 to 0.852 for short-axis image segmentation, compared to the baseline U-net

Learning under Distributed Weak Supervision

The availability of training data for supervision is a frequently encountered bottleneck of medical image analysis methods. While typically established by a clinical expert rater, the increase in acquired imaging data renders traditional pixel-wise segmentations less feasible. In this paper, we examine the use of a crowdsourcing platform for the distribution of super-pixel weak annotation tasks and collect such annotations from a crowd of non-expert raters. The crowd annotations are subsequently used for training a fully convolutional neural network to address the problem of fetal brain segmentation in T2-weighted MR images. Using this approach we report encouraging results compared to highly targeted, fully supervised methods and potentially address a frequent problem impeding image analysis research

TBI lesion segmentation in head CT: impact of preprocessing and data augmentation

Automatic segmentation of lesions in head CT provides keyinformation for patient management, prognosis and disease monitoring.Despite its clinical importance, method development has mostly focusedon multi-parametric MRI. Analysis of the brain in CT is challengingdue to limited soft tissue contrast and its mono-modal nature. We studythe under-explored problem of fine-grained CT segmentation of multiplelesion types (core, blood, oedema) in traumatic brain injury (TBI). Weobserve that preprocessing and data augmentation choices greatly impactthe segmentation accuracy of a neural network, yet these factors arerarely thoroughly assessed in prior work. We design an empirical studythat extensively evaluates the impact of different data preprocessing andaugmentation methods. We show that these choices can have an impactof up to 18% DSC. We conclude that resampling to isotropic resolutionyields improved performance, skull-stripping can be replaced by using theright intensity window, and affine-to-atlas registration is not necessaryif we use sufficient spatial augmentation. Since both skull-stripping andaffine-to-atlas registration are susceptible to failure, we recommend theiralternatives to be used in practice. We believe this is the first work toreport results for fine-grained multi-class segmentation of TBI in CT. Ourfindings may inform further research in this under-explored yet clinicallyimportant task of automatic head CT lesion segmentation