83,373 research outputs found
PartDiff: Image Super-resolution with Partial Diffusion Models
Denoising diffusion probabilistic models (DDPMs) have achieved impressive
performance on various image generation tasks, including image
super-resolution. By learning to reverse the process of gradually diffusing the
data distribution into Gaussian noise, DDPMs generate new data by iteratively
denoising from random noise. Despite their impressive performance,
diffusion-based generative models suffer from high computational costs due to
the large number of denoising steps.In this paper, we first observed that the
intermediate latent states gradually converge and become indistinguishable when
diffusing a pair of low- and high-resolution images. This observation inspired
us to propose the Partial Diffusion Model (PartDiff), which diffuses the image
to an intermediate latent state instead of pure random noise, where the
intermediate latent state is approximated by the latent of diffusing the
low-resolution image. During generation, Partial Diffusion Models start
denoising from the intermediate distribution and perform only a part of the
denoising steps. Additionally, to mitigate the error caused by the
approximation, we introduce "latent alignment", which aligns the latent between
low- and high-resolution images during training. Experiments on both magnetic
resonance imaging (MRI) and natural images show that, compared to plain
diffusion-based super-resolution methods, Partial Diffusion Models
significantly reduce the number of denoising steps without sacrificing the
quality of generation
Hypernetwork functional image representation
Motivated by the human way of memorizing images we introduce their functional
representation, where an image is represented by a neural network. For this
purpose, we construct a hypernetwork which takes an image and returns weights
to the target network, which maps point from the plane (representing positions
of the pixel) into its corresponding color in the image. Since the obtained
representation is continuous, one can easily inspect the image at various
resolutions and perform on it arbitrary continuous operations. Moreover, by
inspecting interpolations we show that such representation has some properties
characteristic to generative models. To evaluate the proposed mechanism
experimentally, we apply it to image super-resolution problem. Despite using a
single model for various scaling factors, we obtained results comparable to
existing super-resolution methods
Numerical methods for coupled reconstruction and registration in digital breast tomosynthesis.
Digital Breast Tomosynthesis (DBT) provides an insight into the fine details of normal fibroglandular tissues and abnormal lesions by reconstructing a pseudo-3D image of the breast. In this respect, DBT overcomes a major limitation of conventional X-ray mam- mography by reducing the confounding effects caused by the superposition of breast tissue. In a breast cancer screening or diagnostic context, a radiologist is interested in detecting change, which might be indicative of malignant disease. To help automate this task image registration is required to establish spatial correspondence between time points. Typically, images, such as MRI or CT, are first reconstructed and then registered. This approach can be effective if reconstructing using a complete set of data. However, for ill-posed, limited-angle problems such as DBT, estimating the deformation is com- plicated by the significant artefacts associated with the reconstruction, leading to severe inaccuracies in the registration. This paper presents a mathematical framework, which couples the two tasks and jointly estimates both image intensities and the parameters of a transformation. Under this framework, we compare an iterative method and a simultaneous method, both of which tackle the problem of comparing DBT data by combining reconstruction of a pair of temporal volumes with their registration. We evaluate our methods using various computational digital phantoms, uncom- pressed breast MR images, and in-vivo DBT simulations. Firstly, we compare both iter- ative and simultaneous methods to the conventional, sequential method using an affine transformation model. We show that jointly estimating image intensities and parametric transformations gives superior results with respect to reconstruction fidelity and regis- tration accuracy. Also, we incorporate a non-rigid B-spline transformation model into our simultaneous method. The results demonstrate a visually plausible recovery of the deformation with preservation of the reconstruction fidelity
MeshfreeFlowNet: A Physics-Constrained Deep Continuous Space-Time Super-Resolution Framework
We propose MeshfreeFlowNet, a novel deep learning-based super-resolution
framework to generate continuous (grid-free) spatio-temporal solutions from the
low-resolution inputs. While being computationally efficient, MeshfreeFlowNet
accurately recovers the fine-scale quantities of interest. MeshfreeFlowNet
allows for: (i) the output to be sampled at all spatio-temporal resolutions,
(ii) a set of Partial Differential Equation (PDE) constraints to be imposed,
and (iii) training on fixed-size inputs on arbitrarily sized spatio-temporal
domains owing to its fully convolutional encoder. We empirically study the
performance of MeshfreeFlowNet on the task of super-resolution of turbulent
flows in the Rayleigh-Benard convection problem. Across a diverse set of
evaluation metrics, we show that MeshfreeFlowNet significantly outperforms
existing baselines. Furthermore, we provide a large scale implementation of
MeshfreeFlowNet and show that it efficiently scales across large clusters,
achieving 96.80% scaling efficiency on up to 128 GPUs and a training time of
less than 4 minutes.Comment: Supplementary Video: https://youtu.be/mjqwPch9gDo. Accepted to SC2
Deep Networks for Image Super-Resolution with Sparse Prior
Deep learning techniques have been successfully applied in many areas of
computer vision, including low-level image restoration problems. For image
super-resolution, several models based on deep neural networks have been
recently proposed and attained superior performance that overshadows all
previous handcrafted models. The question then arises whether large-capacity
and data-driven models have become the dominant solution to the ill-posed
super-resolution problem. In this paper, we argue that domain expertise
represented by the conventional sparse coding model is still valuable, and it
can be combined with the key ingredients of deep learning to achieve further
improved results. We show that a sparse coding model particularly designed for
super-resolution can be incarnated as a neural network, and trained in a
cascaded structure from end to end. The interpretation of the network based on
sparse coding leads to much more efficient and effective training, as well as a
reduced model size. Our model is evaluated on a wide range of images, and shows
clear advantage over existing state-of-the-art methods in terms of both
restoration accuracy and human subjective quality
Self Super-Resolution for Magnetic Resonance Images using Deep Networks
High resolution magnetic resonance~(MR) imaging~(MRI) is desirable in many
clinical applications, however, there is a trade-off between resolution, speed
of acquisition, and noise. It is common for MR images to have worse
through-plane resolution~(slice thickness) than in-plane resolution. In these
MRI images, high frequency information in the through-plane direction is not
acquired, and cannot be resolved through interpolation. To address this issue,
super-resolution methods have been developed to enhance spatial resolution. As
an ill-posed problem, state-of-the-art super-resolution methods rely on the
presence of external/training atlases to learn the transform from low
resolution~(LR) images to high resolution~(HR) images. For several reasons,
such HR atlas images are often not available for MRI sequences. This paper
presents a self super-resolution~(SSR) algorithm, which does not use any
external atlas images, yet can still resolve HR images only reliant on the
acquired LR image. We use a blurred version of the input image to create
training data for a state-of-the-art super-resolution deep network. The trained
network is applied to the original input image to estimate the HR image. Our
SSR result shows a significant improvement on through-plane resolution compared
to competing SSR methods.Comment: Accepted by IEEE International Symposium on Biomedical Imaging (ISBI)
201
Attribute-Guided Face Generation Using Conditional CycleGAN
We are interested in attribute-guided face generation: given a low-res face
input image, an attribute vector that can be extracted from a high-res image
(attribute image), our new method generates a high-res face image for the
low-res input that satisfies the given attributes. To address this problem, we
condition the CycleGAN and propose conditional CycleGAN, which is designed to
1) handle unpaired training data because the training low/high-res and high-res
attribute images may not necessarily align with each other, and to 2) allow
easy control of the appearance of the generated face via the input attributes.
We demonstrate impressive results on the attribute-guided conditional CycleGAN,
which can synthesize realistic face images with appearance easily controlled by
user-supplied attributes (e.g., gender, makeup, hair color, eyeglasses). Using
the attribute image as identity to produce the corresponding conditional vector
and by incorporating a face verification network, the attribute-guided network
becomes the identity-guided conditional CycleGAN which produces impressive and
interesting results on identity transfer. We demonstrate three applications on
identity-guided conditional CycleGAN: identity-preserving face superresolution,
face swapping, and frontal face generation, which consistently show the
advantage of our new method.Comment: ECCV 201
- …