30 research outputs found
Learning to Extract a Video Sequence from a Single Motion-Blurred Image
We present a method to extract a video sequence from a single motion-blurred
image. Motion-blurred images are the result of an averaging process, where
instant frames are accumulated over time during the exposure of the sensor.
Unfortunately, reversing this process is nontrivial. Firstly, averaging
destroys the temporal ordering of the frames. Secondly, the recovery of a
single frame is a blind deconvolution task, which is highly ill-posed. We
present a deep learning scheme that gradually reconstructs a temporal ordering
by sequentially extracting pairs of frames. Our main contribution is to
introduce loss functions invariant to the temporal order. This lets a neural
network choose during training what frame to output among the possible
combinations. We also address the ill-posedness of deblurring by designing a
network with a large receptive field and implemented via resampling to achieve
a higher computational efficiency. Our proposed method can successfully
retrieve sharp image sequences from a single motion blurred image and can
generalize well on synthetic and real datasets captured with different cameras
Measurement-Consistent Networks via a Deep Implicit Layer for Solving Inverse Problems
End-to-end deep neural networks (DNNs) have become state-of-the-art (SOTA)
for solving inverse problems. Despite their outstanding performance, during
deployment, such networks are sensitive to minor variations in the training
pipeline and often fail to reconstruct small but important details, a feature
critical in medical imaging, astronomy, or defence. Such instabilities in DNNs
can be explained by the fact that they ignore the forward measurement model
during deployment, and thus fail to enforce consistency between their output
and the input measurements. To overcome this, we propose a framework that
transforms any DNN for inverse problems into a measurement-consistent one. This
is done by appending to it an implicit layer (or deep equilibrium network)
designed to solve a model-based optimization problem. The implicit layer
consists of a shallow learnable network that can be integrated into the
end-to-end training. Experiments on single-image super-resolution show that the
proposed framework leads to significant improvements in reconstruction quality
and robustness over the SOTA DNNs