Diffusion models have shown exceptional performance in solving inverse
problems. However, one major limitation is the slow inference time. While
faster diffusion samplers have been developed for unconditional sampling, there
has been limited research on conditional sampling in the context of inverse
problems. In this study, we propose a novel and efficient diffusion sampling
strategy that employs the geometric decomposition of diffusion sampling.
Specifically, we discover that the samples generated from diffusion models can
be decomposed into two orthogonal components: a ``denoised" component obtained
by projecting the sample onto the clean data manifold, and a ``noise" component
that induces a transition to the next lower-level noisy manifold with the
addition of stochastic noise. Furthermore, we prove that, under some conditions
on the clean data manifold, the conjugate gradient update for imposing
conditioning from the denoised signal belongs to the clean manifold, resulting
in a much faster and more accurate diffusion sampling. Our method is applicable
regardless of the parameterization and setting (i.e., VE, VP). Notably, we
achieve state-of-the-art reconstruction quality on challenging real-world
medical inverse imaging problems, including multi-coil MRI reconstruction and
3D CT reconstruction. Moreover, our proposed method achieves more than 80 times
faster inference time than the previous state-of-the-art method.Comment: 21 page