Zoom-to-Inpaint: Image Inpainting with High-Frequency Details

Although deep learning has enabled a huge leap forward in image inpainting, current methods are often unable to synthesize realistic high-frequency details. In this paper, we propose applying super-resolution to coarsely reconstructed outputs, refining them at high resolution, and then downscaling the output to the original resolution. By introducing high-resolution images to the refinement network, our framework is able to reconstruct finer details that are usually smoothed out due to spectral bias - the tendency of neural networks to reconstruct low frequencies better than high frequencies. To assist training the refinement network on large upscaled holes, we propose a progressive learning technique in which the size of the missing regions increases as training progresses. Our zoom-in, refine and zoom-out strategy, combined with high-resolution supervision and progressive learning, constitutes a framework-agnostic approach for enhancing high-frequency details that can be applied to any CNN-based inpainting method. We provide qualitative and quantitative evaluations along with an ablation analysis to show the effectiveness of our approach. This seemingly simple, yet powerful approach, outperforms state-of-the-art inpainting methods

MyStyle: A Personalized Generative Prior

We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual. MyStyle allows to reconstruct, enhance and edit images of a specific person, such that the output is faithful to the person's key facial characteristics. Given a small reference set of portrait images of a person (~100), we tune the weights of a pretrained StyleGAN face generator to form a local, low-dimensional, personalized manifold in the latent space. We show that this manifold constitutes a personalized region that spans latent codes associated with diverse portrait images of the individual. Moreover, we demonstrate that we obtain a personalized generative prior, and propose a unified approach to apply it to various ill-posed image enhancement problems, such as inpainting and super-resolution, as well as semantic editing. Using the personalized generative prior we obtain outputs that exhibit high-fidelity to the input images and are also faithful to the key facial characteristics of the individual in the reference set. We demonstrate our method with fair-use images of numerous widely recognizable individuals for whom we have the prior knowledge for a qualitative evaluation of the expected outcome. We evaluate our approach against few-shots baselines and show that our personalized prior, quantitatively and qualitatively, outperforms state-of-the-art alternatives.Comment: Project webpage: https://mystyle-personalized-prior.github.io/, Video: https://youtu.be/QvOdQR3tlO

Biofunctionalization of Large Gold Nanorods Realizes Ultrahigh-Sensitivity Optical Imaging Agents

Gold nanorods (GNRs, ∼ 50 × 15 nm) have been used ubiquitously in biomedicine for their optical properties, and many methods of GNR biofunctionalization have been described. Recently, the synthesis of larger-than-usual GNRs (LGNRs, ∼ 100 × 30 nm) has been demonstrated. However, LGNRs have not been biofunctionalized and therefore remain absent from biomedical literature to date. Here we report the successful biofunctionalization of LGNRs, which produces highly stable particles that exhibit a narrow spectral peak (FWHM ∼100 nm). We further demonstrated that functionalized LGNRs can be used as highly sensitive scattering contrast agents by detecting individual LGNRs in clear liquids. Owing to their increased optical cross sections, we found that LGNRs exhibited up to 32-fold greater backscattering than conventional GNRs. We leveraged these enhanced optical properties to detect LGNRs in the vasculature of live tumor-bearing mice. With LGNR contrast enhancement, we were able to visualize tumor blood vessels at depths that were otherwise undetectable. We expect that the particles reported herein will enable immediate sensitivity improvements in a wide array of biomedical imaging and sensing techniques that rely on conventional GNRs