210 research outputs found
Real-time Controllable Denoising for Image and Video
Controllable image denoising aims to generate clean samples with human
perceptual priors and balance sharpness and smoothness. In traditional
filter-based denoising methods, this can be easily achieved by adjusting the
filtering strength. However, for NN (Neural Network)-based models, adjusting
the final denoising strength requires performing network inference each time,
making it almost impossible for real-time user interaction. In this paper, we
introduce Real-time Controllable Denoising (RCD), the first deep image and
video denoising pipeline that provides a fully controllable user interface to
edit arbitrary denoising levels in real-time with only one-time network
inference. Unlike existing controllable denoising methods that require multiple
denoisers and training stages, RCD replaces the last output layer (which
usually outputs a single noise map) of an existing CNN-based model with a
lightweight module that outputs multiple noise maps. We propose a novel Noise
Decorrelation process to enforce the orthogonality of the noise feature maps,
allowing arbitrary noise level control through noise map interpolation. This
process is network-free and does not require network inference. Our experiments
show that RCD can enable real-time editable image and video denoising for
various existing heavy-weight models without sacrificing their original
performance.Comment: CVPR 202
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Instruction tuning large language model (LLM) on image-text pairs has
achieved unprecedented vision-language multimodal abilities. However, their
vision-language alignments are only built on image-level, the lack of
region-level alignment limits their advancements to fine-grained multimodal
understanding. In this paper, we propose instruction tuning on
region-of-interest. The key design is to reformulate the bounding box as the
format of spatial instruction. The interleaved sequences of visual features
extracted by the spatial instruction and the language embedding are input to
LLM, and trained on the transformed region-text data in instruction tuning
format. Our region-level vision-language model, termed as GPT4RoI, brings brand
new conversational and interactive experience beyond image-level understanding.
(1) Controllability: Users can interact with our model by both language and
spatial instructions to flexibly adjust the detail level of the question. (2)
Capacities: Our model supports not only single-region spatial instruction but
also multi-region. This unlocks more region-level multimodal capacities such as
detailed region caption and complex region reasoning. (3) Composition: Any
off-the-shelf object detector can be a spatial instruction provider so as to
mine informative object attributes from our model, like color, shape, material,
action, relation to other objects, etc. The code, data, and demo can be found
at https://github.com/jshilong/GPT4RoI.Comment: Code has been released at https://github.com/jshilong/GPT4Ro
Nuclear Magnetic Resonance Measurements in High Flat-top Pulsed Magnetic Field up to 40 T at WHMFC
Nuclear magnetic resonance (NMR) technique benefits from high magnetic field
not only due to the field-enhanced measurement sensitivity and resolution, but
also because it is a powerful tool to investigate field-induced physics in
modern material science. In this study, we successfully performed NMR
measurements in high flat-top pulsed magnetic field (FTPMF) up to 40 T. A
two-stage corrected FTPMF with fluctuation less than 10 mT and duration longer
than 9 ms was established. Besides, a Giga-Hz NMR spectrometer and a sample
probe suitable for pulsed-field condition were developed. Both
free-induction-decay and spin-echo sequences were exploited for the
measurements. The derived Nb NMR results show that the stability and
homogeneity of the FTPMF reach an order of 10 ppm / 10 ms and 10 ppm /
10 mm respectively, which is approaching a degree of maturity for some
researches on condensed matter physics.Comment: 8 pages, 9 figure
- …