106 research outputs found
StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models
Content and style (C-S) disentanglement is a fundamental problem and critical
challenge of style transfer. Existing approaches based on explicit definitions
(e.g., Gram matrix) or implicit learning (e.g., GANs) are neither interpretable
nor easy to control, resulting in entangled representations and less satisfying
results. In this paper, we propose a new C-S disentangled framework for style
transfer without using previous assumptions. The key insight is to explicitly
extract the content information and implicitly learn the complementary style
information, yielding interpretable and controllable C-S disentanglement and
style transfer. A simple yet effective CLIP-based style disentanglement loss
coordinated with a style reconstruction prior is introduced to disentangle C-S
in the CLIP image space. By further leveraging the powerful style removal and
generative ability of diffusion models, our framework achieves superior results
than state of the art and flexible C-S disentanglement and trade-off control.
Our work provides new insights into the C-S disentanglement in style transfer
and demonstrates the potential of diffusion models for learning
well-disentangled C-S characteristics.Comment: Accepted by ICCV 202
Fast Learning Radiance Fields by Shooting Much Fewer Rays
Learning radiance fields has shown remarkable results for novel view
synthesis. The learning procedure usually costs lots of time, which motivates
the latest methods to speed up the learning procedure by learning without
neural networks or using more efficient data structures. However, these
specially designed approaches do not work for most of radiance fields based
methods. To resolve this issue, we introduce a general strategy to speed up the
learning procedure for almost all radiance fields based methods. Our key idea
is to reduce the redundancy by shooting much fewer rays in the multi-view
volume rendering procedure which is the base for almost all radiance fields
based methods. We find that shooting rays at pixels with dramatic color change
not only significantly reduces the training burden but also barely affects the
accuracy of the learned radiance fields. In addition, we also adaptively
subdivide each view into a quadtree according to the average rendering error in
each node in the tree, which makes us dynamically shoot more rays in more
complex regions with larger rendering error. We evaluate our method with
different radiance fields based methods under the widely used benchmarks.
Experimental results show that our method achieves comparable accuracy to the
state-of-the-art with much faster training.Comment: Accepted by lEEE Transactions on lmage Processing 2023. Project Page:
https://zparquet.github.io/Fast-Learning . Code:
https://github.com/zParquet/Fast-Learnin
MicroAST: Towards Super-Fast Ultra-Resolution Arbitrary Style Transfer
Arbitrary style transfer (AST) transfers arbitrary artistic styles onto
content images. Despite the recent rapid progress, existing AST methods are
either incapable or too slow to run at ultra-resolutions (e.g., 4K) with
limited resources, which heavily hinders their further applications. In this
paper, we tackle this dilemma by learning a straightforward and lightweight
model, dubbed MicroAST. The key insight is to completely abandon the use of
cumbersome pre-trained Deep Convolutional Neural Networks (e.g., VGG) at
inference. Instead, we design two micro encoders (content and style encoders)
and one micro decoder for style transfer. The content encoder aims at
extracting the main structure of the content image. The style encoder, coupled
with a modulator, encodes the style image into learnable dual-modulation
signals that modulate both intermediate features and convolutional filters of
the decoder, thus injecting more sophisticated and flexible style signals to
guide the stylizations. In addition, to boost the ability of the style encoder
to extract more distinct and representative style signals, we also introduce a
new style signal contrastive loss in our model. Compared to the state of the
art, our MicroAST not only produces visually superior results but also is 5-73
times smaller and 6-18 times faster, for the first time enabling super-fast
(about 0.5 seconds) AST at 4K ultra-resolutions. Code is available at
https://github.com/EndyWon/MicroAST.Comment: Accepted by AAAI 202
Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning
This paper presents a new adversarial training framework for image inpainting
with segmentation confusion adversarial training (SCAT) and contrastive
learning. SCAT plays an adversarial game between an inpainting generator and a
segmentation network, which provides pixel-level local training signals and can
adapt to images with free-form holes. By combining SCAT with standard global
adversarial training, the new adversarial training framework exhibits the
following three advantages simultaneously: (1) the global consistency of the
repaired image, (2) the local fine texture details of the repaired image, and
(3) the flexibility of handling images with free-form holes. Moreover, we
propose the textural and semantic contrastive learning losses to stabilize and
improve our inpainting model's training by exploiting the feature
representation space of the discriminator, in which the inpainting images are
pulled closer to the ground truth images but pushed farther from the corrupted
images. The proposed contrastive losses better guide the repaired images to
move from the corrupted image data points to the real image data points in the
feature representation space, resulting in more realistic completed images. We
conduct extensive experiments on two benchmark datasets, demonstrating our
model's effectiveness and superiority both qualitatively and quantitatively.Comment: Accepted to AAAI2023, Ora
Abnormal changes of bone metabolism markers with age in children with cerebral palsy
Cerebral palsy (CP) is a broad range of diseases with permanent and nonprogressive motor impairments, carrying a high cost for both the individual and the society. The characteristics of low bone mineral density and high risk of fractures suggest that bone metabolism disorders are present in CP. This study aims to investigate the association between indicators of bone metabolism and children with CP. A total of 139 children (75 children with CP and 64 healthy controls) were included in this cross-sectional study. Participants were divided into three age groups (0–2 years, 2.1–4 years, and 4.1–7 years). All children with CP were diagnosed according to clinical criteria and furtherly divided into clinical subtypes. The levels of total procollagen type I N-terminal propeptide (TPINP), N-MID osteocalcin (OC), beta-crosslaps (β-CTX), 25-hydroxyvitamin D (25-OHD) and parathyroid hormone (PTH) in the serum were measured with corresponding detection kits according to the manufacturer's instructions. Serum levels of TPINP and 25-OHD were lower with older age, whereas β-CTX and PTH were higher with older age. In the CP group, TPINP (age 0–2 years and 2.1–4 years) and OC (age 2.1–4 years) levels were higher, while β-CTX (age 2.1–4 years and 4.1–7 years) and PTH (age 2.1–4 years) values were lower than the control group. In addition, there were no statistically significant differences in the levels of these indicators among the CP subgroups with different clinical characteristics. Our study shows that bone turnover markers, indicators of bone metabolism, in children with CP differ significantly from healthy controls. The indicators we studied changed with age, and they did not correlate with disease severity
Potential of Core-Collapse Supernova Neutrino Detection at JUNO
JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve
Detection of the Diffuse Supernova Neutrino Background with JUNO
As an underground multi-purpose neutrino detector with 20 kton liquid scintillator, Jiangmen Underground Neutrino Observatory (JUNO) is competitive with and complementary to the water-Cherenkov detectors on the search for the diffuse supernova neutrino background (DSNB). Typical supernova models predict 2-4 events per year within the optimal observation window in the JUNO detector. The dominant background is from the neutral-current (NC) interaction of atmospheric neutrinos with 12C nuclei, which surpasses the DSNB by more than one order of magnitude. We evaluated the systematic uncertainty of NC background from the spread of a variety of data-driven models and further developed a method to determine NC background within 15\% with {\it{in}} {\it{situ}} measurements after ten years of running. Besides, the NC-like backgrounds can be effectively suppressed by the intrinsic pulse-shape discrimination (PSD) capabilities of liquid scintillators. In this talk, I will present in detail the improvements on NC background uncertainty evaluation, PSD discriminator development, and finally, the potential of DSNB sensitivity in JUNO
- …