184 research outputs found
Image Sampling with Quasicrystals
We investigate the use of quasicrystals in image sampling. Quasicrystals
produce space-filling, non-periodic point sets that are uniformly discrete and
relatively dense, thereby ensuring the sample sites are evenly spread out
throughout the sampled image. Their self-similar structure can be attractive
for creating sampling patterns endowed with a decorative symmetry. We present a
brief general overview of the algebraic theory of cut-and-project quasicrystals
based on the geometry of the golden ratio. To assess the practical utility of
quasicrystal sampling, we evaluate the visual effects of a variety of
non-adaptive image sampling strategies on photorealistic image reconstruction
and non-photorealistic image rendering used in multiresolution image
representations. For computer visualization of point sets used in image
sampling, we introduce a mosaic rendering technique.Comment: For a full resolution version of this paper, along with supplementary
materials, please visit at
http://www.Eyemaginary.com/Portfolio/Publications.htm
Instant Neural Radiance Fields Stylization
We present Instant Neural Radiance Fields Stylization, a novel approach for
multi-view image stylization for the 3D scene. Our approach models a neural
radiance field based on neural graphics primitives, which use a hash
table-based position encoder for position embedding. We split the position
encoder into two parts, the content and style sub-branches, and train the
network for normal novel view image synthesis with the content and style
targets. In the inference stage, we execute AdaIN to the output features of the
position encoder, with content and style voxel grid features as reference. With
the adjusted features, the stylization of novel view images could be obtained.
Our method extends the style target from style images to image sets of scenes
and does not require additional network training for stylization. Given a set
of images of 3D scenes and a style target(a style image or another set of 3D
scenes), our method can generate stylized novel views with a consistent
appearance at various view angles in less than 10 minutes on modern GPU
hardware. Extensive experimental results demonstrate the validity and
superiority of our method
MM-NeRF: Multimodal-Guided 3D Multi-Style Transfer of Neural Radiance Field
3D style transfer aims to render stylized novel views of 3D scenes with the
specified style, which requires high-quality rendering and keeping multi-view
consistency. Benefiting from the ability of 3D representation from Neural
Radiance Field (NeRF), existing methods learn the stylized NeRF by giving a
reference style from an image. However, they suffer the challenges of
high-quality stylization with texture details for multi-style transfer and
stylization with multimodal guidance. In this paper, we reveal that the same
objects in 3D scenes show various states (color tone, details, etc.) from
different views after stylization since previous methods optimized by
single-view image-based style loss functions, leading NeRF to tend to smooth
texture details, further resulting in low-quality rendering. To tackle these
problems, we propose a novel Multimodal-guided 3D Multi-style transfer of NeRF,
termed MM-NeRF, which achieves high-quality 3D multi-style rendering with
texture details and can be driven by multimodal-style guidance. First, MM-NeRF
adopts a unified framework to project multimodal guidance into CLIP space and
extracts multimodal style features to guide the multi-style stylization. To
relieve the problem of lacking details, we propose a novel Multi-Head Learning
Scheme (MLS), in which each style head predicts the parameters of the color
head of NeRF. MLS decomposes the learning difficulty caused by the
inconsistency of multi-style transfer and improves the quality of stylization.
In addition, the MLS can generalize pre-trained MM-NeRF to any new styles by
adding heads with small training costs (a few minutes). Extensive experiments
on three real-world 3D scene datasets show that MM-NeRF achieves high-quality
3D multi-style stylization with multimodal guidance, keeps multi-view
consistency, and keeps semantic consistency of multimodal style guidance. Codes
will be released later
ControlDreamer: Stylized 3D Generation with Multi-View ControlNet
Recent advancements in text-to-3D generation have significantly contributed
to the automation and democratization of 3D content creation. Building upon
these developments, we aim to address the limitations of current methods in
generating 3D models with creative geometry and styles. We introduce multi-view
ControlNet, a novel depth-aware multi-view diffusion model trained on generated
datasets from a carefully curated text corpus. Our multi-view ControlNet is
then integrated into our two-stage pipeline, ControlDreamer, enabling
text-guided generation of stylized 3D models. Additionally, we present a
comprehensive benchmark for 3D style editing, encompassing a broad range of
subjects, including objects, animals, and characters, to further facilitate
research on diverse 3D generation. Our comparative analysis reveals that this
new pipeline outperforms existing text-to-3D methods as evidenced by human
evaluations and CLIP score metrics.Comment: Project page: https://controldreamer.github.io
AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Control
Neural implicit fields are powerful for representing 3D scenes and generating
high-quality novel views, but it remains challenging to use such implicit
representations for creating a 3D human avatar with a specific identity and
artistic style that can be easily animated. Our proposed method, AvatarCraft,
addresses this challenge by using diffusion models to guide the learning of
geometry and texture for a neural avatar based on a single text prompt. We
carefully design the optimization framework of neural implicit fields,
including a coarse-to-fine multi-bounding box training strategy, shape
regularization, and diffusion-based constraints, to produce high-quality
geometry and texture. Additionally, we make the human avatar animatable by
deforming the neural implicit field with an explicit warping field that maps
the target human mesh to a template human mesh, both represented using
parametric human models. This simplifies animation and reshaping of the
generated avatar by controlling pose and shape parameters. Extensive
experiments on various text descriptions show that AvatarCraft is effective and
robust in creating human avatars and rendering novel views, poses, and shapes.
Our project page is: https://avatar-craft.github.io/.Comment: ICCV 2023 Camera Read
Fast Learning Radiance Fields by Shooting Much Fewer Rays
Learning radiance fields has shown remarkable results for novel view
synthesis. The learning procedure usually costs lots of time, which motivates
the latest methods to speed up the learning procedure by learning without
neural networks or using more efficient data structures. However, these
specially designed approaches do not work for most of radiance fields based
methods. To resolve this issue, we introduce a general strategy to speed up the
learning procedure for almost all radiance fields based methods. Our key idea
is to reduce the redundancy by shooting much fewer rays in the multi-view
volume rendering procedure which is the base for almost all radiance fields
based methods. We find that shooting rays at pixels with dramatic color change
not only significantly reduces the training burden but also barely affects the
accuracy of the learned radiance fields. In addition, we also adaptively
subdivide each view into a quadtree according to the average rendering error in
each node in the tree, which makes us dynamically shoot more rays in more
complex regions with larger rendering error. We evaluate our method with
different radiance fields based methods under the widely used benchmarks.
Experimental results show that our method achieves comparable accuracy to the
state-of-the-art with much faster training.Comment: Accepted by lEEE Transactions on lmage Processing 2023. Project Page:
https://zparquet.github.io/Fast-Learning . Code:
https://github.com/zParquet/Fast-Learnin
- …