59 research outputs found
Deep Generative Models on 3D Representations: A Survey
Generative models, as an important family of statistical modeling, target
learning the observed data distribution via generating new instances. Along
with the rise of neural networks, deep generative models, such as variational
autoencoders (VAEs) and generative adversarial network (GANs), have made
tremendous progress in 2D image synthesis. Recently, researchers switch their
attentions from the 2D space to the 3D space considering that 3D data better
aligns with our physical world and hence enjoys great potential in practice.
However, unlike a 2D image, which owns an efficient representation (i.e., pixel
grid) by nature, representing 3D data could face far more challenges.
Concretely, we would expect an ideal 3D representation to be capable enough to
model shapes and appearances in details, and to be highly efficient so as to
model high-resolution data with fast speed and low memory cost. However,
existing 3D representations, such as point clouds, meshes, and recent neural
fields, usually fail to meet the above requirements simultaneously. In this
survey, we make a thorough review of the development of 3D generation,
including 3D shape generation and 3D-aware image synthesis, from the
perspectives of both algorithms and more importantly representations. We hope
that our discussion could help the community track the evolution of this field
and further spark some innovative ideas to advance this challenging task
Learning Human Mesh Recovery in 3D Scenes
We present a novel method for recovering the absolute pose and shape of a
human in a pre-scanned scene given a single image. Unlike previous methods that
perform sceneaware mesh optimization, we propose to first estimate absolute
position and dense scene contacts with a sparse 3D CNN, and later enhance a
pretrained human mesh recovery network by cross-attention with the derived 3D
scene cues. Joint learning on images and scene geometry enables our method to
reduce the ambiguity caused by depth and occlusion, resulting in more
reasonable global postures and contacts. Encoding scene-aware cues in the
network also allows the proposed method to be optimization-free, and opens up
the opportunity for real-time applications. The experiments show that the
proposed network is capable of recovering accurate and physically-plausible
meshes by a single forward pass and outperforms state-of-the-art methods in
terms of both accuracy and speed.Comment: Accepted to CVPR 2023. Project page: https://zju3dv.github.io/sahmr
4K4D: Real-Time 4D View Synthesis at 4K Resolution
This paper targets high-fidelity and real-time view synthesis of dynamic 3D
scenes at 4K resolution. Recently, some methods on dynamic view synthesis have
shown impressive rendering quality. However, their speed is still limited when
rendering high-resolution images. To overcome this problem, we propose 4K4D, a
4D point cloud representation that supports hardware rasterization and enables
unprecedented rendering speed. Our representation is built on a 4D feature grid
so that the points are naturally regularized and can be robustly optimized. In
addition, we design a novel hybrid appearance model that significantly boosts
the rendering quality while preserving efficiency. Moreover, we develop a
differentiable depth peeling algorithm to effectively learn the proposed model
from RGB videos. Experiments show that our representation can be rendered at
over 400 FPS on the DNA-Rendering dataset at 1080p resolution and 80 FPS on the
ENeRF-Outdoor dataset at 4K resolution using an RTX 4090 GPU, which is 30x
faster than previous methods and achieves the state-of-the-art rendering
quality. Our project page is available at https://zju3dv.github.io/4k4d/.Comment: Project Page: https://zju3dv.github.io/4k4
Topological Magnetoresistance of Magnetic Skyrmionic Bubbles
Magnetic skyrmions offer promising prospects for constructing future
energy-efficient and high-density information technology, leading to extensive
explorations of new skyrmionic materials recently. The topological Hall effect
has been widely adopted as a distinctive marker of skyrmion emergence.
Alternately, here we propose a novel signature of skyrmion state by
quantitatively investigating the magnetoresistance (MR) induced by skyrmionic
bubbles in CeMn2Ge2. An intriguing finding was revealed: the anomalous MR
measured at different temperatures can be normalized into a single curve,
regardless of sample thickness. This behavior can be accurately reproduced by
the recent chiral spin textures MR model. Further analysis of the MR anomaly
allowed us to quantitatively examine the effective magnetic fields of various
scattering channels. Remarkably, the analyses, combined with the Lorentz
transmission electronic microscopy results, indicate that the in-plane
scattering channel with triplet exchange interactions predominantly governs the
magnetotransport in the Bloch-type skyrmionic bubble state. Our results not
only provide insights into the quantum correction on MR induced by skyrmionic
bubble phase, but also present an electrical probing method for studying chiral
spin texture formation, evolution and their topological properties, which opens
up exciting possibilities for identifying new skyrmionic materials and
advancing the methodology for studying chiral spin textures.Comment: 17 pages,5 figures,submitte
Recommended from our members
Somatic SF3B1 hotspot mutation in prolactinomas.
The genetic basis and corresponding clinical relevance of prolactinomas remain poorly understood. Here, we perform whole genome sequencing (WGS) on 21 patients with prolactinomas to detect somatic mutations and then validate the mutations with digital polymerase chain reaction (PCR) analysis of tissue samples from 227 prolactinomas. We identify the same hotspot somatic mutation in splicing factor 3 subunit B1 (SF3B1R625H) in 19.8% of prolactinomas. These patients with mutant prolactinomas display higher prolactin (PRL) levels (p = 0.02) and shorter progression-free survival (PFS) (p = 0.02) compared to patients without the mutation. Moreover, we identify that the SF3B1R625H mutation causes aberrant splicing of estrogen related receptor gamma (ESRRG), which results in stronger binding of pituitary-specific positive transcription factor 1 (Pit-1), leading to excessive PRL secretion. Thus our study validates an important mutation and elucidates a potential mechanism underlying the pathogenesis of prolactinomas that may lead to the development of targeted therapeutics
Chromatic Dispersion Equalization FIR Digital Filter for Coherent Receiver
Chromatic dispersion equalization (CDE) in coherent optical communication systems is extremely critical for subsequent digital signal processing (such as frequency offset estimation and carrier phase recovery). Various methods mentioned in the published literature are not satisfactory when the signal bandwidth is limited. This paper proposes a way of using singular value decomposition least square (SVDLS) to obtain the optimal tap weight of the CDE filter and a method to introduce the adaptive mutation particle swarm optimizer (AMPSO) algorithm into the CDE. We show that the two proposed approaches are based on the best approximation of the frequency domain response of the designed and ideal CDE filter. Compared with the traditional CDE method, which needs to be implemented in the full frequency band, the two methods can be implemented in the narrow frequency band. The simulation shows that the effective bandwidth of the baseband signal is limited by squared-root-raised-cosine (SRRC) pulse shaping with a roll-off factor of 0.25 in different modulation formats (DP-QPSK, DP-16 QAM, DP-64 QAM) when the number of taps of the filter is 131, which is 37.5% less than the full frequency band. The designed filter is superior to the existing filter in terms of filtering effect and implementation complexity
Low-Complexity Chromatic Dispersion Equalization FIR Digital Filter for Coherent Receiver
This paper proposes a novel and efficient low-complexity chromatic dispersion equalizer (CDE) based on finite impulse response (FIR) filter architecture for polarization-multiplexed coherent optical communication systems. The FIR filter coefficients are optimized by weights to reduce the energy leakage caused by the truncation effect, and then quantization is used uniformly to reduce the number of real number additions and real number multiplications by utilizing the diversity of the quantized coefficients. Using Optisystem 15 to build a coherent optical communication system for simulation and experimental demonstration, the results show that after the filter coefficients are optimized by weights. Compared with the time-domain chromatic dispersion equalizer (TD-CDE), the proposed design has a lower bit error rate (BER) and better equalization effect. When the transmission distance is 4000 km and the system quantization stages M = 16, the multiplication operation and addition operations reduce computing resources by 99% and 43%, and the BER only increases by 5%. Compared with frequency-domain chromatic dispersion equalizer (FD-CDE), widely used in long-distance communication, the multiplication operation reduces computing resources by 30%. The proposed method provides a new idea for high-performance CDE in long-distance coherent optical communication systems
- …