3 research outputs found
Mixed Neural Voxels for Fast Multi-view Video Synthesis
Synthesizing high-fidelity videos from real-world multi-view input is
challenging because of the complexities of real-world environments and highly
dynamic motions. Previous works based on neural radiance fields have
demonstrated high-quality reconstructions of dynamic scenes. However, training
such models on real-world scenes is time-consuming, usually taking days or
weeks. In this paper, we present a novel method named MixVoxels to better
represent the dynamic scenes with fast training speed and competitive rendering
qualities. The proposed MixVoxels represents the 4D dynamic scenes as a mixture
of static and dynamic voxels and processes them with different networks. In
this way, the computation of the required modalities for static voxels can be
processed by a lightweight model, which essentially reduces the amount of
computation, especially for many daily dynamic scenes dominated by the static
background. To separate the two kinds of voxels, we propose a novel variation
field to estimate the temporal variance of each voxel. For the dynamic voxels,
we design an inner-product time query method to efficiently query multiple time
steps, which is essential to recover the high-dynamic motions. As a result,
with 15 minutes of training for dynamic scenes with inputs of 300-frame videos,
MixVoxels achieves better PSNR than previous methods. Codes and trained models
are available at https://github.com/fengres/mixvoxelsComment: ICCV 2023 (Oral
Deep cascade gradient RBF networks with output-relevant feature extraction and adaptation for nonlinear and nonstationary processes
The main challenge for industrial predictive models is how to effectively deal with big data from high-dimensional processes with nonstationary characteristics. Although deep networks, such as the stacked autoencoder (SAE), can learn useful features from massive data with multilevel architecture, it is difficult to adapt them online to track fast time-varying process dynamics. To integrate feature learning and online adaptation, this paper proposes a deep cascade gradient radial basis function (GRBF) network for online modeling and prediction of nonlinear and nonstationary processes. The proposed deep learning method consists of three modules. First, a preliminary prediction result is generated by a GRBF weak predictor, which is further combined with raw input data for feature extraction. By incorporating the prior weak prediction information, deep output-relevant features are extracted using a SAE. Online prediction is finally produced upon the extracted features with a GRBF predictor, whose weights and structure are updated online to capture fast time-varying process characteristics. Three real-world industrial case studies demonstrate that the proposed deep cascade GRBF network outperforms existing state-of-the-art online modeling approaches as well as deep networks, in terms of both online prediction accuracy and computational complexity