9,658 research outputs found
Light field image processing: an overview
Light field imaging has emerged as a technology allowing to capture richer visual information from our world. As opposed to traditional photography, which captures a 2D projection of the light in the scene integrating the angular domain, light fields collect radiance from rays in all directions, demultiplexing the angular information lost in conventional photography. On the one hand, this higher dimensional representation of visual data offers powerful capabilities for scene understanding, and substantially improves the performance of traditional computer vision problems such as depth sensing, post-capture refocusing, segmentation, video stabilization, material classification, etc. On the other hand, the high-dimensionality of light fields also brings up new challenges in terms of data capture, data compression, content editing, and display. Taking these two elements together, research in light field image processing has become increasingly popular in the computer vision, computer graphics, and signal processing communities. In this paper, we present a comprehensive overview and discussion of research in this field over the past 20 years. We focus on all aspects of light field image processing, including basic light field representation and theory, acquisition, super-resolution, depth estimation, compression, editing, processing algorithms for light field display, and computer vision applications of light field data
Theoretical Engineering and Satellite Comlink of a PTVD-SHAM System
This paper focuses on super helical memory system's design, 'Engineering,
Architectural and Satellite Communications' as a theoretical approach of an
invention-model to 'store time-data'. The current release entails three
concepts: 1- an in-depth theoretical physics engineering of the chip including
its, 2- architectural concept based on VLSI methods, and 3- the time-data
versus data-time algorithm. The 'Parallel Time Varying & Data Super-helical
Access Memory' (PTVD-SHAM), possesses a waterfall effect in its architecture
dealing with the process of voltage output-switch into diverse logic and
quantum states described as 'Boolean logic & image-logic', respectively.
Quantum dot computational methods are explained by utilizing coiled carbon
nanotubes (CCNTs) and CNT field effect transistors (CNFETs) in the chip's
architecture. Quantum confinement, categorized quantum well substrate, and
B-field flux involvements are discussed in theory. Multi-access of coherent
sequences of 'qubit addressing' in any magnitude, gained as pre-defined, here
e.g., the 'big O notation' asymptotically confined into singularity while
possessing a magnitude of 'infinity' for the orientation of array displacement.
Gaussian curvature of k(k<0) is debated in aim of specifying the
2D electron gas characteristics, data storage system for defining short and
long time cycles for different CCNT diameters where space-time continuum is
folded by chance for the particle. Precise pre/post data timing for, e.g.,
seismic waves before earthquake mantle-reach event occurrence, including time
varying self-clocking devices in diverse geographic locations for radar systems
is illustrated in the Subsections of the paper. The theoretical fabrication
process, electromigration between chip's components is discussed as well.Comment: 50 pages, 10 figures (3 multi-figures), 2 tables. v.1: 1 postulate
entailing hypothetical ideas, design and model on future technological
advances of PTVD-SHAM. The results of the previous paper [arXiv:0707.1151v6],
are extended in order to prove some introductory conjectures in theoretical
engineering advanced to architectural analysi
Super-resolution of 3-dimensional scenes
Super-resolution is an image enhancement method that increases the resolution of images and video. Previously this technique could only be applied to 2D scenes. The super-resolution algorithm developed in this thesis creates high-resolution views of 3-dimensional scenes, using low-resolution images captured from varying, unknown positions
Neural View-Interpolation for Sparse Light Field Video
We suggest representing light field (LF) videos as "one-off" neural networks (NN), i.e., a learned mapping from view-plus-time coordinates to high-resolution color values, trained on sparse views. Initially, this sounds like a bad idea for three main reasons: First, a NN LF will likely have less quality than a same-sized pixel basis representation. Second, only few training data, e.g., 9 exemplars per frame are available for sparse LF videos. Third, there is no generalization across LFs, but across view and time instead. Consequently, a network needs to be trained for each LF video. Surprisingly, these problems can turn into substantial advantages: Other than the linear pixel basis, a NN has to come up with a compact, non-linear i.e., more intelligent, explanation of color, conditioned on the sparse view and time coordinates. As observed for many NN however, this representation now is interpolatable: if the image output for sparse view coordinates is plausible, it is for all intermediate, continuous coordinates as well. Our specific network architecture involves a differentiable occlusion-aware warping step, which leads to a compact set of trainable parameters and consequently fast learning and fast execution
From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm
Neural radiance fields (NeRF) have shown great success in novel view
synthesis. However, recovering high-quality details from real-world scenes is
still challenging for the existing NeRF-based approaches, due to the potential
imperfect calibration information and scene representation inaccuracy. Even
with high-quality training frames, the synthetic novel views produced by NeRF
models still suffer from notable rendering artifacts, such as noise and blur.
To address this, we propose NeRFLiX, a general NeRF-agnostic restorer paradigm
that learns a degradation-driven inter-viewpoint mixer. Specially, we design a
NeRF-style degradation modeling approach and construct large-scale training
data, enabling the possibility of effectively removing NeRF-native rendering
artifacts for deep neural networks. Moreover, beyond the degradation removal,
we propose an inter-viewpoint aggregation framework that fuses highly related
high-quality training images, pushing the performance of cutting-edge NeRF
models to entirely new levels and producing highly photo-realistic synthetic
views. Based on this paradigm, we further present NeRFLiX++ with a stronger
two-stage NeRF degradation simulator and a faster inter-viewpoint mixer,
achieving superior performance with significantly improved computational
efficiency. Notably, NeRFLiX++ is capable of restoring photo-realistic
ultra-high-resolution outputs from noisy low-resolution NeRF-rendered views.
Extensive experiments demonstrate the excellent restoration ability of
NeRFLiX++ on various novel view synthesis benchmarks.Comment: 17 pages, 16 figures. Project Page:
https://redrock303.github.io/nerflix_plus/. arXiv admin note: text overlap
with arXiv:2303.0691
3次元画像の高画質化・高機能化に向けた解像度変換処理の研究
学位の種別:課程博士University of Tokyo(東京大学
Neural Radiance Fields: Past, Present, and Future
The various aspects like modeling and interpreting 3D environments and
surroundings have enticed humans to progress their research in 3D Computer
Vision, Computer Graphics, and Machine Learning. An attempt made by Mildenhall
et al in their paper about NeRFs (Neural Radiance Fields) led to a boom in
Computer Graphics, Robotics, Computer Vision, and the possible scope of
High-Resolution Low Storage Augmented Reality and Virtual Reality-based 3D
models have gained traction from res with more than 1000 preprints related to
NeRFs published. This paper serves as a bridge for people starting to study
these fields by building on the basics of Mathematics, Geometry, Computer
Vision, and Computer Graphics to the difficulties encountered in Implicit
Representations at the intersection of all these disciplines. This survey
provides the history of rendering, Implicit Learning, and NeRFs, the
progression of research on NeRFs, and the potential applications and
implications of NeRFs in today's world. In doing so, this survey categorizes
all the NeRF-related research in terms of the datasets used, objective
functions, applications solved, and evaluation criteria for these applications.Comment: 413 pages, 9 figures, 277 citation
FPO++: Efficient Encoding and Rendering of Dynamic Neural Radiance Fields by Analyzing and Enhancing Fourier PlenOctrees
Fourier PlenOctrees have shown to be an efficient representation for
real-time rendering of dynamic Neural Radiance Fields (NeRF). Despite its many
advantages, this method suffers from artifacts introduced by the involved
compression when combining it with recent state-of-the-art techniques for
training the static per-frame NeRF models. In this paper, we perform an
in-depth analysis of these artifacts and leverage the resulting insights to
propose an improved representation. In particular, we present a novel density
encoding that adapts the Fourier-based compression to the characteristics of
the transfer function used by the underlying volume rendering procedure and
leads to a substantial reduction of artifacts in the dynamic model.
Furthermore, we show an augmentation of the training data that relaxes the
periodicity assumption of the compression. We demonstrate the effectiveness of
our enhanced Fourier PlenOctrees in the scope of quantitative and qualitative
evaluations on synthetic and real-world scenes
- …