3,061 research outputs found
In-Network View Synthesis for Interactive Multiview Video Systems
To enable Interactive multiview video systems with a minimum view-switching
delay, multiple camera views are sent to the users, which are used as reference
images to synthesize additional virtual views via depth-image-based rendering.
In practice, bandwidth constraints may however restrict the number of reference
views sent to clients per time unit, which may in turn limit the quality of the
synthesized viewpoints. We argue that the reference view selection should
ideally be performed close to the users, and we study the problem of in-network
reference view synthesis such that the navigation quality is maximized at the
clients. We consider a distributed cloud network architecture where data stored
in a main cloud is delivered to end users with the help of cloudlets, i.e.,
resource-rich proxies close to the users. In order to satisfy last-hop
bandwidth constraints from the cloudlet to the users, a cloudlet re-samples
viewpoints of the 3D scene into a discrete set of views (combination of
received camera views and virtual views synthesized) to be used as reference
for the synthesis of additional virtual views at the client. This in-network
synthesis leads to better viewpoint sampling given a bandwidth constraint
compared to simple selection of camera views, but it may however carry a
distortion penalty in the cloudlet-synthesized reference views. We therefore
cast a new reference view selection problem where the best subset of views is
defined as the one minimizing the distortion over a view navigation window
defined by the user under some transmission bandwidth constraints. We show that
the view selection problem is NP-hard, and propose an effective polynomial time
algorithm using dynamic programming to solve the optimization problem.
Simulation results finally confirm the performance gain offered by virtual view
synthesis in the network
Learning to Synthesize a 4D RGBD Light Field from a Single Image
We present a machine learning algorithm that takes as input a 2D RGB image
and synthesizes a 4D RGBD light field (color and depth of the scene in each ray
direction). For training, we introduce the largest public light field dataset,
consisting of over 3300 plenoptic camera light fields of scenes containing
flowers and plants. Our synthesis pipeline consists of a convolutional neural
network (CNN) that estimates scene geometry, a stage that renders a Lambertian
light field using that geometry, and a second CNN that predicts occluded rays
and non-Lambertian effects. Our algorithm builds on recent view synthesis
methods, but is unique in predicting RGBD for each light field ray and
improving unsupervised single image depth estimation by enforcing consistency
of ray depths that should intersect the same scene point. Please see our
supplementary video at https://youtu.be/yLCvWoQLnmsComment: International Conference on Computer Vision (ICCV) 201
AI-Generated Images as Data Source: The Dawn of Synthetic Era
The advancement of visual intelligence is intrinsically tethered to the
availability of large-scale data. In parallel, generative Artificial
Intelligence (AI) has unlocked the potential to create synthetic images that
closely resemble real-world photographs. This prompts a compelling inquiry: how
much visual intelligence could benefit from the advance of generative AI? This
paper explores the innovative concept of harnessing these AI-generated images
as new data sources, reshaping traditional modeling paradigms in visual
intelligence. In contrast to real data, AI-generated data exhibit remarkable
advantages, including unmatched abundance and scalability, the rapid generation
of vast datasets, and the effortless simulation of edge cases. Built on the
success of generative AI models, we examine the potential of their generated
data in a range of applications, from training machine learning models to
simulating scenarios for computational modeling, testing, and validation. We
probe the technological foundations that support this groundbreaking use of
generative AI, engaging in an in-depth discussion on the ethical, legal, and
practical considerations that accompany this transformative paradigm shift.
Through an exhaustive survey of current technologies and applications, this
paper presents a comprehensive view of the synthetic era in visual
intelligence. A project associated with this paper can be found at
https://github.com/mwxely/AIGS .Comment: 20 pages, 11 figure
- …