49 research outputs found
Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework
Light field (LF) cameras provide perspective information of scenes by taking
directional measurements of the focusing light rays. The raw outputs are
usually dark with additive camera noise, which impedes subsequent processing
and applications. We propose a novel LF denoising framework based on
anisotropic parallax analysis (APA). Two convolutional neural networks are
jointly designed for the task: first, the structural parallax synthesis network
predicts the parallax details for the entire LF based on a set of anisotropic
parallax features. These novel features can efficiently capture the high
frequency perspective components of a LF from noisy observations. Second, the
view-dependent detail compensation network restores non-Lambertian variation to
each LF view by involving view-specific spatial energies. Extensive experiments
show that the proposed APA LF denoiser provides a much better denoising
performance than state-of-the-art methods in terms of visual quality and in
preservation of parallax details
Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising
The high-dimensional nature of the 4-D light field (LF) poses great
challenges in achieving efficient and effective feature embedding, that
severely impacts the performance of downstream tasks. To tackle this crucial
issue, in contrast to existing methods with empirically-designed architectures,
we propose a probabilistic-based feature embedding (PFE), which learns a
feature embedding architecture by assembling various low-dimensional
convolution patterns in a probability space for fully capturing spatial-angular
information. Building upon the proposed PFE, we then leverage the intrinsic
linear imaging model of the coded aperture camera to construct a
cycle-consistent 4-D LF reconstruction network from coded measurements.
Moreover, we incorporate PFE into an iterative optimization framework for 4-D
LF denoising. Our extensive experiments demonstrate the significant superiority
of our methods on both real-world and synthetic 4-D LF images, both
quantitatively and qualitatively, when compared with state-of-the-art methods.
The source code will be publicly available at
https://github.com/lyuxianqiang/LFCA-CR-NET
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
We present a compact but effective CNN model for optical flow, called
PWC-Net. PWC-Net has been designed according to simple and well-established
principles: pyramidal processing, warping, and the use of a cost volume. Cast
in a learnable feature pyramid, PWC-Net uses the cur- rent optical flow
estimate to warp the CNN features of the second image. It then uses the warped
features and features of the first image to construct a cost volume, which is
processed by a CNN to estimate the optical flow. PWC-Net is 17 times smaller in
size and easier to train than the recent FlowNet2 model. Moreover, it
outperforms all published optical flow methods on the MPI Sintel final pass and
KITTI 2015 benchmarks, running at about 35 fps on Sintel resolution (1024x436)
images. Our models are available on https://github.com/NVlabs/PWC-Net.Comment: CVPR 2018 camera ready version (with github link to Caffe and PyTorch
code
Widening the view angle of auto-multiscopic display, denoising low brightness light field data and 3D reconstruction with delicate details
This doctoral thesis will present the results of my work into widening the viewing angle
of the auto-multiscopic display, denoising light filed data the enhancement of captured
light filed data captured in low light circumstance, and the attempts on reconstructing
the subject surface with delicate details from microscopy image sets.
The automultiscopic displays carefully control the distribution of emitted light over
space, direction (angle) and time so that even a static image displayed can encode
parallax across viewing directions (light field). This allows simultaneous observation by
multiple viewers, each perceiving 3D from their own (correct) perspective. Currently,
the illusion can only be effectively maintained over a narrow range of viewing angles.
We propose and analyze a simple solution to widen the range of viewing angles for
automultiscopic displays that use parallax barriers. We insert a refractive medium, with
a high refractive index, between the display and parallax barriers. The inserted medium
warps the exitant lightfield in a way that increases the potential viewing angle. We
analyze the consequences of this warp and build a prototype with a 93% increase in
the effective viewing angle. Additionally, we developed an integral images synthesis
method that can address the refraction introduced by the inserted medium efficiently
without the use of ray tracing.
Capturing light field image with a short exposure time is preferable for eliminating
the motion blur but it also leads to low brightness in a low light environment, which
results in a low signal noise ratio. Most light field denoising methods apply regular 2D
image denoising method to the sub-aperture images of a 4D light field directly, but it
is not suitable for focused light field data whose sub-aperture image resolution is too
low to be applied regular denoising methods. Therefore, we propose a deep learning
denoising method based on micro lens images of focused light field to denoise the depth
map and the original micro lens image set simultaneously, and achieved high quality
total focused images from the low focused light field data.
In areas like digital museum, remote researching, 3D reconstruction with delicate
details of subjects is desired and technology like 3D reconstruction based on macro
photography has been used successfully for various purposes. We intend to push it
further by using microscope rather than macro lens, which is supposed to be able to
capture the microscopy level details of the subject. We design and implement a scanning
method which is able to capture microscopy image set from a curve surface based on
robotic arm, and the 3D reconstruction method suitable for the microscopy image set
Artificial Intelligence in the Creative Industries: A Review
This paper reviews the current state of the art in Artificial Intelligence
(AI) technologies and applications in the context of the creative industries. A
brief background of AI, and specifically Machine Learning (ML) algorithms, is
provided including Convolutional Neural Network (CNNs), Generative Adversarial
Networks (GANs), Recurrent Neural Networks (RNNs) and Deep Reinforcement
Learning (DRL). We categorise creative applications into five groups related to
how AI technologies are used: i) content creation, ii) information analysis,
iii) content enhancement and post production workflows, iv) information
extraction and enhancement, and v) data compression. We critically examine the
successes and limitations of this rapidly advancing technology in each of these
areas. We further differentiate between the use of AI as a creative tool and
its potential as a creator in its own right. We foresee that, in the near
future, machine learning-based AI will be adopted widely as a tool or
collaborative assistant for creativity. In contrast, we observe that the
successes of machine learning in domains with fewer constraints, where AI is
the `creator', remain modest. The potential of AI (or its developers) to win
awards for its original creations in competition with human creatives is also
limited, based on contemporary technologies. We therefore conclude that, in the
context of creative industries, maximum benefit from AI will be derived where
its focus is human centric -- where it is designed to augment, rather than
replace, human creativity
Light field reconstruction from multi-view images
Kang Han studied recovering the 3D world from multi-view images. He proposed several algorithms to deal with occlusions in depth estimation and effective representations in view rendering. the proposed algorithms can be used for many innovative applications based on machine intelligence, such as autonomous driving and Metaverse
Advances in Image Processing, Analysis and Recognition Technology
For many decades, researchers have been trying to make computers’ analysis of images as effective as the system of human vision is. For this purpose, many algorithms and systems have previously been created. The whole process covers various stages, including image processing, representation and recognition. The results of this work can be applied to many computer-assisted areas of everyday life. They improve particular activities and provide handy tools, which are sometimes only for entertainment, but quite often, they significantly increase our safety. In fact, the practical implementation of image processing algorithms is particularly wide. Moreover, the rapid growth of computational complexity and computer efficiency has allowed for the development of more sophisticated and effective algorithms and tools. Although significant progress has been made so far, many issues still remain, resulting in the need for the development of novel approaches