14,307 research outputs found
Steered mixture-of-experts for light field images and video : representation and coding
Research in light field (LF) processing has heavily increased over the last decade. This is largely driven by the desire to achieve the same level of immersion and navigational freedom for camera-captured scenes as it is currently available for CGI content. Standardization organizations such as MPEG and JPEG continue to follow conventional coding paradigms in which viewpoints are discretely represented on 2-D regular grids. These grids are then further decorrelated through hybrid DPCM/transform techniques. However, these 2-D regular grids are less suited for high-dimensional data, such as LFs. We propose a novel coding framework for higher-dimensional image modalities, called Steered Mixture-of-Experts (SMoE). Coherent areas in the higher-dimensional space are represented by single higher-dimensional entities, called kernels. These kernels hold spatially localized information about light rays at any angle arriving at a certain region. The global model consists thus of a set of kernels which define a continuous approximation of the underlying plenoptic function. We introduce the theory of SMoE and illustrate its application for 2-D images, 4-D LF images, and 5-D LF video. We also propose an efficient coding strategy to convert the model parameters into a bitstream. Even without provisions for high-frequency information, the proposed method performs comparable to the state of the art for low-to-mid range bitrates with respect to subjective visual quality of 4-D LF images. In case of 5-D LF video, we observe superior decorrelation and coding performance with coding gains of a factor of 4x in bitrate for the same quality. At least equally important is the fact that our method inherently has desired functionality for LF rendering which is lacking in other state-of-the-art techniques: (1) full zero-delay random access, (2) light-weight pixel-parallel view reconstruction, and (3) intrinsic view interpolation and super-resolution
Neural coding of naturalistic motion stimuli
We study a wide field motion sensitive neuron in the visual system of the
blowfly {\em Calliphora vicina}. By rotating the fly on a stepper motor outside
in a wooded area, and along an angular motion trajectory representative of
natural flight, we stimulate the fly's visual system with input that approaches
the natural situation. The neural response is analyzed in the framework of
information theory, using methods that are free from assumptions. We
demonstrate that information about the motion trajectory increases as the light
level increases over a natural range. This indicates that the fly's brain
utilizes the increase in photon flux to extract more information from the
photoreceptor array, suggesting that imprecision in neural signals is dominated
by photon shot noise in the physical input, rather than by noise generated
within the nervous system itself.Comment: 15 pages, 4 figure
Robustness of Planar Fourier Capture Arrays to Colour Changes and Lost Pixels
Planar Fourier capture arrays (PFCAs) are optical sensors built entirely in
standard microchip manufacturing flows. PFCAs are composed of ensembles of
angle sensitive pixels (ASPs) that each report a single coefficient of the
Fourier transform of the far-away scene. Here we characterize the performance
of PFCAs under the following three non-optimal conditions. First, we show that
PFCAs can operate while sensing light of a wavelength other than the design
point. Second, if only a randomly-selected subset of 10% of the ASPs are
functional, we can nonetheless reconstruct the entire far-away scene using
compressed sensing. Third, if the wavelength of the imaged light is unknown, it
can be inferred by demanding self-consistency of the outputs.Comment: 15 pages including cover page, 12 figures, associated with the 9th
International Conference on Position Sensitive Detector
Backwards is the way forward: feedback in the cortical hierarchy predicts the expected future
Clark offers a powerful description of the brain as a prediction machine, which offers progress on two distinct levels. First, on an abstract conceptual level, it provides a unifying framework for perception, action, and cognition (including subdivisions such as attention, expectation, and imagination). Second, hierarchical prediction offers progress on a concrete descriptive level for testing and constraining conceptual elements and mechanisms of predictive coding models (estimation of predictions, prediction errors, and internal models)
Accurate Light Field Depth Estimation with Superpixel Regularization over Partially Occluded Regions
Depth estimation is a fundamental problem for light field photography
applications. Numerous methods have been proposed in recent years, which either
focus on crafting cost terms for more robust matching, or on analyzing the
geometry of scene structures embedded in the epipolar-plane images. Significant
improvements have been made in terms of overall depth estimation error;
however, current state-of-the-art methods still show limitations in handling
intricate occluding structures and complex scenes with multiple occlusions. To
address these challenging issues, we propose a very effective depth estimation
framework which focuses on regularizing the initial label confidence map and
edge strength weights. Specifically, we first detect partially occluded
boundary regions (POBR) via superpixel based regularization. Series of
shrinkage/reinforcement operations are then applied on the label confidence map
and edge strength weights over the POBR. We show that after weight
manipulations, even a low-complexity weighted least squares model can produce
much better depth estimation than state-of-the-art methods in terms of average
disparity error rate, occlusion boundary precision-recall rate, and the
preservation of intricate visual features
- …