648 research outputs found
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.Comment: Conference on Computer Vision and Pattern Recognition Workshop
Convolutional sparse coding for high dynamic range imaging
Current HDR acquisition techniques are based on either (i) fusing multibracketed, low dynamic range (LDR) images, (ii) modifying existing hardware and capturing different exposures simultaneously with multiple sensors, or (iii) reconstructing a single image with spatially-varying pixel exposures. In this paper, we propose a novel algorithm to recover high-quality HDRI images from a single, coded exposure. The proposed reconstruction method builds on recently-introduced ideas of convolutional sparse coding (CSC); this paper demonstrates how to make CSC practical for HDR imaging. We demonstrate that the proposed algorithm achieves higher-quality reconstructions than alternative methods, we evaluate optical coding schemes, analyze algorithmic parameters, and build a prototype coded HDR camera that demonstrates the utility of convolutional sparse HDRI coding with a custom hardware platform
Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors
Video frame interpolation (VFI) enables many important applications thatmight involve the temporal domain, such as slow motion playback, or the spatialdomain, such as stop motion sequences. We are focusing on the former task,where one of the key challenges is handling high dynamic range (HDR) scenes inthe presence of complex motion. To this end, we explore possible advantages ofdual-exposure sensors that readily provide sharp short and blurry longexposures that are spatially registered and whose ends are temporally aligned.This way, motion blur registers temporally continuous information on the scenemotion that, combined with the sharp reference, enables more precise motionsampling within a single camera shot. We demonstrate that this facilitates amore complex motion reconstruction in the VFI task, as well as HDR framereconstruction that so far has been considered only for the originally capturedframes, not in-between interpolated frames. We design a neural network trainedin these tasks that clearly outperforms existing solutions. We also propose ametric for scene motion complexity that provides important insights into theperformance of VFI methods at the test time.<br
HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains
High dynamic range (HDR) video reconstruction is attracting more and more
attention due to the superior visual quality compared with those of low dynamic
range (LDR) videos. The availability of LDR-HDR training pairs is essential for
the HDR reconstruction quality. However, there are still no real LDR-HDR pairs
for dynamic scenes due to the difficulty in capturing LDR-HDR frames
simultaneously. In this work, we propose to utilize a staggered sensor to
capture two alternate exposure images simultaneously, which are then fused into
an HDR frame in both raw and sRGB domains. In this way, we build a large scale
LDR-HDR video dataset with 85 scenes and each scene contains 60 frames. Based
on this dataset, we further propose a Raw-HDRNet, which utilizes the raw LDR
frames as inputs. We propose a pyramid flow-guided deformation convolution to
align neighboring frames. Experimental results demonstrate that 1) the proposed
dataset can improve the HDR reconstruction performance on real scenes for three
benchmark networks; 2) Compared with sRGB inputs, utilizing raw inputs can
further improve the reconstruction quality and our proposed Raw-HDRNet is a
strong baseline for raw HDR reconstruction. Our dataset and code will be
released after the acceptance of this paper
DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction
Due to hardware constraints, standard off-the-shelf digital cameras suffers
from low dynamic range (LDR) and low frame per second (FPS) outputs. Previous
works in high dynamic range (HDR) video reconstruction uses sequence of
alternating exposure LDR frames as input, and align the neighbouring frames
using optical flow based networks. However, these methods often result in
motion artifacts in challenging situations. This is because, the alternate
exposure frames have to be exposure matched in order to apply alignment using
optical flow. Hence, over-saturation and noise in the LDR frames results in
inaccurate alignment. To this end, we propose to align the input LDR frames
using a pre-trained video frame interpolation network. This results in better
alignment of LDR frames, since we circumvent the error-prone exposure matching
step, and directly generate intermediate missing frames from the same exposure
inputs. Furthermore, it allows us to generate high FPS HDR videos by
recursively interpolating the intermediate frames. Through this work, we
propose to use video frame interpolation for HDR video reconstruction, and
present the first method to generate high FPS HDR videos. Experimental results
demonstrate the efficacy of the proposed framework against optical flow based
alignment methods, with an absolute improvement of 2.4 PSNR value on standard
HDR video datasets [1], [2] and further benchmark our method for high FPS HDR
video generation.Comment: ICPR 202
LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction
As demands for high-quality videos continue to rise, high-resolution and
high-dynamic range (HDR) imaging techniques are drawing attention. To generate
an HDR video from low dynamic range (LDR) images, one of the critical steps is
the motion compensation between LDR frames, for which most existing works
employed the optical flow algorithm. However, these methods suffer from flow
estimation errors when saturation or complicated motions exist. In this paper,
we propose an end-to-end HDR video composition framework, which aligns LDR
frames in the feature space and then merges aligned features into an HDR frame,
without relying on pixel-domain optical flow. Specifically, we propose a
luminance-based alignment network for HDR (LAN-HDR) consisting of an alignment
module and a hallucination module. The alignment module aligns a frame to the
adjacent reference by evaluating luminance-based attention, excluding color
information. The hallucination module generates sharp details, especially for
washed-out areas due to saturation. The aligned and hallucinated features are
then blended adaptively to complement each other. Finally, we merge the
features to generate a final HDR frame. In training, we adopt a temporal loss,
in addition to frame reconstruction losses, to enhance temporal consistency and
thus reduce flickering. Extensive experiments demonstrate that our method
performs better or comparable to state-of-the-art methods on several
benchmarks.Comment: ICCV 202
Super resolution and dynamic range enhancement of image sequences
Camera producers try to increase the spatial resolution of a camera by reducing size of sites on sensor array. However, shot noise causes the signal to noise ratio drop as sensor sites get smaller. This fact motivates resolution enhancement to be performed through software. Super resolution (SR) image reconstruction aims to combine degraded images of a scene in order to form an image which has higher resolution than all observations. There is a demand for high resolution images in biomedical imaging, surveillance, aerial/satellite imaging and high-definition TV (HDTV) technology. Although extensive research has been conducted in SR, attention has not been given to increase the resolution of images under illumination changes. In this study, a unique framework is proposed to increase the spatial resolution and dynamic range of a video sequence using Bayesian and Projection onto Convex Sets (POCS) methods. Incorporating camera response function estimation into image reconstruction allows dynamic range enhancement along with spatial resolution improvement. Photometrically varying input images complicate process of projecting observations onto common grid by violating brightness constancy. A contrast invariant feature transform is proposed in this thesis to register input images with high illumination variation. Proposed algorithm increases the repeatability rate of detected features among frames of a video. Repeatability rate is increased by computing the autocorrelation matrix using the gradients of contrast stretched input images. Presented contrast invariant feature detection improves repeatability rate of Harris corner detector around %25 on average. Joint multi-frame demosaicking and resolution enhancement is also investigated in this thesis. Color constancy constraint set is devised and incorporated into POCS framework for increasing resolution of color-filter array sampled images. Proposed method provides fewer demosaicking artifacts compared to existing POCS method and a higher visual quality in final image
- …