85 research outputs found
Universal Demosaicking of Color Filter Arrays
A large number of color filter arrays (CFAs), periodic or aperiodic, have been proposed. To reconstruct images from all different CFAs and compare their imaging quality, a universal demosaicking method is needed. This paper proposes a new universal demosaicking method based on inter-pixel chrominance capture and optimal demosaicking transformation. It skips the commonly used step to estimate the luminance component at each pixel, and thus, avoids the associated estimation error. Instead, we directly use the acquired CFA color intensity at each pixel as an input component. Two independent chrominance components are estimated at each pixel based on the interpixel chrominance in the window, which is captured with the difference of CFA color values between the pixel of interest and its neighbors. Two mechanisms are employed for the accurate estimation: distance-related and edge-sensing weighting to reflect the confidence levels of the inter-pixel chrominance components, and pseudoinverse-based estimation from the components in a window. Then from the acquired CFA color component and two estimated chrominance components, the three primary colors are reconstructed by a linear color transform, which is optimized for the least transform error. Our experiments show that the proposed method is much better than other published universal demosaicking methods.National Key Basic Research Project of China (973 Program) [2015CB352303, 2011CB302400]; National Natural Science Foundation (NSF) of China [61071156, 61671027]SCI(E)[email protected]; [email protected]; [email protected]; [email protected]
Efficient Encoding of Wireless Capsule Endoscopy Images Using Direct Compression of Colour Filter Array Images
Since its invention in 2001, wireless capsule endoscopy (WCE) has played an important role in the endoscopic examination of the gastrointestinal tract. During this period, WCE has undergone tremendous advances in technology, making it the first-line modality for diseases from bleeding to cancer in the small-bowel. Current research efforts are focused on evolving WCE to include functionality such as drug delivery, biopsy, and active locomotion. For the integration of these functionalities into WCE, two critical prerequisites are the image quality enhancement and the power consumption reduction. An efficient image compression solution is required to retain the highest image quality while reducing the transmission power. The issue is more challenging due to the fact that image sensors in WCE capture images in Bayer Colour filter array (CFA) format. Therefore, standard compression engines provide inferior compression performance.
The focus of this thesis is to design an optimized image compression pipeline to encode the capsule endoscopic (CE) image efficiently in CFA format. To this end, this thesis proposes two image compression schemes.
First, a lossless image compression algorithm is proposed consisting of an optimum reversible colour transformation, a low complexity prediction model, a corner clipping mechanism and a single context adaptive Golomb-Rice entropy encoder. The derivation of colour transformation that provides the best performance for a given prediction model is considered as an optimization problem. The low complexity prediction model works in raster order fashion and requires no buffer memory. The application of colour transformation yields lower inter-colour correlation and allows the efficient independent encoding of the colour components.
The second compression scheme in this thesis is a lossy compression algorithm with a integer discrete cosine transformation at its core. Using the statistics obtained from a large dataset of CE image, an optimum colour transformation is derived using the principal component analysis (PCA). The transformed coefficients are quantized using optimized quantization table, which was designed with a focus to discard medically irrelevant information. A fast demosaicking algorithm is developed to reconstruct the colour image from the lossy CFA image in the decoder. Extensive experiments and comparisons with state-of-the-art lossless image compression methods establish the superiority of the proposed compression methods as simple and efficient image compression algorithm. The lossless algorithm can transmit the image in a lossless manner within the available bandwidth. On the other hand, performance evaluation of lossy compression algorithm indicates that it can deliver high quality images at low transmission power and low computation costs
InSPECtor: an end-to-end design framework for compressive pixelated hyperspectral instruments
Classic designs of hyperspectral instrumentation densely sample the spatial
and spectral information of the scene of interest. Data may be compressed after
the acquisition. In this paper we introduce a framework for the design of an
optimized, micro-patterned snapshot hyperspectral imager that acquires an
optimized subset of the spatial and spectral information in the scene. The data
is thereby compressed already at the sensor level, but can be restored to the
full hyperspectral data cube by the jointly optimized reconstructor. This
framework is implemented with TensorFlow and makes use of its automatic
differentiation for the joint optimization of the layout of the micro-patterned
filter array as well as the reconstructor. We explore the achievable
compression ratio for different numbers of filter passbands, number of scanning
frames, and filter layouts using data collected by the Hyperscout instrument.
We show resulting instrument designs that take snapshot measurements without
losing significant information while reducing the data volume, acquisition
time, or detector space by a factor of 40 as compared to classic, dense
sampling. The joint optimization of a compressive hyperspectral imager design
and the accompanying reconstructor provides an avenue to substantially reduce
the data volume from hyperspectral imagers.Comment: 23 pages, 12 figures, published in Applied Optic
Fast Two-step Blind Optical Aberration Correction
The optics of any camera degrades the sharpness of photographs, which is a
key visual quality criterion. This degradation is characterized by the
point-spread function (PSF), which depends on the wavelengths of light and is
variable across the imaging field. In this paper, we propose a two-step scheme
to correct optical aberrations in a single raw or JPEG image, i.e., without any
prior information on the camera or lens. First, we estimate local Gaussian blur
kernels for overlapping patches and sharpen them with a non-blind deblurring
technique. Based on the measurements of the PSFs of dozens of lenses, these
blur kernels are modeled as RGB Gaussians defined by seven parameters. Second,
we remove the remaining lateral chromatic aberrations (not contemplated in the
first step) with a convolutional neural network, trained to minimize the
red/green and blue/green residual images. Experiments on both synthetic and
real images show that the combination of these two stages yields a fast
state-of-the-art blind optical aberration compensation technique that competes
with commercial non-blind algorithms.Comment: 28 pages, 20 figures, accepted at ECCV'22 as a poste
Content Authentication for Neural Imaging Pipelines: End-to-end Optimization of Photo Provenance in Complex Distribution Channels
Forensic analysis of digital photo provenance relies on intrinsic traces left
in the photograph at the time of its acquisition. Such analysis becomes
unreliable after heavy post-processing, such as down-sampling and
re-compression applied upon distribution in the Web. This paper explores
end-to-end optimization of the entire image acquisition and distribution
workflow to facilitate reliable forensic analysis at the end of the
distribution channel. We demonstrate that neural imaging pipelines can be
trained to replace the internals of digital cameras, and jointly optimized for
high-fidelity photo development and reliable provenance analysis. In our
experiments, the proposed approach increased image manipulation detection
accuracy from 45% to over 90%. The findings encourage further research towards
building more reliable imaging pipelines with explicit provenance-guaranteeing
properties.Comment: Camera ready + supplement, CVPR'1
- …