Embedded FIR filter design for real-time refocusing using a standard plenoptic video camera

Copyright 2014 Society of Photo-Optical Instrumentation Engineers and IS&T—The Society for Imaging Science and Technology. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.A novel and low-cost embedded hardware architecture for real-time refocusing based on a standard plenoptic camera is presented in this study. The proposed layout design synthesizes refocusing slices directly from micro images by omitting the process for the commonly used sub-aperture extraction. Therefore, intellectual property cores, containing switch controlled Finite Impulse Response (FIR) filters, are developed and applied to the Field Programmable Gate Array (FPGA) XC6SLX45 from Xilinx. Enabling the hardware design to work economically, the FIR filters are composed of stored product as well as upsampling and interpolation techniques in order to achieve an ideal relation between image resolution, delay time, power consumption and the demand of logic gates. The video output is transmitted via High-Definition Multimedia Interface (HDMI) with a resolution of 720p at a frame rate of 60 fps conforming to the HD ready standard. Examples of the synthesized refocusing slices are presented

Integral form 4-D light field filters using Xilinx FPGAs and 45 nm CMOS technology

Four-dimensional (4-D) infinite impulse response frequency hyper-planar filter and a digital VLSI architecture for real time light field based depth filtering applications is proposed. A signal flow graph based on discrete spatial integrators is introduced, which leads to improved sensitivity properties for perturbations in filter coefficients. First order sensitivity analysis of filter transfer function shows a 92.9\ua0% reduction of maximum gain error in frequency response with 12 bits of fractional precision, when compared with a direct-form architecture. Prototype FPGA hardware-in-the-loop co-simulations are performed for two different light field geometries. Register transfer level design validation is carried out via FPGA hardware emulation with a host computer providing memory buffers, and the full-design emulation is carried out on a standalone Berkely Emulation Engine (BEE3), operating at 36.44 and 37.31\ua0MHz for the two light field geometries, respectively. 45\ua0nm CMOS implementation is carried out up to the synthesis level, yielding operating frequencies of 154.4 and 153.3\ua0MHz (correspondingly frame rates of 1.15 and 18.286\ua0Hz) for the two light field geometries, respectively

VLSI Architecture for 4-D Depth Filtering

We propose the application of light field cameras and depth-selective 4-D IIR filtering to enable video surveillance, leveraging the post-capture depth-selective filtering enabled by computational photography. Novel ultralow-complexity differential-form depth-selective 4-D IIR filter algorithms and their corresponding architectures are proposed for processing 4-D light fields. Practical results are presented for real-world video sequences, and a CMOS VLSI implementation of the arithmetic processing elements is synthesized. The architecture shows 86.66, 78.94% reduction in multipliers and adders compared to direct-form structure and delivers 26 frames/s for light fields of size 16×16×128×128