Efficient Real-Time Architectures and FPGA Implementations of Histogram-Based Median Filters for High Definition Videos

Digital filtering plays an important role in many signal processing applications. Filtering is performed to recover the original signal from its corrupted version. Median filter is a non-linear digital filter that replaces a sample in a given window by the median value of the samples in the window. For images corrupted with impulse noise, median filter provides a very high quality of filtered images. Several modifications of median filters have been proposed and implemented to achieve high image quality compared to that provided by conventional median filters. When these filters are implemented on hardware platforms such as FPGAs, the performance parameters, namely, the area, power and operating frequency should be taken into consideration in addition to the quality of the filtered image. Therefore, efficient implementation of median filters on FPGAs for image and video processing algorithms has been a topic of much interest. The existing hardware-based median filters for high definition video formats do not always satisfy the real-time throughput requirements or are inefficient with respect to hardware performance parameters, such as the area and frequency. This is due to the fact that most of the existing techniques use sorting-based median calculation, which results in a low hardware performance. In this thesis, architectures that use histogram-based median computation, which is a non-sorting-based operation, are designed with a view of efficient hardware implementation. This is carried out in two parts. We design and implement efficient architectures that satisfy the real-time throughput requirements of full high definition (FHD) videos in the first part and that of ultra high definition (UHD) videos in the second part. In the first part, an efficient real-time histogram-based median filter that uses the concept of bit-plane-slicing and adaptive switching median filter (ASMF) is designed and implemented on FPGAs. We term this architecture as hybrid architecture for median filtering (HAMF). The proposed HAMF computes an approximate median, since it uses only the most significant B-bits of the pixel values for median calculation. As a result, the algorithmic level implementation of the proposed HAMF results in a slight degradation in the filtered image quality compared to that provided by ASMF. The proposed HAMF provides a significant improvement over ASMF in terms of the area and operating frequency, when implemented on different generation FPGAs. Analysis of the different parameters, such as the number of bit-planes used in the computation of the median and the number of pipelining stages, is carried out to study the trade-off between the quality of the filtered image and hardware performance. Although the FPGA implementation of the proposed HAMF provides a very high operating frequency, the quality of the images filtered by its algorithmic level implementation decreases with increasing window size and noise density. This filter may be suitable for applications that require FHD filtering with cost constraints, but not for applications where the output image quality is as important as the hardware performance. Hence, in the second part, we design an efficient and real-time architecture of the hierarchical histogram-based median filter (HHMF). The proposed architecture is designed using a full synchronous pipeline, a synchronous accumulate-and-compare unit, and is scalable. The FPGA implementation of the proposed architecture of HHMF can perform real-time filtering of 4K and 8K UHD videos. The quality of the image filtered by HHMF is not compromised as in the case of HAMF, since HHMF uses all the bit-planes and computes the actual median. Although the FPGA implementation of HHMF results in more area utilization, the proposed implementation is more economical than a GPU-based HHMF implementation and provides a better throughput

Efficient Architecture and Implementation of Vector Median Filter in Co-Design Context

This work presents an efficient fast parallel architecture of the Vector Median Filter (VMF) using combined hardware/software (HW/SW) implementation. The hardware part of the system is implemented using VHDL language, whereas the software part is developed using C/C++ language. The software part of the embedded system uses the NIOS-II softcore processor and the operating system used is μClinux. The comparison between the software and HW/SW solutions shows that adding a hardware part in the design attempts to speed up the filtering process compared to the software solution. This efficient embedded system implementation can perform well in several image processing applications

ReS²tAC—UAV-borne real-time SGM stereo optimized for embedded ARM and CUDA devices

With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV

ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

With the emergence of low-cost robotic systems, such as unmanned aerial vehicle, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK unmanned aerial vehicle (UAV), demonstrating its suitability for real-time stereo processing onboard a UAV

FPGA based secure and noiseless image transmission using LEA and optimized bilateral filter

In today’s world, the transmission of secured and noiseless image is a difficult task. Therefore, effective strategies are important to secure the data or secret image from the attackers. Besides, denoising approaches are important to obtain noise-free images. For this, an effective crypto-steganography method based on Lightweight Encryption Algorithm (LEA) and Modified Least Significant Bit (MLSB) method for secured transmission is proposed. Moreover, a bilateral filter-based Whale Optimization Algorithm (WOA) is used for image denoising. Before image transmission, the secret image is encrypted by the LEA algorithm and embedded into the cover image using Discrete Wavelet Transform (DWT) and MLSB technique. After the image transmission, the extraction process is performed to recover the secret image. Finally, a bilateral filter-WOA is used to remove the noise from the secret image. The Verilog code for the proposed model is designed and simulated in Xilinx software. Finally, the simulation results show that the proposed filtering technique has superior performance than conventional bilateral filter and Gaussian filter in terms of Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM)

Energy Efficient Loop Unrolling for Low-Cost FPGAs

Many embedded applications implement block ciphers and sorting and searching algorithms which use multiple loop iterations for computation. These applications often demand low power operation. The power consumption of designs varies with the implementation choices made by designers. The sequential implementation of loop operations consumes minimal area, but latency and clock power are high. Alternatively, loop unrolling causes high glitch power. In this work, we propose a low area overhead approach for unrolling loop iterations that exhibits reduced glitch power. A latch based glitch filter is introduced that reduces the propagation of glitches from one iteration to next. We explore the optimal number of filters to be inserted for different applications that give a good balance between area and power. We also implement partial unrolling with glitch filters. This approach consumes less area while still giving energy savings comparable to the fully unrolled implementation. Our approach is targeted to Xilinx and Altera FPGAs. We simulate different implementation choices and compare energy results to evaluate the savings. We demonstrate our approach on SIMON-128 and AES-256 block ciphers and a sorting algorithm. We prototype our design on Xilinx Artix-7 and Altera Cyclone-IV-GX FPGA development boards and measure the actual power savings. Results show up-to 90% dynamic energy reduction in Xilinx designs, and 97% reduction in Altera designs with our glitch filtering approach due to glitch power reduction

Implementation of Cost Efficient Image Enhancement Technique Reduce Speckle in Ultrasound Images

Speckle is a granular multiplicative noise that reduces the resolution and contrast of the image there by degrading the diagnostic accuracy of the Ultrasound image. Speckle reduction technique has to be followed to enhance the quality of ultrasound image [3].Speckle noise occurs in all coherent imaging systems, such as ultrasound images. The speckle noise in ultrasound images is often considered as undesirable and has a negative impact on clinical practitioners for diagnosis. Because of the signal-dependent nature of the speckle intensity, speckle noise in ultrasound imaging requires specific handling. So, any ultrasound speckle de-noising method must be designed in such a way that the speckle noise be suppressed without smearing the edges. In other words, any speckle de-noising method must preserve both the edges and structural details of the image and its quality [8].Digital image enhancement techniques are to improving the visual quality of images. Main objective of image enhancement is to process an image so that result is more suitable than original image for specific application. This paper presents real time hardware image enhancement techniques using field programmable gate array (FPGA) [10].It presents architecture for filters pixel by pixel and regions filters for image processing using Xilinx System Generator (XSG). This architecture offer an alternative through a graphical user interface that combines MATLAB, Simulink and XSG and explore important aspects concerned to hardware implementation