Simultaneous image color correction and enhancement using particle swarm optimization

Color images captured under various environments are often not ready to deliver the desired quality due to adverse effects caused by uncontrollable illumination settings. In particular, when the illuminate color is not known a priori, the colors of the objects may not be faithfully reproduced and thus impose difficulties in subsequent image processing operations. Color correction thus becomes a very important pre-processing procedure where the goal is to produce an image as if it is captured under uniform chromatic illumination. On the other hand, conventional color correction algorithms using linear gain adjustments focus only on color manipulations and may not convey the maximum information contained in the image. This challenge can be posed as a multi-objective optimization problem that simultaneously corrects the undesirable effect of illumination color cast while recovering the information conveyed from the scene. A variation of the particle swarm optimization algorithm is further developed in the multi-objective optimization perspective that results in a solution achieving a desirable color balance and an adequate delivery of information. Experiments are conducted using a collection of color images of natural objects that were captured under different lighting conditions. Results have shown that the proposed method is capable of delivering images with higher quality. © 2013 Elsevier Ltd. All rights reserved

Intensity preserving cast removal in color images using particle swarm optimization

In this paper, we present an optimal image enhancement technique for color cast images by preserving their intensity. There are methods which improves the appearance of the affected images under different cast like red, green, blue etc but up to some extent. The proposed color cast method is corrected by using transformation function based on gamma values. These optimal values of gamma are obtained through particle swarm optimization (PSO). This technique preserves the image intensity and maintains the originality of color by satisfying the modified gray world assumptions. For the performance analysis, the image distance metric criteria of CIELAB color space is used. The effectiveness of the proposed approach is illustrated by testing the proposed method on color cast images. It has been found that distance between the reference image and the corrected proposed image is negligible. The calculated value of image distance depicts that the enhanced image results of the proposed algorithm are closer to the reference images in comparison with other existing methods

고해상도 CMOS 이미지 센서를 위한 나노광학소자

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2021.8. 이병호.Image sensor is a device that converts electromagnetic waves scattered by the objects or environment into electric signals. Recently, in the mobile device and autonomous vehicle industries, multiple image sensors having different purposes are required for a single device. In particular, image sensors with more than 100 million pixels are being developed in response to the development of a display to a high resolution of 8K or more. However, due to the limited space of the mobile device, the size of pixels constituting the sensor must be reduced for a high-resolution image sensor, which causes factors that reduce image quality, such as a decrease in light efficiency, a decrease in quantum efficiency, and color interference. Metasurface is a device that modulates electromagnetic waves through an array of antennas smaller than wavelength. It has been proposed as a device that replaces the color filter, lens, and photodiode constituting the optical system of the image sensor. However, the performance of the metasurface corresponding to the miniaturized pixel size was limited by the operating principle that requires several array of nano-antennas. In this dissertation, I present a metasurface optical device that can improve the image quality of an existing image sensor composed of micropixels. First, an absorption type color filter that suppresses reflection is discussed. The reflection that inevitably occurs in the conventional metasurface color filter elements causes a flare phenomenon in the captured image. In this dissertation, I design a color filter that transmits only a specific band and absorbs the rest of the absorption resonant band of a hyperbolic metamaterial antenna using a particle swarm optimization method. In particular, I present a Bayer pattern color filter with a pixel size of 255 nm. Second, I introduce a color distribution meta-surface to increase the light efficiency of the image sensor. Since the photodiode converts light having energy above the band gap into an electric signal, an absorption type color filter is used for color classification in image sensor. This means that the total light efficiency of the image sensor is limited to 33% by the blue, green, and red filters constituting one pixel. Accordingly, a freeform metasurface device is designed that exceeds the conventional optical efficiency limit by distributing light incident on the sub-pixel in different directions according to color. Finally, an optical confinement device capable of increasing signal-to-noise ratio (SNR) in low-illuminance at near-infrared is presented. Through the funnel-shaped plasmonic aperture, the light is focused on a volume much smaller than the wavelength. The focused electric and magnetic fields interact with the spatially distributed semiconductors, which achieve a Purcell effect enhanced by the presence of the metasurface. This dissertation is expected to overcome the conventional nanophotonic devices for image sensors and become a cornerstone of the development of micropixel or nanopixel image sensors. Furthermore, it is expected to contribute to building a new image sensor platform that will replace the optical system constituting the image sensor with metasurface.이미지 센서는 환경에 의해 산란되는 전자기파를 전기신호로 바꾸는 소자로, 최근 모바일 기기와 자율 주행 자동차 산업에서 단일 디바이스에 다른 목적을 가진 이미지 센서들이 요구되고 있다. 특히, 디스플레이가 8K 이상의 고해상도로 발전함에 대응하여 1억화소 이상의 이미지 센서가 개발되고 있다. 그러나, 모바일 기기의 제한된 공간에 의해 고해상도 이미지 센서를 위해서는 센서를 구성하는 픽셀의 크기를 줄여야 하며, 이는 광 효율 감소, 양자 효율 감소, 색 간섭 등의 화질을 감소시키는 요소들을 야기한다. 메타표면은 파장보다 작은 안테나들의 배열을 통해 전자기파를 변조해주는 소자로, 이미지 센서의 광학 시스템을 구성하는 색 필터, 렌즈, 포토 다이오드를 대체하는 소자로 제안되었다. 하지만, 소형화 된 픽셀 크기에 대응하는 메타표면은 나노 안테나의 동작원리와 배열의 한계에 의해 성능이 제한되었다. 본 논문에서는 초소형 픽셀로 구성된 기존 이미지 센서에 대한 화질을 높일 수 있는 메타표면 광학소자를 제시한다. 첫째로, 반사를 억제하는 흡수형 색 필터에 대해서 논의한다. 기존 메타표면 색 필터 소자에서 필연적으로 발생하는 내부 반사는 찍은 이미지에서 플레어 현상을 유발한다. 본 논문에서는 쌍곡 메타물질 안테나의 흡수 공진 대역을 입자 무리 최적화 방식을 이용해 특정 대역 만을 투과하고 나머지는 흡수하는 색 필터를 설계한다. 특히, 255 nm 크기 픽셀의 베이어 패턴 색 필터를 제시한다. 둘째로, 이미지 센서의 광 효율을 높이기 위한 색 분배 메타표면을 제시한다. 이미지 센서의 포토 다이오드는 밴드 갭 이상의 에너지를 가지는 빛에 대해 전기신호로 변환하므로, 색 구분을 위해 흡수형 색 필터를 사용한다. 이는 하나의 픽셀을 구성하는 청, 녹, 적색 필터에 의해 이미지 센서의 전체 광 효율이 33 %로 제한되는 것을 의미한다. 따라서, 서브 픽셀에 입사하는 빛을 색에 따라 다른 방향으로 빛을 분배하여 기존의 광 효율 한계를 넘어서는 자유형 메타표면 소자를 설계한다. 마지막으로, 저조도의 근적외선에서 신호 대 잡음비를 높일 수 있는 광 집속 소자를 제시한다. 깔대기 모양의 플라즈모닉 개구를 통해 빛을 파장보다 매우 작은 크기의 영역에 집중시킨다. 집속된 전기장과 자기장은 공간적으로 분포된 반도체와 상호작용함으로써, 메타표면의 존재에 따라 강화된 Purcell 효과를 얻는다. 본 박사학위 논문은 이미지 센서를 위한 기존의 제한된 메타표면 소자를 극복하고, 초소형 픽셀의 이미지 센서 개발의 초석이 될 것으로 기대된다. 나아가, 이미지 센서를 구성하는 광학 시스템을 메타표면으로 대체할 새로운 플랫폼을 구축하는 것에 기여할 것으로 기대된다.Chapter 1 Introduction 1 1.1 Overview of CMOS image sensors 1 1.2 Toward high-resolution miniaturized pixel 2 1.3 Nanophotonic elements for high-resolution camera 3 1.4 Dissertation overview 5 Chapter 2 Light interaction with subwavelength antennas 7 2.1 Overview of plasmonic antenna 7 2.2 Overview of dielectric metasurface 9 2.3 Overview of hyperbolic metamaterials 11 Chapter 3. Absorptive metasurface color filter based on hyperbolic metamaterial for noise reduction 14 3.1 Introduction 14 3.2 Principle of hyperbolic metamaterial absorbers 17 3.3 Absorptive color filter design based on particle swarm optimization method 19 3.4 Numerical analysis on optimized metasurface color filters 23 3.4.1 Single color filter optimization 23 3.4.2 Angle tolerance for optimized metasurface color filters 26 3.5 Sub-micron metasurface color filter array 29 3.6 Conclusion 35 Chapter 4 High-efficient full-color pixel array based on freeform nanostructures for high-resolution image sensor 37 4.1 Introduction 37 4.2 Optimization of metasurface full-color splitter 40 4.3 Implementation of color splitters 46 4.4 Image quality evaluation 52 4.5 Discussion about off-axis color splitters 55 4.6 Conclusion 59 Chapter 5 Plasmonic metasurface cavity for simultaneous enhancement of optical electric and magnetic fields 60 5.1 Introduction 60 5.2 Working principle and numerical results 63 5.2.1 Principle of funnel-shaped metasurface cavity 63 5.2.2 Discussion 67 5.3 Experimental results 69 5.4 Purcell effect 72 5.5 Conclusion 74 Chapter 6 Conclusion 75 Appendix 78 A.1 Colorimetry 78 A.2 Color difference CIEDE2000 79 B. Related work 80 Bibliography 81박

Improving images in turbid water through enhanced color correction and particle swarm-intelligence fusion (CCPF)

When light travels through a water medium, selective attenuation and scattering have a profound impact on the underwater image. These limitations reduce image quality and impede the ability to perform visual tasks. The suggested integrated color correction with intelligence fusion of particle swarm technique (CCPF) is designed with four phases. The first phase presents a novel way to make improvement on underwater color cast. Limit the improvement to only red color channel. In the second phase, an image is then neutralized from the original image by brightness reconstruction technique resulting in enhancing the image contrast. Next, the mean adjustment based on particle swarm intelligence is implemented to improve the image detail. With the swarm intelligence method, the mean values of inferior color channels are shifted to be close to the mean value of a good color channel. Lastly, a fusion between the brightness reconstructed histogram and modified mean particle swarm intelligence histogram is applied to balance the image color. Analysis of underwater images taken in different depths shows that the proposed CCPF method improves the quality of the output image in terms of neutralizing effectiveness and details accuracy, consequently, significantly outperforming the other state-of-the-art methods. The proposed CCPF approach produces highest average entropy value of 7.823 and average UIQM value of 6.287

An iterative method based FLC-SLM system design for forming multiple complex structures simultaneously in 3D volume with tissue

Complex structure formation and fast focusing of light inside or through turbid media is a challenging task due to refractive index heterogeneity, random light scattering and speckle noise formation. Here, we have proposed a weighted-mutation assisted genetic algorithm (WMA-GA) with an R-squared metric based fitness function that advances the contrast, resolution, focuses light tightly and does fast convergence for both simple and complex structure formation through the scattering media. As a compatible system with the binary WMA-GA, we have presented a fast, cost-effective, and robust iterative wavefront shaping system design with an affordable ferroelectric liquid crystal (FLC) based binary-phase spatial light modulator (SLM). The proposed wavefront shaping system design has been used to construct multiple complex hetero-structures simultaneously in 3D volume by an optimized single phase-mask. The WMA-GA and the prototype system have been validated with 120, 220, 450, and 600 grit ground glass diffusers along with 323, 588, and 852 {\mu}m thick fresh chicken tissues including fluorescence in it. We have demonstrated the robustness of the proposed method to control the photon-in and photon-out from a localized fluorescent dye embedded in the tissue. The detailed results show that the proposed class of algorithm-backed integrated system converges fast with higher contrast and advances the resolution.Comment: 17 pages, 13 figure

Prediction model of alcohol intoxication from facial temperature dynamics based on K-means clustering driven by evolutionary computing

Alcohol intoxication is a significant phenomenon, affecting many social areas, including work procedures or car driving. Alcohol causes certain side effects including changing the facial thermal distribution, which may enable the contactless identification and classification of alcohol-intoxicated people. We adopted a multiregional segmentation procedure to identify and classify symmetrical facial features, which reliably reflects the facial-temperature variations while subjects are drinking alcohol. Such a model can objectively track alcohol intoxication in the form of a facial temperature map. In our paper, we propose the segmentation model based on the clustering algorithm, which is driven by the modified version of the Artificial Bee Colony (ABC) evolutionary optimization with the goal of facial temperature features extraction from the IR (infrared radiation) images. This model allows for a definition of symmetric clusters, identifying facial temperature structures corresponding with intoxication. The ABC algorithm serves as an optimization process for an optimal cluster's distribution to the clustering method the best approximate individual areas linked with gradual alcohol intoxication. In our analysis, we analyzed a set of twenty volunteers, who had IR images taken to reflect the process of alcohol intoxication. The proposed method was represented by multiregional segmentation, allowing for classification of the individual spatial temperature areas into segmentation classes. The proposed method, besides single IR image modelling, allows for dynamical tracking of the alcohol-temperature features within a process of intoxication, from the sober state up to the maximum observed intoxication level.Web of Science118art. no. 99

Signals and Images in Sea Technologies

Life below water is the 14th Sustainable Development Goal (SDG) envisaged by the United Nations and is aimed at conserving and sustainably using the oceans, seas, and marine resources for sustainable development. It is not difficult to argue that signals and image technologies may play an essential role in achieving the foreseen targets linked to SDG 14. Besides increasing the general knowledge of ocean health by means of data analysis, methodologies based on signal and image processing can be helpful in environmental monitoring, in protecting and restoring ecosystems, in finding new sensor technologies for green routing and eco-friendly ships, in providing tools for implementing best practices for sustainable fishing, as well as in defining frameworks and intelligent systems for enforcing sea law and making the sea a safer and more secure place. Imaging is also a key element for the exploration of the underwater world for various scopes, ranging from the predictive maintenance of sub-sea pipelines and other infrastructure projects, to the discovery, documentation, and protection of sunken cultural heritage. The scope of this Special Issue encompasses investigations into techniques and ICT approaches and, in particular, the study and application of signal- and image-based methods and, in turn, exploration of the advantages of their application in the previously mentioned areas

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available