Algorithms for the enhancement of dynamic range and colour constancy of digital images & video

One of the main objectives in digital imaging is to mimic the capabilities of the human eye, and perhaps, go beyond in certain aspects. However, the human visual system is so versatile, complex, and only partially understood that no up-to-date imaging technology has been able to accurately reproduce the capabilities of the it. The extraordinary capabilities of the human eye have become a crucial shortcoming in digital imaging, since digital photography, video recording, and computer vision applications have continued to demand more realistic and accurate imaging reproduction and analytic capabilities. Over decades, researchers have tried to solve the colour constancy problem, as well as extending the dynamic range of digital imaging devices by proposing a number of algorithms and instrumentation approaches. Nevertheless, no unique solution has been identified; this is partially due to the wide range of computer vision applications that require colour constancy and high dynamic range imaging, and the complexity of the human visual system to achieve effective colour constancy and dynamic range capabilities. The aim of the research presented in this thesis is to enhance the overall image quality within an image signal processor of digital cameras by achieving colour constancy and extending dynamic range capabilities. This is achieved by developing a set of advanced image-processing algorithms that are robust to a number of practical challenges and feasible to be implemented within an image signal processor used in consumer electronics imaging devises. The experiments conducted in this research show that the proposed algorithms supersede state-of-the-art methods in the fields of dynamic range and colour constancy. Moreover, this unique set of image processing algorithms show that if they are used within an image signal processor, they enable digital camera devices to mimic the human visual system s dynamic range and colour constancy capabilities; the ultimate goal of any state-of-the-art technique, or commercial imaging device

ViPS: Visual processing system for medical imaging

Imaging has become an indispensable tool in modern medicine. Various powerful and expensive platforms to study medical imaging applications appear in recent years. In this article, we design and propose a Visual Processing System (ViPS) that processes medical imaging applications efficiently. ViPS provides a user-friendly programming environment and high-performance architecture to perform image analysis, features extraction and object recognition for complex real-time images or videos. The data structure of image or video is described in the program memory using pattern descriptors; ViPS uses specialized 3D memory structure to handle complex images or videos and processes them on microprocessors or application specific hardware accelerators. The proposed system is highly reliable in terms of cost, performance, and power. ViPS based system is implemented and tested on a Xilinx Virtex-7 FPGA VC707 Evaluation Kit. The performance of ViPS is compared with the Intel i7 multi-core, GPU Jetson TK1 Embedded Development Kit with 192 CUDA cores based graphic systems. When compared with the Intel and GPU-based systems, the results show that ViPS performs real-time video reconstruction at 2x and 1.45x of higher frame rate, achieves 14.6x to 4.8x of speedup while executing different image processing applications and 20.3% and 12.6% of speedup for video processing algorithms respectively.Peer Reviewe

A High-Performance System Architecture for Medical Imaging

Medical imaging is classified into different modalities such as ultrasound, X-ray, computed tomography (CT), positron emission tomography (PET), magnetic resonance imaging (MRI), single-photon emission tomography (SPECT), nuclear medicine (NM), mammography, and fluoroscopy. Medical imaging includes various imaging diagnostic and treatment techniques and methods to model the human body, and therefore, performs an essential role to improve the health care of the community. Medical imaging, scans (such as X-Ray, CT, etc.) are essential in a variety of medical health-care environments. With the enhanced health-care management and increase in availability of medical imaging equipment, the number of global imaging-based systems is growing. Effective, safe, and high-quality imaging is essential for the medical decision-making. In this chapter, we proposed a medical imaging-based high-performance hardware architecture and software programming toolkit called high-performance medical imaging system (HPMIS). The HPMIS can perform medical image registration, storage, and processing in hardware with the support of C/C++ function calls. The system is easy to program and gives high performance to different medical imaging applications

Impairing one sensory modality enhances another by reconfiguring peptidergic signalling in Caenorhabditis elegans

Animals that lose one sensory modality often show augmented responses to other sensory inputs. The mechanisms underpinning this cross-modal plasticity are poorly understood. We probe such mechanisms by performing a forward genetic screen for mutants with enhanced O2 perception in Caenorhabditis elegans. Multiple mutants exhibiting increased O2 responsiveness concomitantly show defects in other sensory responses. One mutant, qui-1, defective in a conserved NACHT/WD40 protein, abolishes pheromone-evoked Ca2+ responses in the ADL pheromone-sensing neurons. At the same time, ADL responsiveness to pre-synaptic input from O2-sensing neurons is heightened in qui-1, and other sensory defective mutants, resulting in enhanced neurosecretion although not increased Ca2+ responses. Expressing qui-1 selectively in ADL rescues both the qui-1 ADL neurosecretory phenotype and enhanced escape from 21% O2. Profiling ADL neurons in qui-1 mutants highlights extensive changes in gene expression, notably of many neuropeptide receptors. We show that elevated ADL expression of the conserved neuropeptide receptor NPR-22 is necessary for enhanced ADL neurosecretion in qui-1 mutants, and is sufficient to confer increased ADL neurosecretion in control animals. Sensory loss can thus confer cross-modal plasticity by changing the peptidergic connectome

Face recognition by means of advanced contributions in machine learning

Face recognition (FR) has been extensively studied, due to both scientific fundamental challenges and current and potential applications where human identification is needed. FR systems have the benefits of their non intrusiveness, low cost of equipments and no useragreement requirements when doing acquisition, among the most important ones. Nevertheless, despite the progress made in last years and the different solutions proposed, FR performance is not yet satisfactory when more demanding conditions are required (different viewpoints, blocked effects, illumination changes, strong lighting states, etc). Particularly, the effect of such non-controlled lighting conditions on face images leads to one of the strongest distortions in facial appearance. This dissertation addresses the problem of FR when dealing with less constrained illumination situations. In order to approach the problem, a new multi-session and multi-spectral face database has been acquired in visible, Near-infrared (NIR) and Thermal infrared (TIR) spectra, under different lighting conditions. A theoretical analysis using information theory to demonstrate the complementarities between different spectral bands have been firstly carried out. The optimal exploitation of the information provided by the set of multispectral images has been subsequently addressed by using multimodal matching score fusion techniques that efficiently synthesize complementary meaningful information among different spectra. Due to peculiarities in thermal images, a specific face segmentation algorithm has been required and developed. In the final proposed system, the Discrete Cosine Transform as dimensionality reduction tool and a fractional distance for matching were used, so that the cost in processing time and memory was significantly reduced. Prior to this classification task, a selection of the relevant frequency bands is proposed in order to optimize the overall system, based on identifying and maximizing independence relations by means of discriminability criteria. The system has been extensively evaluated on the multispectral face database specifically performed for our purpose. On this regard, a new visualization procedure has been suggested in order to combine different bands for establishing valid comparisons and giving statistical information about the significance of the results. This experimental framework has more easily enabled the improvement of robustness against training and testing illumination mismatch. Additionally, focusing problem in thermal spectrum has been also addressed, firstly, for the more general case of the thermal images (or thermograms), and then for the case of facialthermograms from both theoretical and practical point of view. In order to analyze the quality of such facial thermograms degraded by blurring, an appropriate algorithm has been successfully developed. Experimental results strongly support the proposed multispectral facial image fusion, achieving very high performance in several conditions. These results represent a new advance in providing a robust matching across changes in illumination, further inspiring highly accurate FR approaches in practical scenarios.El reconeixement facial (FR) ha estat àmpliament estudiat, degut tant als reptes fonamentals científics que suposa com a les aplicacions actuals i futures on requereix la identificació de les persones. Els sistemes de reconeixement facial tenen els avantatges de ser no intrusius,presentar un baix cost dels equips d’adquisició i no la no necessitat d’autorització per part de l’individu a l’hora de realitzar l'adquisició, entre les més importants. De totes maneres i malgrat els avenços aconseguits en els darrers anys i les diferents solucions proposades, el rendiment del FR encara no resulta satisfactori quan es requereixen condicions més exigents (diferents punts de vista, efectes de bloqueig, canvis en la il·luminació, condicions de llum extremes, etc.). Concretament, l'efecte d'aquestes variacions no controlades en les condicions d'il·luminació sobre les imatges facials condueix a una de les distorsions més accentuades sobre l'aparença facial. Aquesta tesi aborda el problema del FR en condicions d'il·luminació menys restringides. Per tal d'abordar el problema, hem adquirit una nova base de dades de cara multisessió i multiespectral en l'espectre infraroig visible, infraroig proper (NIR) i tèrmic (TIR), sota diferents condicions d'il·luminació. En primer lloc s'ha dut a terme una anàlisi teòrica utilitzant la teoria de la informació per demostrar la complementarietat entre les diferents bandes espectrals objecte d’estudi. L'òptim aprofitament de la informació proporcionada pel conjunt d'imatges multiespectrals s'ha abordat posteriorment mitjançant l'ús de tècniques de fusió de puntuació multimodals, capaces de sintetitzar de manera eficient el conjunt d’informació significativa complementària entre els diferents espectres. A causa de les característiques particulars de les imatges tèrmiques, s’ha requerit del desenvolupament d’un algorisme específic per la segmentació de les mateixes. En el sistema proposat final, s’ha utilitzat com a eina de reducció de la dimensionalitat de les imatges, la Transformada del Cosinus Discreta i una distància fraccional per realitzar les tasques de classificació de manera que el cost en temps de processament i de memòria es va reduir de forma significa. Prèviament a aquesta tasca de classificació, es proposa una selecció de les bandes de freqüències més rellevants, basat en la identificació i la maximització de les relacions d'independència per mitjà de criteris discriminabilitat, per tal d'optimitzar el conjunt del sistema. El sistema ha estat àmpliament avaluat sobre la base de dades de cara multiespectral, desenvolupada pel nostre propòsit. En aquest sentit s'ha suggerit l’ús d’un nou procediment de visualització per combinar diferents bandes per poder establir comparacions vàlides i donar informació estadística sobre el significat dels resultats. Aquest marc experimental ha permès més fàcilment la millora de la robustesa quan les condicions d’il·luminació eren diferents entre els processos d’entrament i test. De forma complementària, s’ha tractat la problemàtica de l’enfocament de les imatges en l'espectre tèrmic, en primer lloc, pel cas general de les imatges tèrmiques (o termogrames) i posteriorment pel cas concret dels termogrames facials, des dels punt de vista tant teòric com pràctic. En aquest sentit i per tal d'analitzar la qualitat d’aquests termogrames facials degradats per efectes de desenfocament, s'ha desenvolupat un últim algorisme. Els resultats experimentals recolzen fermament que la fusió d'imatges facials multiespectrals proposada assoleix un rendiment molt alt en diverses condicions d’il·luminació. Aquests resultats representen un nou avenç en l’aportació de solucions robustes quan es contemplen canvis en la il·luminació, i esperen poder inspirar a futures implementacions de sistemes de reconeixement facial precisos en escenaris no controlats.Postprint (published version

Modulaarisen graafipohjaisen kuvankäsittelyjärjestelmän verifiointi

Electronic devices today have become complex. Any non-trivial device consists of both hardware and software. Tightening time to market and cost requirements put pressure on the development process of the devices. Software and hardware needs to be developed concurrently and must be verified in an early phase of product development. This thesis introduces a graph based image processing system. Image processing system is a complex system that usually consists of software, firmware and hardware. The possibilities and methods of graph verification are investigated in this thesis. Graphs can be used to handle the complexity of the system by encapsulating the functionality of the underlying implementations. Graphs provide modularity and configurability that can be utilized in the development and verification of the system. Reuse of software is increased due to the consistent and defined nature of graphs and their vertices. Software development shift left can be enabled by performing graph vertex verification in isolation by using pre-silicon development platforms. In this thesis, image processing system graphs were also used in a real life product development project. Graph verification was initiated early in the product development. Shift left was exercised by utilizing the graph verification in several pre-silicon platforms. Functional, performance and stability testing was implemented. Both complete graphs and their vertices were verified in isolation. Graph verification provided many benefits to the product development. Implementations could be tested in several different environments in isolation using only a light test framework. Issues could be found and fixed early. Performance bottlenecks could be pinpointed and acted upon. With the foundations laid in this project, it would be possible in the future to take more advantage of graphs. More advanced automated image quality testing would allow efficient verification. Finer granularity graphs would allow more configurability and more focused testing. Shift left could be further increased by adapting the development of the algorithms to use graphs. This would lower the gap between algorithms and actual vertex implementations and also introduce the available test infrastructure to algorithm development

Non-parametric Methods for Automatic Exposure Control, Radiometric Calibration and Dynamic Range Compression

Imaging systems are essential to a wide range of modern day applications. With the continuous advancement in imaging systems, there is an on-going need to adapt and improve the imaging pipeline running inside the imaging systems. In this thesis, methods are presented to improve the imaging pipeline of digital cameras. Here we present three methods to improve important phases of the imaging process, which are (i) ``Automatic exposure adjustment'' (ii) ``Radiometric calibration'' (iii) ''High dynamic range compression''. These contributions touch the initial, intermediate and final stages of imaging pipeline of digital cameras. For exposure control, we propose two methods. The first makes use of CCD-based equations to formulate the exposure control problem. To estimate the exposure time, an initial image was acquired for each wavelength channel to which contrast adjustment techniques were applied. This helps to recover a reference cumulative distribution function of image brightness at each channel. The second method proposed for automatic exposure control is an iterative method applicable for a broad range of imaging systems. It uses spectral sensitivity functions such as the photopic response functions for the generation of a spectral power image of the captured scene. A target image is then generated using the spectral power image by applying histogram equalization. The exposure time is hence calculated iteratively by minimizing the squared difference between target and the current spectral power image. Here we further analyze the method by performing its stability and controllability analysis using a state space representation used in control theory. The applicability of the proposed method for exposure time calculation was shown on real world scenes using cameras with varying architectures. Radiometric calibration is the estimate of the non-linear mapping of the input radiance map to the output brightness values. The radiometric mapping is represented by the camera response function with which the radiance map of the scene is estimated. Our radiometric calibration method employs an L1 cost function by taking advantage of Weisfeld optimization scheme. The proposed calibration works with multiple input images of the scene with varying exposure. It can also perform calibration using a single input with few constraints. The proposed method outperforms, quantitatively and qualitatively, various alternative methods found in the literature of radiometric calibration. Finally, to realistically represent the estimated radiance maps on low dynamic range display (LDR) devices, we propose a method for dynamic range compression. Radiance maps generally have higher dynamic range (HDR) as compared to the widely used display devices. Thus, for display purposes, dynamic range compression is required on HDR images. Our proposed method generates few LDR images from the HDR radiance map by clipping its values at different exposures. Using contrast information of each LDR image generated, the method uses an energy minimization approach to estimate the probability map of each LDR image. These probability maps are then used as label set to form final compressed dynamic range image for the display device. The results of our method were compared qualitatively and quantitatively with those produced by widely cited and professionally used methods