A Study of Colour Rendering in the In-Camera Imaging Pipeline

Consumer cameras such as digital single-lens reflex camera (DSLR) and smartphone cameras have onboard hardware that applies a series of processing steps to transform the initial captured raw sensor image to the final output image that is provided to the user. These processing steps collectively make up the in-camera image processing pipeline. This dissertation aims to study the processing steps related to colour rendering which can be categorized into two stages. The first stage is to convert an image's sensor-specific raw colour space to a device-independent perceptual colour space. The second stage is to further process the image into a display-referred colour space and includes photo-finishing routines to make the image appear visually pleasing to a human. This dissertation makes four contributions towards the study of camera colour rendering. The first contribution is the development of a software-based research platform that closely emulates the in-camera image processing pipeline hardware. This platform allows the examination of the various image states of the captured image as it is processed from the sensor response to the final display output. Our second contribution is to demonstrate the advantage of having access to intermediate image states within the in-camera pipeline that provide more accurate colourimetric consistency among multiple cameras. Our third contribution is to analyze the current colourimetric method used by consumer cameras and to propose a modification that is able to improve its colour accuracy. Our fourth contribution is to describe how to customize a camera imaging pipeline using machine vision cameras to produce high-quality perceptual images for dermatological applications. The dissertation concludes with a summary and future directions

Mathematical Approaches for Image Enhancement Problems

This thesis develops novel techniques that can solve some image enhancement problems using theoretically and technically proven and very useful mathematical tools to image processing such as wavelet transforms, partial differential equations, and variational models. Three subtopics are mainly covered. First, color image denoising framework is introduced to achieve high quality denoising results by considering correlations between color components while existing denoising approaches can be plugged in flexibly. Second, a new and efficient framework for image contrast and color enhancement in the compressed wavelet domain is proposed. The proposed approach is capable of enhancing both global and local contrast and brightness as well as preserving color consistency. The framework does not require inverse transform for image enhancement since linear scale factors are directly applied to both scaling and wavelet coefficients in the compressed domain, which results in high computational efficiency. Also contaminated noise in the image can be efficiently reduced by introducing wavelet shrinkage terms adaptively in different scales. The proposed method is able to enhance a wavelet-coded image computationally efficiently with high image quality and less noise or other artifact. The experimental results show that the proposed method produces encouraging results both visually and numerically compared to some existing approaches. Finally, image inpainting problem is discussed. Literature review, psychological analysis, and challenges on image inpainting problem and related topics are described. An inpainting algorithm using energy minimization and texture mapping is proposed. Mumford-Shah energy minimization model detects and preserves edges in the inpainting domain by detecting both the main structure and the detailed edges. This approach utilizes faster hierarchical level set method and guarantees convergence independent of initial conditions. The estimated segmentation results in the inpainting domain are stored in segmentation map, which is referred by a texture mapping algorithm for filling textured regions. We also propose an inpainting algorithm using wavelet transform that can expect better global structure estimation of the unknown region in addition to shape and texture properties since wavelet transforms have been used for various image analysis problems due to its nice multi-resolution properties and decoupling characteristics

Textural Difference Enhancement based on Image Component Analysis

In this thesis, we propose a novel image enhancement method to magnify the textural differences in the images with respect to human visual characteristics. The method is intended to be a preprocessing step to improve the performance of the texture-based image segmentation algorithms. We propose to calculate the six Tamura's texture features (coarseness, contrast, directionality, line-likeness, regularity and roughness) in novel measurements. Each feature follows its original understanding of the certain texture characteristic, but is measured by some local low-level features, e.g., direction of the local edges, dynamic range of the local pixel intensities, kurtosis and skewness of the local image histogram. A discriminant texture feature selection method based on principal component analysis (PCA) is then proposed to find the most representative characteristics in describing textual differences in the image. We decompose the image into pairwise components representing the texture characteristics strongly and weakly, respectively. A set of wavelet-based soft thresholding methods are proposed as the dictionaries of morphological component analysis (MCA) to sparsely highlight the characteristics strongly and weakly from the image. The wavelet-based thresholding methods are proposed in pair, therefore each of the resulted pairwise components can exhibit one certain characteristic either strongly or weakly. We propose various wavelet-based manipulation methods to enhance the components separately. For each component representing a certain texture characteristic, a non-linear function is proposed to manipulate the wavelet coefficients of the component so that the component is enhanced with the corresponding characteristic accentuated independently while having little effect on other characteristics. Furthermore, the above three methods are combined into a uniform framework of image enhancement. Firstly, the texture characteristics differentiating different textures in the image are found. Secondly, the image is decomposed into components exhibiting these texture characteristics respectively. Thirdly, each component is manipulated to accentuate the corresponding texture characteristics exhibited there. After re-combining these manipulated components, the image is enhanced with the textural differences magnified with respect to the selected texture characteristics. The proposed textural differences enhancement method is used prior to both grayscale and colour image segmentation algorithms. The convincing results of improving the performance of different segmentation algorithms prove the potential of the proposed textural difference enhancement method

Textural Difference Enhancement based on Image Component Analysis

In this thesis, we propose a novel image enhancement method to magnify the textural differences in the images with respect to human visual characteristics. The method is intended to be a preprocessing step to improve the performance of the texture-based image segmentation algorithms. We propose to calculate the six Tamura's texture features (coarseness, contrast, directionality, line-likeness, regularity and roughness) in novel measurements. Each feature follows its original understanding of the certain texture characteristic, but is measured by some local low-level features, e.g., direction of the local edges, dynamic range of the local pixel intensities, kurtosis and skewness of the local image histogram. A discriminant texture feature selection method based on principal component analysis (PCA) is then proposed to find the most representative characteristics in describing textual differences in the image. We decompose the image into pairwise components representing the texture characteristics strongly and weakly, respectively. A set of wavelet-based soft thresholding methods are proposed as the dictionaries of morphological component analysis (MCA) to sparsely highlight the characteristics strongly and weakly from the image. The wavelet-based thresholding methods are proposed in pair, therefore each of the resulted pairwise components can exhibit one certain characteristic either strongly or weakly. We propose various wavelet-based manipulation methods to enhance the components separately. For each component representing a certain texture characteristic, a non-linear function is proposed to manipulate the wavelet coefficients of the component so that the component is enhanced with the corresponding characteristic accentuated independently while having little effect on other characteristics. Furthermore, the above three methods are combined into a uniform framework of image enhancement. Firstly, the texture characteristics differentiating different textures in the image are found. Secondly, the image is decomposed into components exhibiting these texture characteristics respectively. Thirdly, each component is manipulated to accentuate the corresponding texture characteristics exhibited there. After re-combining these manipulated components, the image is enhanced with the textural differences magnified with respect to the selected texture characteristics. The proposed textural differences enhancement method is used prior to both grayscale and colour image segmentation algorithms. The convincing results of improving the performance of different segmentation algorithms prove the potential of the proposed textural difference enhancement method

Image Quality Evaluation in Lossy Compressed Images

This research focuses on the quantification of image quality in lossy compressed images, exploring the impact of digital artefacts and scene characteristics upon image quality evaluation. A subjective paired comparison test was implemented to assess perceived quality of JPEG 2000 against baseline JPEG over a range of different scene types. Interval scales were generated for both algorithms, which indicated a subjective preference for JPEG 2000, particularly at low bit rates, and these were confirmed by an objective distortion measure. The subjective results did not follow this trend for some scenes however, and both algorithms were found to be scene dependent as a result of the artefacts produced at high compression rates. The scene dependencies were explored from the interval scale results, which allowed scenes to be grouped according to their susceptibilities to each of the algorithms. Groupings were correlated with scene measures applied in a linked study. A pilot study was undertaken to explore perceptibility thresholds of JPEG 2000 of the same set of images. This work was developed with a further experiment to investigate the thresholds of perceptibility and acceptability of higher resolution JPEG 2000 compressed images. A set of images was captured using a professional level full-frame Digital Single Lens Reflex camera, using a raw workflow and carefully controlled image-processing pipeline. The scenes were quantified using a set of simple scene metrics to classify them according to whether they were average, higher than, or lower than average, for a number of scene properties known to affect image compression and perceived image quality; these were used to make a final selection of test images. Image fidelity was investigated using the method of constant stimuli to quantify perceptibility thresholds and just noticeable differences (JNDs) of perceptibility. Thresholds and JNDs of acceptability were also quantified to explore suprathreshold quality evaluation. The relationships between the two thresholds were examined and correlated with the results from the scene measures, to identify more or less susceptible scenes. It was found that the level and differences between the two thresholds was an indicator of scene dependency and could be predicted by certain types of scene characteristics. A third study implemented the soft copy quality ruler as an alternative psychophysical method, by matching the quality of compressed images to a set of images varying in a single attribute, separated by known JND increments of quality. The imaging chain and image processing workflow were evaluated using objective measures of tone reproduction and spatial frequency response. An alternative approach to the creation of ruler images was implemented and tested, and the resulting quality rulers were used to evaluate a subset of the images from the previous study. The quality ruler was found to be successful in identifying scene susceptibilities and observer sensitivity. The fourth investigation explored the implementation of four different image quality metrics. These were the Modular Image Difference Metric, the Structural Similarity Metric, The Multi-scale Structural Similarity Metric and the Weighted Structural Similarity Metric. The metrics were tested against the subjective results and all were found to have linear correlation in terms of predictability of image quality

Diagnosis of skin cancer using novel computer vision and deep learning techniques

Recent years have noticed an increase in the total number of skin cancer cases and it is projected to grow exponentially, however mortality rate of malignant melanoma can be decreased if it is diagnosed and treated in its early stage. Notwithstanding the fact that visual similarity between benign and malignant lesions makes the task of diagnosis difficult even for an expert dermatologist, thereby increasing the chances of false prediction. This dissertation proposes two novel methods of computer-aided diagnosis for the classification of malignant lesion. The first method pre-processes the acquired image by the Dull razor method (for digital hair removal) and histogram equalisation. Henceforth the image is segmented by the proposed method using LR-fuzzy logic and it achieves an accuracy, sensitivity and specificity of 96.50%, 97.50% and 96.25% for the PH2 dataset; 96.16%, 91.88% and 98.26% for the ISIC 2017 dataset; 95.91%, 91.62% and 97.37% for ISIC 2018 dataset respectively. Furthermore, the image is classified by the modified You Only Look Once (YOLO v3) classifier and it yields an accuracy, sensitivity and specificity of 98.16%, 95.43%, and 99.50% respectively. The second method enhances the images by removing digital artefacts and histogram equalisation. Thereafter, triangular neutrosophic number (TNN) is used for segmentation of lesion, which achieves an accuracy, sensitivity, and specificity of 99.00%, 97.50%, 99.38% for PH2; 98.83%, 98.48%, 99.01% for ISIC 2017; 98.56%, 98.50%, 98.58% for ISIC 2018; and 97.86%, 97.56%, 97.97% for ISIC 2019 dataset respectively. Furthermore, data augmentation is performed by the addition of artefacts and noise to the training dataset and rotating the images at an angle of 650, 1350, and 2150 such that the training dataset is increased to 92838 from 30946 images. Additionally, a novel classifier based on inception and residual module is trained over augmented dataset and it is able to achieve an accuracy, sensitivity and specificity of 99.50%, 100%, 99.38% for PH2; 99.33%, 98.48%, 99.75% for ISIC 2017; 98.56%, 97.61%, 98.88% for ISIC 2018 and 98.04%, 96.67%, 98.52% for ISIC 2019 dataset respectively. Later in our dissertation, the proposed methods are deployed into real-time mobile applications, therefore enabling the users to diagnose the suspected lesion with ease and accuracy

Development of a fast and accurate method for the segmentation of diabetic foot ulcer images

A Dissertation Submitted in Partial Fulfilment of the Requirements for the Degree of Master’s in Information and Communication Science and Engineering of the Nelson Mandela African Institution of Science and TechnologyGlobally, Diabetic Foot Ulcers (DFUs) are among the major sources of morbidity and death among people diagnosed with diabetes. Diabetic foot ulcers are the leading diabetes-related complications that result in non-traumatic lower-limb amputations among these patients. Being a serious health concern, DFUs present a significant therapeutic challenge to specialists, particularly in countries with limited health resources and where the vast majority of patients are admitted to healthcare facilities when the ulcers have fully advanced. Clinical practices currently employed to assess and treat DFU are mostly based on the vigilance of both the patient and clinician. These practices have been proved to experience major limitations which include less accurate assessment methods, time-consuming diagnostic procedures, and relatively high treatment costs. Digital image processing is thus a potential solution to address issues of the inaccuracy of visual assessment as well as minimizing consecutive patient visits to the clinics. Image processing techniques for ulcer assessment have thus been a center of study in various works of literature. In the available works of literature, these methods include measuring the ulcer area as well as using a medical digital photography scheme. The most notable drawbacks of such approaches include system complexity, complex-exhaustive training phases, and high computational cost. Inspired by the weaknesses of the existing techniques, this study proposes a segmentation method that incorporates a hybrid diffusion-steered functional derived from the Total variation and the Perona-Malik diffusivities, which have been reported that they can effectively capture semantic features in images. Empirical results from the experiments that were carried out in the MATLAB environment show that the proposed method generates clearer segmented outputs with higher perceptual and objective qualities. More importantly, the proposed method offers lower computational times—an advantage that gives more insights into the possible application of the method in time-sensitive tasks

Expert System with an Embedded Imaging Module for Diagnosing Lung Diseases

Lung diseases are one of the major causes of suffering and death in the world. Improved survival rate could be obtained if the diseases can be detected at its early stage. Specialist doctors with the expertise and experience to interpret medical images and diagnose complex lung diseases are scarce. In this work, a rule-based expert system with an embedded imaging module is developed to assist the general physicians in hospitals and clinics to diagnose lung diseases whenever the services of specialist doctors are not available. The rule-based expert system contains a large knowledge base of data from various categories such as patient's personal and medical history, clinical symptoms, clinical test results and radiological information. An imaging module is integrated into the expert system for the enhancement of chest X-Ray images. The goal of this module is to enhance the chest X-Ray images so that it can provide details similar to more expensive methods such as MRl and CT scan. A new algorithm which is a modified morphological grayscale top hat transform is introduced to increase the visibility of lung nodules in chest X-Rays. Fuzzy inference technique is used to predict the probability of malignancy of the nodules. The output generated by the expert system was compared with the diagnosis made by the specialist doctors. The system is able to produce results\ud which are similar to the diagnosis made by the doctors and is acceptable by clinical standards

Digital Image Processing

This book presents several recent advances that are related or fall under the umbrella of 'digital image processing', with the purpose of providing an insight into the possibilities offered by digital image processing algorithms in various fields. The presented mathematical algorithms are accompanied by graphical representations and illustrative examples for an enhanced readability. The chapters are written in a manner that allows even a reader with basic experience and knowledge in the digital image processing field to properly understand the presented algorithms. Concurrently, the structure of the information in this book is such that fellow scientists will be able to use it to push the development of the presented subjects even further