Skin image illumination modeling and chromophore identication for melanoma diagnosis

International audienceThe presence of illumination variation in dermatological images has a negative impact on the automatic detection and analysis of cutaneous lesions. This paper proposes a new illumination modeling and chromophore identication method to correct lighting variation in skin lesion images, as well as to extract melanin and hemoglobin concentrations of human skin, based on an adaptive bilateral decomposition and a weighted polynomial curve tting, with the knowledge of a multi-layered skin model. Different from state-of-the-art approaches based on the Lambert law, the proposed method, considering both specular reection and diffuse reection of the skin, enables us to address highlight and strong shading effects usually existing in skin color images captured in an uncontrolled environment. The derived melanin and hemoglobin indices, directly relating to the pathological tissue conditions, tend to be less inuenced by external imaging factors and are more efcient in describing pigmentation distributions. Experiments show that the proposed method gave better visual results and superior lesion segmentation, when compared to two other illumination correction algorithms, both designed specically for dermatological images. For computer-aided diagnosis of melanoma, sensitivity achieves 85.52% when using our chromophore descriptors, which is 8~20% higher than those derived from other color descriptors. This demonstrates the benet of the proposed method for automatic skin disease analysis

Melanin and Hemoglobin Identification for Skin Disease Analysis

International audienceThis paper proposes a novel method to extract melanin and hemoglobin concentrations of human skin, using bilateral decomposition with the knowledge of a multiple layered skin model and absorbance characteristics of major chromophores. Different from state-of-art approaches, the proposed method enables to address highlight and strong shading usually existing in skin color images captured under uncontrolled environment. The derived melanin and hemoglobin indices, directly related to the pathological tissue conditions, tend to be less influenced by external imaging factors and are effective for describing pigmentation distributions. Experiments demonstrate the value of the proposed method for computer-aided diagnosis of different skin diseases. The diagnostic accuracy of melanoma increases by 9-15% for conventional RGB lesion images, compared to techniques using other color descriptors. The discrimination of inflammatory acne and hyperpigmentation reveals acne stage, which would be useful for acne severity evaluation. It is expected that this new method will prove useful for other skin disease analysis

Laparoscopic Image Recovery and Stereo Matching

Laparoscopic imaging can play a significant role in the minimally invasive surgical procedure. However, laparoscopic images often suffer from insufficient and irregular light sources, specular highlight surfaces, and a lack of depth information. These problems can negatively influence the surgeons during surgery, and lead to erroneous visual tracking and potential surgical risks. Thus, developing effective image-processing algorithms for laparoscopic vision recovery and stereo matching is of significant importance. Most related algorithms are effective on nature images, but less effective on laparoscopic images. The first purpose of this thesis is to restore low-light laparoscopic vision, where an effective image enhancement method is proposed by identifying different illumination regions and designing the enhancement criteria for desired image quality. This method can enhance the low-light region by reducing noise amplification during the enhancement process. In addition, this thesis also proposes a simplified Retinex optimization method for non-uniform illumination enhancement. By integrating the prior information of the illumination and reflectance into the optimization process, this method can significantly enhance the dark region while preserving naturalness, texture details, and image structures. Moreover, due to the replacement of the total variation term with two $l_2$-norm terms, the proposed algorithm has a significant computational advantage. Second, a global optimization method for specular highlight removal from a single laparoscopic image is proposed. This method consists of a modified dichromatic reflection model and a novel diffuse chromaticity estimation technique. Due to utilizing the limited color variation of the laparoscopic image, the estimated diffuse chromaticity can approximate the true diffuse chromaticity, which allows us to effectively remove the specular highlight with texture detail preservation. Third, a robust edge-preserving stereo matching method is proposed, based on sparse feature matching, left and right illumination equalization, and refined disparity optimization processes. The sparse feature matching and illumination equalization techniques can provide a good disparity map initialization so that our refined disparity optimization can quickly obtain an accurate disparity map. This approach is particularly promising on surgical tool edges, smooth soft tissues, and surfaces with strong specular highlight

Hybrid-Rendering Techniques in GPU

Ray tracing has long been the holy grail of real time rendering. This technique, commonly used for photo realism, simulates the physical behavior of light, at the cost of being computationally heavy. With the introduction of Nvidia RTX graphic card family, which provides hardware support for ray tracing, this technique started to look like a reality for real time. However, the same problems that afflicted the usage of this technique remain, and even with specialized hardware it is still extremely expensive. To account for these drawbacks, researchers and developers pair this technique with rasterization and denoising. This results in a hybrid system that tries to join the best of both worlds, having both photo realistic quality and real time performance. In this work we intend on further exploring hybrid render systems, offering a review of the state of the art with a special focus on real time ray tracing and our own hybrid implementation with photo realistic quality and real time performance (>30 fps), implemented using the Vulkan API. In this project, we highlight the detailed analysis of the impacts of History Rectification (Variance Color Clamping) on the temporal filter component of the denoising system and how to overcome the introduced artifacts. Additionally, we also highlight the analysis of the introduction of a separable blur on the spatial filter and the introduction of Reinhard Tone Mapping prior to denoising, consequently improving this procedure

Highlights Analysis System (HAnS) for low dynamic range to high dynamic range conversion of cinematic low dynamic range content

We propose a novel and efficient algorithm for detection of specular reflections and light sources (highlights) in cinematic content. The detection of highlights is important for reconstructing them properly in the conversion of the low dynamic range (LDR) to high dynamic range (HDR) content. Highlights are often difficult to be distinguished from bright diffuse surfaces, due to their brightness being reduced in the conventional LDR content production. Moreover, the cinematic LDR content is subject to the artistic use of effects that change the apparent brightness of certain image regions (e.g. limiting depth of field, grading, complex multi-lighting setup, etc.). To ensure the robustness of highlights detection to these effects, the proposed algorithm goes beyond considering only absolute brightness and considers five different features. These features are: the size of the highlight relative to the size of the surrounding image structures, the relative contrast in the surrounding of the highlight, its absolute brightness expressed through the luminance (luma feature), through the saturation in the color space (maxRGB feature) and through the saturation in white (minRGB feature). We evaluate the algorithm on two different image data-sets. The first one is a publicly available LDR image data-set without cinematic content, which allows comparison to the broader State of the art. Additionally, for the evaluation on cinematic content, we create an image data-set consisted of manually annotated cinematic frames and real-world images. For the purpose of demonstrating the proposed highlights detection algorithm in a complete LDR-to-HDR conversion pipeline, we additionally propose a simple inverse-tone-mapping algorithm. The experimental analysis shows that the proposed approach outperforms conventional highlights detection algorithms on both image data-sets, achieves high quality reconstruction of the HDR content and is suited for use in LDR-to-HDR conversion