A survey, review, and future trends of skin lesion segmentation and classification

The Computer-aided Diagnosis or Detection (CAD) approach for skin lesion analysis is an emerging field of research that has the potential to alleviate the burden and cost of skin cancer screening. Researchers have recently indicated increasing interest in developing such CAD systems, with the intention of providing a user-friendly tool to dermatologists to reduce the challenges encountered or associated with manual inspection. This article aims to provide a comprehensive literature survey and review of a total of 594 publications (356 for skin lesion segmentation and 238 for skin lesion classification) published between 2011 and 2022. These articles are analyzed and summarized in a number of different ways to contribute vital information regarding the methods for the development of CAD systems. These ways include: relevant and essential definitions and theories, input data (dataset utilization, preprocessing, augmentations, and fixing imbalance problems), method configuration (techniques, architectures, module frameworks, and losses), training tactics (hyperparameter settings), and evaluation criteria. We intend to investigate a variety of performance-enhancing approaches, including ensemble and post-processing. We also discuss these dimensions to reveal their current trends based on utilization frequencies. In addition, we highlight the primary difficulties associated with evaluating skin lesion segmentation and classification systems using minimal datasets, as well as the potential solutions to these difficulties. Findings, recommendations, and trends are disclosed to inform future research on developing an automated and robust CAD system for skin lesion analysis

Domain-inspired image processing and computer vision to support deep-sea benthic ecology

Optical imagery is a necessary methodological tool for ecological research within marine environments, particularly in deeper waters. For benthic (seafloor) surveys, interpretation of image data is crucial to creating high-resolution maps of seabed habitats. This is fundamental to marine spatial planning and mitigating long-term damage of anthropogenic stressors such as growing resource demand, climate change and pollution. However there are numerous, and significant, issues in extracting a reliable ground-truth from imagery to support this process. Analysis of benthic images is difficult, due in part to the extreme variation and inconsistency in image quality - caused by complex interactions between light and water. It is also time-consuming. This thesis is dedicated to providing solutions to manage these challenges, from a strong perspective of the end-user. Specifically, we aim to improve the annotation of benthic habitats from imagery in terms of quality, consistency and efficiency. Throughout, we consider the purpose the imagery serves and work closely with end-users to best optimize our solutions. First, and for the majority of this thesis, we investigate image processing techniques to improve the appearance of image features important for habitat classification. We find that tone mapping is an effective and simple (and thus accessible) method through which to improve image quality for interpretation. We describe beneficial (expert-informed) properties for brightness distributions in underwater images and introduce a novel tone-mapping algorithm, Weibull Tone Mapping (WTM), to enhance benthic images. WTM theory operates within general constraints that model image requirements (properties) specified by image analysts, yet possesses a suitable degree of flexibility and customisation. As a tool, WTM provides analysts with a fast and ‘user-friendly’ method to improve benthic habitat classification. Second, we consider computer vision methods that could automatically identify benthic habitats in imagery, relieving the analysis bottleneck. We find that baseline transfer learning of machine learning models, with limited optimization, will better facilitate adoption by novice users, yet still provides a powerful means to swiftly extract and assess benthic data

Automatic human face detection in color images

Automatic human face detection in digital image has been an active area of research over the past decade. Among its numerous applications, face detection plays a key role in face recognition system for biometric personal identification, face tracking for intelligent human computer interface (HCI), and face segmentation for object-based video coding. Despite significant progress in the field in recent years, detecting human faces in unconstrained and complex images remains a challenging problem in computer vision. An automatic system that possesses a similar capability as the human vision system in detecting faces is still a far-reaching goal. This thesis focuses on the problem of detecting human laces in color images. Although many early face detection algorithms were designed to work on gray-scale Images, strong evidence exists to suggest face detection can be done more efficiently by taking into account color characteristics of the human face. In this thesis, we present a complete and systematic face detection algorithm that combines the strengths of both analytic and holistic approaches to face detection. The algorithm is developed to detect quasi-frontal faces in complex color Images. This face class, which represents typical detection scenarios in most practical applications of face detection, covers a wide range of face poses Including all in-plane rotations and some out-of-plane rotations. The algorithm is organized into a number of cascading stages including skin region segmentation, face candidate selection, and face verification. In each of these stages, various visual cues are utilized to narrow the search space for faces. In this thesis, we present a comprehensive analysis of skin detection using color pixel classification, and the effects of factors such as the color space, color classification algorithm on segmentation performance. We also propose a novel and efficient face candidate selection technique that is based on color-based eye region detection and a geometric face model. This candidate selection technique eliminates the computation-intensive step of window scanning often employed In holistic face detection, and simplifies the task of detecting rotated faces. Besides various heuristic techniques for face candidate verification, we developface/nonface classifiers based on the naive Bayesian model, and investigate three feature extraction schemes, namely intensity, projection on face subspace and edge-based. Techniques for improving face/nonface classification are also proposed, including bootstrapping, classifier combination and using contextual information. On a test set of face and nonface patterns, the combination of three Bayesian classifiers has a correct detection rate of 98.6% at a false positive rate of 10%. Extensive testing results have shown that the proposed face detector achieves good performance in terms of both detection rate and alignment between the detected faces and the true faces. On a test set of 200 images containing 231 faces taken from the ECU face detection database, the proposed face detector has a correct detection rate of 90.04% and makes 10 false detections. We have found that the proposed face detector is more robust In detecting in-plane rotated laces, compared to existing face detectors. +D2

Yleiskäyttöisen kuvankäsittelyliukuhihnan suunnittelu ja toteutus kamerapuhelimiin

Valtaosa nykyisistä edullisen hintaluokan matkapuhelimista sisältää kameran, joka käyttää yksinkertaista optiikkaa ja halpaa kamerasensoria. Riittävän kuvanlaadun takaamiseksi kuvaa käsitellään kuvankäsittelyalgoritmeilla, jotka yhdessä muodostavat kuvankäsittelyliukuhihnan. Paras suorituskyky saavutetaan yleensä rautapohjaisella liukuhihnalla. Ohjelmistopohjaisia ratkaisuja voidaan kuitenkin suosia tuotantokustannusten minimoimiseksi ja liukuhihnan joustavuuden sekä laajennettavuuden parantamiseksi. Jotta liukuhihnan muistinkulutus voidaan minimoida vähämuistisessa ympäristössä, on järkevää toteuttaa ohjelmistopohjainen kuvankäsittelyliukuhihna käyttäen juovapuskureita. Juovapuskurit monimutkaistavat liukuhihnan hallintaa, mikä kuitenkin on ratkaistavissa automaation avulla. Tämä diplomityö esittelee yleiskäyttöisen ohjelmistokehyksen juovapuskuripohjaiselle kuvankäsittelyliukuhihnalle. Ohjelmistokehys soveltuu vähämuistisiin ympäristöihin ja helpottaa kuvankäsittelyalgoritmien lisäämistä, poistamista ja muuttamista sekä muita liukuhihnan hallintatehtäviä merkittävästi. Säästöt kehitysajassa voivat olla jopa kuukausia. Ohjelmistokehyksen suorituskykyä ja muistinkäyttöä verrataan nykyisiin toteutuksiin käyttäen todellista kuvankäsittelyliukuhihnaa testitapauksena. Työssä pohditaan myös, kuinka juovapuskuripohjainen liukuhihna voitaisiin rinnakkaistaa entistä paremman suorituskyvyn saavuttamiseksi moniydinpuhelimissa. Rinnakkaistetun liukuhihan toteuttamiseksi esitellään kaksi lähestymistapaa: dataviipaloitu liukuhihna ja työjonopohjainen liukuhihna. Tutkimus osoittaa, että ohjelmistokehys säästää muistia yli 99% verrattuna perinteisiin toteutuksiin, jotka käyttävät ping-pong puskurointia täyden koon kuvapuskureilla. Toteutettu ohjelmistokehys parantaa myös suorituskykyä paremman välimuistin käytön ansiosta ja lisää kuvankäsittelyliukuhihnan joustavuutta useilla erilaisilla konfiguraatioilla.The majority of the current affordable mobile devices contain a camera with simple optics and a low-cost camera sensor. In these devices, the quality of the captured images is made acceptable with various image processing algorithms that together form an image reconstruction pipeline. The best performance is often achieved with a hardware pipeline, but software implementations can be preferred to minimize production costs and to maximize flexibility. To minimize memory consumption in such a limited-resource environment, it is reasonable to implement a software-based image reconstruction pipeline using line buffers. The line buffers complicate the management of the pipeline, which, however, can be solved by increasing development tool automatization. This Thesis presents a generic software framework for a line-buffer-based image reconstruction pipeline. The presented framework is capable of operating in low-memory environments and significantly eases algorithm insertions, changes of processing order, and other pipeline management tasks. The savings in development time can be even months. The performance and memory usage of the software framework is compared to contemporary implementations by using a real image reconstruction pipeline as the test case. The Thesis also discusses how the line-buffer-based pipeline could be parallelized to achieve improved performance on multi-core devices. Two promising approaches are considered: slice-based parallelization and work-queue-based parallelization. The experiments show that the software framework offers over 99% memory savings compared with traditional implementations using a ping-pong buffer scheme with full-sized image buffers. The implemented framework also enhances processing performance due to better cache usage and increases flexibility with various pipeline configurations

High-performance hardware accelerators for image processing in space applications

Mars is a hard place to reach. While there have been many notable success stories in getting probes to the Red Planet, the historical record is full of bad news. The success rate for actually landing on the Martian surface is even worse, roughly 30%. This low success rate must be mainly credited to the Mars environment characteristics. In the Mars atmosphere strong winds frequently breath. This phenomena usually modifies the lander descending trajectory diverging it from the target one. Moreover, the Mars surface is not the best place where performing a safe land. It is pitched by many and close craters and huge stones, and characterized by huge mountains and hills (e.g., Olympus Mons is 648 km in diameter and 27 km tall). For these reasons a mission failure due to a landing in huge craters, on big stones or on part of the surface characterized by a high slope is highly probable. In the last years, all space agencies have increased their research efforts in order to enhance the success rate of Mars missions. In particular, the two hottest research topics are: the active debris removal and the guided landing on Mars. The former aims at finding new methods to remove space debris exploiting unmanned spacecrafts. These must be able to autonomously: detect a debris, analyses it, in order to extract its characteristics in terms of weight, speed and dimension, and, eventually, rendezvous with it. In order to perform these tasks, the spacecraft must have high vision capabilities. In other words, it must be able to take pictures and process them with very complex image processing algorithms in order to detect, track and analyse the debris. The latter aims at increasing the landing point precision (i.e., landing ellipse) on Mars. Future space-missions will increasingly adopt Video Based Navigation systems to assist the entry, descent and landing (EDL) phase of space modules (e.g., spacecrafts), enhancing the precision of automatic EDL navigation systems. For instance, recent space exploration missions, e.g., Spirity, Oppurtunity, and Curiosity, made use of an EDL procedure aiming at following a fixed and precomputed descending trajectory to reach a precise landing point. This approach guarantees a maximum landing point precision of 20 km. By comparing this data with the Mars environment characteristics, it is possible to understand how the mission failure probability still remains really high. A very challenging problem is to design an autonomous-guided EDL system able to even more reduce the landing ellipse, guaranteeing to avoid the landing in dangerous area of Mars surface (e.g., huge craters or big stones) that could lead to the mission failure. The autonomous behaviour of the system is mandatory since a manual driven approach is not feasible due to the distance between Earth and Mars. Since this distance varies from 56 to 100 million of km approximately due to the orbit eccentricity, even if a signal transmission at the light speed could be possible, in the best case the transmission time would be around 31 minutes, exceeding so the overall duration of the EDL phase. In both applications, algorithms must guarantee self-adaptability to the environmental conditions. Since the Mars (and in general the space) harsh conditions are difficult to be predicted at design time, these algorithms must be able to automatically tune the internal parameters depending on the current conditions. Moreover, real-time performances are another key factor. Since a software implementation of these computational intensive tasks cannot reach the required performances, these algorithms must be accelerated via hardware. For this reasons, this thesis presents my research work done on advanced image processing algorithms for space applications and the associated hardware accelerators. My research activity has been focused on both the algorithm and their hardware implementations. Concerning the first aspect, I mainly focused my research effort to integrate self-adaptability features in the existing algorithms. While concerning the second, I studied and validated a methodology to efficiently develop, verify and validate hardware components aimed at accelerating video-based applications. This approach allowed me to develop and test high performance hardware accelerators that strongly overcome the performances of the actual state-of-the-art implementations. The thesis is organized in four main chapters. Chapter 2 starts with a brief introduction about the story of digital image processing. The main content of this chapter is the description of space missions in which digital image processing has a key role. A major effort has been spent on the missions in which my research activity has a substantial impact. In particular, for these missions, this chapter deeply analizes and evaluates the state-of-the-art approaches and algorithms. Chapter 3 analyzes and compares the two technologies used to implement high performances hardware accelerators, i.e., Application Specific Integrated Circuits (ASICs) and Field Programmable Gate Arrays (FPGAs). Thanks to this information the reader may understand the main reasons behind the decision of space agencies to exploit FPGAs instead of ASICs for high-performance hardware accelerators in space missions, even if FPGAs are more sensible to Single Event Upsets (i.e., transient error induced on hardware component by alpha particles and solar radiation in space). Moreover, this chapter deeply describes the three available space-grade FPGA technologies (i.e., One-time Programmable, Flash-based, and SRAM-based), and the main fault-mitigation techniques against SEUs that are mandatory for employing space-grade FPGAs in actual missions. Chapter 4 describes one of the main contribution of my research work: a library of high-performance hardware accelerators for image processing in space applications. The basic idea behind this library is to offer to designers a set of validated hardware components able to strongly speed up the basic image processing operations commonly used in an image processing chain. In other words, these components can be directly used as elementary building blocks to easily create a complex image processing system, without wasting time in the debug and validation phase. This library groups the proposed hardware accelerators in IP-core families. The components contained in a same family share the same provided functionality and input/output interface. This harmonization in the I/O interface enables to substitute, inside a complex image processing system, components of the same family without requiring modifications to the system communication infrastructure. In addition to the analysis of the internal architecture of the proposed components, another important aspect of this chapter is the methodology used to develop, verify and validate the proposed high performance image processing hardware accelerators. This methodology involves the usage of different programming and hardware description languages in order to support the designer from the algorithm modelling up to the hardware implementation and validation. Chapter 5 presents the proposed complex image processing systems. In particular, it exploits a set of actual case studies, associated with the most recent space agency needs, to show how the hardware accelerator components can be assembled to build a complex image processing system. In addition to the hardware accelerators contained in the library, the described complex system embeds innovative ad-hoc hardware components and software routines able to provide high performance and self-adaptable image processing functionalities. To prove the benefits of the proposed methodology, each case study is concluded with a comparison with the current state-of-the-art implementations, highlighting the benefits in terms of performances and self-adaptability to the environmental conditions

Color in context and spatial color computation

The purpose of this dissertation is to contribute in the field of spatial color computation models.We begin introducing an overview about different approaches in the definitionof computational models of color in digital imaging. In particular, we present a recent accurate mathematical definition of the Retinex algorithm, that lead to the definition of a new computational model called Random Spray Retinex (RSR). We then introduce the tone mapping problem, discussing the need for color computation in the implementation of a perceptual correct computational model. At this aim we will present the HDR Retinex algorithm, that addresses tone mappingand color constancy at the same time. In the end, we present some experiments analyzing the influence of HDR Retinex spatial color computation on tristimulus colors obtained using different Color Matching Functions (CMFs) on spectral luminance distribution generated by a photometric raytracer

Discrete Wavelet Transforms

The discrete wavelet transform (DWT) algorithms have a firm position in processing of signals in several areas of research and industry. As DWT provides both octave-scale frequency and spatial timing of the analyzed signal, it is constantly used to solve and treat more and more advanced problems. The present book: Discrete Wavelet Transforms: Algorithms and Applications reviews the recent progress in discrete wavelet transform algorithms and applications. The book covers a wide range of methods (e.g. lifting, shift invariance, multi-scale analysis) for constructing DWTs. The book chapters are organized into four major parts. Part I describes the progress in hardware implementations of the DWT algorithms. Applications include multitone modulation for ADSL and equalization techniques, a scalable architecture for FPGA-implementation, lifting based algorithm for VLSI implementation, comparison between DWT and FFT based OFDM and modified SPIHT codec. Part II addresses image processing algorithms such as multiresolution approach for edge detection, low bit rate image compression, low complexity implementation of CQF wavelets and compression of multi-component images. Part III focuses watermaking DWT algorithms. Finally, Part IV describes shift invariant DWTs, DC lossless property, DWT based analysis and estimation of colored noise and an application of the wavelet Galerkin method. The chapters of the present book consist of both tutorial and highly advanced material. Therefore, the book is intended to be a reference text for graduate students and researchers to obtain state-of-the-art knowledge on specific applications