Graph Spectral Image Processing

Recent advent of graph signal processing (GSP) has spurred intensive studies of signals that live naturally on irregular data kernels described by graphs (e.g., social networks, wireless sensor networks). Though a digital image contains pixels that reside on a regularly sampled 2D grid, if one can design an appropriate underlying graph connecting pixels with weights that reflect the image structure, then one can interpret the image (or image patch) as a signal on a graph, and apply GSP tools for processing and analysis of the signal in graph spectral domain. In this article, we overview recent graph spectral techniques in GSP specifically for image / video processing. The topics covered include image compression, image restoration, image filtering and image segmentation

Superpixel labeling for medical image segmentation

openNowadays, most methods for image segmentation consider images in a pixel- wise manner, which is a huge job and also time-consuming. On the other hand, superpixel labeling can make the segmentation task easier in some aspects. First, superpixels carry more information than pixels because they usually follow the edges present in the image. Furthermore, superpixels have perceptual meaning, and finally, they can be very useful in computationally demanding problems, since by mapping pixels to superpixels we are reducing the complexity of the problem. In this thesis, we propose to do superpixel-wise labeling on two med- ical image datasets including ISIC Lesion Skin and Chest X-ray, then we feed them to the U-Net Convolutional Neural Network (CNN) DoubleU-Net and Dual-Aggregation Transformer (DuAT) network to segment our images in term of superpixels. Three different methods of labeling are used in this thesis: Su- perpixel labeling, Extended Superpixel Labeling (Distance-base Labeling), and Random Walk Superpixel labeling. The Superpixel labeled ground truths are used just for training. For the evaluation, we consider the original image and also the original binary ground truth. We considered four different superpixel algorithms, namely Simple Linear Iterative Clustering (SLIC), Felsenszwalb Hut- tenlocher (FH), QuickShift (QS) , and Superpixels Extracted via Energy-Driven Sampling (SEEDS). We evaluate the segmentation result with metrics such as Dice Coefficient, Precision, Intersection Over Union (IOU), and Sensitivity. Our results show the accuracy of 0.89 and 0.95 percent in dice coefficient for skin lesion and chest X-ray datasets respectively. Key Words: Superpixels, Medical Images, U-Net, DoubleU-Net, Image seg- mentation, CNN, DuAT, SEEDS.Nowadays, most methods for image segmentation consider images in a pixel- wise manner, which is a huge job and also time-consuming. On the other hand, superpixel labeling can make the segmentation task easier in some aspects. First, superpixels carry more information than pixels because they usually follow the edges present in the image. Furthermore, superpixels have perceptual meaning, and finally, they can be very useful in computationally demanding problems, since by mapping pixels to superpixels we are reducing the complexity of the problem. In this thesis, we propose to do superpixel-wise labeling on two med- ical image datasets including ISIC Lesion Skin and Chest X-ray, then we feed them to the U-Net Convolutional Neural Network (CNN) DoubleU-Net and Dual-Aggregation Transformer (DuAT) network to segment our images in term of superpixels. Three different methods of labeling are used in this thesis: Su- perpixel labeling, Extended Superpixel Labeling (Distance-base Labeling), and Random Walk Superpixel labeling. The Superpixel labeled ground truths are used just for training. For the evaluation, we consider the original image and also the original binary ground truth. We considered four different superpixel algorithms, namely Simple Linear Iterative Clustering (SLIC), Felsenszwalb Hut- tenlocher (FH), QuickShift (QS) , and Superpixels Extracted via Energy-Driven Sampling (SEEDS). We evaluate the segmentation result with metrics such as Dice Coefficient, Precision, Intersection Over Union (IOU), and Sensitivity. Our results show the accuracy of 0.89 and 0.95 percent in dice coefficient for skin lesion and chest X-ray datasets respectively. Key Words: Superpixels, Medical Images, U-Net, DoubleU-Net, Image seg- mentation, CNN, DuAT, SEEDS

An optimal factor analysis approach to improve the wavelet-based image resolution enhancement techniques

The existing wavelet-based image resolution enhancement techniques have many assumptions, such as limitation of the way to generate low-resolution images and the selection of wavelet functions, which limits their applications in different fields. This paper initially identifies the factors that effectively affect the performance of these techniques and quantitatively evaluates the impact of the existing assumptions. An approach called Optimal Factor Analysis employing the genetic algorithm is then introduced to increase the applicability and fidelity of the existing methods. Moreover, a new Figure of Merit is proposed to assist the selection of parameters and better measure the overall performance. The experimental results show that the proposed approach improves the performance of the selected image resolution enhancement methods and has potential to be extended to other methods

Interactive segmentation: a scalable superpixel-based method

This paper addresses the problem of interactive multiclass segmentation of images. We propose a fast and efficient new interactive segmentation method called superpixel alpha fusion (SaF). From a few strokes drawn by a user over an image, this method extracts relevant semantic objects. To get a fast calculation and an accurate segmentation, SaF uses superpixel oversegmentation and support vector machine classification. We compare SaF with competing algorithms by evaluating its performances on reference benchmarks. We also suggest four new datasets to evaluate the scalability of interactive segmentation methods, using images from some thousand to several million pixels. We conclude with two applications of SaF

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Panchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Panchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area

An Optimal Factor Analysis Approach to Improve the Wavelet-based Image Resolution Enhancement Techniques

The existing wavelet-based image resolution enhancement techniques have many assumptions, such as limitation of the way to generate low-resolution images and the selection of wavelet functions, which limits their applications in different fields. This paper initially identifies the factors that effectively affect the performance of these techniques and quantitatively evaluates the impact of the existing assumptions. An approach called Optimal Factor Analysis employing the genetic algorithm is then introduced to increase the applicability and fidelity of the existing methods. Moreover, a new Figure of Merit is proposed to assist the selection of parameters and better measure the overall performance. The experimental results show that the proposed approach improves the performance of the selected image resolution enhancement methods and has potential to be extended to other methods

Robust density modelling using the student's t-distribution for human action recognition

The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is highly sensitive to outliers. The Gaussian distribution is also often used as base component of graphical models for recognising human actions in the videos (hidden Markov model and others) and the presence of outliers can significantly affect the recognition accuracy. In contrast, the Student's t-distribution is more robust to outliers and can be exploited to improve the recognition rate in the presence of abnormal data. In this paper, we present an HMM which uses mixtures of t-distributions as observation probabilities and show how experiments over two well-known datasets (Weizmann, MuHAVi) reported a remarkable improvement in classification accuracy. © 2011 IEEE

3D Segmentation & Measurement of Macular Holes

Macular holes are blinding conditions where a hole develops in the central part of retina, resulting in reduced central vision. The prognosis and treatment options are related to a number of variables including the macular hole size and shape. In this work we introduce a method to segment and measure macular holes in three-dimensional (3D) data. High-resolution spectral domain optical coherence tomography (SD-OCT) allows precise imaging of the macular hole geometry in three dimensions, but the measurement of these by human observers is time consuming and prone to high inter- and intra-observer variability, being characteristically measured in 2D rather than 3D. This work introduces several novel techniques to automatically retrieve accurate 3D measurements of the macular hole, including surface area, base area, base diameter, top area, top diameter, height, and minimum diameter. Specifically, it is introducing a multi-scale 3D level set segmentation approach based on a state-of-the-art level set method, and introducing novel curvature-based cutting and 3D measurement procedures. The algorithm is fully automatic, and we validate the extracted measurements both qualitatively and quantitatively, where the results show the method to be robust across a variety of scenarios. A segmentation software package is presented for targeting medical and biological applications, with a high level of visual feedback and several usability enhancements over existing packages. Specifically, it is providing a substantially faster graphics processing unit (GPU) implementation of the local Gaussian distribution fitting (LGDF) energy model, which can segment inhomogeneous objects with poorly defined boundaries as often encountered in biomedical images. It also provides interactive brushes to guide the segmentation process in a semi-automated framework. The speed of implementation allows us to visualise the active surface in real-time with a built-in ray tracer, where users may halt evolution at any timestep to correct implausible segmentation by painting new blocking regions or new seeds. Quantitative and qualitative validation is presented, demonstrating the practical efficacy of the interactive elements for a variety of real-world datasets. The size of macular holes is known to be one of the strongest predictors of surgical success both anatomically and functionally. Furthermore, it is used to guide the choice of treatment, the optimum surgical approach and to predict outcome. Our automated 3D image segmentation algorithm has extracted 3D shape-based macular hole measurements and described the dimensions and morphology. Our approach is able to robustly and accurately measure macular hole dimensions. This thesis is considered as a significant contribution for clinical applications particularly in the field of macular hole segmentation and shape analysis