Video Deraining Mechanism by Preserving the Temporal Consistency and Intrinsic Properties of Rain Streaks

The problem of video deraining is one of the most focused research areas where several techniques are introduced to achieve higher visual quality in the results. The process of video deraining tries to eliminate the presence of rain streaks in the videos so that the overall quality of the video can be enhanced. The existing frameworks tried to accurately eliminate the rain streaks in videos, but there is still room for improvement to preserve the temporal consistencies and intrinsic properties of rain streaks. This work does the job with the combination of handcrafted and deep priors that deeply extract and identify the nature of rain streaks in different dimensions. The proposed work included three main steps: Prior extraction, Derain modelling and optimization. Four major priors are extracted from the frames where the gradient prior (GP) and sparse prior (SP) are extracted from the rain streaks, and the smooth temporal prior (STP) and deep prior (DP) are extracted from the clean video. The unidirectional total variation (UTV) is applied to extract the GP, and the L1 normalization method is followed to extract the SP and STP. The DP is then extracted from the clean frames using the residual gated recurrent deraining network (Res-GRRN) model based on deep learning. Derain modelling is carried out based on the extracted priors, and the stochastic alternating direction multiplier method (SADMM) algorithm is utilized to solve the optimization problem. The proposed approach is then implemented in python and evaluated using the real-world dataset. The overall PSNR achieved by the proposed approach is 39.193dB, which is more optimal than the existing methods

Neural Gradient Regularizer

Owing to its significant success, the prior imposed on gradient maps has consistently been a subject of great interest in the field of image processing. Total variation (TV), one of the most representative regularizers, is known for its ability to capture the sparsity of gradient maps. Nonetheless, TV and its variants often underestimate the gradient maps, leading to the weakening of edges and details whose gradients should not be zero in the original image. Recently, total deep variation (TDV) has been introduced, assuming the sparsity of feature maps, which provides a flexible regularization learned from large-scale datasets for a specific task. However, TDV requires retraining when the image or task changes, limiting its versatility. In this paper, we propose a neural gradient regularizer (NGR) that expresses the gradient map as the output of a neural network. Unlike existing methods, NGR does not rely on the sparsity assumption, thereby avoiding the underestimation of gradient maps. NGR is applicable to various image types and different image processing tasks, functioning in a zero-shot learning fashion, making it a versatile and plug-and-play regularizer. Extensive experimental results demonstrate the superior performance of NGR over state-of-the-art counterparts for a range of different tasks, further validating its effectiveness and versatility

Image Enhancement via Deep Spatial and Temporal Networks

Image enhancement is a classic problem in computer vision and has been studied for decades. It includes various subtasks such as super-resolution, image deblurring, rain removal and denoise. Among these tasks, image deblurring and rain removal have become increasingly active, as they play an important role in many areas such as autonomous driving, video surveillance and mobile applications. In addition, there exists connection between them. For example, blur and rain often degrade images simultaneously, and the performance of their removal rely on the spatial and temporal learning. To help generate sharp images and videos, in this thesis, we propose efficient algorithms based on deep neural networks for solving the problems of image deblurring and rain removal. In the first part of this thesis, we study the problem of image deblurring. Four deep learning based image deblurring methods are proposed. First, for single image deblurring, a new framework is presented which firstly learns how to transfer sharp images to realistic blurry images via a learning-to-blur Generative Adversarial Network (GAN) module, and then trains a learning-to-deblur GAN module to learn how to generate sharp images from blurry versions. In contrast to prior work which solely focuses on learning to deblur, the proposed method learns to realistically synthesize blurring effects using unpaired sharp and blurry images. Second, for video deblurring, spatio-temporal learning and adversarial training methods are used to recover sharp and realistic video frames from input blurry versions. 3D convolutional kernels on the basis of deep residual neural networks are employed to capture better spatio-temporal features, and train the proposed network with both the content loss and adversarial loss to drive the model to generate realistic frames. Third, the problem of extracting sharp image sequences from a single motion-blurred image is tackled. A detail-aware network is presented, which is a cascaded generator to handle the problems of ambiguity, subtle motion and loss of details. Finally, this thesis proposes a level-attention deblurring network, and constructs a new large-scale dataset including images with blur caused by various factors. We use this dataset to evaluate current deep deblurring methods and our proposed method. In the second part of this thesis, we study the problem of image deraining. Three deep learning based image deraining methods are proposed. First, for single image deraining, the problem of joint removal of raindrops and rain streaks is tackled. In contrast to most of prior works which solely focus on the raindrops or rain streaks removal, a dual attention-in-attention model is presented, which removes raindrops and rain streaks simultaneously. Second, for video deraining, a novel end-to-end framework is proposed to obtain the spatial representation, and temporal correlations based on ResNet-based and LSTM-based architectures, respectively. The proposed method can generate multiple deraining frames at a time, which outperforms the state-of-the-art methods in terms of quality and speed. Finally, for stereo image deraining, a deep stereo semantic-aware deraining network is proposed for the first time in computer vision. Different from the previous methods which only learn from pixel-level loss function or monocular information, the proposed network advances image deraining by leveraging semantic information and visual deviation between two views

Image Analysis and Processing With Applications in Proteomics and Medicine

Στην παρούσα διατριβή παρουσιάζονται αυτόματοι αλγόριθμοι ανάλυσης εικόνας για την κατάτμηση διαφόρων τύπων εικόνων, με έμφαση στις εικόνες πρωτεομικής και στις ιατρικές εικόνες. Οι προτεινόμενοι αλγόριθμοι βασίζονται στις αρχές των παραμορφώσιμων μοντέλων. Η διατριβή εστιάζει σε δύο κυρίως στόχους: 1) στην επίλυση του σημαντικού προβλήματος της αυτόματης παραμετροποίησης στην κατάτμηση εικόνας, 2) στην διατύπωση ενός ολοκληρωμένου μοντέλου κατάτμησης εικόνων πρωτεομικής. Η πρώτη συνεισφορά είναι ένα πρωτότυπο πλαίσιο αυτόματης παραμετροποίησης των ενεργών περιγραμμάτων περιοχής. Το πλαίσιο εμπλουτίζει τα αποτελέσματα με αντικειμενικότητα και απελευθερώνει τους τελικούς χρήστες από την επίπονη διαδικασία της εμπειρικής ρύθμισης. Εφαρμόζεται σε διάφορους τύπους ιατρικών εικόνων και παραμένει ανεπηρέαστο στις τροποποιήσεις των ρυθμίσεων των συσκευών λήψης των εικόνων αυτών. Τα πειραματικά αποτελέσματα καταδεικνύουν ότι το προτεινόμενο πλαίσιο διατηρεί υψηλή την ποιότητα κατάτμησης, συγκρίσιμη με εκείνη που επιτυγχάνεται με εμπειρική παραμετροποίηση. Η δεύτερη συνεισφορά είναι ένα αυτόματο μοντέλο βασιζόμενο στα ενεργά περιγράμματα για την κατάτμηση εικόνων πρωτεομικής. Το μοντέλο αντιμετωπίζει σημαντικά προβλήματα συμπεριλαμβανομένων των γραμμών, τεχνουργημάτων, αχνών και επικαλυπτομένων κηλίδων. Ακόμη, παρέχει εναλλακτική λύση στην επιρρεπή σε σφάλματα διαδικασία της χειρωνακτικής επεξεργασίας που απαιτείται στα υπάρχοντα πακέτα λογισμικού. Τα πειραματικά αποτελέσματα καταδεικνύουν ότι το προτεινόμενο μοντέλο υπερτερεί των υπαρχόντων πακέτων λογισμικού σε ποσοτικές μετρικές εντοπισμού και κατάτμησης.This thesis introduces unsupervised image analysis algorithms for the segmentation of several types of images, with an emphasis on proteomics and medical images. Τhe presented algorithms are tailored upon the principles of deformable models. Two objectives are pursued: 1) the core issue of unsupervised parameterization in image segmentation, 2) the formulation of a complete model for the segmentation of proteomics images. The first contribution is a novel framework for automated parameterization of region-based active contours. The presented framework endows segmentation results with objectivity and sets domain users free from the cumbersome process of empirical adjustment. It is applicable on various medical imaging modalities and remains insensitive on alterations in the settings of acquisition devices. The experimental results demonstrate that the presented framework maintains a high segmentation quality, comparable to the one obtained with empirical parameterization. The second contribution is an unsupervised active contour-based model for the segmentation of proteomics images. The presented model copes with crucial issues including streaks, artifacts, faint and overlapping spots. Moreover, it provides an alternate to the error-prone process of manual editing, required in state-of-the-art software packages. The experimental results demonstrate that the proposed model outperforms software packages in terms of detection and segmentation quantity metrics

image analysis and processing with applications in proteomics and medicine

This thesis introduces unsupervised image analysis algorithms for the segmentation of several types of images, with an emphasis on proteomics and medical images. Τhe presented algorithms are tailored upon the principles of deformable models and more specific region-based active contours. Two different objectives are pursued. The first is the core issue of unsupervised parameterization in image segmentation, whereas the second is the formulation of a complete model for the segmentation of proteomics images, which is the first to exploit the appealing attributes of active contours. The first major contribution of this thesis is a novel framework for the automated parameterization of region-based active contours. The presented framework aims to endow segmentation results with objectivity and robustness as well as to set domain users free from the cumbersome and time-consuming process of empirical adjustment. It is applicable on various medical imaging modalities and remains insensitive on alterations in the settings of the acquisition devices. The experimental results demonstrate that the presented framework maintains a segmentation quality which is comparable to the one obtained with empirical parameterization. The second major contribution of this thesis is an unsupervised active contour-based model for the segmentation of proteomics images. The presented model copes with crucial issues in 2D-GE image analysis including streaks, artifacts, faint and overlapping spots. In addition, it provides an alternate to the laborious, error-prone process of manual editing, which is required in state-of-the-art 2D-GE image analysis software packages. The experimental results demonstrate that the presented model outperforms 2D-GE image analysis software packages in terms of detection and segmentation quantity metrics

Measuring and Modeling Fluid Dynamic Processes using Digital Image Sequence Analysis

In this thesis novel motion models have been developed and incorporated into an extended parameter estimation framework that allows to accurately estimate the parameters and regularize them if needed. The performance of this framework has been increased to real time and implemented on inexpensive graphics hardware. Confidence and situation measures have been designed to discard inaccurate estimates. A phase field approach was developed to estimate piecewise smooth motion while detecting object boundaries at the same time. These algorithmic improvements have been successfully applied to three areas of fluid dynamics: air-sea interaction, microfluidics and plant physiology. At the ocean surface, the fluxes of heat and momentum have been measured with thermographic techniques, both spatially and temporally highly resolved. These measurement techniques present milestones for research in air-sea interaction, where point measurements and particle based laboratory measurements represent the state-of-the art. Calculations were done with two models, both making complement assumptions. Still, results derived from both models agree remarkably well. Measurements were conducted in laboratory settings as well as in the field. Microfluidic flow was measured with a new approach to molecular tagging velocimetry that explicitly models Taylor dispersion. This has lead to an increase in accuracy and applicability. Inaccuracies and problems of previous approaches due to Taylor dispersion were successfully evaded. Ground truth test measurements have been conducted, proving the accuracy of this novel technique. For the first time, flow velocities were measured in the xylem of plant leaves with active thermography. This represents a technique for measuring these flows on extended leaf areas on free standing plants, minimizing the impact caused by the measurement. Ground truth measurements on perfused leafs were performed. Measurements were also conducted on free standing plants in a climatic chamber, to measure xylem flows and relate flow velocities to environmental parameter. With a cuvette, environmental factors were varied locally. These measurements underlined the sensitivity of the new approach. A linear relationship in between flow rates and xylem diameter was found