A Convex Model for Edge-Histogram Specification with Applications to Edge-preserving Smoothing

The goal of edge-histogram specification is to find an image whose edge image has a histogram that matches a given edge-histogram as much as possible. Mignotte has proposed a non-convex model for the problem [M. Mignotte. An energy-based model for the image edge-histogram specification problem. IEEE Transactions on Image Processing, 21(1):379--386, 2012]. In his work, edge magnitudes of an input image are first modified by histogram specification to match the given edge-histogram. Then, a non-convex model is minimized to find an output image whose edge-histogram matches the modified edge-histogram. The non-convexity of the model hinders the computations and the inclusion of useful constraints such as the dynamic range constraint. In this paper, instead of considering edge magnitudes, we directly consider the image gradients and propose a convex model based on them. Furthermore, we include additional constraints in our model based on different applications. The convexity of our model allows us to compute the output image efficiently using either Alternating Direction Method of Multipliers or Fast Iterative Shrinkage-Thresholding Algorithm. We consider several applications in edge-preserving smoothing including image abstraction, edge extraction, details exaggeration, and documents scan-through removal. Numerical results are given to illustrate that our method successfully produces decent results efficiently

A unified framework for document image restoration

Ph.DDOCTOR OF PHILOSOPH

Medical image synthesis using generative adversarial networks: towards photo-realistic image synthesis

This proposed work addresses the photo-realism for synthetic images. We introduced a modified generative adversarial network: StencilGAN. It is a perceptually-aware generative adversarial network that synthesizes images based on overlaid labelled masks. This technique can be a prominent solution for the scarcity of the resources in the healthcare sector

Sparse Gradient Optimization and its Applications in Image Processing

Millions of digital images are captured by imaging devices on a daily basis. The way imaging devices operate follows an integral process from which the information of the original scene needs to be estimated. The estimation is done by inverting the integral process of the imaging device with the use of optimization techniques. This linear inverse problem, the inversion of the integral acquisition process, is at the heart of several image processing applications such as denoising, deblurring, inpainting, and super-resolution. We describe in detail the use of linear inverse problems in these applications. We review and compare several state-of-the-art optimization algorithms that invert this integral process. Linear inverse problems are usually very difficult to solve. Therefore, additional prior assumptions need to be introduced to successfully estimate the output signal. Several priors have been suggested in the research literature, with the Total Variation (TV) being one of the most prominent. In this thesis, we review another prior, the l0 pseudo-norm over the gradient domain. This prior allows full control over how many non-zero gradients are retained to approximate prominent structures of the image. We show the superiority of the l0 gradient prior over the TV prior in recovering genuinely piece-wise constant signals. The l0 gradient prior has shown to produce state-of-the-art results in edge-preserving image smoothing. Moreover, this general prior can be applied to several other applications, such as edge extraction, clip-art JPEG artifact removal, non-photorealistic image rendering, detail magnification, and tone mapping. We review and evaluate several state-of-the-art algorithms that solve the optimization problem based on the l0 gradient prior. Subsequently we apply the l0 gradient prior to two applications where we show superior results as compared to the current state-of-the-art. The first application is that of single-image reflection removal. Existing solutions to this problem have shown limited success because of the highly ill-posed nature of the problem. We show that the standard l0 gradient prior with a modified data-fidelity term based on the Laplacian operator is able to sufficiently remove unwanted reflections from images in many realistic scenarios. We conduct extensive experiments and show that our method outperforms the state-of-the-art. In the second application of haze removal from visible-NIR image pairs we propose a novel optimization framework, where the prior term penalizes the number of non-zero gradients of the difference between the output and the NIR image. Due to the longer wavelengths of NIR, an image taken in the NIR spectrum suffers significantly less from haze artifacts. Using this prior term, we are able to transfer details from the haze-free NIR image to the final result. We show that our formulation provides state-of-the-art results compared to haze removal methods that use a single image and also to those that are based on visible-NIR image pairs

On Using and Improving Gradient Domain Processing for Image Enhancement

Ph.DDOCTOR OF PHILOSOPH

Exploiting Spatio-Temporal Coherence for Video Object Detection in Robotics

This paper proposes a method to enhance video object detection for indoor environments in robotics. Concretely, it exploits knowledge about the camera motion between frames to propagate previously detected objects to successive frames. The proposal is rooted in the concepts of planar homography to propose regions of interest where to find objects, and recursive Bayesian filtering to integrate observations over time. The proposal is evaluated on six virtual, indoor environments, accounting for the detection of nine object classes over a total of ∼ 7k frames. Results show that our proposal improves the recall and the F1-score by a factor of 1.41 and 1.27, respectively, as well as it achieves a significant reduction of the object categorization entropy (58.8%) when compared to a two-stage video object detection method used as baseline, at the cost of small time overheads (120 ms) and precision loss (0.92).</p

Digital Restoration of Damaged Historical Parchment

In this thesis we describe the development of a pipeline for digitally restoring damaged historical parchment. The work was carried out in collaboration with London Metropolitan Archives (LMA), who are in possession of an extremely valuable 17th century document called The Great Parchment Book. This book served as the focus of our project and throughout this thesis we demonstrate our methods on its folios. Our aim was to expose the content of the book in a legible form so that it can be properly catalogued and studied. Our approach begins by acquiring an accurate digitisation of the pages. We have developed our own 3D reconstruction pipeline detailed in Chapter 5 in which each parchment is imaged using a hand-held digital-SLR camera, and the resulting image set is used to generate a high-resolution textured 3D reconstruction of each parchment. Investigation into methods for flatting the parchments demonstrated an analogy with surface parametrization. Flattening the entire parchment globally with various existing parametrization algorithms is problematic, as discussed in Chapters 4, 6, and 7, since this approach is blind to the distortion undergone by the parchment. We propose two complementary approaches to deal with this issue. Firstly, exploiting the fact that a reader will only ever inspect a small area of the folio at a given time, we proposed a method for performing local undistortion of the parchments inside an interactive viewer application. The application, described in Chapter 6, allows a user to browse a parchment folio as the application un-distorts in real-time the area of the parchment currently under inspection. It also allows the user to refer back to the original image set of the parchment to help with resolving ambiguities in the reconstruction and to deal with issues of provenance. Secondly, we proposed a method for estimating the actual deformation undergone by each parchment when it was damaged by using cues in the text. Since the text was originally written in straight lines and in a roughly uniform script size, we can detect the the variation in text orientation and size and use this information to estimate the deformation. in Chapter 7 we then show how this deformation can be inverted by posing the problem as a Poisson mesh deformation, and solving it in a way that guarantees local injectivity, to generate a globally flattened and undistorted image of each folio. We also show how these images can optionally be colour corrected to remove the shading cues baked into the reconstruction texture, and the discolourations in the parchment itself, to further improve legibility and give a more complete impression that the parchment has been restored. The methods we have developed have been very well received by London Metropolitan Archives, as well the the larger archival community. We have used the methods to digitise the entire Great Parchment Book, and have demonstrated our global flattening method on eight folios. As of the time of writing of this thesis, our methods are being used to virtually restore all of the remaining folios of the Great Parchment Book. Staff at LMA are also investigating potential future directions by experimenting with other interesting documents in their collections, and are exploring the possibility of setting up a service which would give access to our methods to other archival institutions with similarly damaged documents

PDE-based image compression based on edges and optimal data

This thesis investigates image compression with partial differential equations (PDEs) based on edges and optimal data. It first presents a lossy compression method for cartoon-like images. Edges together with some adjacent pixel values are extracted and encoded. During decoding, information not covered by this data is reconstructed by PDE-based inpainting with homogeneous diffusion. The result is a compression codec based on perceptual meaningful image features which is able to outperform JPEG and JPEG2000. In contrast, the second part of the thesis focuses on the optimal selection of inpainting data. The proposed methods allow to recover a general image from only 4% of all pixels almost perfectly, even with homogeneous diffusion inpainting. A simple conceptual encoding shows the potential of an optimal data selection for image compression: The results beat the quality of JPEG2000 when anisotropic diffusion is used for inpainting. Finally, the thesis shows that the combination of the concepts allows for further improvements.Die vorliegende Arbeit untersucht die Bildkompression mit partiellen Differentialgleichungen (PDEs), basierend auf Kanten und optimalen Daten. Sie stellt zunächst ein verlustbehaftetes Kompressionsverfahren für cartoonartige Bilder vor. Dazu werden Kanten zusammen mit einigen benachbarten Pixelwerten extrahiert und anschließend kodiert. Während der Dekodierung, werden Informationen, die durch die gespeicherten Daten nicht abgedeckt sind, mittels PDE-basiertem Inpainting mit homogenener Diffusion rekonstruiert. Das Ergebnis ist ein Kompressionscodec, der auf visuell bedeutsamen Bildmerkmalen basiert und in der Lage ist, die Qualität von JPEG und JPEG2000 zu übertreffen. Im Gegensatz dazu konzentriert sich der zweite Teil der Arbeit auf die optimale Auswahl von Inpaintingdaten. Die vorgeschlagenen Methoden ermöglichen es, ein gewöhnliches Bild aus nur 4% aller Pixel nahezu perfekt wiederherzustellen, selbst mit homogenem Diffusionsinpainting. Eine einfache konzeptuelle Kodierung zeigt das Potential einer optimierten Datenauswahl auf: Die Ergebnisse übersteigen die Qualität von JPEG2000, sofern das Inpainting mit einem anisotropen Diffusionsprozess erfolgt. Schließlich zeigt die Arbeit, dass weitere Verbesserungen durch die Kombination der Konzepte erreicht werden können