Joint Image Reconstruction and Segmentation Using the Potts Model

We propose a new algorithmic approach to the non-smooth and non-convex Potts problem (also called piecewise-constant Mumford-Shah problem) for inverse imaging problems. We derive a suitable splitting into specific subproblems that can all be solved efficiently. Our method does not require a priori knowledge on the gray levels nor on the number of segments of the reconstruction. Further, it avoids anisotropic artifacts such as geometric staircasing. We demonstrate the suitability of our method for joint image reconstruction and segmentation. We focus on Radon data, where we in particular consider limited data situations. For instance, our method is able to recover all segments of the Shepp-Logan phantom from $7$ angular views only. We illustrate the practical applicability on a real PET dataset. As further applications, we consider spherical Radon data as well as blurred data

Structure-aware image denoising, super-resolution, and enhancement methods

Denoising, super-resolution and structure enhancement are classical image processing applications. The motive behind their existence is to aid our visual analysis of raw digital images. Despite tremendous progress in these fields, certain difficult problems are still open to research. For example, denoising and super-resolution techniques which possess all the following properties, are very scarce: They must preserve critical structures like corners, should be robust to the type of noise distribution, avoid undesirable artefacts, and also be fast. The area of structure enhancement also has an unresolved issue: Very little efforts have been put into designing models that can tackle anisotropic deformations in the image acquisition process. In this thesis, we design novel methods in the form of partial differential equations, patch-based approaches and variational models to overcome the aforementioned obstacles. In most cases, our methods outperform the existing approaches in both quality and speed, despite being applicable to a broader range of practical situations.Entrauschen, Superresolution und Strukturverbesserung sind klassische Anwendungen der Bildverarbeitung. Ihre Existenz bedingt sich in dem Bestreben, die visuelle Begutachtung digitaler Bildrohdaten zu unterstützen. Trotz erheblicher Fortschritte in diesen Feldern bedürfen bestimmte schwierige Probleme noch weiterer Forschung. So sind beispielsweise Entrauschungsund Superresolutionsverfahren, welche alle der folgenden Eingenschaften besitzen, sehr selten: die Erhaltung wichtiger Strukturen wie Ecken, Robustheit bezüglich der Rauschverteilung, Vermeidung unerwünschter Artefakte und niedrige Laufzeit. Auch im Gebiet der Strukturverbesserung liegt ein ungelöstes Problem vor: Bisher wurde nur sehr wenig Forschungsaufwand in die Entwicklung von Modellen investieret, welche anisotrope Deformationen in bildgebenden Verfahren bewältigen können. In dieser Arbeit entwerfen wir neue Methoden in Form von partiellen Differentialgleichungen, patch-basierten Ansätzen und Variationsmodellen um die oben erwähnten Hindernisse zu überwinden. In den meisten Fällen übertreffen unsere Methoden nicht nur qualitativ die bisher verwendeten Ansätze, sondern lösen die gestellten Aufgaben auch schneller. Zudem decken wir mit unseren Modellen einen breiteren Bereich praktischer Fragestellungen ab

Data-driven reconstruction methods for photoacoustic tomography:Learning structures by structured learning

Photoacoustic tomography (PAT) is an imaging technique with potential applications in various fields of biomedicine. By visualising vascular structures, PAT could help in the detection and diagnosis of diseases related to their dysregulation. In PAT, tissue is illuminated by light. After entering the tissue, the light undergoes scattering and absorption. The absorbed energy is transformed into an initial pressure by the photoacoustic effect, which travels to ultrasound detectors outside the tissue.This thesis is concerned with the inverse problem of the described physical process: what was the initial pressure in the tissue that gave rise to the detected pressure outside? The answer to this question is difficult to obtain when light penetration in tissue is not sufficient, the measurements are corrupted, or only a small number of detectors can be used in a limited geometry. For decades, the field of variational methods has come up with new approaches to solve these kind of problems. these kind of problems: the combination of new theory and clever algorithms has led to improved numerical results in many image reconstruction problems. In the past five years, previously state-of-the-art results were greatly surpassed by combining variational methods with artificial neural networks, a form of artificial intelligence.In this thesis we investigate several ways of combining data-driven artificial neural networks with model-driven variational methods. We combine the topics of photoacoustic tomography, inverse problems and artificial neural networks.Chapter 3 treats the variational problem in PAT and provides a framework in which hand-crafted regularisers can easily be compared. Both directional and higher-order total variation methods show improved results over direct methods for PAT with structures resembling vasculature.Chapter 4 provides a method to jointly solve the PAT reconstruction and segmentation problem for absorbing structures resembling vasculature. Artificial neural networks are embodied in the algorithmic structure of primal-dual methods, which are a popular way to solve variational problems. It is shown that a diverse training set is of utmost importance to solve multiple problems with one learned algorithm.Chapter 5 provides a convergence analysis for data-consistent networks, which combine classical regularisation methods with artificial neural networks. Numerical results are shown for an inverse problem that couples the Radon transform with a saturation problem for biomedical images.Chapter 6 explores the idea of fully-learned reconstruction by connecting two nonlinear autoencoders. By enforcing a dimensionality reduction in the artificial neural network, a joint manifold for measurements and images is learned. The method, coined learned SVD, provides advantages over other fully-learned methods in terms of interpretability and generalisation. Numerical results show high-quality reconstructions, even in the case where no information on the forward process is used.In this thesis, several ways of combining model-based methods with data-driven artificial neural networks were investigated. The resulting hybrid methods showed improved tomography reconstructions. By allowing data to improve a structured method, deeper vascular structures could be imaged with photoacoustic tomography.<br/

3-D data handling and registration of multiple modality medical images

The many different clinical imaging modalities used in diagnosis and therapy deliver two different types of information: morphological and functional. Clinical interpretation can be assisted and enhanced by combining such information (e.g. superimposition or fusion). The handling of such data needs to be performed in 3-D. Various methods for registration developed by other authors are reviewed and compared. Many of these are based on registering external reference markers, and are cumbersome and present significant problems to both patients and operators. Internal markers have also been used, but these may be very difficult to identify. Alternatively, methods based on the external surface of an object have been developed which eliminate some of the problems associated with the other methods. Thus the methods which have been extended, developed, and described here, are based primarily on the fitting of surfaces, as determined from images obtained from the different modalities to be registered. Annex problems to that of the surface fitting are those of surface detection and display. Some segmentation and surface reconstruction algorithms have been developed to identify the surface to be registered. Surface and volume rendering algorithms have also been implemented to facilitate the display of clinical results. An iterative surface fitting algorithm has been developed based on the minimization of a least squares distance (LSD) function, using the Powell method and alternative minimization algorithms. These algorithms and the qualities of fit so obtained were intercompared. Some modifications were developed to enhance the speed of convergence, to improve the accuracy, and to enhance the display of results during the process of fitting. A common problem with all such methods was found to be the choice of the starting point (the initial transformation parameters) and the avoidance of local minima which often require manual operator intervention. The algorithm was modified to apply a global minimization by using a cumulative distance error in a sequentially terminated process in order to speed up the time of evaluating of each search location. An extension of the algorithm into multi-resolution (scale) space was also implemented. An initial global search is performed at coarse resolution for the 3-D surfaces of both modalities where an appropriate threshold is defined to reject likely mismatch transformations by testing of only a limited subset of surface points. This process is used to define the set of points in the transformation space to be used for the next level of resolution, again with appropriately chosen threshold levels, and continued down to the finest resolution level. All these processes were evaluated using sets of well defined image models. The assessment of this algorithm for 3-D surface registration of data from (3-D) MRI with MRI, MRI with PET, MRI with SPECT, and MRI with CT data is presented, and clinical examples are illustrated and assessed. In the current work, the data from multi-modality imaging of two different types phantom (e.g. Hoffman brain phantom, Jaszczak phantom), thirty routinely imaged patients and volunteer subjects, and ten patients with setting external markers on their head were used to assess and verify 3-D registration. The accuracy of the sequential multi-resolution method obtained by the distance values of 4-10 selected reference points on each data set gave an accuracy of 1.44±0.42 mm for MR-MR, 1.82±0.65 for MR-CT, 2.38±0.88 for MR-PET, and 3.17±1.12 for MR-SPECT registration. The cost of this process was determined to be of the order of 200 seconds (on a Micro-VAX II), although this is highly dependent on some adjustable parameters of the process (e.g. threshold and the size of the geometrical transformation space) by which the accuracy is aimed