Γ-Convergence of an Ambrosio-Tortorelli approximation scheme for image segmentation

Given an image u0, the aim of minimising the Mumford-Shah functional is to find a decomposition of the image domain into sub-domains and a piecewise smooth approximation u of u0 such that u varies smoothly within each sub-domain. Since the Mumford-Shah functional is highly non-smooth, regularizations such as the Ambrosio-Tortorelli approximation can be considered which is one of the most computationally efficient approximations of the Mumford-Shah functional for image segmentation. While very impressive numerical results have been achieved in a large range of applications when minimising the functional, no analytical results are currently available for minimizers of the functional in the piecewise smooth setting, and this is the goal of this work. Our main result is the Γ-convergence of the Ambrosio-Tortorelli approximation of the Mumford-Shah functional for piecewise smooth approximations. This requires the introduction of an appropriate function space. As a consequence of our Γ-convergence result, we can infer the convergence of minimizers of the respective functionals

Directional Sinogram Inpainting for Limited Angle Tomography

In this paper we propose a new joint model for the reconstruction of tomography data under limited angle sampling regimes. In many applications of Tomography, e.g. Electron Microscopy and Mammography, physical limitations on acquisition lead to regions of data which cannot be sampled. Depending on the severity of the restriction, reconstructions can contain severe, characteristic, artefacts. Our model aims to address these artefacts by inpainting the missing data simultaneously with the reconstruction. Numerically, this problem naturally evolves to require the minimisation of a non-convex and non-smooth functional so we review recent work in this topic and extend results to fit an alternating (block) descent framework. \oldtext{We illustrate the effectiveness of this approach with numerical experiments on two synthetic datasets and one Electron Microscopy dataset.} \newtext{We perform numerical experiments on two synthetic datasets and one Electron Microscopy dataset. Our results show consistently that the joint inpainting and reconstruction framework can recover cleaner and more accurate structural information than the current state of the art methods

Combined First and Second Order Total Variation Inpainting using Split Bregman

In this article we discuss the implementation of the combined first and second order total variation inpainting that was introduced by Papafitsoros and Schdönlieb. We describe the algorithm we use (split Bregman) in detail, and we give some examples that indicate the difference between pure first and pure second order total variation inpainting

Research data supporting "Deep learning as optimal control problems"

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalise. We provide data and associate code for some of the examples reported in the paper.We acknowledge support from the Leverhulme Trust EarlyCareer Fellowship ECF-2016-611 ’Learning from mistakes: a supervised feedback-loop for imaging applications', the Leverhulme Trust project on Breaking the non-convexity barrier, the Philip Leverhulme Prize, the EPSRC grant No. EP/M00483X/1, the EPSRC Centre No. EP/N014588/1, the European Union Horizon 2020 research and innovation programmes under the MarieSkodowska-Curie grant agreement No. 777826 NoMADS and No. 691070 CHiPS,the Cantab Capital Institute for the Mathematics of Information and the Alan Turing Institute. We gratefully acknowledge the support of NVIDIA Corporation with the donation of a Quadro P6000 and a Titan Xp GPUs used for this research. We thank the SPIRIT project (no. 231632) under the Research Council of Norway FRIPRO funding scheme. We also would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programmes 'Variational methods and effective algorithms for imaging and vision' (2017) and 'Geometry, compatibility and structure preservation in computational differential equations' (2019) where work on this paper was undertaken. This work was supported by EPSRC grant no EP/K032208/1

A multi-contrast MRI approach to thalamus segmentation

Thalamic alterations occur in many neurological disorders including Alzheimer's disease, Parkinson's disease and multiple sclerosis. Routine interventions to improve symptom severity in movement disorders, for example, often consist of surgery or deep brain stimulation to diencephalic nuclei. Therefore, accurate delineation of grey matter thalamic subregions is of the upmost clinical importance. MRI is highly appropriate for structural segmentation as it provides different views of the anatomy from a single scanning session. Though with several contrasts potentially available, it is also of increasing importance to develop new image segmentation techniques that can operate multi-spectrally. We hereby propose a new segmentation method for use with multi-modality data, which we evaluated for automated segmentation of major thalamic subnuclear groups using T-1-weighted, T2*-weighted and quantitative susceptibility mapping (QSM) information. The proposed method consists of four steps: Highly iterative image co-registration, manual segmentation on the average training-data template, supervised learning for pattern recognition, and a final convex optimisation step imposing further spatial constraints to refine the solution. This led to solutions in greater agreement with manual segmentation than the standard Morel atlas based approach. Furthermore, we show that the multi-contrast approach boosts segmentation performances. We then investigated whether prior knowledge using the training-template contours could further improve convex segmentation accuracy and robustness, which led to highly precise multi-contrast segmentations in single subjects. This approach can be extended to most 3D imaging data types and any region of interest discernible in single scans or multi-subject templates

Directional Sinogram Inpainting for Limited Angle Tomography

In this paper we propose a new joint model for the reconstruction of tomography data under limited angle sampling regimes. In many applications of Tomography, e.g. Electron Microscopy and Mammography, physical limitations on acquisition lead to regions of data which cannot be sampled. Depending on the severity of the restriction, reconstructions can contain severe, characteristic, artefacts. Our model aims to address these artefacts by inpainting the missing data simultaneously with the reconstruction. Numerically, this problem naturally evolves to require the minimisation of a non-convex and non-smooth functional so we review recent work in this topic and extend results to fit an alternating (block) descent framework. \oldtext{We illustrate the effectiveness of this approach with numerical experiments on two synthetic datasets and one Electron Microscopy dataset.} \newtext{We perform numerical experiments on two synthetic datasets and one Electron Microscopy dataset. Our results show consistently that the joint inpainting and reconstruction framework can recover cleaner and more accurate structural information than the current state of the art methods

Enhancing joint reconstruction and segmentation with non-convex Bregman iteration

All imaging modalities such as computed tomography, emission tomography and magnetic resonance imaging require a reconstruction approach to produce an image. A common image processing task for applications that utilise those modalities is image segmentation, typically performed posterior to the reconstruction. Recently, the idea of tackling both problems jointly has been proposed. We explore a new approach that combines reconstruction and segmentation in a unified framework. We derive a variational model that consists of a total variation regularised reconstruction from undersampled measurements and a Chan-Vese-based segmentation. We extend the variational regularisation scheme to a Bregman iteration framework to improve the reconstruction and therefore the segmentation. We develop a novel alternating minimisation scheme that solves the non-convex optimisation problem with provable convergence guarantees. Our results for synthetic and real data show that both reconstruction and segmentation are improved compared to the classical sequential approach.</p