Three-dimensional Segmentation of Trees Through a Flexible Multi-Class Graph Cut Algorithm (MCGC)

Developing a robust algorithm for automatic individual tree crown (ITC) detection from airborne laser scanning datasets is important for tracking the responses of trees to anthropogenic change. Such approaches allow the size, growth and mortality of individual trees to be measured, enabling forest carbon stocks and dynamics to be tracked and understood. Many algorithms exist for structurally simple forests including coniferous forests and plantations. Finding a robust solution for structurally complex, species-rich tropical forests remains a challenge; existing segmentation algorithms often perform less well than simple area-based approaches when estimating plot-level biomass. Here we describe a Multi-Class Graph Cut (MCGC) approach to tree crown delineation. This uses local three-dimensional geometry and density information, alongside knowledge of crown allometries, to segment individual tree crowns from airborne LiDAR point clouds. Our approach robustly identifies trees in the top and intermediate layers of the canopy, but cannot recognise small trees. From these three-dimensional crowns, we are able to measure individual tree biomass. Comparing these estimates to those from permanent inventory plots, our algorithm is able to produce robust estimates of hectare-scale carbon density, demonstrating the power of ITC approaches in monitoring forests. The flexibility of our method to add additional dimensions of information, such as spectral reflectance, make this approach an obvious avenue for future development and extension to other sources of three-dimensional data, such as structure from motion datasets.Jonathan Williams holds a NERC studentship [NE/N008952/1] which is a CASE partnership with support from Royal Society for the Protection of Birds (RSPB). David Coomes was supported by an International Academic Fellowship from the Leverhulme Trust. Carola-Bibiane Schoenlieb was supported by the RISE projects CHiPS and NoMADS, the Cantab Capital Institute for the Mathematics of Information and the Alan Turing Institute. We gratefully acknowledge the support of NVIDIA Corporation with the donation of a Quadro P6000 GPU used for this research

Γ-Convergence of an Ambrosio-Tortorelli approximation scheme for image segmentation

Given an image u0, the aim of minimising the Mumford-Shah functional is to find a decomposition of the image domain into sub-domains and a piecewise smooth approximation u of u0 such that u varies smoothly within each sub-domain. Since the Mumford-Shah functional is highly non-smooth, regularizations such as the Ambrosio-Tortorelli approximation can be considered which is one of the most computationally efficient approximations of the Mumford-Shah functional for image segmentation. While very impressive numerical results have been achieved in a large range of applications when minimising the functional, no analytical results are currently available for minimizers of the functional in the piecewise smooth setting, and this is the goal of this work. Our main result is the Γ-convergence of the Ambrosio-Tortorelli approximation of the Mumford-Shah functional for piecewise smooth approximations. This requires the introduction of an appropriate function space. As a consequence of our Γ-convergence result, we can infer the convergence of minimizers of the respective functionals

Directional Sinogram Inpainting for Limited Angle Tomography

In this paper we propose a new joint model for the reconstruction of tomography data under limited angle sampling regimes. In many applications of Tomography, e.g. Electron Microscopy and Mammography, physical limitations on acquisition lead to regions of data which cannot be sampled. Depending on the severity of the restriction, reconstructions can contain severe, characteristic, artefacts. Our model aims to address these artefacts by inpainting the missing data simultaneously with the reconstruction. Numerically, this problem naturally evolves to require the minimisation of a non-convex and non-smooth functional so we review recent work in this topic and extend results to fit an alternating (block) descent framework. \oldtext{We illustrate the effectiveness of this approach with numerical experiments on two synthetic datasets and one Electron Microscopy dataset.} \newtext{We perform numerical experiments on two synthetic datasets and one Electron Microscopy dataset. Our results show consistently that the joint inpainting and reconstruction framework can recover cleaner and more accurate structural information than the current state of the art methods

Combined First and Second Order Total Variation Inpainting using Split Bregman

In this article we discuss the implementation of the combined first and second order total variation inpainting that was introduced by Papafitsoros and Schdönlieb. We describe the algorithm we use (split Bregman) in detail, and we give some examples that indicate the difference between pure first and pure second order total variation inpainting

Research data supporting the publication "Choose your path wisely: gradient descent in a Bregman distance framework"

This is the research data that accompanies the publication "Choose your path wisely: gradient descent in a Bregman distance framework". It contains code for all numerical examples given in the numerical results section of the associated publication. Note that the parallel MRI example and the classification example require additional external data. The corresponding URLs can be found in the associated publicationThis work was funded by the Leverhulme Trust Early Career Fellowship ’Learning from mistakes: a supervised feedback-loop for imaging applications’, the Isaac Newton Trust, the Engineering and Physical Sciences Research Council (EPSRC) ’EP/K009745/1’, the Leverhulme Trust project ’Breaking the non-convexity barrier’, the EPSRC grant ’EP/M00483X/1’, the EPSRC centre ’EP/N014588/1’, the Cantab Capital Institute for the Mathematics of Information and CHiPS (Horizon 2020 RISE project grant)

Research data supporting "Deep learning as optimal control problems"

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalise. We provide data and associate code for some of the examples reported in the paper.We acknowledge support from the Leverhulme Trust EarlyCareer Fellowship ECF-2016-611 ’Learning from mistakes: a supervised feedback-loop for imaging applications', the Leverhulme Trust project on Breaking the non-convexity barrier, the Philip Leverhulme Prize, the EPSRC grant No. EP/M00483X/1, the EPSRC Centre No. EP/N014588/1, the European Union Horizon 2020 research and innovation programmes under the MarieSkodowska-Curie grant agreement No. 777826 NoMADS and No. 691070 CHiPS,the Cantab Capital Institute for the Mathematics of Information and the Alan Turing Institute. We gratefully acknowledge the support of NVIDIA Corporation with the donation of a Quadro P6000 and a Titan Xp GPUs used for this research. We thank the SPIRIT project (no. 231632) under the Research Council of Norway FRIPRO funding scheme. We also would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the programmes 'Variational methods and effective algorithms for imaging and vision' (2017) and 'Geometry, compatibility and structure preservation in computational differential equations' (2019) where work on this paper was undertaken. This work was supported by EPSRC grant no EP/K032208/1

A multi-contrast MRI approach to thalamus segmentation

Thalamic alterations occur in many neurological disorders including Alzheimer's disease, Parkinson's disease and multiple sclerosis. Routine interventions to improve symptom severity in movement disorders, for example, often consist of surgery or deep brain stimulation to diencephalic nuclei. Therefore, accurate delineation of grey matter thalamic subregions is of the upmost clinical importance. MRI is highly appropriate for structural segmentation as it provides different views of the anatomy from a single scanning session. Though with several contrasts potentially available, it is also of increasing importance to develop new image segmentation techniques that can operate multi-spectrally. We hereby propose a new segmentation method for use with multi-modality data, which we evaluated for automated segmentation of major thalamic subnuclear groups using T-1-weighted, T2*-weighted and quantitative susceptibility mapping (QSM) information. The proposed method consists of four steps: Highly iterative image co-registration, manual segmentation on the average training-data template, supervised learning for pattern recognition, and a final convex optimisation step imposing further spatial constraints to refine the solution. This led to solutions in greater agreement with manual segmentation than the standard Morel atlas based approach. Furthermore, we show that the multi-contrast approach boosts segmentation performances. We then investigated whether prior knowledge using the training-template contours could further improve convex segmentation accuracy and robustness, which led to highly precise multi-contrast segmentations in single subjects. This approach can be extended to most 3D imaging data types and any region of interest discernible in single scans or multi-subject templates