Contraintes géométriques et topologiques pour la segmentation d’images médicales : approches hybrides variationnelles et par apprentissage profond

Image segmentation is a central process in computer vision, especially for medical image analysis. When planning a radiotherapy treatment, it is necessary to segment the target tumour as well as adjacent healthy organs (so-called organs at risk). Although convolutional neural networks exhibit accurate segmentations, some artefacts remain (isolated pixels, holes etc.). Thus, incorporating prior knowledge into a segmentation process, whether it be topological prescriptions such as the number of related components, the (partial) convexity of the boundary of an object, or geometrical constraints via, for example, the penalisation of the volume by constraints, is critical. In particular, when one wishes to preserve contextual relationships between objects and obtain a segmentation that is homeomorphic to a known a priori. Inspired by this observation, this thesis aims to provide a hybrid variational/deep learning framework including geometric and topological constraints in the training of convolutional neural networks, in the form of a penalty in the loss function. The objective is to improve the quality of medical image segmentations, for which the contours of the objects to be segmented are not well defined. Thus, a first model includes geometric constraints through a regularisation based on the weighted total variation, a volume/area penalty and a Mumford-Shah term. In a second model, we interpret the segmentation process as a registration task pairing the ground truth and the image to be labelled, based on non-linear elasticity principles. The application of incompressibility conditions on the determinant of the Jacobian matrix of the deformation guarantees preservation of volume and topology, without self-intersection of the material. Theoretical results highlighting the mathematical soundness of the models are provided, as well as an analysis of appropriate numerical algorithms based on a splitting strategy and yielding subproblems that admit, for the most part, closed form solutions. The experiments are mainly conducted on the SegTHOR database which contains thoracic CT scans of patients treated by radiotherapy and aim to segment 4 organs at risk to be preserved from radiation. They demonstrate that our methods provide significant improvements over existing unconstrained approaches, both in terms of quantitative criteria such as the measurement of region overlap and qualitative assessment, especially when the classes are unbalanced.La segmentation d’images constitue un traitement central de la vision par ordinateur, et particulièrement pour l’analyse d’images médicales. Lors de la planification d’un traitement par radiothérapie, il est nécessaire de segmenter la tumeur cible ainsi que les organes sains adjacents (appelés organes à risque). Si les réseaux de neurones convolutifs exhibent des segmentations précises, certains artefacts subsistent néanmoins (pixels isolés, trous etc.). Ainsi, l’inclusion d’informations a priori dans une tâche de segmentation, qu’il s’agisse de contraintes topologiques telles que le nombre de composantes connexes, la convexité partielle de la frontière d’un objet, ou de prescriptions géométriques via par exemple la pénalisation du volume par des contraintes, s’avère critique. Notamment, lorsqu’on souhaite préserver les relations contextuelles entre les objets et obtenir une segmentation homéomorphe à un a priori connu. Motivé par cette observation, ce travail de thèse vise à fournir un cadre hybride variationnel/apprentissage profond incluant des contraintes géométriques et topologiques dans l’apprentissage des réseaux de neurones convolutifs, sous la forme d’une pénalisation dans la fonction de perte. L’objectif réside dans l’amélioration de la qualité des segmentations d’images médicales, pour lesquelles les contours des objets à segmenter ne sont pas bien définis. Ainsi, un premier modèle inclut des contraintes géométriques par le biais d’une régularisation bâtie sur la variation totale pondérée, d’une pénalisation du volume/de l’aire, et d’un terme d’attache aux données de Mumford-Shah. Dans un second modèle, nous interprétons le processus de segmentation comme une tâche de recalage appariant la vérité terrain et l’image à étiqueter, fondée sur des principes d’élasticité non linéaire. L’application de conditions d’incompressibilité sur le déterminant de la matrice jacobienne de la déformation garantit la préservation du volume et de la topologie, sans auto-intersection de la matière. Des résultats théoriques soulignant la solidité mathématique des modèles sont fournis, ainsi qu’une analyse des algorithmes numériques appropriés basés sur une stratégie de séparation de variables et donnant des sous-problèmes qui admettent, pour la plupart, des solutions closed form. Les expériences sont principalement menées sur la base de données SegTHOR qui contient des scanners thoraciques de patients soignés par radiothérapie et dont on cherche à segmenter 4 organes à risque à préserver des rayons. Elles démontrent que nos méthodes apportent des améliorations significatives par rapport aux approches non contraintes existantes, à la fois en termes de critères quantitatifs tels que la mesure du chevauchement des régions et d’évaluation qualitative, en particulier lorsque les classes sont déséquilibrées

Geometric and topological constraints for medical image segmentation : hybrid variational-based and deep learning-based approaches

La segmentation d’images constitue un traitement central de la vision par ordinateur, et particulièrement pour l’analyse d’images médicales. Lors de la planification d’un traitement par radiothérapie, il est nécessaire de segmenter la tumeur cible ainsi que les organes sains adjacents (appelés organes à risque). Si les réseaux de neurones convolutifs exhibent des segmentations précises, certains artefacts subsistent néanmoins (pixels isolés, trous etc.). Ainsi, l’inclusion d’informations a priori dans une tâche de segmentation, qu’il s’agisse de contraintes topologiques telles que le nombre de composantes connexes, la convexité partielle de la frontière d’un objet, ou de prescriptions géométriques via par exemple la pénalisation du volume par des contraintes, s’avère critique. Notamment, lorsqu’on souhaite préserver les relations contextuelles entre les objets et obtenir une segmentation homéomorphe à un a priori connu. Motivé par cette observation, ce travail de thèse vise à fournir un cadre hybride variationnel/apprentissage profond incluant des contraintes géométriques et topologiques dans l’apprentissage des réseaux de neurones convolutifs, sous la forme d’une pénalisation dans la fonction de perte. L’objectif réside dans l’amélioration de la qualité des segmentations d’images médicales, pour lesquelles les contours des objets à segmenter ne sont pas bien définis. Ainsi, un premier modèle inclut des contraintes géométriques par le biais d’une régularisation bâtie sur la variation totale pondérée, d’une pénalisation du volume/de l’aire, et d’un terme d’attache aux données de Mumford-Shah. Dans un second modèle, nous interprétons le processus de segmentation comme une tâche de recalage appariant la vérité terrain et l’image à étiqueter, fondée sur des principes d’élasticité non linéaire. L’application de conditions d’incompressibilité sur le déterminant de la matrice jacobienne de la déformation garantit la préservation du volume et de la topologie, sans auto-intersection de la matière. Des résultats théoriques soulignant la solidité mathématique des modèles sont fournis, ainsi qu’une analyse des algorithmes numériques appropriés basés sur une stratégie de séparation de variables et donnant des sous-problèmes qui admettent, pour la plupart, des solutions closed form. Les expériences sont principalement menées sur la base de données SegTHOR qui contient des scanners thoraciques de patients soignés par radiothérapie et dont on cherche à segmenter 4 organes à risque à préserver des rayons. Elles démontrent que nos méthodes apportent des améliorations significatives par rapport aux approches non contraintes existantes, à la fois en termes de critères quantitatifs tels que la mesure du chevauchement des régions et d’évaluation qualitative, en particulier lorsque les classes sont déséquilibrées.Image segmentation is a central process in computer vision, especially for medical image analysis. When planning a radiotherapy treatment, it is necessary to segment the target tumour as well as adjacent healthy organs (so-called organs at risk). Although convolutional neural networks exhibit accurate segmentations, some artefacts remain (isolated pixels, holes etc.). Thus, incorporating prior knowledge into a segmentation process, whether it be topological prescriptions such as the number of related components, the (partial) convexity of the boundary of an object, or geometrical constraints via, for example, the penalisation of the volume by constraints, is critical. In particular, when one wishes to preserve contextual relationships between objects and obtain a segmentation that is homeomorphic to a known a priori. Inspired by this observation, this thesis aims to provide a hybrid variational/deep learning framework including geometric and topological constraints in the training of convolutional neural networks, in the form of a penalty in the loss function. The objective is to improve the quality of medical image segmentations, for which the contours of the objects to be segmented are not well defined. Thus, a first model includes geometric constraints through a regularisation based on the weighted total variation, a volume/area penalty and a Mumford-Shah term. In a second model, we interpret the segmentation process as a registration task pairing the ground truth and the image to be labelled, based on non-linear elasticity principles. The application of incompressibility conditions on the determinant of the Jacobian matrix of the deformation guarantees preservation of volume and topology, without self-intersection of the material. Theoretical results highlighting the mathematical soundness of the models are provided, as well as an analysis of appropriate numerical algorithms based on a splitting strategy and yielding subproblems that admit, for the most part, closed form solutions. The experiments are mainly conducted on the SegTHOR database which contains thoracic CT scans of patients treated by radiotherapy and aim to segment 4 organs at risk to be preserved from radiation. They demonstrate that our methods provide significant improvements over existing unconstrained approaches, both in terms of quantitative criteria such as the measurement of region overlap and qualitative assessment, especially when the classes are unbalanced

Contraintes géométriques et topologiques pour la segmentation d’images médicales : approches hybrides variationnelles et par apprentissage profond

Image segmentation is a central process in computer vision, especially for medical image analysis. When planning a radiotherapy treatment, it is necessary to segment the target tumour as well as adjacent healthy organs (so-called organs at risk). Although convolutional neural networks exhibit accurate segmentations, some artefacts remain (isolated pixels, holes etc.). Thus, incorporating prior knowledge into a segmentation process, whether it be topological prescriptions such as the number of related components, the (partial) convexity of the boundary of an object, or geometrical constraints via, for example, the penalisation of the volume by constraints, is critical. In particular, when one wishes to preserve contextual relationships between objects and obtain a segmentation that is homeomorphic to a known a priori. Inspired by this observation, this thesis aims to provide a hybrid variational/deep learning framework including geometric and topological constraints in the training of convolutional neural networks, in the form of a penalty in the loss function. The objective is to improve the quality of medical image segmentations, for which the contours of the objects to be segmented are not well defined. Thus, a first model includes geometric constraints through a regularisation based on the weighted total variation, a volume/area penalty and a Mumford-Shah term. In a second model, we interpret the segmentation process as a registration task pairing the ground truth and the image to be labelled, based on non-linear elasticity principles. The application of incompressibility conditions on the determinant of the Jacobian matrix of the deformation guarantees preservation of volume and topology, without self-intersection of the material. Theoretical results highlighting the mathematical soundness of the models are provided, as well as an analysis of appropriate numerical algorithms based on a splitting strategy and yielding subproblems that admit, for the most part, closed form solutions. The experiments are mainly conducted on the SegTHOR database which contains thoracic CT scans of patients treated by radiotherapy and aim to segment 4 organs at risk to be preserved from radiation. They demonstrate that our methods provide significant improvements over existing unconstrained approaches, both in terms of quantitative criteria such as the measurement of region overlap and qualitative assessment, especially when the classes are unbalanced.La segmentation d’images constitue un traitement central de la vision par ordinateur, et particulièrement pour l’analyse d’images médicales. Lors de la planification d’un traitement par radiothérapie, il est nécessaire de segmenter la tumeur cible ainsi que les organes sains adjacents (appelés organes à risque). Si les réseaux de neurones convolutifs exhibent des segmentations précises, certains artefacts subsistent néanmoins (pixels isolés, trous etc.). Ainsi, l’inclusion d’informations a priori dans une tâche de segmentation, qu’il s’agisse de contraintes topologiques telles que le nombre de composantes connexes, la convexité partielle de la frontière d’un objet, ou de prescriptions géométriques via par exemple la pénalisation du volume par des contraintes, s’avère critique. Notamment, lorsqu’on souhaite préserver les relations contextuelles entre les objets et obtenir une segmentation homéomorphe à un a priori connu. Motivé par cette observation, ce travail de thèse vise à fournir un cadre hybride variationnel/apprentissage profond incluant des contraintes géométriques et topologiques dans l’apprentissage des réseaux de neurones convolutifs, sous la forme d’une pénalisation dans la fonction de perte. L’objectif réside dans l’amélioration de la qualité des segmentations d’images médicales, pour lesquelles les contours des objets à segmenter ne sont pas bien définis. Ainsi, un premier modèle inclut des contraintes géométriques par le biais d’une régularisation bâtie sur la variation totale pondérée, d’une pénalisation du volume/de l’aire, et d’un terme d’attache aux données de Mumford-Shah. Dans un second modèle, nous interprétons le processus de segmentation comme une tâche de recalage appariant la vérité terrain et l’image à étiqueter, fondée sur des principes d’élasticité non linéaire. L’application de conditions d’incompressibilité sur le déterminant de la matrice jacobienne de la déformation garantit la préservation du volume et de la topologie, sans auto-intersection de la matière. Des résultats théoriques soulignant la solidité mathématique des modèles sont fournis, ainsi qu’une analyse des algorithmes numériques appropriés basés sur une stratégie de séparation de variables et donnant des sous-problèmes qui admettent, pour la plupart, des solutions closed form. Les expériences sont principalement menées sur la base de données SegTHOR qui contient des scanners thoraciques de patients soignés par radiothérapie et dont on cherche à segmenter 4 organes à risque à préserver des rayons. Elles démontrent que nos méthodes apportent des améliorations significatives par rapport aux approches non contraintes existantes, à la fois en termes de critères quantitatifs tels que la mesure du chevauchement des régions et d’évaluation qualitative, en particulier lorsque les classes sont déséquilibrées

Data approximation : mathematical modelling and numerical simulations

International audienc

A geometrically-constrained deep network for CT image segmentation

International audienceIncorporating prior knowledge into a segmentation process, whether it be geometrical constraints such as volume penalisation, (partial) convexity enforcement, or topological prescriptions to preserve the contextual relations between objects, proves to improve accuracy in medical image segmentation, in particular when addressing the issue of weak boundary definition. Motivated by this observation, the proposed contribution aims to include geometrical constraints in the training of convolutional neural networks in the form of a penalty in the loss function. These geometrical constraints take several forms and encompass level curve alignment through the weighted total variation component, an area penalisation phrased as a hard constraint in the modelling, and an intensity homogeneity criterion based on a combination of the standard Dice loss with the piecewise constant Mumford-Shah model. The mathematical formulation yields a non-smooth non-convex optimisation problem, which rules out conventional smooth optimisation techniques and leads us to adopt a Lagrangian setting. The application falls within the scope of organ-at-risk segmentation in CT (Computed Tomography) images, in the context of radiotherapy planning. Experiments demonstrate that our method provides significant improvements over existing non-constrained approaches

On the Inclusion of Topological Requirements in CNNs for Semantic Segmentation Applied to Radiotherapy

International audienceThe incorporation of prior knowledge into a medical segmentation task allows to compensate for the issue of weak boundary definition and to be more in line with anatomical reality even though the data do not explicitly show these characteristics. This motivation underlies the proposed contribution which aims to provide a unified variational framework involving topological requirements in the training of convolutional neural networks through the design of a suitable penalty in the loss function. More precisely, these topological constraints are implicitly enforced by viewing the segmentation assignment as a registration task between the considered image and its associated ground truth under incompressibility condition, making them homeomorphic. The application falls within the scope of organ-at-risk segmentation in CT (Computed Tomography) images, in the context of radiotherapy planning

Enforcing Geometrical Priors in Deep Networks for Semantic Segmentation Applied to Radiotherapy Planning

International audienceIncorporating prior knowledge into a segmentation process, whether it is geometrical constraints such as volume penalisation,(partial) convexity enforcement, or topological prescriptions to preserve the contextual relations between objects, provesto improve accuracy in medical image segmentation, in particular when addressing the issue of weak boundary definition.Motivated by this observation, the proposed contribution aims to provide a unified variational framework including geometricalconstraints in the training of convolutional neural networks in the form of a penalty in the loss function. These geometricalconstraints take several forms and encompass level curve alignment through the integration of the weighted total variation, anarea penalisation phrased as a hard constraint in the modelling, and an intensity homogeneity criterion based on a combinationof the standard Dice loss with the piecewise constant Mumford–Shah model. The mathematical formulation yields a non-smooth non-convex optimisation problem, which rules out conventional smooth optimisation techniques and leads us to adopta Lagrangian setting. The application falls within the scope of organ-at-risk segmentation in CT (computed tomography)images, in the context of radiotherapy planning. Experiments demonstrate that our method provides significant improvements(i) over existing non-constrained approaches both in terms of quantitative criteria, such as the measure of overlap, andqualitative assessment (spatial regularisation/coherency, fewer outliers), (ii) over in-layer constrained deep convolutionalnetworks, and shows a certain degree of versatility

Educated consumers don’t believe artificial meat is the solution to the problems with the meat industry

The production ofin vitromeat by cell culture has been suggested by some scientists as one solution to address the majorchallenges facing our society. Firstly, consumers would like the meat industry to reduce potential discomfort of animals onmodern farms, or even to avoid killing animals to eat them. Secondly, citizens would like meat producers to reduce potentialenvironmental deterioration by livestock and finally, there is a need to reduce world hunger by increasing protein resourceswhile the global population is predicted to grow rapidly. According to its promoters, artificial meat has a potential to makeeating animals unnecessary, to reduce carbon footprint of meat production and to satisfy all the nutritional needs and desiresof consumers and citizens. To check these assumptions, a total of 817 educated people (mainly scientists and students)were interviewed worldwide by internet in addition to 865 French educated people. We also interviewed 208 persons(mainly scientists) after an oral presentation regarding artificial meat. Results of the three surveys were similar, but differedbetween males and females. More than half of the respondents believed that “artificial meat” was feasible and realistic.However, there was no majority to think that artificial meat will be healthy and tasty, except respondents who were in favourof artificial meat. A large majority of the respondents believed that the meat industry is facing important problems relatedto the protection of the environment, animal welfare or inefficient meat production to feed humanity. However, respondentsdid not believe that artificial meat will be the solution to solve the mentioned problems with the meat industry, especiallyrespondents who were against artificial meat. The vast majority of consumers wished to continue to eat meat even theywould accept to consume less meat in a context of increasing food needs. Only a minority of respondents (from 5 to 11%)would recommend or accept to eatin vitromeat instead of meat produced from farm animals. Despite these limitations,38 to 47% of the respondents would continue to support research on artificial meat, but a majority of them believed thatartificial meat will not be accepted by consumers in the future, except for respondents who were in favour of artificial meat.We speculated that the apparent contradictory answers to this survey expressed the fact that people trust scientists who are supposed to continuously discover new technologies potentially useful in a long term future for the human beings, butpeople also expressed concern for their health and were not convinced that artificial meat will be tasty, safe and healthyenough to be accepted by consumer