Advanced Computational Methods for Oncological Image Analysis.

The Special Issue "Advanced Computational Methods for Oncological Image Analysis", published for the Journal of Imaging, covered original research papers about state-of-the-art and novel algorithms and methodologies, as well as applications of computational methods for oncological image analysis, ranging from radiogenomics to deep learning [...]

Advanced Computational Methods for Oncological Image Analysis

[Cancer is the second most common cause of death worldwide and encompasses highly variable clinical and biological scenarios. Some of the current clinical challenges are (i) early diagnosis of the disease and (ii) precision medicine, which allows for treatments targeted to specific clinical cases. The ultimate goal is to optimize the clinical workflow by combining accurate diagnosis with the most suitable therapies. Toward this, large-scale machine learning research can define associations among clinical, imaging, and multi-omics studies, making it possible to provide reliable diagnostic and prognostic biomarkers for precision oncology. Such reliable computer-assisted methods (i.e., artificial intelligence) together with clinicians’ unique knowledge can be used to properly handle typical issues in evaluation/quantification procedures (i.e., operator dependence and time-consuming tasks). These technical advances can significantly improve result repeatability in disease diagnosis and guide toward appropriate cancer care. Indeed, the need to apply machine learning and computational intelligence techniques has steadily increased to effectively perform image processing operations—such as segmentation, co-registration, classification, and dimensionality reduction—and multi-omics data integration.

Advanced machine learning methods for oncological image analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally- invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head- neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra- dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis

Advanced computational methods for oncological image analysis

: The Special Issue "Advanced Computational Methods for Oncological Image Analysis", published for the Journal of Imaging, covered original research papers about state-of-the-art and novel algorithms and methodologies, as well as applications of computational methods for oncological image analysis, ranging from radiogenomics to deep learning [...]

Tumor Growth Modeling in Oncological Image Analysis

International audienceno abstrac

Advanced Machine Learning Methods for Oncological Image Analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally-invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head-neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra-dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis.Cancer är en global hälsoutmaning som uppskattas ansvara för cirka 10 miljoner dödsfall i hela världen, bara under året 2020. Framsteg inom medicinsk bildtagning och hårdvaruutveckling de senaste tre decennierna har banat vägen för moderna medicinska bildgivande system vars upplösningsförmåga tillåter att fånga information om tumörers anatomi, fysiologi, funktion samt metabolism. Medicinsk bildanalys har därför fått en mer betydelserik roll i klinikers dagliga rutiner inom onkologin, för bland annat screening, diagnostik, uppföljning av behandling samt icke-invasiv utvärdering av sjukdomsprognoser. Sjukvårdens behov av medicinska bilder har lett till att det nu på sjukhusen finns en enorm mängd medicinska bilder på alla moderna sjukhus. Med hänsyn till den viktiga roll medicinsk bilddata spelar i dagens sjukvård, samt den mängd manuellt arbete som behöver göras för att analysera den mängd data som genereras varje dag, så har utvecklingen av digitala verktyg för att för att automatiskt eller semi-automatiskt analysera  bilddatan alltid haft stort intresse. Därför har en rad maskininlärningsverktyg utvecklats för analys av onkologisk data, för att gripa sig an läkares repetitiva vardagssysslor. Den här avhandlingen syftar att bidra till fältet “onkologisk bildanalys” genom att föreslå nya sätt att kvantifiera tumörers egenskaper från medicinsk bilddata. Specifikt, är denna avhandling baserad på sex artiklar där de första två har fokus att presentera nya metoder för segmentering av tumörer, och de resterande fyra ämnar att utveckla kvantitativa biomarkörer för cancerdiagnostik och prognos. Huvudsyftet för “Studie I” har varit att utveckla en djupinlärnings-pipeline vars syfte är att fånga lungpatalogiers anatomier (inklusive lungtumörer) samt integrera detta med djupa neurala nätverk för segmentering för att nyttja det första nätverkets utfall för att förbättra segmenteringskvalitén. Den föreslagna pipelinen testades på flertalet dataset och numeriska analyser visar en överlägsna resultat för den föreslagna “prior-medvetna” djupinlärningsmetoden. “Studie II” ämnar att ta sig an ett viktig problem som övervakade segmenteringsmetoder ställs inför: ett beroende av enorma annoterade dataset. I denna studie föreslås en icke-övervakad segmenteringsmetod som baseras på konceptet “ifyllnad” (“inpainting”) för att segmentera tumörer i områdena: lungor samt huvud och hals i bilder från olika modaliteter. Den föreslagna metoden lyckas bättre än en familj väletablerade icke-oövervakade segmenteringsmodeller. “Studie III” och “Studie IV” försöker automatiskt diskriminera benigna lungtumörer från maligna tumörer genom att analysera bilder från LDCT (lågdos-CT). I “Studie III“ föreslås ett djupt neuralt nätverk för klassificering vars grafstruktur tillåter lokal analys av tumörens inbördes heterogeniteter samt en helhetsbild från global kontextuell information. “Studie IV” försöker utvärdera noggrant utvalda metoder som grundar sig på att extrahera anatomiska särdrag från medicinska bilder. I studien jämförs konventionella “radiomics”-metoder med särdrag från neurala nätverk samt en kombination av båda på samma dataset. Resultat från studien visar att en kombination av särdrag från djupa neurala nätverk samt “radiomics” kan ge bättre resultat i klassificeringsproblemet. “Studie V” har fokus på tidig bedömning av lungtumörers respons på behandling genom att utveckla ett set nya fysiologisk observerbara särdrag. Den presenterade metoden har använts för att kvantifiera förändringar i tumörers karaktär i PET-CT-undersökningar för att predicera patienters prognos två år efter senaste behandling. Metoden jämförts mot konventionella “radiomics” och utvärderingen visar att den föreslagna metoden ger förbättrade resultat. Till skilnad från “Studie V”, som fokuserar på att lösa ett binärt klassificeringsproblem, så försöker “Studie VI” predicera överlevnadsgraden hos patienter med lung- samt huvud och hals-cancer genom att undersöka neurala nätverk med sfäriska faltningsoperationer. Metoden jämförs mot, bland annat, “radiomics” och visar liknande resultat för analys på samma dataset, men bättre resultat för analys på olika dataset. Sammanfattningsvis så utnyttjar de sex studierna olika medicinska bildgivande system samt en mängd olika bildbehandling- och maskininlärningstekniker för att utveckla verktyg för att kvantifierar tumörers egenskaper, som kan underlätta fastställande av diagnos och prognos.QC 2022-08-29</p

Advanced Machine Learning Methods for Oncological Image Analysis

Cancer is a major public health problem, accounting for an estimated 10 million deaths worldwide in 2020 alone. Rapid advances in the field of image acquisition and hardware development over the past three decades have resulted in the development of modern medical imaging modalities that can capture high-resolution anatomical, physiological, functional, and metabolic quantitative information from cancerous organs. Therefore, the applications of medical imaging have become increasingly crucial in the clinical routines of oncology, providing screening, diagnosis, treatment monitoring, and non/minimally-invasive evaluation of disease prognosis. The essential need for medical images, however, has resulted in the acquisition of a tremendous number of imaging scans. Considering the growing role of medical imaging data on one side and the challenges of manually examining such an abundance of data on the other side, the development of computerized tools to automatically or semi-automatically examine the image data has attracted considerable interest. Hence, a variety of machine learning tools have been developed for oncological image analysis, aiming to assist clinicians with repetitive tasks in their workflow. This thesis aims to contribute to the field of oncological image analysis by proposing new ways of quantifying tumor characteristics from medical image data. Specifically, this thesis consists of six studies, the first two of which focus on introducing novel methods for tumor segmentation. The last four studies aim to develop quantitative imaging biomarkers for cancer diagnosis and prognosis. The main objective of Study I is to develop a deep learning pipeline capable of capturing the appearance of lung pathologies, including lung tumors, and integrating this pipeline into the segmentation networks to leverage the segmentation accuracy. The proposed pipeline was tested on several comprehensive datasets, and the numerical quantifications show the superiority of the proposed prior-aware DL framework compared to the state of the art. Study II aims to address a crucial challenge faced by supervised segmentation models: dependency on the large-scale labeled dataset. In this study, an unsupervised segmentation approach is proposed based on the concept of image inpainting to segment lung and head-neck tumors in images from single and multiple modalities. The proposed autoinpainting pipeline shows great potential in synthesizing high-quality tumor-free images and outperforms a family of well-established unsupervised models in terms of segmentation accuracy. Studies III and IV aim to automatically discriminate the benign from the malignant pulmonary nodules by analyzing the low-dose computed tomography (LDCT) scans. In Study III, a dual-pathway deep classification framework is proposed to simultaneously take into account the local intra-nodule heterogeneities and the global contextual information. Study IV seeks to compare the discriminative power of a series of carefully selected conventional radiomics methods, end-to-end Deep Learning (DL) models, and deep features-based radiomics analysis on the same dataset. The numerical analyses show the potential of fusing the learned deep features into radiomic features for boosting the classification power. Study V focuses on the early assessment of lung tumor response to the applied treatments by proposing a novel feature set that can be interpreted physiologically. This feature set was employed to quantify the changes in the tumor characteristics from longitudinal PET-CT scans in order to predict the overall survival status of the patients two years after the last session of treatments. The discriminative power of the introduced imaging biomarkers was compared against the conventional radiomics, and the quantitative evaluations verified the superiority of the proposed feature set. Whereas Study V focuses on a binary survival prediction task, Study VI addresses the prediction of survival rate in patients diagnosed with lung and head-neck cancer by investigating the potential of spherical convolutional neural networks and comparing their performance against other types of features, including radiomics. While comparable results were achieved in intra-dataset analyses, the proposed spherical-based features show more predictive power in inter-dataset analyses. In summary, the six studies incorporate different imaging modalities and a wide range of image processing and machine-learning techniques in the methods developed for the quantitative assessment of tumor characteristics and contribute to the essential procedures of cancer diagnosis and prognosis.Cancer är en global hälsoutmaning som uppskattas ansvara för cirka 10 miljoner dödsfall i hela världen, bara under året 2020. Framsteg inom medicinsk bildtagning och hårdvaruutveckling de senaste tre decennierna har banat vägen för moderna medicinska bildgivande system vars upplösningsförmåga tillåter att fånga information om tumörers anatomi, fysiologi, funktion samt metabolism. Medicinsk bildanalys har därför fått en mer betydelserik roll i klinikers dagliga rutiner inom onkologin, för bland annat screening, diagnostik, uppföljning av behandling samt icke-invasiv utvärdering av sjukdomsprognoser. Sjukvårdens behov av medicinska bilder har lett till att det nu på sjukhusen finns en enorm mängd medicinska bilder på alla moderna sjukhus. Med hänsyn till den viktiga roll medicinsk bilddata spelar i dagens sjukvård, samt den mängd manuellt arbete som behöver göras för att analysera den mängd data som genereras varje dag, så har utvecklingen av digitala verktyg för att för att automatiskt eller semi-automatiskt analysera  bilddatan alltid haft stort intresse. Därför har en rad maskininlärningsverktyg utvecklats för analys av onkologisk data, för att gripa sig an läkares repetitiva vardagssysslor. Den här avhandlingen syftar att bidra till fältet “onkologisk bildanalys” genom att föreslå nya sätt att kvantifiera tumörers egenskaper från medicinsk bilddata. Specifikt, är denna avhandling baserad på sex artiklar där de första två har fokus att presentera nya metoder för segmentering av tumörer, och de resterande fyra ämnar att utveckla kvantitativa biomarkörer för cancerdiagnostik och prognos. Huvudsyftet för “Studie I” har varit att utveckla en djupinlärnings-pipeline vars syfte är att fånga lungpatalogiers anatomier (inklusive lungtumörer) samt integrera detta med djupa neurala nätverk för segmentering för att nyttja det första nätverkets utfall för att förbättra segmenteringskvalitén. Den föreslagna pipelinen testades på flertalet dataset och numeriska analyser visar en överlägsna resultat för den föreslagna “prior-medvetna” djupinlärningsmetoden. “Studie II” ämnar att ta sig an ett viktig problem som övervakade segmenteringsmetoder ställs inför: ett beroende av enorma annoterade dataset. I denna studie föreslås en icke-övervakad segmenteringsmetod som baseras på konceptet “ifyllnad” (“inpainting”) för att segmentera tumörer i områdena: lungor samt huvud och hals i bilder från olika modaliteter. Den föreslagna metoden lyckas bättre än en familj väletablerade icke-oövervakade segmenteringsmodeller. “Studie III” och “Studie IV” försöker automatiskt diskriminera benigna lungtumörer från maligna tumörer genom att analysera bilder från LDCT (lågdos-CT). I “Studie III“ föreslås ett djupt neuralt nätverk för klassificering vars grafstruktur tillåter lokal analys av tumörens inbördes heterogeniteter samt en helhetsbild från global kontextuell information. “Studie IV” försöker utvärdera noggrant utvalda metoder som grundar sig på att extrahera anatomiska särdrag från medicinska bilder. I studien jämförs konventionella “radiomics”-metoder med särdrag från neurala nätverk samt en kombination av båda på samma dataset. Resultat från studien visar att en kombination av särdrag från djupa neurala nätverk samt “radiomics” kan ge bättre resultat i klassificeringsproblemet. “Studie V” har fokus på tidig bedömning av lungtumörers respons på behandling genom att utveckla ett set nya fysiologisk observerbara särdrag. Den presenterade metoden har använts för att kvantifiera förändringar i tumörers karaktär i PET-CT-undersökningar för att predicera patienters prognos två år efter senaste behandling. Metoden jämförts mot konventionella “radiomics” och utvärderingen visar att den föreslagna metoden ger förbättrade resultat. Till skilnad från “Studie V”, som fokuserar på att lösa ett binärt klassificeringsproblem, så försöker “Studie VI” predicera överlevnadsgraden hos patienter med lung- samt huvud och hals-cancer genom att undersöka neurala nätverk med sfäriska faltningsoperationer. Metoden jämförs mot, bland annat, “radiomics” och visar liknande resultat för analys på samma dataset, men bättre resultat för analys på olika dataset. Sammanfattningsvis så utnyttjar de sex studierna olika medicinska bildgivande system samt en mängd olika bildbehandling- och maskininlärningstekniker för att utveckla verktyg för att kvantifierar tumörers egenskaper, som kan underlätta fastställande av diagnos och prognos.QC 2022-08-29</p