Fully-automated Body Composition Analysis in Routine CT Imaging Using 3D Semantic Segmentation Convolutional Neural Networks

Body tissue composition is a long-known biomarker with high diagnostic and prognostic value in cardiovascular, oncological and orthopaedic diseases, but also in rehabilitation medicine or drug dosage. In this study, the aim was to develop a fully automated, reproducible and quantitative 3D volumetry of body tissue composition from standard CT examinations of the abdomen in order to be able to offer such valuable biomarkers as part of routine clinical imaging. Therefore an in-house dataset of 40 CTs for training and 10 CTs for testing were fully annotated on every fifth axial slice with five different semantic body regions: abdominal cavity, bones, muscle, subcutaneous tissue, and thoracic cavity. Multi-resolution U-Net 3D neural networks were employed for segmenting these body regions, followed by subclassifying adipose tissue and muscle using known hounsfield unit limits. The S{\o}rensen Dice scores averaged over all semantic regions was 0.9553 and the intra-class correlation coefficients for subclassified tissues were above 0.99. Our results show that fully-automated body composition analysis on routine CT imaging can provide stable biomarkers across the whole abdomen and not just on L3 slices, which is historically the reference location for analysing body composition in the clinical routine

Large‑scale analysis of iliopsoas muscle volumes in the UK Biobank

Psoas muscle measurements are frequently used as markers of sarcopenia and predictors of health. Manually measured cross-sectional areas are most commonly used, but there is a lack of consistency regarding the position of the measurement and manual annotations are not practical for large population studies. We have developed a fully automated method to measure iliopsoas muscle volume (comprised of the psoas and iliacus muscles) using a convolutional neural network. Magnetic resonance images were obtained from the UK Biobank for 5000 participants, balanced for age, gender and BMI. Ninety manual annotations were available for model training and validation. The model showed excellent performance against out-of-sample data (average dice score coefficient of 0.9046 ± 0.0058 for six-fold cross-validation). Iliopsoas muscle volumes were successfully measured in all 5000 participants. Iliopsoas volume was greater in male compared with female subjects. There was a small but significant asymmetry between left and right iliopsoas muscle volumes. We also found that iliopsoas volume was significantly related to height, BMI and age, and that there was an acceleration in muscle volume decrease in men with age. Our method provides a robust technique for measuring iliopsoas muscle volume that can be applied to large cohorts

First PACS‐integrated artificial intelligence‐based software tool for rapid and fully automatic analysis of body composition from CT in clinical routine

Background: To externally evaluate the first picture archiving communications system (PACS)-integrated artificial intelligence (AI)-based workflow, trained to automatically detect a predefined computed tomography (CT) slice at the third lumbar vertebra (L3) and automatically perform complete image segmentation for analysis of CT body composition and to compare its performance with that of an established semi-automatic segmentation tool regarding speed and accuracy of tissue area calculation. Methods: For fully automatic analysis of body composition with L3 recognition, U-Nets were trained (Visage) and compared with a conventional image segmentation software (TomoVision). Tissue was differentiated into psoas muscle, skeletal muscle, visceral adipose tissue (VAT) and subcutaneous adipose tissue (SAT). Mid-L3 level images from randomly selected DICOM slice files of 20 CT scans acquired with various imaging protocols were segmented with both methods. Results: Success rate of AI-based L3 recognition was 100%. Compared with semi-automatic, fully automatic AI-based image segmentation yielded relative differences of 0.22% and 0.16% for skeletal muscle, 0.47% and 0.49% for psoas muscle, 0.42% and 0.42% for VAT and 0.18% and 0.18% for SAT. AI-based fully automatic segmentation was significantly faster than semi-automatic segmentation (3 ± 0 s vs. 170 ± 40 s, P < 0.001, for User 1 and 152 ± 40 s, P < 0.001, for User 2). Conclusion: Rapid fully automatic AI-based, PACS-integrated assessment of body composition yields identical results without transfer of critical patient data. Additional metabolic information can be inserted into the patient’s image report and offered to the referring clinicians

AATCT-IDS: A Benchmark Abdominal Adipose Tissue CT Image Dataset for Image Denoising, Semantic Segmentation, and Radiomics Evaluation

Methods: In this study, a benchmark \emph{Abdominal Adipose Tissue CT Image Dataset} (AATTCT-IDS) containing 300 subjects is prepared and published. AATTCT-IDS publics 13,732 raw CT slices, and the researchers individually annotate the subcutaneous and visceral adipose tissue regions of 3,213 of those slices that have the same slice distance to validate denoising methods, train semantic segmentation models, and study radiomics. For different tasks, this paper compares and analyzes the performance of various methods on AATTCT-IDS by combining the visualization results and evaluation data. Thus, verify the research potential of this data set in the above three types of tasks. Results: In the comparative study of image denoising, algorithms using a smoothing strategy suppress mixed noise at the expense of image details and obtain better evaluation data. Methods such as BM3D preserve the original image structure better, although the evaluation data are slightly lower. The results show significant differences among them. In the comparative study of semantic segmentation of abdominal adipose tissue, the segmentation results of adipose tissue by each model show different structural characteristics. Among them, BiSeNet obtains segmentation results only slightly inferior to U-Net with the shortest training time and effectively separates small and isolated adipose tissue. In addition, the radiomics study based on AATTCT-IDS reveals three adipose distributions in the subject population. Conclusion: AATTCT-IDS contains the ground truth of adipose tissue regions in abdominal CT slices. This open-source dataset can attract researchers to explore the multi-dimensional characteristics of abdominal adipose tissue and thus help physicians and patients in clinical practice. AATCT-IDS is freely published for non-commercial purpose at: \url{https://figshare.com/articles/dataset/AATTCT-IDS/23807256}.Comment: 17 pages, 7 figure

Deep neural network for automatic volumetric segmentation of whole-body CT images for body composition assessment

Background & aims: Body composition analysis on CT images is a valuable tool for sarcopenia assessment. We aimed to develop and validate a deep neural network applicable to whole-body CT images of PET-CT scan for the automatic volumetric segmentation of body composition. Methods: For model development, one hundred whole-body or torso 18F-fluorodeoxyglucose PET-CT scans of 100 patients were retrospectively included. Two radiologists semi-automatically labeled the following seven body components in every CT image slice, providing a total of 46,967 image slices from the 100 scans for training the 3D U-Net (training, 39,268 slices; tuning, 3116 slices; internal validation, 4583 slices): skin, bone, muscle, abdominal visceral fat, subcutaneous fat, internal organs with vessels, and central nervous system. The segmentation accuracy was assessed using reference masks from three external datasets: two Korean centers (4668 and 4796 image slices from 20 CT scans, each) and a French public dataset (3763 image slices from 24 CT scans). The 3D U-Net-driven values were clinically validated using bioelectrical impedance analysis (BIA) and by assessing the model's diagnostic performance for sarcopenia in a community-based elderly cohort (n = 522). Results: The 3D U-Net achieved accurate body composition segmentation with an average dice similarity coefficient of 96.5%-98.9% for all masks and 92.3%-99.3% for muscle, abdominal visceral fat, and subcutaneous fat in the validation datasets. The 3D U-Net-derived torso volume of skeletal muscle and fat tissue and the average area of those tissues in the waist were correlated with BIA-derived appendicular lean mass (correlation coefficients: 0.71 and 0.72, each) and fat mass (correlation coefficients: 0.95 and 0.93, each). The 3D U-Net-derived average areas of skeletal muscle and fat tissue in the waist were independently associated with sarcopenia (P < .001, each) with adjustment for age and sex, providing an area under the curve of 0.858 (95% CI, 0.815 to 0.901). Conclusions: This deep neural network model enabled the automatic volumetric segmentation of body composition on whole-body CT images, potentially expanding adjunctive sarcopenia assessment on PET-CT scan and volumetric assessment of metabolism in whole-body muscle and fat tissues.ope

Cultivate Quantitative Magnetic Resonance Imaging Methods to Measure Markers of Health and Translate to Large Scale Cohort Studies

Magnetic Resonance Imaging (MRI) is an indispensable tool in healthcare and research, with a growing demand for its services. The appeal of MRI stems from its non-ionizing radiation nature, ability to generate high-resolution images of internal organs and structures without invasive procedures, and capacity to provide quantitative assessments of tissue properties such as ectopic fat, body composition, and organ volume. All without long term side effects. Nine published papers are submitted which show the cultivation of quantitative measures of ectopic fat within the liver and pancreas using MRI, and the process of validating whole-body composition and organ volume measurements. All these techniques have been translated into large-scale studies to improve health measurements in large population cohorts. Translating this work into large-scale studies, including the use of artificial intelligence, is included. Additionally, an evaluation accompanies these published studies, appraising the evolution of these quantitative MRI techniques from the conception to their application in large cohort studies. Finally, this appraisal provides a summary of future work on crowdsourcing of ground truth training data to facilitate its use in wider applications of artificial intelligence.In conclusion, this body of work presents a portfolio of evidence to fulfil the requirements of a PhD by published works at the University of Salford

Prediction of Pre-Operative Local Staging and Optimising Treatment Response to Neoadjuvant Therapy in Colorectal Cancer.

The presence of abnormal Lymph Nodes (LNs) in patients with colorectal cancer is an essential determinant of prognosis and guides treatment options (surgical and medical). Staging with Computed Tomography (CT) is somewhat inaccurate in determining true nodal status. As a result, either approximate estimates must be made on imaging, or definitive nodal staging determined by surgical resection before recommendations about the risk vs benefit of chemotherapy can be made reliably. Patients with advanced rectal cancer are commonly referred for neoadjuvant therapy as part of standard care treatment protocols based on Magnetic Resonance Imaging (MRI) local staging. Following neoadjuvant therapy, many patients then undergo surgical resection. However, a significant proportion achieve a complete Clinical Response (cCR) with modern neoadjuvant treatment, and these patients are increasingly offered non-operative management and surveillance with the goal of organ preservation. Accurate clinical staging parameters and predictive markers of tumour response may help guide more personalised treatment strategies and identify potential candidates for non-operative management more accurately. Within the past decade, a promising new strategy termed Total Neoadjuvant Therapy (TNT) has been shown to improve compliance with chemotherapy, by delivering this sequentially with chemoradiotherapy prior to surgery in patients with rectal cancer. TNT has the potential to reduce distant failure risk and provide significantly higher rates of pathological Complete Response (pCR) and cCR with an opportunity to manage patients non-operatively, however, optimal treatment sequencing of radiotherapy and chemotherapy remains somewhat unclear. Pre-operative prediction of nodal status in colon cancer, neoadjuvant treatment response in rectal cancer, as well as optimal sequencing of neoadjuvant therapy, represent major areas of weakness in current treatment paradigms in colorectal surgical oncology. Furthermore, they are all areas of active research, and frequently tie in together during Multi-Disciplinary Team meeting (MDT) discussions in clinical practice. The aims of this thesis are: Firstly, to investigate Artificial Intelligence (AI) models for prediction of LN status on preoperative staging CT in patients with colon cancer. Secondly, to identify pre-treatment factors predictive of Complete Response (CR) following neoadjuvant therapy in patients with Locally Advanced Rectal Cancer (LARC), specifically sarcopenia, clinical and biochemical factors. Lastly, to determine whether a Personalised Total Neoadjuvant Therapy (pTNT) protocol with sequencing tailored to the clinical stage at presentation results in better short-term oncological outcomes compared to a uniform protocol for all patients with advanced rectal cancer. To achieve these aims, two meta-analyses were performed to identify the gaps in the field of AI LN detection. The first, focused on the accuracy of deep learning algorithms and radiomics models compared with radiologist assessment in the diagnosis of lymphadenopathy in patients with abdominopelvic malignancies and the second solely focused on colorectal cancer. Subsequently, a deep learning model was developed to assess LN status on staging CT in patients with colon cancer, and the model’s performance was compared with baseline results of a prospective study evaluating the accuracy of preoperative staging. A systemic review and meta-analysis were performed to identify and assess AI segmentation models able to predict sarcopenia using CT scans. Following this, an institutional colorectal cancer database was interrogated to determine if sarcopenia or clinical and biochemical markers were associated with tumour response in patients with LARC. Prospective data was collected on patients in two hospitals who underwent pTNT based on their clinical staging at presentation for the treatment of advanced rectal cancer. A cohort study was performed to summarise tumour response, chemotherapy compliance and the toxicity profile of patients. An additional multicentre retrospective cohort analysis comparing pTNT over a 3-year period to a historical cohort of randomised control trial patients who had extended chemotherapy in the wait period (xCRT) or standard long course Chemoradiotherapy (sCRT) was conducted. The two meta-analyses determined that deep learning assessment of LNs demonstrated the greatest potential for assessment of LN without the need for surgery, with MRI for rectal cancer and CT in colon cancer providing the greatest accuracy. Our clinical studies demonstrated that radiological assessment remains the most effective preoperative method of staging LNs, with histology considered the gold standard. Deep learning assessment using a ResNet-50 framework is limited to very low accuracy and specificity in detecting abnormal LNs when compared to the radiologist’s assessment. It is likely that the poor performance of the deep learning model is attributed to the lack of features extracted from the CT scans. The meta-analysis found that deep learning segmentation models can accurately predict sarcopenia using CT scans. However, sarcopenia was not found to be a predictor of pCR in patients with LARC. The clinical predictors of good tumour response after neoadjuvant therapy for rectal cancer were found to be a clinical T2 stage and Body Mass Index (BMI) ≥25kg/m2. Pre-treatment biochemical markers were not predictive of tumour response after neoadjuvant therapy for rectal cancer. Our research found that over 40% of the patients who underwent pTNT for the treatment of advanced rectal cancer demonstrated a complete response in the primary tumour (pCR and/or cCR) resulting in a high rate of organ preservation. Furthermore, 45% of the patients with stage M1 disease achieved a complete M1 response. Compliance with chemotherapy was over 95% and toxicity was lower than expected. When comparing a pTNT approach with xCRT or sCRT in patients with LARC, there was a significant difference in complete response and cCR rate favouring the pTNT group compared to the xCRT and sCRT groups. In conclusion, these results suggest that a deep learning model with a ResNet-50 framework does not serve as a reliable staging tool for the prediction of LN status using preoperative staging CT in patients with colon cancer. Despite a large volume of research, the ability to predict which patients are likely to achieve a complete response by measuring pre-treatment sarcopenia, clinical and biochemical markers remains elusive. Early results of a pTNT approach tailoring sequencing of neoadjuvant chemotherapy to disease risk at presentation are encouraging and compare favourably to xCRT and sCRT in patients with advanced rectal cancer.Thesis (Ph.D.) -- University of Adelaide, School of Medicine, 202