Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review

The medical image analysis field has traditionally been focused on the development of organ-, and disease-specific methods. Recently, the interest in the development of more 20 comprehensive computational anatomical models has grown, leading to the creation of multi-organ models. Multi-organ approaches, unlike traditional organ-specific strategies, incorporate inter-organ relations into the model, thus leading to a more accurate representation of the complex human anatomy. Inter-organ relations are not only spatial, but also functional and physiological. Over the years, the strategies 25 proposed to efficiently model multi-organ structures have evolved from the simple global modeling, to more sophisticated approaches such as sequential, hierarchical, or machine learning-based models. In this paper, we present a review of the state of the art on multi-organ analysis and associated computation anatomy methodology. The manuscript follows a methodology-based classification of the different techniques 30 available for the analysis of multi-organs and multi-anatomical structures, from techniques using point distribution models to the most recent deep learning-based approaches. With more than 300 papers included in this review, we reflect on the trends and challenges of the field of computational anatomy, the particularities of each anatomical region, and the potential of multi-organ analysis to increase the impact of 35 medical imaging applications on the future of healthcare.Comment: Paper under revie

Segmentation of pelvic structures from preoperative images for surgical planning and guidance

Prostate cancer is one of the most frequently diagnosed malignancies globally and the second leading cause of cancer-related mortality in males in the developed world. In recent decades, many techniques have been proposed for prostate cancer diagnosis and treatment. With the development of imaging technologies such as CT and MRI, image-guided procedures have become increasingly important as a means to improve clinical outcomes. Analysis of the preoperative images and construction of 3D models prior to treatment would help doctors to better localize and visualize the structures of interest, plan the procedure, diagnose disease and guide the surgery or therapy. This requires efficient and robust medical image analysis and segmentation technologies to be developed. The thesis mainly focuses on the development of segmentation techniques in pelvic MRI for image-guided robotic-assisted laparoscopic radical prostatectomy and external-beam radiation therapy. A fully automated multi-atlas framework is proposed for bony pelvis segmentation in MRI, using the guidance of MRI AE-SDM. With the guidance of the AE-SDM, a multi-atlas segmentation algorithm is used to delineate the bony pelvis in a new \ac{MRI} where there is no CT available. The proposed technique outperforms state-of-the-art algorithms for MRI bony pelvis segmentation. With the SDM of pelvis and its segmented surface, an accurate 3D pelvimetry system is designed and implemented to measure a comprehensive set of pelvic geometric parameters for the examination of the relationship between these parameters and the difficulty of robotic-assisted laparoscopic radical prostatectomy. This system can be used in both manual and automated manner with a user-friendly interface. A fully automated and robust multi-atlas based segmentation has also been developed to delineate the prostate in diagnostic MR scans, which have large variation in both intensity and shape of prostate. Two image analysis techniques are proposed, including patch-based label fusion with local appearance-specific atlases and multi-atlas propagation via a manifold graph on a database of both labeled and unlabeled images when limited labeled atlases are available. The proposed techniques can achieve more robust and accurate segmentation results than other multi-atlas based methods. The seminal vesicles are also an interesting structure for therapy planning, particularly for external-beam radiation therapy. As existing methods fail for the very onerous task of segmenting the seminal vesicles, a multi-atlas learning framework via random decision forests with graph cuts refinement has further been proposed to solve this difficult problem. Motivated by the performance of this technique, I further extend the multi-atlas learning to segment the prostate fully automatically using multispectral (T1 and T2-weighted) MR images via hybrid \ac{RF} classifiers and a multi-image graph cuts technique. The proposed method compares favorably to the previously proposed multi-atlas based prostate segmentation. The work in this thesis covers different techniques for pelvic image segmentation in MRI. These techniques have been continually developed and refined, and their application to different specific problems shows ever more promising results.Open Acces

Registration and Deformable Model-Based Neck Muscles Segmentation and 3D Reconstruction

Whiplash is a very common ailment encountered in clinical practice that is usually a result of vehicle accidents but also domestic activities and sports injuries. It is normally caused when neck organs (specifically muscles) are impaired. Whiplash-associated disorders include acute headaches, neck pain, stiffness, arm dislocation, abnormal sensations, and auditory and optic problems, the persistence of which may be chronic or acute. Insurance companies compensate almost fifty percent of claims lodged due to whiplash injury through compulsory third party motor insurance. The morphological structures of neck muscles undergo hypertrophy or atrophy following damage caused to them by accidents. Before any medical treatment is applied , any such change needs to be known which requires 3D visualization of the neck muscles through a proper segmentation of them because the neck contains many other sensitive organs such as nerves, blood vessels, the spinal cord and trachea. The segmentation of neck muscles in medical images is a more challenging task than those of other muscles and organs due to their similar densities and compactness, low resolutions and contrast in medical images, anatomical variabilities among individuals, noise, inhomogeneity of medical images and false boundaries created by intra-muscular fat. Traditional segmentation algorithms, such as those used in thresholding and clustering-based methods, are not applicable in this project and also not suitable for medical images. Although there are some techniques available in clinical research for segmenting muscles such as thigh, tongue, leg, hip and pectoral ones, to the best of author's knowledge, there are no methods available for segmenting neck muscles due to the challenges described above. In the first part of this dissertation, an atlas-based method for segmenting MR images, which uses linear and non-linear registration frameworks, is proposed, with output from the registration process further refined by a novel parametric deformable model. The proposed method is tested on real clinical data of both healthy and non-healthy individuals. During the last few decades, registration- and deformable model-based segmentation methods have been very popular for medical image segmentation due to their incorporation of prior information. While registration-based segmentation techniques can preserve topologies of objects in an image, accuracy of atlas-based segmentation depends mainly on an effective registration process. In this study, the registration framework is designed in a novel way in which images are initially registered by a distinct 3D affine transformation and then aligned by a local elastic geometrical transformation based on discrete cosines and registered firstly slice-wise and then block-wise. The numbers of motion parameters are changed in three different steps per frame. This proposed registration framework can handle anatomical variabilities and pathologies by confining its parameters in local regions. Also, as warping of the framework relies on number of motion parameters, similarities between two images, gradients of floating image and coordinate mesh grid values, it can easily manage pathological and anatomical variabilities using a hierarchical parameter scheme. The labels transferred from atlas can be improved by deformable model-based segmentation. Although geometric deformable models have been widely used in many biomedical applications over recent years, they cannot work in the context of neck muscles segmentation due to noise, background clutter and similar objects touching each other. Another important drawback of geometric deformable models is that they are many times slower than parametric deformable ones. Therefore, the segmentation results produced by the registration process are ameliorated using a multiple-object parametric deformable model which is discussed in detail in the second part of this thesis. This algorithm uses a novel Gaussian potential energy distribution which can adapt to topological changes and does not require re-parameterization. Also, it incorporates a new overlap removal technique which ensures that there are no overlaps or gaps inside an object. Furthermore, stopping criteria of vertices are designed so that difference between boundaries of the deformable model and actual object is minimal. The multiple-object parametric deformable model is also applied in a template contours propagation-based segmentation technique, as discussed in the third part of this dissertation. This method is semi-automatic, whereby a manual delineation of middle image in a MRI data set is required. It can handle anatomical variabilities more easily than atlas-based segmentation because it can segment any individual's data irrespective of his/her age, weight and height with low computational complexity and it does not depend on other data as it operates semi-automatically. In it, initial model contour resides close to the object's boundary, with degree of closeness dependent on slice thicknesses and gaps between the slices

Improving Radiotherapy Targeting for Cancer Treatment Through Space and Time

Radiotherapy is a common medical treatment in which lethal doses of ionizing radiation are preferentially delivered to cancerous tumors. In external beam radiotherapy, radiation is delivered by a remote source which sits several feet from the patient\u27s surface. Although great effort is taken in properly aligning the target to the path of the radiation beam, positional uncertainties and other errors can compromise targeting accuracy. Such errors can lead to a failure in treating the target, and inflict significant toxicity to healthy tissues which are inadvertently exposed high radiation doses. Tracking the movement of targeted anatomy between and during treatment fractions provides valuable localization information that allows for the reduction of these positional uncertainties. Inter- and intra-fraction anatomical localization data not only allows for more accurate treatment setup, but also potentially allows for 1) retrospective treatment evaluation, 2) margin reduction and modification of the dose distribution to accommodate daily anatomical changes (called `adaptive radiotherapy\u27), and 3) targeting interventions during treatment (for example, suspending radiation delivery while the target it outside the path of the beam). The research presented here investigates the use of inter- and intra-fraction localization technologies to improve radiotherapy to targets through enhanced spatial and temporal accuracy. These technologies provide significant advancements in cancer treatment compared to standard clinical technologies. Furthermore, work is presented for the use of localization data acquired from these technologies in adaptive treatment planning, an investigational technique in which the distribution of planned dose is modified during the course of treatment based on biological and/or geometrical changes of the patient\u27s anatomy. The focus of this research is directed at abdominal sites, which has historically been central to the problem of motion management in radiation therapy

Multiparametric Magnetic Resonance Imaging Artificial Intelligence Pipeline for Oropharyngeal Cancer Radiotherapy Treatment Guidance

Oropharyngeal cancer (OPC) is a widespread disease and one of the few domestic cancers that is rising in incidence. Radiographic images are crucial for assessment of OPC and aid in radiotherapy (RT) treatment. However, RT planning with conventional imaging approaches requires operator-dependent tumor segmentation, which is the primary source of treatment error. Further, OPC expresses differential tumor/node mid-RT response (rapid response) rates, resulting in significant differences between planned and delivered RT dose. Finally, clinical outcomes for OPC patients can also be variable, which warrants the investigation of prognostic models. Multiparametric MRI (mpMRI) techniques that incorporate simultaneous anatomical and functional information coupled to artificial intelligence (AI) approaches could improve clinical decision support for OPC by providing immediately actionable clinical rationale for adaptive RT planning. If tumors could be reproducibly segmented, rapid response could be classified, and prognosis could be reliably determined, overall patient outcomes would be optimized to improve the therapeutic index as a function of more risk-adapted RT volumes. Consequently, there is an unmet need for automated and reproducible imaging which can simultaneously segment tumors and provide predictive value for actionable RT adaptation. This dissertation primarily seeks to explore and optimize image processing, tumor segmentation, and patient outcomes in OPC through a combination of advanced imaging techniques and AI algorithms. In the first specific aim of this dissertation, we develop and evaluate mpMRI pre-processing techniques for use in downstream segmentation, response prediction, and outcome prediction pipelines. Various MRI intensity standardization and registration approaches were systematically compared and benchmarked. Moreover, synthetic image algorithms were developed to decrease MRI scan time in an effort to optimize our AI pipelines. We demonstrated that proper intensity standardization and image registration can improve mpMRI quality for use in AI algorithms, and developed a novel method to decrease mpMRI acquisition time. Subsequently, in the second specific aim of this dissertation, we investigated underlying questions regarding the implementation of RT-related auto-segmentation. Firstly, we quantified interobserver variability for an unprecedented large number of observers for various radiotherapy structures in several disease sites (with a particular emphasis on OPC) using a novel crowdsourcing platform. We then trained an AI algorithm on a series of extant matched mpMRI datasets to segment OPC primary tumors. Moreover, we validated and compared our best model\u27s performance to clinical expert observers. We demonstrated that AI-based mpMRI OPC tumor auto-segmentation offers decreased variability and comparable accuracy to clinical experts, and certain mpMRI input channel combinations could further improve performance. Finally, in the third specific aim of this dissertation, we predicted OPC primary tumor mid-therapy (rapid) treatment response and prognostic outcomes. Using co-registered pre-therapy and mid-therapy primary tumor manual segmentations of OPC patients, we generated and characterized treatment sensitive and treatment resistant pre-RT sub-volumes. These sub-volumes were used to train an AI algorithm to predict individual voxel-wise treatment resistance. Additionally, we developed an AI algorithm to predict OPC patient progression free survival using pre-therapy imaging from an international data science competition (ranking 1st place), and then translated these approaches to mpMRI data. We demonstrated AI models could be used to predict rapid response and prognostic outcomes using pre-therapy imaging, which could help guide treatment adaptation, though further work is needed. In summary, the completion of these aims facilitates the development of an image-guided fully automated OPC clinical decision support tool. The resultant deliverables from this project will positively impact patients by enabling optimized therapeutic interventions in OPC. Future work should consider investigating additional imaging timepoints, imaging modalities, uncertainty quantification, perceptual and ethical considerations, and prospective studies for eventual clinical implementation. A dynamic version of this dissertation is publicly available and assigned a digital object identifier through Figshare (doi: 10.6084/m9.figshare.22141871)

Book of Abstracts 15th International Symposium on Computer Methods in Biomechanics and Biomedical Engineering and 3rd Conference on Imaging and Visualization

In this edition, the two events will run together as a single conference, highlighting the strong connection with the Taylor & Francis journals: Computer Methods in Biomechanics and Biomedical Engineering (John Middleton and Christopher Jacobs, Eds.) and Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization (JoãoManuel R.S. Tavares, Ed.). The conference has become a major international meeting on computational biomechanics, imaging andvisualization. In this edition, the main program includes 212 presentations. In addition, sixteen renowned researchers will give plenary keynotes, addressing current challenges in computational biomechanics and biomedical imaging. In Lisbon, for the first time, a session dedicated to award the winner of the Best Paper in CMBBE Journal will take place. We believe that CMBBE2018 will have a strong impact on the development of computational biomechanics and biomedical imaging and visualization, identifying emerging areas of research and promoting the collaboration and networking between participants. This impact is evidenced through the well-known research groups, commercial companies and scientific organizations, who continue to support and sponsor the CMBBE meeting series. In fact, the conference is enriched with five workshops on specific scientific topics and commercial software.info:eu-repo/semantics/draf

Textural features for bladder cancer definition on CT images

Genitourinary cancer refers to the presence of tumours in the genital or urinary organs such as bladder, kidney and prostate. In 2008 the worldwide incidence of bladder cancer was 382,600 with a mortality of 150,282. Radiotherapy is one of the main treatment choices for genitourinary cancer where accurate delineation of the gross tumour volume (GTV) on computed tomography (CT) images is crucial for the success of this treatment. Limited CT resolution and contrast in soft tissue organs make this difficult and has led to significant inter- and intra- clinical variability in defining the extent of the GTV, especially at the junctions of different organs. In addition the introduction of new imaging techniques and modalities has significantly increased the number of the medical images that require contouring. More advanced image processing is required to help reduce contouring variability and assist in handling the increased volume of data. In this thesis image analysis methodologies were used to extract low-level features such as entropy, moment and correlation from radiotherapy planning CT images. These distinctive features were identified and used for defining the GTV and to implement a fully-automatic contouring system. The first key contribution is to demonstrate that second-order statistics from co-occurrence matrices (GTSDM) give higher accuracy in classifying soft tissue regions of interest (ROIs) into GTV and non-GTV. Loadings of the principal components (PCs) of the GTSDM features were found to be consistent over different patients. Exhaustive feature selection suggested that entropies and correlations produced consistently larger areas under receiver operating characteristic (AUROC) curves than first-order features. The second significant contribution is to demonstrate that in the bladder-prostate junction, where the largest inter-clinical variability is observed, the second-order principal entropy from stationery wavelet denoised CT images (DPE) increased the saliency of the bladder prostate junction. As a result thresholding of the DPE produced good agreement between gold standard clinical contours and those produced by this approach with Dice coefficients. The third contribution is to implement a fully automatic and reproducible system for bladder cancer GTV auto-contouring based on classifying second-order statistics. The Dice similarity coefficients (DSCs) were employed to evaluate the automatic contours. It was found that in the mid-range of the bladder the automatic contours are accurate, but in the inferior and superior ends of bladder automatic contours were more likely to have small DSCs with clinical contours, which reconcile with the fact of clinical variability in defining GTVs. A novel male bladder probability atlas was constructed based on the clinical contours and volume estimation from the classification results. Registration of the classification results with this probabilistic atlas consistently increases the DSCs of the inferior slices