The optimisation of deep neural networks for segmenting multiple knee joint tissues from MRIs.

Automated semantic segmentation of multiple knee joint tissues is desirable to allow faster and more reliable analysis of large datasets and to enable further downstream processing e.g. automated diagnosis. In this work, we evaluate the use of conditional Generative Adversarial Networks (cGANs) as a robust and potentially improved method for semantic segmentation compared to other extensively used convolutional neural network, such as the U-Net. As cGANs have not yet been widely explored for semantic medical image segmentation, we analysed the effect of training with different objective functions and discriminator receptive field sizes on the segmentation performance of the cGAN. Additionally, we evaluated the possibility of using transfer learning to improve the segmentation accuracy. The networks were trained on i) the SKI10 dataset which comes from the MICCAI grand challenge "Segmentation of Knee Images 2010″, ii) the OAI ZIB dataset containing femoral and tibial bone and cartilage segmentations of the Osteoarthritis Initiative cohort and iii) a small locally acquired dataset (Advanced MRI of Osteoarthritis (AMROA) study) consisting of 3D fat-saturated spoiled gradient recalled-echo knee MRIs with manual segmentations of the femoral, tibial and patellar bone and cartilage, as well as the cruciate ligaments and selected peri-articular muscles. The Sørensen-Dice Similarity Coefficient (DSC), volumetric overlap error (VOE) and average surface distance (ASD) were calculated for segmentation performance evaluation. DSC ≥ 0.95 were achieved for all segmented bone structures, DSC ≥ 0.83 for cartilage and muscle tissues and DSC of ≈0.66 were achieved for cruciate ligament segmentations with both cGAN and U-Net on the in-house AMROA dataset. Reducing the receptive field size of the cGAN discriminator network improved the networks segmentation performance and resulted in segmentation accuracies equivalent to those of the U-Net. Pretraining not only increased segmentation accuracy of a few knee joint tissues of the fine-tuned dataset, but also increased the network's capacity to preserve segmentation capabilities for the pretrained dataset. cGAN machine learning can generate automated semantic maps of multiple tissues within the knee joint which could increase the accuracy and efficiency for evaluating joint health.European Union's Horizon 2020 Framework Programme [grant number 761214] Addenbrooke’s Charitable Trust (ACT) National Institute of Health Research (NIHR) Cambridge Biomedical Research Centre University of Cambridge Cambridge University Hospitals NHS Foundation Trust GSK VARSITY: PHD STUDENTSHIP Funder reference: 300003198

Quantitative Magnetic Resonance Imaging and Analysis of Articular Cartilage and Osteoarthritis

MRI plays an important role in the continuing search for a sensitive osteoarthritis (OA) imaging biomarker able to detect early, pre-morphological alterations in cartilage composition. Determining the compositional recovery pattern of cartilage following acute joint loading could potentially present a more sensitive biomarker for defining cartilage health [1]. However, only a limited amount of studies have assessed both the immediate effect of joint loading on cartilage, as well as its post-loading recovery. In addition, when assessing the compositional responses of cartilage to joint loading, previous studies usually did not incorporate the measurement error of the used quantitative MRI technique into their analysis. Therefore, an uncertainty persists whether or not compositional MRI techniques are sensitive enough to measure changes in water and macromolecular content of cartilage, or if previous studies were merely measuring noise. Consequently, an objective of this thesis is to increase our understanding of and reliability in quantitative T2 and T1ρ relaxation time mapping to detect compositional responses of cartilage following a joint loading activity. Furthermore, to obtain the quantitative morphological and compositional measures of cartilage, detailed region-specific delineation of cartilage is required. This delineation (or segmentation) of cartilage is laborious and time-consuming as it is usually performed manually by an expert observer. Many new advances in image analysis, particularly those in convolutional neural networks (CNNs) and deep learning, have enabled a time-efficient semi- or fully-automated alternative to this process [2, 3]. This thesis explores the utility of deep CNNs generated segmentations for accurate surface-based analysis of cartilage morphology and composition from knee MRIs as well as of cortical bone thickness from knee CTs. Chapter 1 will provide an introduction into the structure and biomechanics of articular cartilage and the role of MRI in imaging the degenerative joint disorder, osteoarthritis as well as the effects of different joint loading activities on cartilage morphology and composition. Chapter 2 explains the principle of MRI and the pulse sequences used in the following chapter for the morphometric and compositional assessment of articular cartilage. Chapter 3 describes the use of 3D Cartilage Surface Mapping (3D-CaSM) [3] to assess variations in cartilage T1ρ and T2 relaxation times of young, healthy participants following a mild, unilateral stepping activity. By evaluating and incorporating the intrasessional repeatability of the T1ρ and T2 mapping techniques, I aim to highlight those cartilage areas experiencing exercise-induced compositional changes greater than measurement error. A significant amount of time is needed to manually segment the regions-of-interest required to perform the 3D-CaSM used in Chapter 3. Therefore, in Chapter 4, I assessed the use of deep convolutional neural networks for automating the segmentation process for multiple knee joint tissues simultaneous and increase the time-efficiency for evaluating knee MR datasets. I evaluated the use of a conditional Generative Adversarial Network (cGAN) as a potentially improved method for automated segmentation compared to the widely used convolutional neural network, U-Net. In Chapter 5 I combined the 3D-CaSM and automated segmentation methods presented in Chapters 3 and 4, respectively to assess the use of fully automatic segmentations of femoral and tibial bone-cartilage structures for accurate surface-based analysis of cartilage morphology and composition on knee MR images. This was performed on publicly available data from the Osteoarthritis Initiative, a multicentre observational study with expert manual segmentations provided by the Zuse Institute in Berlin. Chapter 6 describes an automated pipeline for subchondral cortical bone thickness mapping from knee CT data. I developed a method of using automated segmentations of articular cartilage and bone from knee MRI data to determine the periarticular bone surface which is covered by cartilage. This surface was then used to perform cortical bone thickness measurements on corresponding CT data. I validated this pipeline using data from the EU-funded, multi-centre observational study called Applied Private-Public partneRship enabling OsteoArthritis Clinical Headway (APPROACH). Chapter 7 summarises the main conclusions and contributions of the works presented in this thesis as well as providing directions for future work.PhD Studentship funded by GlaxoSmithKlin

Mechanical Characterisation and Computational Modelling of Spinal Ligaments

Low back pain is a common complaint in people of all ages. The long-term success rates of many surgical devices to treat the spine have been relatively low and improved methods of pre-clinical testing of these devices are therefore needed. Sheep spine models are commonly employed in pre-clinical research studies for the evaluation of spinal devices. The anterior and posterior longitudinal ligaments (ALL and PLL) provide passive stability to the spine, however, limited studies have been conducted to characterise the mechanical properties of the ovine longitudinal ligaments or compare them to the human. Moreover, previous studies have derived material properties for the human ALL and PLL directly from force-displacement data, assuming uniform cross sectional area and length, and these values have been used extensively in finite element models of the spine for the analysis of clinical interventions. The aim of this study was to develop a methodology to test and compare the stiffness of human and ovine spinal longitudinal ligaments and to uniquely combine experimental and specimen-specific finite element (FE) modelling approaches to determine the ligament mechanical properties. The methodology was developed on ovine thoracic spines and then applied to human thoracic spines. The spines were dissected into functional spinal units (FSUs) with the posterior elements removed and imaged under micro computed tomography (µCT). The specimens were sectioned through the disc to leave only either the ALL or PLL intact and tested in tension to determine the stiffness. The µCT images from each FSU were used to build specimen-specific FE models of the ligaments and bony attachments. Hyper-elastic material models were used to represent the ligament behaviour. Initial values for the material model were derived using mean cross sectional area (CSA) and length (L), with the assumption that ligament was uniaxially loaded. The parameters were then iteratively changed until a best fit to the corresponding experimental load-displacement data was found for each specimen. The stiffness of the ligaments for the ovine specimens were found to be higher than for the human specimens. This may have implications for the use of ovine FSUs for preclinical testing of devices. There was poor agreement between the material parameters derived from FE models and the initial values derived by assuming a mean CSA and L. This work demonstrates that a specimen-specific image-based approach needs to be applied to derive the elastic properties of the ligaments due to their non-uniform shape and cross-sectional area

Modeling and Simulation in Engineering

This book provides an open platform to establish and share knowledge developed by scholars, scientists, and engineers from all over the world, about various applications of the modeling and simulation in the design process of products, in various engineering fields. The book consists of 12 chapters arranged in two sections (3D Modeling and Virtual Prototyping), reflecting the multidimensionality of applications related to modeling and simulation. Some of the most recent modeling and simulation techniques, as well as some of the most accurate and sophisticated software in treating complex systems, are applied. All the original contributions in this book are jointed by the basic principle of a successful modeling and simulation process: as complex as necessary, and as simple as possible. The idea is to manipulate the simplifying assumptions in a way that reduces the complexity of the model (in order to make a real-time simulation), but without altering the precision of the results

Analysis of MRI for Knee Osteoarthritis using Machine Learning

Approximately 8.5 million people in the UK (13.5% of the population) have osteoarthritis (OA) in one or both knees, with more than 6 million people in the UK suffering with painful osteoarthritis of the knee. In addition, an ageing population implies that an estimated 17 million people (twice as many as in 2012) are likely to be living with OA by 2030. Despite this, there exists no disease modifying drugs for OA and structural OA in MRI is poorly characterised. This motivates research to develop biomarkers and tools to aid osteoarthritis diagnosis from MRI of the knee. Previously many solutions for learning biomarkers have relied upon hand-crafted features to characterise and diagnose osteoarthritis from MRI. The methods proposed in this thesis are scalable and use machine learning to characterise large populations of the OAI dataset, with one experiment applying an algorithm to over 10,000 images. Studies of this size enable subtle characteristics of the dataset to be learnt and model many variations within a population. We present data-driven algorithms to learn features to predict OA from the appearance of the articular cartilage. An unsupervised manifold learning algorithm is used to compute a low dimensional representation of knee MR data which we propose as an imaging marker of OA. Previous metrics introduced for OA diagnosis are loosely based on the research communities intuition of the structural causes of OA progression, including morphological measures of the articular cartilage such as the thickness and volume. We demonstrate that there is a strong correlation between traditional morphological measures of the articular cartilage and the biomarkers identified using the manifold learning algorithm that we propose (R 2 = 0.75). The algorithm is extended to create biomarkers for different regions and sequences. A combination of these markers is proposed to yield a diagnostic imaging biomarker with superior performance. The diagnostic biomarkers presented are shown to improve upon hand-crafted morphological measure of disease status presented in the literature, a linear discriminant analysis (LDA) classification for early stage diagnosis of knee osteoarthritis results with an AUC of 0.9. From the biomarker discovery experiments we identified that intensity based affine registration of knee MRIs is not sufficiently robust for large scale image analysis, approximately 5% of these registrations fail. We have developed fast algorithms to compute robust affine transformations of knee MRI, which enables accurate pairwise registrations in large datasets. We model the population of images as a non-linear manifold, a registration is defined by the shortest geodesic path over the manifold representation. We identify sources of error in our manifold representation and propose fast mitigation strategies by checking for consistency across the manifold and by utilising multiple paths. These mitigation strategies are shown to improve registration accuracy and can be computed in less than 2 seconds with current architecture.Open Acces

Development of procedures for the design, optimization and manufacturing of customized orthopaedic and trauma implants: Geometrical/anatomical modelling from 3D medical imaging

Tese de Doutoramento (Programa Doutoral em Engenharia Biomédica)The introduction of imaging techniques in 1970 is one of the most relevant historical milestones in modern medicine. Medical imaging techniques have dramatically changed our understanding of the Human anatomy and physiology. The ability to non-invasively extract visual information allowed, not only the three-dimensional representation of the internal organs and musculo-skeletal system, but also the simulation of surgical procedures, the execution of computer aided surgeries, the development of more accurate biomechanical models, the development of custom-made implants, among others. The combination of the most advanced medical imaging systems with the most advanced CAD and CAM techniques, may allow the development of custom-made implants that meet patient-speci c traits. The geometrical and functional optimization of these devices may increase implant life-expectancy, especially in patients with marked deviations from the anatomical standards. In the implant customization protocol from medical image data, there are several steps that need to be followed in a sequential way, namely: Medical Image Processing and Recovering; Accurate Image Segmentation and 3D Surface Model Generation; Geometrical Customization based on CAD and CAE techniques; FEA Optimization of the Implant Geometry; and Manufacturing using CAD-CAM Technologies. This work aims to develop the necessary procedures for custom implant development from medical image data. This includes the extraction of highly accurate three-dimensional representation of the musculo-skeletal system from the Computed Tomography imaging, and the development of customized implants, given the speci c requirements of the target anatomy, and the applicable best practices found in the literature. A two-step segmentation protocol is proposed. In the rst step the region of interest is pre-segmented in order to obtain a good approximation to the desired geometry. Next, a fully automatic segmentation re nement is applied to obtain a more accurate representation of the target domain. The re nement step is composed by several sub-steps, more precisely, the recovery of the original image, considering the limiting resolution of the imaging system; image cropping; image interpolation; and segmentation re nement over the up-sampled domain. Highly accurate segmentations of the target domain were obtained with the proposed pipeline. The limiting factor to the accurate description of the domain accuracy is the image acquisition process, rather the following image processing, segmentation and surface meshing steps. The new segmentation pipeline was used in the development of three tailor-made implants, namely, a tibial nailing system, a mandibular implant, and a Total Hip Replacement system. Implants optimization is carried with Finite Element Analysis, considering the critical loading conditions that may be applied to each implant in working conditions. The new tibial nailing system is able of sustaining critical loads without implant failure; the new mandibular endoprosthesis that allows the recovery of the natural stress and strain elds observed in intact mandibles; and the Total Hip Replacement system that showed comparable strain shielding levels as commercially available stems. In summary, in the present thesis the necessary procedures for custom implant design are investigated, and new algorithms proposed. The guidelines for the characterization of the image acquisition, image processing, image segmentation and 3D reconstruction are presented and discussed. This new image processing pipeline is applied and validated in the development of the three abovementioned customized implants, for di erent medical applications and that satisfy speci c anatomical needs.Um dos principais marcos da história moderna da medicina e a introdução da imagem médica, em meados da década de 1970. As tecnologias de imagem permitiram aumentar e potenciar o nosso conhecimento acerca da anatomia e fisiologia do corpo Humano. A capacidade de obter informação imagiológica de forma não invasiva permitiu, não são a representação tridimensional de órgãos e do sistema músculo-esquelético, mas também a simulação de procedimentos cirúrgicos, a realização de cirurgias assistidas por computador, a criação de modelos biomecânicos mais realistas, a criação de implantes personalizados, entre outros. A conjugação dos sistemas mais avançados de imagem medica com as técnicas mais avançadas de modelação e maquinagem, pode permitir o desenvolvimento de implantes personalizados mais otimizados, que vão de encontro as especificidades de cada paciente. Por sua vez, a otimização geométrica e biomecânica destes dispositivos pode permitir, quer o aumento da sua longevidade, quer o tratamento de pessoas com estruturas anatómicas que se afastam dos padrões normais. O processo de modelação de implantes a partir da imagem medica passa por um conjunto de procedimentos a adotar, sequencialmente, ate ao produto final, a saber: Processamento e Recuperação de Imagem; Segmentação de Imagem e Reconstrução tridimensional da Região de Interesse; Modelação Geométrica do Implante; Simulação Numérica para a Otimização da Geometria; a Maquinagem do Implante. Este trabalho visa o desenvolvimento dos procedimentos necessários para a criação de implantes personalizados a partir da imagem medica, englobando a extração de modelos ósseos geométricos rigorosos a partir de imagens de Tomografia Computorizada e, a partir desses modelos, desenvolver implantes personalizados baseados nas melhores praticas existentes na literatura e que satisfaçam as especificidades da anatomia do paciente. Assim, apresenta-se e discute-se um novo procedimento de segmentação em dois passos. No primeiro e feita uma pre-segmentação que visa obter uma aproximação iniciala região de interesse. De seguida, um procedimento de refinamento da segmentação totalmente automático e aplicada a segmentação inicial para obter uma descrição mais precisa do domínio de interesse. O processo de refinamento da segmentação e constituído por vários procedimentos, designadamente: recuperação da imagem original, tendo em consideração a resolução limitante do sistema de imagem; o recorte da imagem na vizinhança da região pre-segmentada; a interpolação da região de interesse; e o refinamento da segmentação aplicando a técnica de segmentação Level-Sets sobre o domínio interpolado. O procedimento de segmentação permitiu extrair modelos extremamente precisos a partir da informação imagiológica. Os resultados revelam que o fator limitante a descrição do domínio e o processo de aquisição de imagem, em detrimento dos diversos passos de processamento subsequentes. O novo protocolo de segmentação foi utilizado no desenvolvimento de três implantes personalizados, a saber: um sistema de fixação interna para a tíbia; um implante mandibular; e um sistema para a Reconstrução Total da articulação da Anca. A otimização do comportamento mecânico dos implantes foi feita utilizado o Método dos Elementos Finitos, tendo em conta os carregamentos críticos a que estes podem estar sujeitos durante a sua vida útil. O sistema de fixação interna para a tíbia e capaz de suportar os carregamentos críticos, sem que a sua integridade mecânica seja comprometida; o implante mandibular permite recuperar os campos de tensão e deformação observados em mandíbulas intactas; e a Prótese Total da Anca apresenta níveis de strain shielding ao longo do fémur proximal comparáveis com os níveis observados em dispositivos comercialmente disponíveis. Em suma, nesta tese de Doutoramento são investigados e propostos novos procedimentos para o projeto de implantes feitos por medida. São apresentadas e discutidas as linhas orientadoras para a caracterização precisa do sistema de aquisição de imagem, para o processamento de imagem, para a segmentação, e para a reconstrução 3D das estruturas anatómicas a partir da imagem medica. Este conjunto de linhas orientadoras é aplicado e validado no desenvolvimento de três implantes personalizados, citados anteriormente, para aplicações médicas distintas e que satisfazem as necessidades anatómicas específicas de cada paciente.Fundação para a Ciência e Tecnologia (FCT

Optimization of Operation Sequencing in CAPP Using Hybrid Genetic Algorithm and Simulated Annealing Approach

In any CAPP system, one of the most important process planning functions is selection of the operations and corresponding machines in order to generate the optimal operation sequence. In this paper, the hybrid GA-SA algorithm is used to solve this combinatorial optimization NP (Non-deterministic Polynomial) problem. The network representation is adopted to describe operation and sequencing flexibility in process planning and the mathematical model for process planning is described with the objective of minimizing the production time. Experimental results show effectiveness of the hybrid algorithm that, in comparison with the GA and SA standalone algorithms, gives optimal operation sequence with lesser computational time and lesser number of iterations

Optimization of Operation Sequencing in CAPP Using Hybrid Genetic Algorithm and Simulated Annealing Approach

In any CAPP system, one of the most important process planning functions is selection of the operations and corresponding machines in order to generate the optimal operation sequence. In this paper, the hybrid GA-SA algorithm is used to solve this combinatorial optimization NP (Non-deterministic Polynomial) problem. The network representation is adopted to describe operation and sequencing flexibility in process planning and the mathematical model for process planning is described with the objective of minimizing the production time. Experimental results show effectiveness of the hybrid algorithm that, in comparison with the GA and SA standalone algorithms, gives optimal operation sequence with lesser computational time and lesser number of iterations