3D approximation of scapula bone shape from 2D X-ray images using landmark-constrained statistical shape model fitting

Two-dimensional X-ray imaging is the dominant imaging modality in low-resource countries despite the existence of three-dimensional (3D) imaging modalities. This is because fewer hospitals in low-resource countries can afford the 3D imaging systems as their acquisition and operation costs are higher. However, 3D images are desirable in a range of clinical applications, for example surgical planning. The aim of this research was to develop a tool for 3D approximation of scapula bone from 2D X-ray images using landmark-constrained statistical shape model fitting. First, X-ray stereophotogrammetry was used to reconstruct the 3D coordinates of points located on 2D X-ray images of the scapula, acquired from two perspectives. A suitable calibration frame was used to map the image coordinates to their corresponding 3D realworld coordinates. The 3D point localization yielded average errors of (0.14, 0.07, 0.04) mm in the X, Y and Z coordinates respectively, and an absolute reconstruction error of 0.19 mm. The second phase assessed the reproducibility of the scapula landmarks reported by Ohl et al. (2010) and Borotikar et al. (2015). Only three (the inferior angle, acromion and the coracoid process) of the eight reproducible landmarks considered were selected as these were identifiable from the two different perspectives required for X-ray stereophotogrammetry in this project. For the last phase, an approximation of a scapula was produced with the aid of a statistical shape model (SSM) built from a training dataset of 84 CT scapulae. This involved constraining an SSM to the 3D reconstructed coordinates of the selected reproducible landmarks from 2D X-ray images. Comparison of the approximate model with a CT-derived ground truth 3D segmented volume resulted in surface-to-surface average distances of 4.28 mm and 3.20 mm, using three and sixteen landmarks respectively. Hence, increasing the number of landmarks produces a posterior model that makes better predictions of patientspecific reconstructions. An average Euclidean distance of 1.35 mm was obtained between the three selected landmarks on the approximation and the corresponding landmarks on the CT image. Conversely, a Euclidean distance of 5.99 mm was obtained between the three selected landmarks on the original SSM and corresponding landmarks on the CT image. The Euclidean distances confirm that a posterior model moves closer to the CT image, hence it reduces the search space for a more exact patient-specific 3D reconstruction by other fitting algorithms

A deep learning algorithm for contour detection in synthetic 2D biplanar X-ray images of the scapula: towards improved 3D reconstruction of the scapula

Three-dimensional (3D) reconstruction from X-ray images using statistical shape models (SSM) provides a cost-effective way of increasing the diagnostic utility of two-dimensional (2D) X-ray images, especially in low-resource settings. The landmark-constrained model fitting approach is one way to obtain patient-specific models from a statistical model. This approach requires an accurate selection of corresponding features, usually landmarks, from the bi-planar X-ray images. However, X-ray images are 2D representations of 3D anatomy with super-positioned structures, which confounds this approach. The literature shows that detection and use of contours to locate corresponding landmarks within biplanar X-ray images can address this limitation. The aim of this research project was to train and validate a deep learning algorithm for detection the contour of a scapula in synthetic 2D bi-planar Xray images. Synthetic bi-planar X-ray images were obtained from scapula mesh samples with annotated landmarks generated from a validated SSM obtained from the Division of Biomedical Engineering, University of Cape Town. This was followed by the training of two convolutional neural network models as the first objective of the project; the first model was trained to predict the lateral (LAT) scapula image given the anterior-posterior (AP) image. The second model was trained to predict the AP image given the LAT image. The trained models had an average Dice coefficient value of 0.926 and 0.964 for the predicted LAT and AP images, respectively. However, the trained models did not generalise to the segmented real X-ray images of the scapula. The second objective was to perform landmark-constrained model fitting using the corresponding landmarks embedded in the predicted images. To achieve this objective, the 2D landmark locations were transformed into 3D coordinates using the direct linear transformation. The 3D point localization yielded average errors of (0.35, 0.64, 0.72) mm in the X, Y and Z directions, respectively, and a combined coordinate error of 1.16 mm. The reconstructed landmarks were used to reconstruct meshes that had average surface-to-surface distances of 3.22 mm and 1.72 mm for 3 and 6 landmarks, respectively. The third objective was to reconstruct the scapula mesh using matching points on the scapula contour in the bi-planar images. The average surface-to-surface distances of the reconstructed meshes with 8 matching contour points and 6 corresponding landmarks of the same meshes were 1.40 and 1.91 mm, respectively. In summary, the deep learning models were able to learn the mapping between the bi-planar images of the scapula. Increasing the number of corresponding landmarks from the bi-planar images resulted into better 3D reconstructions. However, obtaining these corresponding landmarks was non-trivial, necessitating the use of matching points selected from the scapulae contours. The results from the latter approach signal a need to explore contour matching methods to obtain more corresponding points in order to improve the scapula 3D reconstruction using landmark-constrained model fitting

Three Dimensional Nonlinear Statistical Modeling Framework for Morphological Analysis

This dissertation describes a novel three-dimensional (3D) morphometric analysis framework for building statistical shape models and identifying shape differences between populations. This research generalizes the use of anatomical atlases on more complex anatomy as in case of irregular, flat bones, and bones with deformity and irregular bone growth. The foundations for this framework are: 1) Anatomical atlases which allow the creation of homologues anatomical models across populations; 2) Statistical representation for output models in a compact form to capture both local and global shape variation across populations; 3) Shape Analysis using automated 3D landmarking and surface matching. The proposed framework has various applications in clinical, forensic and physical anthropology fields. Extensive research has been published in peer-reviewed image processing, forensic anthropology, physical anthropology, biomedical engineering, and clinical orthopedics conferences and journals. The forthcoming discussion of existing methods for morphometric analysis, including manual and semi-automatic methods, addresses the need for automation of morphometric analysis and statistical atlases. Explanations of these existing methods for the construction of statistical shape models, including benefits and limitations of each method, provide evidence of the necessity for such a novel algorithm. A novel approach was taken to achieve accurate point correspondence in case of irregular and deformed anatomy. This was achieved using a scale space approach to detect prominent scale invariant features. These features were then matched and registered using a novel multi-scale method, utilizing both coordinate data as well as shape descriptors, followed by an overall surface deformation using a new constrained free-form deformation. Applications of output statistical atlases are discussed, including forensic applications for the skull sexing, as well as physical anthropology applications, such as asymmetry in clavicles. Clinical applications in pelvis reconstruction and studying of lumbar kinematics and studying thickness of bone and soft tissue are also discussed

Towards a framework for multi class statistical modelling of shape, intensity, and kinematics in medical images

Statistical modelling has become a ubiquitous tool for analysing of morphological variation of bone structures in medical images. For radiological images, the shape, relative pose between the bone structures and the intensity distribution are key features often modelled separately. A wide range of research has reported methods that incorporate these features as priors for machine learning purposes. Statistical shape, appearance (intensity profile in images) and pose models are popular priors to explain variability across a sample population of rigid structures. However, a principled and robust way to combine shape, pose and intensity features has been elusive for four main reasons: 1) heterogeneity of the data (data with linear and non-linear natural variation across features); 2) sub-optimal representation of three-dimensional Euclidean motion; 3) artificial discretization of the models; and 4) lack of an efficient transfer learning process to project observations into the latent space. This work proposes a novel statistical modelling framework for multiple bone structures. The framework provides a latent space embedding shape, pose and intensity in a continuous domain allowing for new approaches to skeletal joint analysis from medical images. First, a robust registration method for multi-volumetric shapes is described. Both sampling and parametric based registration algorithms are proposed, which allow the establishment of dense correspondence across volumetric shapes (such as tetrahedral meshes) while preserving the spatial relationship between them. Next, the framework for developing statistical shape-kinematics models from in-correspondence multi-volumetric shapes embedding image intensity distribution, is presented. The framework incorporates principal geodesic analysis and a non-linear metric for modelling the spatial orientation of the structures. More importantly, as all the features are in a joint statistical space and in a continuous domain; this permits on-demand marginalisation to a region or feature of interest without training separate models. Thereafter, an automated prediction of the structures in images is facilitated by a model-fitting method leveraging the models as priors in a Markov chain Monte Carlo approach. The framework is validated using controlled experimental data and the results demonstrate superior performance in comparison with state-of-the-art methods. Finally, the application of the framework for analysing computed tomography images is presented. The analyses include estimation of shape, kinematic and intensity profiles of bone structures in the shoulder and hip joints. For both these datasets, the framework is demonstrated for segmentation, registration and reconstruction, including the recovery of patient-specific intensity profile. The presented framework realises a new paradigm in modelling multi-object shape structures, allowing for probabilistic modelling of not only shape, but also relative pose and intensity as well as the correlations that exist between them. Future work will aim to optimise the framework for clinical use in medical image analysis

Articulated Statistical Shape Modelling of the Shoulder Joint

The shoulder joint is the most mobile and unstable joint in the human body. This makes it vulnerable to soft tissue pathologies and dislocation. Insight into the kinematics of the joint may enable improved diagnosis and treatment of different shoulder pathologies. Shoulder joint kinematics can be influenced by the articular geometry of the joint. The aim of this project was to develop an analysis framework for shoulder joint kinematics via the use of articulated statistical shape models (ASSMs). Articulated statistical shape models extend conventional statistical shape models by combining the shape variability of anatomical objects collected from different subjects (statistical shape models), with the physical variation of pose between the same objects (articulation). The developed pipeline involved manual annotation of anatomical landmarks selected on 3D surface meshes of scapulae and humeri and establishing dense surface correspondence across these data through a registration process. The registration was performed using a Gaussian process morphable model fitting approach. In order to register two objects separately, while keeping their shape and kinematics relationship intact, one of the objects (scapula) was fixed leaving the other (humerus) to be mobile. All the pairs of registered humeri and scapulae were brought back to their native imaged position using the inverse of the associated registration transformation. The glenohumeral rotational center and local anatomic coordinate system of the humeri and scapulae were determined using the definitions suggested by the International Society of Biomechanics. Three motions (flexion, abduction, and internal rotation) were generated using Euler angle sequences. The ASSM of the model was built using principal component analysis and validated. The validation results show that the model adequately estimated the shape and pose encoded in the training data. Developing ASSM of the shoulder joint helps to define the statistical shape and pose parameters of the gleno humeral articulating surfaces. An ASSM of the shoulder joint has potential applications in the analysis and investigation of population-wide joint posture variation and kinematics. Such analyses may include determining and quantifying abnormal articulation of the joint based on the range of motion; understanding of detailed glenohumeral joint function and internal joint measurement; and diagnosis of shoulder pathologies. Future work will involve developing a protocol for encoding the shoulder ASSM with real, rather than handcrafted, pose variation

Computer Aided Tools for the Design and Planning of Personalized Shoulder Arthroplasty

La artroplastia de hombro es el tercer procedimiento de reemplazo articular más común, después de la artroplastia de rodilla y cadera, y actualmentees el de más rápido crecimiento en el campo ortopédico. Las principales opciones quirúrgicas incluyen la artroplastia total de hombro (TSA), en la quese restaura la anatomía articular normal, y, para pacientes con un manguito rotador completamente desgarrado, la artroplastia inversa de hombro (RSA), en la que la bola y la cavidad de la articulación glenohumeral se cambian. A pesar del progreso reciente y los avances en el diseño, las tasas de complicaciones reportadas para RSA son más altas que las de la artroplastia de hombro convencional. Un enfoque específico para el paciente, en el que los médicos adaptan el tratamiento quirúrgico a las características del mismo y al estado preoperatorio, por ejemplo mediante implantes personalizados y planificación previa, puede ayudar a reducir los problemas postoperatorios y mejorar el resultado funcional. El objetivo principal de esta tesis es desarrollar y evaluar métodos novedosos para RSA personalizado, utilizando tecnologías asistidas por ordenador de última generación para estandarizar y automatizar las fases de diseño y planificación.Los implantes personalizados son una solución adecuada para el tratamiento de pacientes con pérdida extensa de hueso glenoideo. Sin embargo, los ingenieros clínicos se enfrentan a muchas variables en el diseño de implantes (número y tipo de tornillos, superficie de contacto, etc.) y una gran variabilidad anatómica y patológica. Actualmente, no existen herramientas objetivas para guiarlos a la hora de elegir el diseño óptimo, es decir, con suficiente estabilidad inicial del implante, lo que hace que el proceso de diseño sea tedioso, lento y dependiente del usuario. En esta tesis, se desarrolló una simulación de Virtual Bench Test (VBT) utilizando un modelo de elementos finitos para evaluar automáticamente la estabilidad inicial de los implantes de hombro personalizados. A través de un experimento de validación, se demostró que los ingenieros clínicos pueden utilizar el resultado de Virtual Bench Test como referencia para respaldar sus decisiones y adaptaciones durante el proceso de diseño del implante.Al diseñar implantes de hombro, el conocimiento de la morfología y la calidad ósea de la escápula en toda la población es fundamental. En particular, se tienen en cuenta las regiones con la mejor reserva ósea (hueso cortical) para definir la posición y orientación de los orificios de los tornillos, mientras se busca una fijación óptima. Como alternativa a las mediciones manuales, cuya generalización está limitada por el análisis de pequeños subconjuntos de pacientes potenciales, Statistical Shape Models (SSMs) se han utilizado comúnmente para describir la variabilidad de la forma dentro de una población. Sin embargo, estos SSMs normalmente no contienen información sobre el grosor cortical.Por lo tanto, se desarrolló una metodología para combinar la forma del hueso escapular y la morfología de la cortical en un SSM. Primero, se presentó y evaluó un método para estimar el espesor cortical, a partir de un análisis de perfil de Hounsfield Unit (HU). Luego, utilizando 32 escápulas sanas segmentadas manualmente, se creó y evaluó un modelo de forma estadística que incluía información de la cortical. La herramienta desarrollada se puede utilizar para implantar virtualmente un nuevo diseño y probar su congruencia dentro de una población virtual generada, reduciendo así el número de iteraciones de diseño y experimentos con cadáveres.Las mediciones del alargamiento de los músculos deltoides y del manguito rotador durante la planificación quirúrgica pueden ayudar a los médicos aseleccionar un diseño y una posición de implante adecuados. Sin embargo, tal evaluación requiere la indicación de puntos anatómicos como referencia para los puntos de unión de los músculos, un proceso que requiere mucho tiempo y depende del usuario, ya que a menudo se realiza manualmente. Además, las imágenes médicas, que se utilizan normalmente para la artroplastia de hombro,contienen en su mayoría solo el húmero proximal, lo que hace imposible indicarlos puntos de unión de los músculos que se encuentran fuera del campo de visión de la exploración. Por lo tanto, se desarrolló y evaluó un método totalmente automatizado, basado en SSM, para medir la elongación del deltoides y del manguito rotador. Su aplicabilidad clínica se demostró mediante la evaluación del rendimiento de la estimación automatizada de la elongación muscular para un conjunto de articulaciones artríticas del hombro utilizadas para la planificación preoperatoria de RSA, lo que confirma que es una herramienta adecuada para los cirujanos a la hora de evaluar y refinar las decisiones clínicas.En esta investigación, se dio un paso importante en la dirección de un enfoque más personalizado de la artroplastia inversa de hombro, en el que el manejo quirúrgico, es decir, el diseño y la posición del implante, se adapta a las características específicas del paciente y al estado preoperatorio. Al aplicar tecnologías asistidas por computadora en la práctica clínica, el proceso de diseño y planificación se puede automatizar y estandarizar, reduciendo así los costos y los plazos de entrega. Además, gracias a los métodos novedosos presentados en esta tesis, esperamos en el futuro una adopción más amplia del enfoque personalizado, con importantes beneficios tanto para los cirujanos como para los pacientes.Shoulder arthroplasty is the third most common joint replacement procedure, after knee and hip arthroplasty, and currently the most rapidly growing one in the orthopaedic field. The main surgical options include total shoulder arthroplasty (TSA), in which the normal joint anatomy is restored, and, for patients with a completely torn rotator cuff, reverse shoulder arthroplasty (RSA), in which the ball and the socket of the glenohumeral joint are switched. Despite the recent progress and advancement in design, the reported rates of complication for RSA are higher than those of conventional shoulder arthroplasty. A patient-specific approach, in which clinicians adapt the surgical management to patient characteristics and preoperative condition, e.g. through custom implants and pre-planning, can help to reduce postoperative problems and improve the functional outcome. The main goal of this thesis is to develop and evaluate novel methods for personalized RSA, using state-of-the-art computer aided technologies to standardize and automate the design and planning phases. Custom implants are a suitable solution when treating patients with extensive glenoid bone loss. However, clinical engineers are confronted with an enormous implant design space (number and type of screws, contact surface, etc.) and large anatomical and pathological variability. Currently, no objective tools exist to guide them when choosing the optimal design, i.e. with sufficient initial implant stability, thus making the design process tedious, time-consuming, and user-dependent. In this thesis, a Virtual Bench Test (VBT) simulation was developed using a finite element model to automatically evaluate the initial stability of custom shoulder implants. Through a validation experiment, it was shown that the virtual test bench output can be used by clinical engineers as a reference to support their decisions and adaptations during the implant design process. When designing shoulder implants, knowledge about bone morphology and bone quality of the scapula throughout a certain population is fundamental. In particular, regions with the best bone stock (cortical bone) are taken into account to define the position and orientation of the screw holes, while aiming for an optimal fixation. As an alternative to manual measurements, whose generalization is limited by the analysis of small sub-sets of the potential patients, Statistical Shape Models (SSMs) have been commonly used to describe shape variability within a population. However, these SSMs typically do not contain information about cortical thickness. Therefore, a methodology to combine scapular bone shape and cortex morphology in an SSM was developed. First, a method to estimate cortical thickness, starting from a profile analysis of Hounsfield Unit (HU), was presented and evaluated. Then, using 32 manually segmented healthy scapulae, a statistical shape model including cortical information was created and assessed. The developed tool can be used to virtually implant a new design and test its congruency inside a generated virtual population, thus reducing the number of design iterations and cadaver labs. Measurements of deltoid and rotator cuff muscle elongation during surgical planning can help clinicians to select a suitable implant design and position. However, such an assessment requires the indication of anatomical landmarks as a reference for the muscle attachment points, a process that is time-consuming and user-dependent, since often performed manually. Additionally, the medical images, which are normally used for shoulder arthroplasty, mostly contain only the proximal humerus, making it impossible to indicate those muscle attachment points which lie outside of the field of view of the scan. Therefore, a fully-automated method, based on SSM, for measuring deltoid and rotator cuff elongation was developed and evaluated. Its clinical applicability was demonstrated by assessing the performance of the automated muscle elongation estimation for a set of arthritic shoulder joints used for preoperative planning of RSA, thus confirming it a suitable tool for surgeons when evaluating and refining clinical decisions. In this research, a major step was taken into the direction of a more personalized approach to Reverse Shoulder Arthroplasty, in which the surgical management, i.e. implant design and position, is adapted to the patient-specific characteristics and preoperative condition. By applying computer aided technologies in the clinical practice, design and planning process can be automated and standardized, thus reducing costs and lead times. Additionally, thanks to the novel methods presented in this thesis, we expect in the future a wider adoption of the personalized approach, with important benefits both for surgeons and patients.<br /

Applications of a Biomechanical Patient Model for Adaptive Radiation Therapy

Biomechanical patient modeling incorporates physical knowledge of the human anatomy into the image processing that is required for tracking anatomical deformations during adaptive radiation therapy, especially particle therapy. In contrast to standard image registration, this enforces bio-fidelic image transformation. In this thesis, the potential of a kinematic skeleton model and soft tissue motion propagation are investigated for crucial image analysis steps in adaptive radiation therapy. The first application is the integration of the kinematic model in a deformable image registration process (KinematicDIR). For monomodal CT scan pairs, the median target registration error based on skeleton landmarks, is smaller than (1.6 ± 0.2) mm. In addition, the successful transferability of this concept to otherwise challenging multimodal registration between CT and CBCT as well as CT and MRI scan pairs is shown to result in median target registration error in the order of 2 mm. This meets the accuracy requirement for adaptive radiation therapy and is especially interesting for MR-guided approaches. Another aspect, emerging in radiotherapy, is the utilization of deep-learning-based organ segmentation. As radiotherapy-specific labeled data is scarce, the training of such methods relies heavily on augmentation techniques. In this work, the generation of synthetically but realistically deformed scans used as Bionic Augmentation in the training phase improved the predicted segmentations by up to 15% in the Dice similarity coefficient, depending on the training strategy. Finally, it is shown that the biomechanical model can be built-up from automatic segmentations without deterioration of the KinematicDIR application. This is essential for use in a clinical workflow

Automatic image analysis of C-arm Computed Tomography images for ankle joint surgeries

Open reduction and internal fixation is a standard procedure in ankle surgery for treating a fractured fibula. Since fibula fractures are often accompanied by an injury of the syndesmosis complex, it is essential to restore the correct relative pose of the fibula relative to the adjoining tibia for the ligaments to heal. Otherwise, the patient might experience instability of the ankle leading to arthritis and ankle pain and ultimately revision surgery. Incorrect positioning referred to as malreduction of the fibula is assumed to be one of the major causes of unsuccessful ankle surgery. 3D C-arm imaging is the current standard procedure for revealing malreduction of fractures in the operating room. However, intra-operative visual inspection of the reduction result is complicated due to high inter-individual variation of the ankle anatomy and rather based on the subjective experience of the surgeon. A contralateral side comparison with the patient’s uninjured ankle is recommended but has not been integrated into clinical routine due to the high level of radiation exposure it incurs. This thesis presents the first approach towards a computer-assisted intra-operative contralateral side comparison of the ankle joint. The focus of this thesis was the design, development and validation of a software-based prototype for a fully automatic intra-operative assistance system for orthopedic surgeons. The implementation does not require an additional 3D C-arm scan of the uninjured ankle, thus reducing time consumption and cumulative radiation dose. A 3D statistical shape model (SSM) is used to reconstruct a 3D surface model from three 2D fluoroscopic projections representing the uninjured ankle. To this end, a 3D SSM segmentation is performed on the 3D image of the injured ankle to gain prior knowledge of the ankle. A 3D convolutional neural network (CNN) based initialization method was developed and its outcome was incorporated into the SSM adaption step. Segmentation quality was shown to be improved in terms of accuracy and robustness compared to the pure intensity-based SSM. This allows us to overcome the limitations of the previously proposed methods, namely inaccuracy due to metal artifacts and the lack of device-to-patient orientation of the C-arm. A 2D-CNN is employed to extract semantic knowledge from all fluoroscopic projection images. This step of the pipeline both creates features for the subsequent reconstruction and also helps to pre-initialize the 3D-SSM without user interaction. A 2D-3D multi-bone reconstruction method has been developed which uses distance maps of the 2D features for fast and accurate correspondence optimization and SSM adaption. This is the central and most crucial component of the workflow. This is the first time that a bone reconstruction method has been applied to the complex ankle joint and the first reconstruction method using CNN based segmentations as features. The reconstructed 3D-SSM of the uninjured ankle can be back-projected and visualized in a workflow-oriented manner to procure clear visualization of the region of interest, which is essential for the evaluation of the reduction result. The surgeon can thus directly compare an overlay of the contralateral ankle with the injured ankle. The developed methods were evaluated individually using data sets acquired during a cadaver study and representative clinical data acquired during fibular reduction. A hierarchical evaluation was designed to assess the inaccuracies of the system on different levels and to identify major sources of error. The overall evaluation performed on eleven challenging clinical datasets acquired for manual contralateral side comparison showed that the system is capable of accurately reconstructing 3D surface models of the uninjured ankle solely using three projection images. A mean Hausdorff distance of 1.72 mm was measured when comparing the reconstruction result to the ground truth segmentation and almost achieved the high required clinical accuracy of 1-2 mm. The overall error of the pipeline was mainly attributed to inaccuracies in the 2D-CNN segmentation. The consistency of these results requires further validation on a larger dataset. The workflow proposed in this thesis establishes the first approach to enable automatic computer-assisted contralateral side comparison in ankle surgery. The feasibility of the proposed approach was proven on a limited amount of clinical cases and has already yielded good results. The next important step is to alleviate the identified bottlenecks in the approach by providing more training data in order to further improve the accuracy. In conclusion, the new approach presented gives the chance to guide the surgeon during the reduction process, improve the surgical outcome while avoiding additional radiation exposure and reduce the number of revision surgeries in the long term