Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

2D/3D image registration to align a 3D volume and 2D X-ray images is a challenging problem due to its ill-posed nature and various artifacts presented in 2D X-ray images. In this paper, we propose a multi-agent system with an auto attention mechanism for robust and efficient 2D/3D image registration. Specifically, an individual agent is trained with dilated Fully Convolutional Network (FCN) to perform registration in a Markov Decision Process (MDP) by observing a local region, and the final action is then taken based on the proposals from multiple agents and weighted by their corresponding confidence levels. The contributions of this paper are threefold. First, we formulate 2D/3D registration as a MDP with observations, actions, and rewards properly defined with respect to X-ray imaging systems. Second, to handle various artifacts in 2D X-ray images, multiple local agents are employed efficiently via FCN-based structures, and an auto attention mechanism is proposed to favor the proposals from regions with more reliable visual cues. Third, a dilated FCN-based training mechanism is proposed to significantly reduce the Degree of Freedom in the simulation of registration environment, and drastically improve training efficiency by an order of magnitude compared to standard CNN-based training method. We demonstrate that the proposed method achieves high robustness on both spine cone beam Computed Tomography data with a low signal-to-noise ratio and data from minimally invasive spine surgery where severe image artifacts and occlusions are presented due to metal screws and guide wires, outperforming other state-of-the-art methods (single agent-based and optimization-based) by a large margin.Comment: AAAI 201

Medical image registration by neural networks: a regression-based registration approach

This thesis focuses on the development and evaluation of a registration-by-regression approach for the 3D/2D registration of coronary Computed Tomography Angiography (CTA) and X-ray angiography. This regression-based method relates image features of 2D projection images to the transformation parameters of the 3D image by a nonlinear regression. It treats registration as a regression problem, as an alternative for the traditional iterative approach that often comes with high computational costs and limited capture range. First we presented a survey of the methods with a regression-based registration approach for medical applications, as well as a summary of their main characteristics (Chapter 2). Second, we studied the registration methodology, addressing the input features and the choice of regression model (Chapter 3 and Chapter 4). For that purpose, we evaluated different options using simulated X-ray images generated from coronary artery tree models derived from 3D CTA scans. We also compared the registration-by-regression results with a method based on iterative optimization. Different image features of 2D projections and seven regression techniques were considered. The regression approach for simulated X-rays was shown to be slightly less accurate, but much more robust than the method based on an iterative optimization approach. Neural Networks obtained accurate results and showed to be robust to large initial misalignment. Third, we evaluated the registration-by-regression method using clinical data, integrating the 3D preoperative CTA of the coronary arteries with intraoperative 2D X-ray angiography images (Chapter 5). For the evaluation of the image registration, a gold standard registration was established using an exhaustive search followed by a multi-observer visual scoring procedure. The influence of preprocessing options for the simulated images and the real X-rays was studied. Several image features were also compared. The coronary registration–by-regression results were not satisfactory, resembling manual initialization accuracy. Therefore, the proposed method for this concrete problem and in its current configuration is not sufficiently accurate to be used in the clinical practice. The framework developed enables us to better understand the dependency of the proposed method on the differences between simulated and real images. The main difficulty lies in the substantial differences in appearance between the images used for training (simulated X-rays from 3D coronary models) and the actual images obtained during the intervention (real X-ray angiography). We suggest alternative solutions and recommend to evaluate the registration-by-regression approach in other applications where training data is available that has similar appearance to the eventual test data

Multi-modality cardiac image computing: a survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future

Évaluation d'une technique de compensation du mouvement des artères coronaires à partir d'une séquence angiographique

Une méthode de compensation du mouvement cardiorespiratoire à partir d’un biplan ou d’un monoplan angiographique est proposée. Le mouvement est estimé à partir d’un recalage 3D/2D. Le modèle 3D des artères coronaires est extrait par triangulation à partir de deux vues angiographiques avant l’intervention minimalement invasive. Les paramètres des caméras sont d’abord calibrés par une optimisation non linéaire où l’erreur de reprojection est minimisée. La sélection des artères est réalisée manuellement et la correspondance se calcule à partir de l’intersection de la ligne centrale et la droite épipolaire sur la vue associée. Avec le modèle 3D, le mouvement cardiorespiratoire est compensé à l’aide d’un filtre particulaire. Chaque particule du système correspond aux paramètres de déformations affines appliqués à la projection 2D. Cette projection est comparée à la distance du centre des artères observée sur les angiographies. Cette distance s’obtient par le traitement des images médicales avec une série de filtres. Un filtre de vascularité est d’abord appliqué, suivi par des filtres de débruitage et de binarisation. Finalement, la transformée de distance modifiée est calculée. Un algorithme est aussi présenté afin de segmenter les bifurcations des artères coronaires tout en minimisant le bruit. Les particules sont échantillonnées en favorisant celles minimisant la distance entre les artères et les bifurcations. De manière itérative, le filtre de particules recommence le processus de génération, d’évaluation et d’échantillonnage des particules en contraignant davantage les paramètres de déformation. Un simulateur de mouvement cardiorespiratoire est utilisé en deux temps. Il sert d’abord à construire un gabarit de paramètres de déformation servant à limiter l’espace de génération des particules du système durant tout le cycle cardiaque. Grâce aux positions 3D connues des artères, le simulateur sert aussi à mesurer et valider l’efficacité de la méthode de compensation du mouvement cardiorespiratoire suggérée

Modelo de sistema de soporte a la diagnosis de trastornos osteoarticulares de miembros inferiores utilizando procesamiento de imágenes de rayos X

Los trastornos osteoarticulares aquejan a personas de todas las regiones del mundo sin distinción, ejemplos de ellas son: la osteoporosis y atrosis. La OMS determina la existencia de un incremento de casos en sociedades socioeconómicas más bajas y la Unión Europea establece una estrategia enfocada a entregar salud personalizada en el momento correcto, y brindar una alternativa de prevención oportuna y especifica denominada (PerMed). En este contexto nuestro país necesita aplicar la Medicina Personalizada para diagnosticar a tiempo enfermedades con alta incidencia. La presente investigación busca alinearse a los objetivos de la Medicina Personalizada proporcionando un modelo de sistema de soporte a la diagnosis de trastornos osteoarticulares de miembros inferiores utilizando procesamiento de imágenes de rayos X, teniendo presente la confidencialidad y protección de los datos. El pre-procesamiento de las imágenes de rayos X, permitió eliminar los desafíos de estas imágenes, y posibilito la generación de un gold-standard que sirvió como guía para la segmentación-registro de las estructuras óseas de miembros inferiores. Se utilizaron los modelos estadísticos como: SSM - Statistical Shape Model, SAM – Statistical Appeareance Model, ASM - Active Shape Model y Gradient Profiling en el refinamiento de la etapa de segmentación-registro como parte del entrenamiento y prueba. Estos modelos han sido validados con artículos de investigación presentados en el Capítulo IV con resultados de precisión en la segmentación entre el 74 % y 83 % y para la clasificación de las estructuras óseas dependiendo del objetivo a resolver sea: a) detectar fracturas en el acetábulo, o b) detectar osteoporosis en el fémur proximal, los resultados obtuvieron una precisión de: 73% y 87% respectivamente; y por ultimo para lograr el objetivo de: c) medir la distancia articular, se obtiene un error promedio equivalente a 2.4 px, este es un error aceptable para respaldar el diagnostico de desgaste articular de cadera llamado "osteoartritis de cadera". Asimismo, hubo una mejora significativa en el tiempo de procesamiento comparado con la literatura analizada

Registration-by-regression of coronary CTA and X-ray angiography

We evaluate the integration of 3D preoperative computed tomography angiography of the coronary arteries with intraoperative 2D X-ray angiographies by a recently proposed novel registration-by-regression method. The method relates image features of 2D projection images to the transformation parameters of the 3D image. We compared different sets of features and studied the influence of preprocessing the training set. For the registration evaluation, a gold standard was developed from eight X-ray angiography sequences from six different patients. The alignment quality was measured using the 3D mean target registration error (mTRE). The registration-by-regression method achieved moderate accuracy (median mTRE of 15 mm) on real images. It does therefore not provide yet a complete solution to the 3D–2D registration problem but it could be used as an initialisation method to eliminate the need for manual initialisation.info:eu-repo/semantics/publishedVersio