Fully automated accurate patient positioning in computed tomography using anterior-posterior localizer images and a deep neural network: a dual-center study

Objectives This study aimed to improve patient positioning accuracy by relying on a CT localizer and a deep neural network to optimize image quality and radiation dose. Methods We included 5754 chest CT axial and anterior–posterior (AP) images from two diferent centers, C1 and C2. After pre-processing, images were split into training (80%) and test (20%) datasets. A deep neural network was trained to generate 3D axial images from the AP localizer. The geometric centerlines of patient bodies were indicated by creating a bounding box on the predicted images. The distance between the body centerline, estimated by the deep learning model and ground truth (BCAP), was compared with patient mis-centering during manual positioning (BCMP). We evaluated the performance of our model in terms of distance between the lung centerline estimated by the deep learning model and the ground truth (LCAP). Results The error in terms of BCAP was − 0.75 ± 7.73 mm and 2.06 ± 10.61 mm for C1 and C2, respectively. This error was signifcantly lower than BCMP, which achieved an error of 9.35 ± 14.94 and 13.98 ± 14.5 mm for C1 and C2, respectively. The absolute BCAP was 5.7 ± 5.26 and 8.26 ± 6.96 mm for C1 and C2, respectively. The LCAP metric was 1.56 ± 10.8 and −0.27 ± 16.29 mm for C1 and C2, respectively. The error in terms of BCAP and LCAP was higher for larger patients (p value<0.01). Conclusion The accuracy of the proposed method was comparable to available alternative methods, carrying the advantage of being free from errors related to objects blocking the camera visibility

Med-Query: Steerable Parsing of 9-DoF Medical Anatomies with Query Embedding

Automatic parsing of human anatomies at instance-level from 3D computed tomography (CT) scans is a prerequisite step for many clinical applications. The presence of pathologies, broken structures or limited field-of-view (FOV) all can make anatomy parsing algorithms vulnerable. In this work, we explore how to exploit and conduct the prosperous detection-then-segmentation paradigm in 3D medical data, and propose a steerable, robust, and efficient computing framework for detection, identification, and segmentation of anatomies in CT scans. Considering complicated shapes, sizes and orientations of anatomies, without lose of generality, we present the nine degrees-of-freedom (9-DoF) pose estimation solution in full 3D space using a novel single-stage, non-hierarchical forward representation. Our whole framework is executed in a steerable manner where any anatomy of interest can be directly retrieved to further boost the inference efficiency. We have validated the proposed method on three medical imaging parsing tasks of ribs, spine, and abdominal organs. For rib parsing, CT scans have been annotated at the rib instance-level for quantitative evaluation, similarly for spine vertebrae and abdominal organs. Extensive experiments on 9-DoF box detection and rib instance segmentation demonstrate the effectiveness of our framework (with the identification rate of 97.0% and the segmentation Dice score of 90.9%) in high efficiency, compared favorably against several strong baselines (e.g., CenterNet, FCOS, and nnU-Net). For spine identification and segmentation, our method achieves a new state-of-the-art result on the public CTSpine1K dataset. Last, we report highly competitive results in multi-organ segmentation at FLARE22 competition. Our annotations, code and models will be made publicly available at: https://github.com/alibaba-damo-academy/Med_Query.Comment: updated versio

A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

Deep learning-based post-processing of real-time MRI to assess and quantify dynamic wrist movement in health and disease

While morphologic magnetic resonance imaging (MRI) is the imaging modality of choice for the evaluation of ligamentous wrist injuries, it is merely static and incapable of diagnosing dynamic wrist instability. Based on real-time MRI and algorithm-based image post-processing in terms of convolutional neural networks (CNNs), this study aims to develop and validate an automatic technique to quantify wrist movement. A total of 56 bilateral wrists (28 healthy volunteers) were imaged during continuous and alternating maximum ulnar and radial abduction. Following CNN-based automatic segmentations of carpal bone contours, scapholunate and lunotriquetral gap widths were quantified based on dedicated algorithms and as a function of wrist position. Automatic segmentations were in excellent agreement with manual reference segmentations performed by two radiologists as indicated by Dice similarity coefficients of 0.96 ± 0.02 and consistent and unskewed Bland–Altman plots. Clinical applicability of the framework was assessed in a patient with diagnosed scapholunate ligament injury. Considerable increases in scapholunate gap widths across the range-of-motion were found. In conclusion, the combination of real-time wrist MRI and the present framework provides a powerful diagnostic tool for dynamic assessment of wrist function and, if confirmed in clinical trials, dynamic carpal instability that may elude static assessment using clinical-standard imaging modalities

Deformable Multisurface Segmentation of the Spine for Orthopedic Surgery Planning and Simulation

Purpose: We describe a shape-aware multisurface simplex deformable model for the segmentation of healthy as well as pathological lumbar spine in medical image data. Approach: This model provides an accurate and robust segmentation scheme for the identification of intervertebral disc pathologies to enable the minimally supervised planning and patient-specific simulation of spine surgery, in a manner that combines multisurface and shape statistics-based variants of the deformable simplex model. Statistical shape variation within the dataset has been captured by application of principal component analysis and incorporated during the segmentation process to refine results. In the case where shape statistics hinder detection of the pathological region, user assistance is allowed to disable the prior shape influence during deformation. Results: Results demonstrate validation against user-assisted expert segmentation, showing excellent boundary agreement and prevention of spatial overlap between neighboring surfaces. This section also plots the characteristics of the statistical shape model, such as compactness, generalizability and specificity, as a function of the number of modes used to represent the family of shapes. Final results demonstrate a proof-of-concept deformation application based on the open-source surgery simulation Simulation Open Framework Architecture toolkit. Conclusions: To summarize, we present a deformable multisurface model that embeds a shape statistics force, with applications to surgery planning and simulation

Spine intervertebral disc labeling using a fully convolutional redundant counting model

Labeling intervertebral discs is relevant as it notably enables clinicians to understand the relationship between a patient's symptoms (pain, paralysis) and the exact level of spinal cord injury. However manually labeling those discs is a tedious and user-biased task which would benefit from automated methods. While some automated methods already exist for MRI and CT-scan, they are either not publicly available, or fail to generalize across various imaging contrasts. In this paper we combine a Fully Convolutional Network (FCN) with inception modules to localize and label intervertebral discs. We demonstrate a proof-of-concept application in a publicly-available multi-center and multi-contrast MRI database (n=235 subjects). The code is publicly available at https://github.com/neuropoly/vertebral-labeling-deep-learning.Comment: MIDL 202

Automatic Segmentation of Intramedullary Multiple Sclerosis Lesions

Contexte: La moelle épinière est un composant essentiel du système nerveux central. Elle contient des neurones responsables d’importantes fonctionnalités et assure la transmission d’informations motrices et sensorielles entre le cerveau et le système nerveux périphérique. Un endommagement de la moelle épinière, causé par un choc ou une maladie neurodégénérative, peut mener à un sérieux handicap, pouvant entraîner des incapacités fonctionnelles, de la paralysie et/ou de la douleur. Chez les patients atteints de sclérose en plaques (SEP), la moelle épinière est fréquemment affectée par de l’atrophie et/ou des lésions. L’imagerie par résonance magnétique (IRM) conventionnelle est largement utilisée par des chercheurs et des cliniciens pour évaluer et caractériser, de façon non-invasive, des altérations micro-structurelles. Une évaluation quantitative des atteintes structurelles portées à la moelle épinière (e.g. sévérité de l’atrophie, extension des lésions) est essentielle pour le diagnostic, le pronostic et la supervision sur le long terme de maladies, telles que la SEP. De plus, le développement de biomarqueurs impartiaux est indispensable pour évaluer l’effet de nouveaux traitements thérapeutiques. La segmentation de la moelle épinière et des lésions intramédullaires de SEP sont, par conséquent, pertinentes d’un point de vue clinique, aussi bien qu’une étape nécessaire vers l’interprétation d’images RM multiparamétriques. Cependant, la segmentation manuelle est une tâche extrêmement chronophage, fastidieuse et sujette à des variations inter- et intra-expert. Il y a par conséquent un besoin d’automatiser les méthodes de segmentations, ce qui pourrait faciliter l’efficacité procédures d’analyses. La segmentation automatique de lésions est compliqué pour plusieurs raisons: (i) la variabilité des lésions en termes de forme, taille et position, (ii) les contours des lésions sont la plupart du temps difficilement discernables, (iii) l’intensité des lésions sur des images MR sont similaires à celles de structures visiblement saines. En plus de cela, réaliser une segmentation rigoureuse sur l’ensemble d’une base de données multi-centrique d’IRM est rendue difficile par l’importante variabilité des protocoles d’acquisition (e.g. résolution, orientation, champ de vue de l’image). Malgré de considérables récents développements dans le traitement d’images MR de moelle épinière, il n’y a toujours pas de méthode disponible pouvant fournir une segmentation rigoureuse et fiable de la moelle épinière pour un large spectre de pathologies et de protocoles d’acquisition. Concernant les lésions intramédullaires, une recherche approfondie dans la littérature n’a pas pu fournir une méthode disponible de segmentation automatique. Objectif: Développer un système complètement automatique pour segmenter la moelle épinière et les lésions intramédullaires sur des IRM conventionnelles humaines. Méthode: L’approche présentée est basée de deux réseaux de neurones à convolution mis en cascade. La méthode a été pensée pour faire face aux principaux obstacles que présentent les données IRM de moelle épinière. Le procédé de segmentation a été entrainé et validé sur une base de données privée composée de 1943 images, acquises dans 30 différents centres avec des protocoles hétérogènes. Les sujets scannés comportent 459 sujets sains, 471 patients SEP et 112 avec d’autres pathologies affectant la moelle épinière. Le module de segmentation de la moelle épinière a été comparé à une méthode existante reconnue par la communauté, PropSeg. Résultats: L’approche basée sur les réseaux de neurones à convolution a fourni de meilleurs résultats que PropSeg, atteignant un Dice médian (intervalle inter-quartiles) de 94.6 (4.6) vs. 87.9 (18.3) %. Pour les lésions, notre segmentation automatique a permis d'obtenir un Dice de 60.0 (21.4) % en le comparant à la segmentation manuelle, un ratio de vrai positifs de 83 (34) %, et une précision de 77 (44) %. Conclusion: Une méthode complètement automatique et innovante pour segmenter la moelle épinière et les lésions SEP intramédullaires sur des données IRM a été conçue durant ce projet de maîtrise. La méthode a été abondamment validée sur une base de données clinique. La robustesse de la méthode de segmentation de moelle épinière a été démontrée, même sur des cas pathologiques. Concernant la segmentation des lésions, les résultats sont encourageants, malgré un taux de faux positifs relativement élevé. Je crois en l’impact que peut potentiellement avoir ces outils pour la communauté de chercheurs. Dans cette optique, les méthodes ont été intégrées et documentées dans un logiciel en accès-ouvert, la “Spinal Cord Toolbox”. Certains des outils développés pendant ce projet de Maîtrise sont déjà utilisés par des analyses d’études cliniques, portant sur des patients SEP et sclérose latérale amyotrophique.----------ABSTRACT Context: The spinal cord is a key component of the central nervous system, which contains neurons responsible for complex functions, and ensures the conduction of motor and sensory information between the brain and the peripheral nervous system. Damage to the spinal cord, through trauma or neurodegenerative diseases, can lead to severe impairment, including functional disabilities, paralysis and/or pain. In multiple sclerosis (MS) patients, the spinal cord is frequently affected by atrophy and/or lesions. Conventional magnetic resonance imaging (MRI) is widely used by researchers and clinicians to non-invasively assess and characterize spinal cord microstructural changes. Quantitative assessment of the structural damage to the spinal cord (e.g. atrophy severity, lesion extent) is essential for the diagnosis, prognosis and longitudinal monitoring of diseases, such as MS. Furthermore, the development of objective biomarkers is essential to evaluate the effect of new therapeutic treatments. Spinal cord and intramedullary MS lesions segmentation is consequently clinically relevant, as well as a necessary step towards the interpretation of multi-parametric MR images. However, manual segmentation is highly time-consuming, tedious and prone to intra- and inter-rater variability. There is therefore a need for automated segmentation methods to facilitate the efficiency of analysis pipelines. Automatic lesion segmentation is challenging for various reasons: (i) lesion variability in terms of shape, size and location, (ii) lesion boundaries are most of the time not well defined, (iii) lesion intensities on MR data are confounding with those of normal-appearing structures. Moreover, achieving robust segmentation across multi-center MRI data is challenging because of the broad variability of data features (e.g. resolution, orientation, field of view). Despite recent substantial developments in spinal cord MRI processing, there is still no method available that can yield robust and reliable spinal cord segmentation across the very diverse spinal pathologies and data features. Regarding the intramedullary lesions, a thorough search of the relevant literature did not yield available method of automatic segmentation. Goal: To develop a fully-automatic framework for segmenting the spinal cord and intramedullary MS lesions from conventional human MRI data. Method: The presented approach is based on a cascade of two Convolutional Neural Networks (CNN). The method has been designed to face the main challenges of ‘real world’ spinal cord MRI data. It was trained and validated on a private dataset made up of 1943 MR volumes, acquired in different 30 sites with heterogeneous acquisition protocols. Scanned subjects involve 459 healthy controls, 471 MS patients and 112 with other spinal pathologies. The proposed spinal cord segmentation method was compared to a state-of-the-art spinal cord segmentation method, PropSeg. Results: The CNN-based approach achieved better results than PropSeg, yielding a median (interquartile range) Dice of 94.6 (4.6) vs. 87.9 (18.3) % when compared to the manual segmentation. For the lesion segmentation task, our method provided a median Dice-overlap with the manual segmentation of 60.0 (21.4) %, a lesion-based true positive rate of 83 (34) % and a lesion-based precision de 77 (44) %. Conclusion: An original fully-automatic method to segment the spinal cord and intramedullary MS lesions on MRI data has been devised during this Master’s project. The method was validated extensively against a clinical dataset. The robustness of the spinal cord segmentation has been demonstrated, even on challenging pathological cases. Regarding the lesion segmentation, the results are encouraging despite the fairly high false positive rate. I believe in the potential value of these developed tools for the research community. In this vein, the methods are integrated and documented into an open-source software, the Spinal Cord Toolbox. Some of the tools developed during this Master’s project are already integrated into automated analysis pipelines of clinical studies, including MS and Amyotrophic Lateral Sclerosis patients

Fluoroscopic Navigation for Robot-Assisted Orthopedic Surgery

Robot-assisted orthopedic surgery has gained increasing attention due to its improved accuracy and stability in minimally-invasive interventions compared to a surgeon's manual operation. An effective navigation system is critical, which estimates the intra-operative tool-to-tissue pose relationship to guide the robotic surgical device. However, most existing navigation systems use fiducial markers, such as bone pin markers, to close the calibration loop, which requires a clear line of sight and is not ideal for patients. This dissertation presents fiducial-free, fluoroscopic image-based navigation pipelines for three robot-assisted orthopedic applications: femoroplasty, core decompression of the hip, and transforaminal lumbar epidural injections. We propose custom-designed image intensity-based 2D/3D registration algorithms for pose estimation of bone anatomies, including femur and spine, and pose estimation of a rigid surgical tool and a flexible continuum manipulator. We performed system calibration and integration into a surgical robotic platform. We validated the navigation system's performance in comprehensive simulation and ex vivo cadaveric experiments. Our results suggest the feasibility of applying our proposed navigation methods for robot-assisted orthopedic applications. We also investigated machine learning approaches that can benefit the medical imaging analysis, automate the navigation component or address the registration challenges. We present a synthetic X-ray data generation pipeline called SyntheX, which enables large-scale machine learning model training. SyntheX was used to train feature detection tasks of the pelvis anatomy and the continuum manipulator, which were used to initialize the registration pipelines. Last but not least, we propose a projective spatial transformer module that learns a convex shape similarity function and extends the registration capture range. We believe that our image-based navigation solutions can benefit and inspire related orthopedic robot-assisted system designs and eventually be used in the operating rooms to improve patient outcomes