Segment Anything Model for Medical Images?

The Segment Anything Model (SAM) is the first foundation model for general image segmentation. It designed a novel promotable segmentation task, ensuring zero-shot image segmentation using the pre-trained model via two main modes including automatic everything and manual prompt. SAM has achieved impressive results on various natural image segmentation tasks. However, medical image segmentation (MIS) is more challenging due to the complex modalities, fine anatomical structures, uncertain and complex object boundaries, and wide-range object scales. SAM has achieved impressive results on various natural image segmentation tasks. Meanwhile, zero-shot and efficient MIS can well reduce the annotation time and boost the development of medical image analysis. Hence, SAM seems to be a potential tool and its performance on large medical datasets should be further validated. We collected and sorted 52 open-source datasets, and build a large medical segmentation dataset with 16 modalities, 68 objects, and 553K slices. We conducted a comprehensive analysis of different SAM testing strategies on the so-called COSMOS 553K dataset. Extensive experiments validate that SAM performs better with manual hints like points and boxes for object perception in medical images, leading to better performance in prompt mode compared to everything mode. Additionally, SAM shows remarkable performance in some specific objects and modalities, but is imperfect or even totally fails in other situations. Finally, we analyze the influence of different factors (e.g., the Fourier-based boundary complexity and size of the segmented objects) on SAM's segmentation performance. Extensive experiments validate that SAM's zero-shot segmentation capability is not sufficient to ensure its direct application to the MIS.Comment: 23 pages, 14 figures, 12 table

Improving Radiotherapy Targeting for Cancer Treatment Through Space and Time

Radiotherapy is a common medical treatment in which lethal doses of ionizing radiation are preferentially delivered to cancerous tumors. In external beam radiotherapy, radiation is delivered by a remote source which sits several feet from the patient\u27s surface. Although great effort is taken in properly aligning the target to the path of the radiation beam, positional uncertainties and other errors can compromise targeting accuracy. Such errors can lead to a failure in treating the target, and inflict significant toxicity to healthy tissues which are inadvertently exposed high radiation doses. Tracking the movement of targeted anatomy between and during treatment fractions provides valuable localization information that allows for the reduction of these positional uncertainties. Inter- and intra-fraction anatomical localization data not only allows for more accurate treatment setup, but also potentially allows for 1) retrospective treatment evaluation, 2) margin reduction and modification of the dose distribution to accommodate daily anatomical changes (called `adaptive radiotherapy\u27), and 3) targeting interventions during treatment (for example, suspending radiation delivery while the target it outside the path of the beam). The research presented here investigates the use of inter- and intra-fraction localization technologies to improve radiotherapy to targets through enhanced spatial and temporal accuracy. These technologies provide significant advancements in cancer treatment compared to standard clinical technologies. Furthermore, work is presented for the use of localization data acquired from these technologies in adaptive treatment planning, an investigational technique in which the distribution of planned dose is modified during the course of treatment based on biological and/or geometrical changes of the patient\u27s anatomy. The focus of this research is directed at abdominal sites, which has historically been central to the problem of motion management in radiation therapy

Development of Targeted Liposomal Formulation Approaches for Enhanced Colorectal Cancer Therapy

Colorectal cancer (CRC) is the 4th most commonly detected cancer in the USA. Despite promising advances, the 5-year survival rate for the metastatic disease remains dismal (40⁰C with ultrasound contrast agents and bacterial attachments can improve the real-time chemo-immunotherapy of CRC. Towards these goals, we investigated the following specific aims in murine models of colon cancer: 1) Develop echogenic-LTSL (E-LTSL) for real-time ultrasound-enhanced reporting of tumor temperature and doxorubicin delivery, 2) Utilize tumor homing Salmonella typhimurium for LTSL delivery and enhanced chemo-immunotherapy with High Intensity Focused Ultrasound (HIFU) tumor heating (~42°C), and 3) Investigate the ability of magnetic bacteria Magnetospirillim magneticum (AMB-1) to aid LTSL tumor drug delivery under magnetic guidance. Our data showed that intratumoral vascular contrast of E-LTSL as a function of temperature and doxorubicin delivery was strongly correlated, enabling robust estimation of temporal variation in colon tumor temperature and drug delivery. LTSL attachment didn’t impact Salmonella viability and improved chemo-immunotherapy outcomes in murine colon cancers by promoting the population of M1 macrophages with HIFU heating. Finally, the use of magnetic guidance for AMB-LTSL significantly reduced the colon cancer viability by enhancing cellular and tumor localizations of doxorubicin. In conclusion, we found that multifunctional LTSL formulations significantly improved the CRC treatment outcomes in murine models by aiding the real-time monitoring and removing the resistive and suppressive tumor microenvironment features

Strain ultrasound elastography of aneurysm sac content after randomized endoleak embolization with sclerosing and non-sclerosing chitosan-based hydrogels in a preclinical model

Mise en contexte : La réparation endovasculaire des anévrismes de l’aorte abdominale est limitée par le développement des endofuites, qui nécessite un suivi à long terme par imagerie. L’élastographie sonore de déformation a été proposée comme méthode complémentaire pour aider à la détection des endofuites et la caractérisation des propriétés mécaniques des anévrismes. On s’intéresse ici également à la possibilité de suivre l’embolisation des endofuites, qui est indiquée dans certains cas mais dont le succès est variable. Un nouvel agent d’embolisation a été récemment créé en combinant un hydrogel de chitosane radio-opaque (CH) et le sclérosant tetradecyl sulfate de sodium (STS), qui s’appelle CH-STS. Le CH-STS démontre des propriétés mécaniques in vitro favorables, mais son comportement in vivo et son effet sur l’évolution du sac par rapport à un agent non-sclérosant pourraient être mieux caractérisés. L’objectif de cette étude était la caractérisation des propriétés mécaniques des composantes des endofuites embolisées avec CH-STS et CH avec élastographie sonore de déformation. Méthodologie : Des anévrismes bilatéraux avec endofuites de type I ont été créés au niveau des artères iliaques communes chez neuf chiens. Chez chaque sujet, une endofuite a été embolisée avec CH, et l’autre, avec CH-STS, d’une façon aléatoire et aveugle. Des images d’échographie duplex et des cinéloops pour élastographie sonore de déformation ont été acquis à 1 semaine, 1 mois, 3 mois et (chez 3 sujets) 6 mois post-embolisation. La tomodensitométrie a été faite à 3 mois et (si pertinente) 6 mois post-embolisation. L’histopathologie a été faite au sacrifice. Les études radiologiques et les données d’histopathologie ont été co-enregistrées pour définir trois régions d’intérêt sur les cinéloops : l’agent d’embolisation (au sacrifice), le thrombus intraluminal (au sacrifice) et le sac anévrismal (pendant chaque suivi). L’élastographie sonore de déformation a été faite avec les segmentations par deux observateurs indépendants. La déformation axiale maximale (DAM) a été le critère d’évaluation principal. Les analyses statistiques ont été faites avec des modèles mixtes linéaires généralisés et des coefficients de corrélations intraclasses (ICCs). Résultats : Des endofuites résiduelles ont été trouvées dans 7/9 (77.8%) et 4/9 (44.4%) des anévrismes embolisés avec CH et CH-STS, respectivement. Le CH-STS a eu une DAM 66 % plus basse (p < 0.001) que le CH. Le thrombus a eu une DAM 37% plus basse (p = 0.010) que le CH et 77% plus élevée (p = 0.079) que le CH-STS. Il n’y avait aucune différence entre les thrombi associés avec les deux traitements. Les sacs anévrismaux embolisés avec CH-STS ont eu une DAM 29% plus basse (p < 0.001) que ceux embolisés avec CH. Des endofuites résiduelles ont été associées avec une DAM du sac anévrismal 53% plus élevée (p < 0.001). Le ICC pour la DAM a été de 0.807 entre les deux segmentations. Conclusion : Le CH-STS confère des valeurs de déformations plus basses aux anévrismes embolisés. Les endofuites persistantes sont associées avec des déformations plus élevées du sac anévrismal.Background: Endovascular aneurysm repair (EVAR) is the modality of choice for the treatment of abdominal aortic aneurysms (AAAs). EVAR is limited by the development of endoleaks, which necessitate long-term imaging follow-up. Conventional follow-up modalities suffer from unique limitations. Strain ultrasound elastography (SUE) has been recently proposed as an imaging adjunct to detect endoleaks and to characterize aneurysm mechanical properties. Once detected, certain endoleaks may be treated with embolization; however, success is limited. In this context, the embolic agent CH-STS—containing a chitosan hydrogel and the sclerosant sodium tetradecyl sulphate (STS)—was created. CH-STS demonstrates favorable mechanical properties in vitro; however, its behavior in vivo and impact on sac evolution compared to a non-sclerosing chitosan-based embolic agent (CH) merit further characterization. Purpose: To compare the mechanical properties of the constituents of endoleaks embolized with CH and CH-STS—including the agent, the intraluminal thrombus (ILT), and the overall sac—via SUE. Methods: Bilateral common iliac artery aneurysms with type I endoleaks were created in nine dogs. In each animal, one endoleak was randomly embolized with CH, and the other with CH-STS. Duplex ultrasound (DUS) and radiofrequency cine loops were acquired at 1 week, 1 month, 3 months, and—in 3 subjects—6 months post-embolization. Contrast-enhanced CT was performed at 3 months and—where applicable—6 months post-embolization. Histopathological analysis was performed at time of sacrifice. Radiological studies and histopathological slides were co-registered to identify three regions of interest (ROIs) on the cine loops: embolic agent (at sacrifice), ILT (at sacrifice), and aneurysm sac (at all follow-up times). SUE was performed using segmentations from two independent observers on the cine loops. Maximum axial deformation (MAD) was the main outcome. Statistical analysis was performed using general linear mixed models and intraclass correlation coefficients (ICCs). Results: Residual endoleaks were identified in 7/9 (77.8%) and 4/9 (44.4%) aneurysms embolized with CH and CH-STS, respectively. CH-STS had a 66 % lower MAD (p < 0.001) than CH. The ILT had a 37% lower MAD (p = 0.010) than CH and a 77% greater MAD (p = 0.079; trending towards significance) than CH-STS. There was no difference in the ILT between treatment groups. Aneurysm sacs embolized with CH-STS had a 29% lower MAD (p < 0.001) than those with CH. Residual endoleak increased MAD of the aneurysm sac by 53% (p < 0.001), regardless of the agent used. The ICC for MAD was 0.807 between readers’ segmentations. Conclusion: CH-STS confers lower strain values to embolized aneurysms. Persistent endoleaks result are associated with increased sac strain, which may be useful for clinical follow-up

Deep learning in medical imaging and radiation therapy

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/146980/1/mp13264_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/146980/2/mp13264.pd

A Methodology for Evaluating Image Segmentation Algorithms

The purpose of this paper is to describe a framework for evaluating image segmentation algorithms. Image segmentation consists of object recognition and delineation. For evaluating segmentation methods, three factors - precision (reproducibility), accuracy (agreement with truth), and efficiency (time taken) – need to be considered for both recognition and delineation. To assess precision, we need to choose a figure of merit (FOM), repeat segmentation considering all sources of variation, and determine variations in FOM via statistical analysis. It is impossible usually to establish true segmentation. Hence, to assess accuracy, we need to choose a surrogate of true segmentation and proceed as for precision. To assess efficiency, both the computational and the user time required for algorithm and operator training and for algorithm execution should be measured and analyzed. Precision, accuracy, and efficiency are interdependent. It is difficult to improve one factor without affecting others. Segmentation methods must be compared based on all three factors. The weight given to each factor depends on application

Improvements in four-dimensional and dual energy computed tomography

Dual energy and 4D computed tomography (CT) seek to address some of the limitations in traditional CT imaging. Dual energy CT, among other purposes, allows for the quantification and improved visualization of contrast materials, and 4D CT is often used in radiation therapy applications as it allows for the visualization and quantification of object motion. While much research has been done with these technologies, areas remain for potential improvement, both in preclinical and clinical settings, which will be explored in this dissertation. Preclinical dual energy cone-beam CT (CBCT) can benefit from wider separation between the peak energy of the two energy spectra. Using simulations and an x-ray source with a wide kVp range the contrast to noise ratio and Iodine concentration accuracy and precision were determined from Iodine material images. Improvements of 80% in CNR and 58% in precision were observed in the optimal energy pair of 60kVp/200kVp compared to a standard energy pair of 80kVp/140kVp. In 4D imaging, using projection data to obtain the required respiratory signal (“data driven”) can reduce setup complexity and cost of preclinical respiratory monitoring and reduce clinical 4D CT artifacts. Several clinical data driven 4D CBCT methods were modified for mice. Errors in projection sorting were within 4% of a breathing phase and were statistically less than the previous method for data driven 4D CBCT in mice. In clinical 4D CT, semi-automatically drawn target volumes and artifacts were compared between data driven and standard 4D CT images. Target volumes were shown to be statistically at least as large as standard contours, and artifacts were significantly reduced using the data driven technique. 4D CBCT is promising for use in evaluating tumor motion immediately prior to radiation treatment, but suffers from under sampling artifacts. An iterative volume of interest based reconstruction (I4D VOI) that aims to reduce artifacts without increases in computation time was compared to several other reconstruction techniques using a long scan patient data set. No statistical difference in tumor motion error was observed between I4D VOI and any of the other reconstruction methods. However, potential improvement over non-iterative VOI was demonstrated and computation time was reduced compared to TV minimization

Developing advanced mathematical models for detecting abnormalities in 2D/3D medical structures.

Detecting abnormalities in two-dimensional (2D) and three-dimensional (3D) medical structures is among the most interesting and challenging research areas in the medical imaging field. Obtaining the desired accurate automated quantification of abnormalities in medical structures is still very challenging. This is due to a large and constantly growing number of different objects of interest and associated abnormalities, large variations of their appearances and shapes in images, different medical imaging modalities, and associated changes of signal homogeneity and noise for each object. The main objective of this dissertation is to address these problems and to provide proper mathematical models and techniques that are capable of analyzing low and high resolution medical data and providing an accurate, automated analysis of the abnormalities in medical structures in terms of their area/volume, shape, and associated abnormal functionality. This dissertation presents different preliminary mathematical models and techniques that are applied in three case studies: (i) detecting abnormal tissue in the left ventricle (LV) wall of the heart from delayed contrast-enhanced cardiac magnetic resonance images (MRI), (ii) detecting local cardiac diseases based on estimating the functional strain metric from cardiac cine MRI, and (iii) identifying the abnormalities in the corpus callosum (CC) brain structure—the largest fiber bundle that connects the two hemispheres in the brain—for subjects that suffer from developmental brain disorders. For detecting the abnormal tissue in the heart, a graph-cut mathematical optimization model with a cost function that accounts for the object’s visual appearance and shape is used to segment the the inner cavity. The model is further integrated with a geometric model (i.e., a fast marching level set model) to segment the outer border of the myocardial wall (the LV). Then the abnormal tissue in the myocardium wall (also called dead tissue, pathological tissue, or infarct area) is identified based on a joint Markov-Gibbs random field (MGRF) model of the image and its region (segmentation) map that accounts for the pixel intensities and the spatial interactions between the pixels. Experiments with real in-vivo data and comparative results with ground truth (identified by a radiologist) and other approaches showed that the proposed framework can accurately detect the pathological tissue and can provide useful metrics for radiologists and clinicians. To estimate the strain from cardiac cine MRI, a novel method based on tracking the LV wall geometry is proposed. To achieve this goal, a partial differential equation (PDE) method is applied to track the LV wall points by solving the Laplace equation between the LV contours of each two successive image frames over the cardiac cycle. The main advantage of the proposed tracking method over traditional texture-based methods is its ability to track the movement and rotation of the LV wall based on tracking the geometric features of the inner, mid-, and outer walls of the LV. This overcomes noise sources that come from scanner and heart motion. To identify the abnormalities in the CC from brain MRI, the CCs are aligned using a rigid registration model and are segmented using a shape-appearance model. Then, they are mapped to a simple unified space for analysis. This work introduces a novel cylindrical mapping model, which is conformal (i.e., one to one transformation and bijective), that enables accurate 3D shape analysis of the CC in the cylindrical domain. The framework can detect abnormalities in all divisions of the CC (i.e., splenium, rostrum, genu and body). In addition, it offers a whole 3D analysis of the CC abnormalities instead of only area-based analysis as done by previous groups. The initial classification results based on the centerline length and CC thickness suggest that the proposed CC shape analysis is a promising supplement to the current techniques for diagnosing dyslexia. The proposed techniques in this dissertation have been successfully tested on complex synthetic and MR images and can be used to advantage in many of today’s clinical applications of computer-assisted medical diagnostics and intervention