PET-based artificial intelligence applications in cardiac nuclear medicine.

In the recent years, artificial intelligence (AI) applications have gained interest in the field of cardiovascular medical imaging, including positron emission tomography (PET). The use of AI in cardiac PET imaging is to date limited, although first, important results have been shown, overcoming technical issues, improving diagnostic accuracy and providing prognostic information. In this review we aimed to summarize the state-of-the-art regarding AI applications in cardiovascular PET

Machine learning techniques for arrhythmic risk stratification: a review of the literature

Ventricular arrhythmias (VAs) and sudden cardiac death (SCD) are significant adverse events that affect the morbidity and mortality of both the general population and patients with predisposing cardiovascular risk factors. Currently, conventional disease-specific scores are used for risk stratification purposes. However, these risk scores have several limitations, including variations among validation cohorts, the inclusion of a limited number of predictors while omitting important variables, as well as hidden relationships between predictors. Machine learning (ML) techniques are based on algorithms that describe intervariable relationships. Recent studies have implemented ML techniques to construct models for the prediction of fatal VAs. However, the application of ML study findings is limited by the absence of established frameworks for its implementation, in addition to clinicians’ unfamiliarity with ML techniques. This review, therefore, aims to provide an accessible and easy-to-understand summary of the existing evidence about the use of ML techniques in the prediction of VAs. Our findings suggest that ML algorithms improve arrhythmic prediction performance in different clinical settings. However, it should be emphasized that prospective studies comparing ML algorithms to conventional risk models are needed while a regulatory framework is required prior to their implementation in clinical practice

Classification of ischemia from myocardial polar maps in 15 O-H 2 O cardiac perfusion imaging using a convolutional neural network

We implemented a two-dimensional convolutional neural network (CNN) for classification of polar maps extracted from Carimas (Turku PET Centre, Finland) software used for myocardial perfusion analysis. 138 polar maps from O-15-H2O stress perfusion study in JPEG format from patients classified as ischemic or non-ischemic based on finding obstructive coronary artery disease (CAD) on invasive coronary artery angiography were used. The CNN was evaluated against the clinical interpretation. The classification accuracy was evaluated with: accuracy (ACC), area under the receiver operating characteristic curve (AUC), F1 score (F1S), sensitivity (SEN), specificity (SPE) and precision (PRE). The CNN had a median ACC of 0.8261, AUC of 0.8058, F1S of 0.7647, SEN of 0.6500, SPE of 0.9615 and PRE of 0.9286. In comparison, clinical interpretation had ACC of 0.8696, AUC of 0.8558, F1S of 0.8333, SEN of 0.7500, SPE of 0.9615 and PRE of 0.9375. The CNN classified only 2 cases differently than the clinical interpretation. The clinical interpretation and CNN had similar accuracy in classifying false positives and true negatives. Classification of ischemia is feasible in 15O-H2O stress perfusion imaging using JPEG polar maps alone with a custom CNN and may be useful for the detection of obstructive CAD.</p

임상의사 결정 지원 시스템을 위한 심층 신경망 기반의 심초음파 자동해석에 관한 연구

학위논문(박사) -- 서울대학교대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2022. 8. 김희찬.심초음파 검사는 심장병 진단에 사용되는 중요한 도구이며, 수축기 및 이완기 단계의 심장 이미지를 제공한다. 심초음파 검사를 통해 심방과 심실의 다양한 구조적 이상과, 판막 이상등의 질환을 정량적으로 또는 정성적으로 진단할 수 있다. 심초음파 검사는 비침습적인 특성으로 인하여에 심장 전문의들이 많이 사용하고 있으며, 심장 질환자가 점점 많아지는 추세에 따라 더 많이 사용될 것으로 기대되고 있다. 심초음파 검사는 이러한 안전성과 유용성에도 불구하고, CT나 MRI와는 달리 1)정확한 영상을 얻는데 오랜 훈련기간이 필요하고 2) 영상을 얻을 수 있는 부위와 얻을 수 잇는 단면영상이 제한적이어서 검사 시 놓친 소견은 추후 영상을 감수할 경우에도 발견할 수 없는 특징을 가지고 있다. 이에 다라 측정과 해석의 정량화와 함께 검사상 이상소견을 놓치지 않을 수 있는 보완조치에 대한 요구가 많았고, 이러한 요구에 부응하여 심장전문의를 위한 임상 의사결정 지원 시스템에 대한 많은 연구가 진행되고 있다.. 인공지능의 발달로 인해 어느정도 이러한 요구에 부응할 수 있게 되었다. 이 연구의 흐름은 두가지로 나뉘게 되는데, 첫째는 심장의 구조물들을 분할하여 크기를 측정하고 특이치를 감지하는 정량적인 연구방법과, 병변이 어느 부위에 있는지 이미지 내에서 확인하는 정성적 접근법으로 나뉜다. 기존에는 이 두 연구가 대부분 따로 진행되어 왔으나, 임상의사의 진단 흐름을 고려해 볼 때 이 두가지 모두가 포함되는 임상 의사 결정 지원 시스템의 개발이 필요한 현실이다. 이러한 관점에서 본 학위 논문의 목표는 대규모 코호트 후향적 연구를 통해 AI 기반의 심장 초음파 임상 의사결정 지원 시스템을 개발하고 검증하는 것이다. 데이터는 2016년에서 2021년도 사이에 서울대 병원에서 시행된 2600예의 심초음파검사 영상(정상소견1300명, 병적소견 1300명)를 이용하였다. 정량적분석과 정성적 분석을 모두 고려하기 위해 두개의 네트워크가 개발되었으며, 그 유효성은 환자 데이터로 검증되었다. 먼저 정량적 분석을 위한 이미지 분할을 위해 U-net 기반 딥러닝 네트워크가 개발되었으며, 개발에 필요한 데이터를 위해 심장전문의가 좌심실, 좌심방, 대동맥, 우심실, 좌심실 후벽 및 심실간 중격의 정보를 이미지에 표시를 하였다. 훈련된 네트워크로부터 나온 이미지로부터 6개의 구조물의 직경과 면적을 구하여 벡터화 하였으며, 수축기말 및 이완기말 단계의 프레임 정보를 벡터로부터 추출하였다. 둘째로 정성적 진단을 위한 네트워크 개발을 위해 Resnet152 기반의 CNN을 사용하였다. 이 네트워크의 입력데이터는 정량적 네트워크에서 추출된 수축기말 및 이완기말 정보를 기반으로 10프레임이 추출되었다. 입력데이터가 정상인지 아닌지 구분하도록 했을 뿐 아니라, 마지막 레이어에서 그라디언트 가중 클래스 활성화 매핑(Grad-CAM)방법론을 이용하여 네트워크가 이미지상의 어느 부위를 보고 이상소견으로 분류했는지 시각화 하였다. 그 결과 먼저 정량적 네트워크 성능을 측정하기 위해 환자 1300명의 데이터를 통해 각 구조물의 직경과 관련된 심장질환이 얼마나 잘 검출됐는지 확인하였다. 심실중격, 좌심실 후벽, 대동맥과 관련된 병적소견을 제외하고 다른구조물의 민감도와 특이성은 모두 90% 이상이다. 수축기 말기 및 확장기 말기 위상 검출도 정확했는데, 심장전문의에 의해 선택된 프레임에 비하여 수축기 말기의 경우 평균 0.52 프레임, 확장기 말기의 경우 0.9 프레임의 차이를 보였다. 정성분석을 위한 네트워크의 경우, 첫 번째 네트워크로부터 선택된 위상정보를 바탕으로 10개의 입력데이터를 결정하였고, 무작위로 선택된 10개의 결과를 비교하였다. 그 결과 정확도가 각각 90.33%, 81.16%로 나타났으며, 1차 정량적 네트워크 에서 추출된 수축기말, 이완기말 프레임 정보는 환자를 판별하는 네트워크의 성능 향상에 기여했음을 알 수 있다. 또한 Grad-CAM 결과는 첫 번째 네트워크의 프레임 정보를 기반으로 데이터에서 추출된10 장의 이미지가 입력데이터로 쓰였을 때가 무작위로 추출된 10장의 이미지로 훈련된 네트워크 보다 병변의 위치를 더 정확하게 표시하는 것을 확인하였다. 결론적으로 본 연구는 정량적, 정성적 분석을 위한 AI 기반 심장 초음파 임상의사 결정 지원 시스템을 개발하였으며, 이 시스템이 실현 가능한 것으로 검증되었다.Echocardiography is an indispensable tool for cardiologists in the diagnosis of heart diseases. By echocardiography, various structural abnormalities in the heart can be quantitatively or qualitatively diagnosed. Due to its non-invasiveness, the usage of echocardiography in the diagnosis of heart disease has continuously increased. Despite the increasing role in the cardiology practice, echocardiography requires experience in capturing and knowledge in interpreting images. . Moreover, in contrast to CT or MRI images, important information can be missed once not obtained at the time of examination. Therefore, obtaining and interpreting images should be done simultaneously, or, at least, all obtained images should be audited by the experienced cardiologist before releasing the patient from the examination booth. Because of the peculiar characteristics of echocardiography compared to CT or MRI, there have been incessant demands for the clinical decision support system(CDSS) for echocardiography. With the advance of Artificial Intelligence (AI), there have been several studies regarding decision support systems for echocardiography. The flow of these studies is divided into two approaches: One is the quantitative approach to segment the images and detects an abnormality in size and function. The other is the qualitative approach to detect abnormality in morphology. Unfortunately, most of these two studies have been conducted separately. However, since cardiologists perform quantitative and qualitative analysis simultaneously in analyzing echocardiography, an optimal CDSS needs to be a combination of these two approaches. From this point of view, this study aims to develop and validate an AI-based CDSS for echocardiograms through a large-scale retrospective cohort. Echocardiographic data of 2,600 patients who visited Seoul National University Hospital (1300 cardiac patients and 1300 non-cardiac patients with normal echocardiogram) between 2016 and 2021. Two networks were developed for the quantitative and qualitative analysis, and their usefulnesses were verified with the patient data. First, a U-net based deep learning network was developed for segmentation in the quantitative analysis. Annotated images by the experienced cardiologist with the left ventricle, interventricular septum, left ventricular posterior wall, right ventricle, aorta, and left atrium, were used for training. The diameters and areas of the six structures were obtained and vectorized from the segmentation images, and the frame information at the end-systolic and end-diastolic phases was extracted from the vector. The second network for the qualitative diagnosis was developed using a convolutional neural network (CNN) based on Resnet 152. The input data of this network was extracted from 10 frames of each patient based on end-diastolic and end-systolic phase information extracted from the quantitative network. The network not only distinguished the input data between normal and abnormal but also visualized the location of the abnormality on the image through the Gradient-weighted Class Activation Mapping (Grad-CAM) at the last layer. The performance of the quantitative network in the chamber size and function measurements was assessed in 1300 patients. Sensitivity and specificity were both over 90% except for pathologies related to the left ventricular posterior wall, interventricular septum, and aorta. The end-systolic and end-diastolic phase detection was also accurate, with an average difference of 0.52 frames for the end-systolic and 0.9 frames for the end-diastolic phases. In the case of the network for qualitative analysis, 10 input data were selected based on the phase information determined from the first network, and the results of 10 randomly selected images were compared. As a result, the accuracy was 90.3% and 81.2%, respectively, and the phase information selected from the first network contributed to the improvement of the performance of the network. Also, the results of Grad-CAM confirmed that the network trained with 10 images of data extracted based on the phase information from the first network displays the location of the lesion more accurately than the network trained with 10 randomly selected data. In conclusion, this study proposed an AI-based CDSS for echocardiography in the quantitative and qualitative analysis.ABSTRACT １ CONTENTS v LIST OF TABLES vii LIST OF FIGURES viii CHAPTER 1 1 Introduction 1 1. Introduction 2 1.1. Echocardiogram 2 1.1.1. Diagnosis using Echocardiogram 2 1.1.2. Limitation in Echocardiogram 3 1.1.3. Artificial Intelligence in Echocardiogram 6 1.2. Clinical Background 7 1.2.1. Diagnostic Flow 8 1.2.2. Previous studies and clinical implication of this study 11 1.3. Technical Background 16 1.3.1. Convolutional Neural Network (CNN) 16 1.3.1.1. U-net 18 1.3.1.2. Residual Network 20 1.3.1.3. Gradient-weighted Class Activation Mapping (Grad-CAM) 22 1.4. Unmet Clinical Needs 26 1.5. Objective 27 CHAPTER 2 28 Materials & Methods 28 2. Materials & Methods 29 2.1. Data Description 29 2.2. Annotated Data 32 2.3. Overall Architecture 33 2.3.1. Quantitative Network 35 2.3.2. Qualitative Network 37 2.4. Dice Similarity Score 39 2.5. Intersection over Union 40 CHAPTER 3 41 3. Results & Discussion 42 3.1. Quantitative Network Result 42 3.1.1. Diagnostic results 47 3.1.2. Phase Detection Result 49 3.2. Qualitative Network Results 51 3.2.1. Grad-CAM Result 56 3.3. Limitation 58 3.3.1. Need for external dataset for generalizable network 58 3.3.2. Futurework of the system 59 CHAPTER 4 60 4. Conclusion 61 Abstract in Korean 62 Bibliography 65박

Multi-modality cardiac image computing: a survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, either combining information from different modalities or transferring information across modalities. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, modality selection, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future

Contribuciones de las técnicas machine learning a la cardiología. Predicción de reestenosis tras implante de stent coronario

[ES]Antecedentes: Existen pocos temas de actualidad equiparables a la posibilidad de la tecnología actual para desarrollar las mismas capacidades que el ser humano, incluso en medicina. Esta capacidad de simular los procesos de inteligencia humana por parte de máquinas o sistemas informáticos es lo que conocemos hoy en día como inteligencia artificial. Uno de los campos de la inteligencia artificial con mayor aplicación a día de hoy en medicina es el de la predicción, recomendación o diagnóstico, donde se aplican las técnicas machine learning. Asimismo, existe un creciente interés en las técnicas de medicina de precisión, donde las técnicas machine learning pueden ofrecer atención médica individualizada a cada paciente. El intervencionismo coronario percutáneo (ICP) con stent se ha convertido en una práctica habitual en la revascularización de los vasos coronarios con enfermedad aterosclerótica obstructiva significativa. El ICP es asimismo patrón oro de tratamiento en pacientes con infarto agudo de miocardio; reduciendo las tasas de muerte e isquemia recurrente en comparación con el tratamiento médico. El éxito a largo plazo del procedimiento está limitado por la reestenosis del stent, un proceso patológico que provoca un estrechamiento arterial recurrente en el sitio de la ICP. Identificar qué pacientes harán reestenosis es un desafío clínico importante; ya que puede manifestarse como un nuevo infarto agudo de miocardio o forzar una nueva resvascularización del vaso afectado, y que en casos de reestenosis recurrente representa un reto terapéutico. Objetivos: Después de realizar una revisión de las técnicas de inteligencia artificial aplicadas a la medicina y con mayor profundidad, de las técnicas machine learning aplicadas a la cardiología, el objetivo principal de esta tesis doctoral ha sido desarrollar un modelo machine learning para predecir la aparición de reestenosis en pacientes con infarto agudo de miocardio sometidos a ICP con implante de un stent. Asimismo, han sido objetivos secundarios comparar el modelo desarrollado con machine learning con los scores clásicos de riesgo de reestenosis utilizados hasta la fecha; y desarrollar un software que permita trasladar esta contribución a la práctica clínica diaria de forma sencilla. Para desarrollar un modelo fácilmente aplicable, realizamos nuestras predicciones sin variables adicionales a las obtenidas en la práctica rutinaria. Material: El conjunto de datos, obtenido del ensayo GRACIA-3, consistió en 263 pacientes con características demográficas, clínicas y angiográficas; 23 de ellos presentaron reestenosis a los 12 meses después de la implantación del stent. Todos los desarrollos llevados a cabo se han hecho en Python y se ha utilizado computación en la nube, en concreto AWS (Amazon Web Services). Metodología: Se ha utilizado una metodología para trabajar con conjuntos de datos pequeños y no balanceados, siendo importante el esquema de validación cruzada anidada utilizado, así como la utilización de las curvas PR (precision-recall, exhaustividad-sensibilidad), además de las curvas ROC, para la interpretación de los modelos. Se han entrenado los algoritmos más habituales en la literatura para elegir el que mejor comportamiento ha presentado. Resultados: El modelo con mejores resultados ha sido el desarrollado con un clasificador extremely randomized trees; que superó significativamente (0,77; área bajo la curva ROC a los tres scores clínicos clásicos; PRESTO-1 (0,58), PRESTO-2 (0,58) y TLR (0,62). Las curvas exhaustividad sensibilidad ofrecieron una imagen más precisa del rendimiento del modelo extremely randomized trees que muestra un algoritmo eficiente (0,96) para no reestenosis, con alta exhaustividad y alta sensibilidad. Para un umbral considerado óptimo, de 1,000 pacientes sometidos a implante de stent, nuestro modelo machine learning predeciría correctamente 181 (18%) más casos en comparación con el mejor score de riesgo clásico (TLR). Las variables más importantes clasificadas según su contribución a las predicciones fueron diabetes, enfermedad coronaria en 2 ó más vasos, flujo TIMI post-ICP, plaquetas anormales, trombo post-ICP y colesterol anormal. Finalmente, se ha desarrollado una calculadora para trasladar el modelo a la práctica clínica. La calculadora permite estimar el riesgo individual de cada paciente y situarlo en una zona de riesgo, facilitando la toma de decisión al médico en cuanto al seguimiento adecuado para el mismo. Conclusiones: Aplicado inmediatamente después de la implantación del stent, un modelo machine learning diferencia mejor a aquellos pacientes que presentarán o no reestenosis respecto a los discriminadores clásicos actuales

[¹⁸F]fluorination of biorelevant arylboronic acid pinacol ester scaffolds synthesized by convergence techniques

Aim: The development of small molecules through convergent multicomponent reactions (MCR) has been boosted during the last decade due to the ability to synthesize, virtually without any side-products, numerous small drug-like molecules with several degrees of structural diversity.(1) The association of positron emission tomography (PET) labeling techniques in line with the “one-pot” development of biologically active compounds has the potential to become relevant not only for the evaluation and characterization of those MCR products through molecular imaging, but also to increase the library of radiotracers available. Therefore, since the [18F]fluorination of arylboronic acid pinacol ester derivatives tolerates electron-poor and electro-rich arenes and various functional groups,(2) the main goal of this research work was to achieve the 18F-radiolabeling of several different molecules synthesized through MCR. Materials and Methods: [18F]Fluorination of boronic acid pinacol esters was first extensively optimized using a benzaldehyde derivative in relation to the ideal amount of Cu(II) catalyst and precursor to be used, as well as the reaction solvent. Radiochemical conversion (RCC) yields were assessed by TLC-SG. The optimized radiolabeling conditions were subsequently applied to several structurally different MCR scaffolds comprising biologically relevant pharmacophores (e.g. β-lactam, morpholine, tetrazole, oxazole) that were synthesized to specifically contain a boronic acid pinacol ester group. Results: Radiolabeling with fluorine-18 was achieved with volumes (800 μl) and activities (≤ 2 GBq) compatible with most radiochemistry techniques and modules. In summary, an increase in the quantities of precursor or Cu(II) catalyst lead to higher conversion yields. An optimal amount of precursor (0.06 mmol) and Cu(OTf)2(py)4 (0.04 mmol) was defined for further reactions, with DMA being a preferential solvent over DMF. RCC yields from 15% to 76%, depending on the scaffold, were reproducibly achieved. Interestingly, it was noticed that the structure of the scaffolds, beyond the arylboronic acid, exerts some influence in the final RCC, with electron-withdrawing groups in the para position apparently enhancing the radiolabeling yield. Conclusion: The developed method with high RCC and reproducibility has the potential to be applied in line with MCR and also has a possibility to be incorporated in a later stage of this convergent “one-pot” synthesis strategy. Further studies are currently ongoing to apply this radiolabeling concept to fluorine-containing approved drugs whose boronic acid pinacol ester precursors can be synthesized through MCR (e.g. atorvastatin)