5 research outputs found
Artificial Intelligence and Echocardiography
Artificial intelligence (AI) is evolving in the field of diagnostic medical imaging, including echocardiography. Although the dynamic nature of echocardiography presents challenges beyond those of static images from X-ray, computed tomography, magnetic resonance, and radioisotope imaging, AI has influenced all steps of echocardiography, from image acquisition to automatic measurement and interpretation. Considering that echocardiography often is affected by inter-observer variability and shows a strong dependence on the level of experience, AI could be extremely advantageous in minimizing observer variation and providing reproducible measures, enabling accurate diagnosis. Currently, most reported AI applications in echocardiographic measurement have focused on improved image acquisition and automation of repetitive and tedious tasks; however, the role of AI applications should not be limited to conventional processes. Rather, AI could provide clinically important insights from subtle and non-specific data, such as changes in myocardial texture in patients with myocardial disease. Recent initiatives to develop large echocardiographic databases can facilitate development of AI applications. The ultimate goal of applying AI to echocardiography is automation of the entire process of echocardiogram analysis. Once automatic analysis becomes reliable, workflows in clinical echocardiographic will change radically. The human expert will remain the master controlling the overall diagnostic process, will not be replaced by AI, and will obtain significant support from AI systems to guide acquisition, perform measurements, and integrate and compare data on request.ope
Automated multi-beat tissue Doppler echocardiography analysis using deep neural networks
Tissue Doppler imaging is an essential echocardiographic technique for the non-invasive assessment of myocardial blood velocity. Image acquisition and interpretation are performed by trained operators who visually localise landmarks representing Doppler peak velocities. Current clinical guidelines recommend averaging measurements over several heartbeats. However, this manual process is both time-consuming and disruptive to workflow. An automated system for accurate beat isolation and landmark identification would be highly desirable. A dataset of tissue Doppler images was annotated by three cardiologist experts, providing a gold standard and allowing for observer variability comparisons. Deep neural networks were trained for fully automated predictions on multiple heartbeats and tested on tissue Doppler strips of arbitrary length. Automated measurements of peak Doppler velocities show good BlandβAltman agreement (average standard deviation of 0.40 cm/s) with consensus expert values; less than the inter-observer variability (0.65 cm/s). Performance is akin to individual experts (standard deviation of 0.40 to 0.75 cm/s). Our approach allows forβ>β26 times as many heartbeats to be analysed, compared to a manual approach. The proposed automated models can accurately and reliably make measurements on tissue Doppler images spanning several heartbeats, with performance indistinguishable from that of human experts, but with significantly shorter processing time
Multibeat echocardiographic phase detection using deep neural networks
Background
Accurate identification of end-diastolic and end-systolic frames in echocardiographic cine loops is important, yet challenging, for human experts. Manual frame selection is subject to uncertainty, affecting crucial clinical measurements, such as myocardial strain. Therefore, the ability to automatically detect frames of interest is highly desirable.
Methods
We have developed deep neural networks, trained and tested on multi-centre patient data, for the accurate identification of end-diastolic and end-systolic frames in apical four-chamber 2D multibeat cine loop recordings of arbitrary length. Seven experienced cardiologist experts independently labelled the frames of interest, thereby providing infallible annotations, allowing for observer variability measurements.
Results
When compared with the ground-truth, our model shows an average frame difference of β0.09 Β± 1.10 and 0.11 Β± 1.29 frames for end-diastolic and end-systolic frames, respectively. When applied to patient datasets from a different clinical site, to which the model was blind during its development, average frame differences of β1.34 Β± 3.27 and β0.31 Β± 3.37 frames were obtained for both frames of interest. All detection errors fall within the range of inter-observer variability: [-0.87, β5.51]Β±[2.29, 4.26] and [-0.97, β3.46]Β±[3.67, 4.68] for ED and ES events, respectively.
Conclusions
The proposed automated model can identify multiple end-systolic and end-diastolic frames in echocardiographic videos of arbitrary length with performance indistinguishable from that of human experts, but with significantly shorter processing time
μμμμ¬ κ²°μ μ§μ μμ€ν μ μν μ¬μΈ΅ μ κ²½λ§ κΈ°λ°μ μ¬μ΄μν μλν΄μμ κ΄ν μ°κ΅¬
νμλ
Όλ¬Έ(λ°μ¬) -- μμΈλνκ΅λνμ : 곡과λν νλκ³Όμ λ°μ΄μ€μμ§λμ΄λ§μ 곡, 2022. 8. κΉν¬μ°¬.μ¬μ΄μν κ²μ¬λ μ¬μ₯λ³ μ§λ¨μ μ¬μ©λλ μ€μν λꡬμ΄λ©°, μμΆκΈ° λ° μ΄μκΈ° λ¨κ³μ μ¬μ₯ μ΄λ―Έμ§λ₯Ό μ 곡νλ€. μ¬μ΄μν κ²μ¬λ₯Ό ν΅ν΄ μ¬λ°©κ³Ό μ¬μ€μ λ€μν ꡬ쑰μ μ΄μκ³Ό, νλ§ μ΄μλ±μ μ§νμ μ λμ μΌλ‘ λλ μ μ±μ μΌλ‘ μ§λ¨ν μ μλ€. μ¬μ΄μν κ²μ¬λ λΉμΉ¨μ΅μ μΈ νΉμ±μΌλ‘ μΈνμ¬μ μ¬μ₯ μ λ¬Έμλ€μ΄ λ§μ΄ μ¬μ©νκ³ μμΌλ©°, μ¬μ₯ μ§νμκ° μ μ λ§μμ§λ μΆμΈμ λ°λΌ λ λ§μ΄ μ¬μ©λ κ²μΌλ‘ κΈ°λλκ³ μλ€.
μ¬μ΄μν κ²μ¬λ μ΄λ¬ν μμ μ±κ³Ό μ μ©μ±μλ λΆκ΅¬νκ³ , CTλ MRIμλ λ¬λ¦¬ 1)μ νν μμμ μ»λλ° μ€λ νλ ¨κΈ°κ°μ΄ νμνκ³ 2) μμμ μ»μ μ μλ λΆμμ μ»μ μ μλ λ¨λ©΄μμμ΄ μ νμ μ΄μ΄μ κ²μ¬ μ λμΉ μ견μ μΆν μμμ κ°μν κ²½μ°μλ λ°κ²¬ν μ μλ νΉμ§μ κ°μ§κ³ μλ€. μ΄μ λ€λΌ μΈ‘μ κ³Ό ν΄μμ μ λνμ ν¨κ» κ²μ¬μ μ΄μμ견μ λμΉμ§ μμ μ μλ 보μμ‘°μΉμ λν μκ΅¬κ° λ§μκ³ , μ΄λ¬ν μꡬμ λΆμνμ¬ μ¬μ₯μ λ¬Έμλ₯Ό μν μμ μμ¬κ²°μ μ§μ μμ€ν
μ λν λ§μ μ°κ΅¬κ° μ§νλκ³ μλ€..
μΈκ³΅μ§λ₯μ λ°λ¬λ‘ μΈν΄ μ΄λμ λ μ΄λ¬ν μꡬμ λΆμν μ μκ² λμλ€. μ΄ μ°κ΅¬μ νλ¦μ λκ°μ§λ‘ λλκ² λλλ°, 첫째λ μ¬μ₯μ ꡬ쑰물λ€μ λΆν νμ¬ ν¬κΈ°λ₯Ό μΈ‘μ νκ³ νΉμ΄μΉλ₯Ό κ°μ§νλ μ λμ μΈ μ°κ΅¬λ°©λ²κ³Ό, λ³λ³μ΄ μ΄λ λΆμμ μλμ§ μ΄λ―Έμ§ λ΄μμ νμΈνλ μ μ±μ μ κ·Όλ²μΌλ‘ λλλ€. κΈ°μ‘΄μλ μ΄ λ μ°κ΅¬κ° λλΆλΆ λ°λ‘ μ§νλμ΄ μμΌλ, μμμμ¬μ μ§λ¨ νλ¦μ κ³ λ €ν΄ λ³Ό λ μ΄ λκ°μ§ λͺ¨λκ° ν¬ν¨λλ μμ μμ¬ κ²°μ μ§μ μμ€ν
μ κ°λ°μ΄ νμν νμ€μ΄λ€.
μ΄λ¬ν κ΄μ μμ λ³Έ νμ λ
Όλ¬Έμ λͺ©νλ λκ·λͺ¨ μ½νΈνΈ νν₯μ μ°κ΅¬λ₯Ό ν΅ν΄ AI κΈ°λ°μ μ¬μ₯ μ΄μν μμ μμ¬κ²°μ μ§μ μμ€ν
μ κ°λ°νκ³ κ²μ¦νλ κ²μ΄λ€. λ°μ΄ν°λ 2016λ
μμ 2021λ
λ μ¬μ΄μ μμΈλ λ³μμμ μνλ 2600μμ μ¬μ΄μνκ²μ¬ μμ(μ μμ견1300λͺ
, λ³μ μ견 1300λͺ
)λ₯Ό μ΄μ©νμλ€. μ λμ λΆμκ³Ό μ μ±μ λΆμμ λͺ¨λ κ³ λ €νκΈ° μν΄ λκ°μ λ€νΈμν¬κ° κ°λ°λμμΌλ©°, κ·Έ μ ν¨μ±μ νμ λ°μ΄ν°λ‘ κ²μ¦λμλ€.
λ¨Όμ μ λμ λΆμμ μν μ΄λ―Έμ§ λΆν μ μν΄ U-net κΈ°λ° λ₯λ¬λ λ€νΈμν¬κ° κ°λ°λμμΌλ©°, κ°λ°μ νμν λ°μ΄ν°λ₯Ό μν΄ μ¬μ₯μ λ¬Έμκ° μ’μ¬μ€, μ’μ¬λ°©, λλλ§₯, μ°μ¬μ€, μ’μ¬μ€ νλ²½ λ° μ¬μ€κ° μ€κ²©μ μ 보λ₯Ό μ΄λ―Έμ§μ νμλ₯Ό νμλ€. νλ ¨λ λ€νΈμν¬λ‘λΆν° λμ¨ μ΄λ―Έμ§λ‘λΆν° 6κ°μ ꡬ쑰물μ μ§κ²½κ³Ό λ©΄μ μ ꡬνμ¬ λ²‘ν°ν νμμΌλ©°, μμΆκΈ°λ§ λ° μ΄μκΈ°λ§ λ¨κ³μ νλ μ μ 보λ₯Ό 벑ν°λ‘λΆν° μΆμΆνμλ€.
λμ§Έλ‘ μ μ±μ μ§λ¨μ μν λ€νΈμν¬ κ°λ°μ μν΄ Resnet152 κΈ°λ°μ CNNμ μ¬μ©νμλ€. μ΄ λ€νΈμν¬μ μ
λ ₯λ°μ΄ν°λ μ λμ λ€νΈμν¬μμ μΆμΆλ μμΆκΈ°λ§ λ° μ΄μκΈ°λ§ μ 보λ₯Ό κΈ°λ°μΌλ‘ 10νλ μμ΄ μΆμΆλμλ€. μ
λ ₯λ°μ΄ν°κ° μ μμΈμ§ μλμ§ κ΅¬λΆνλλ‘ νμ λΏ μλλΌ, λ§μ§λ§ λ μ΄μ΄μμ κ·ΈλΌλμΈνΈ κ°μ€ ν΄λμ€ νμ±ν 맀ν(Grad-CAM)λ°©λ²λ‘ μ μ΄μ©νμ¬ λ€νΈμν¬κ° μ΄λ―Έμ§μμ μ΄λ λΆμλ₯Ό λ³΄κ³ μ΄μμ견μΌλ‘ λΆλ₯νλμ§ μκ°ν νμλ€.
κ·Έ κ²°κ³Ό λ¨Όμ μ λμ λ€νΈμν¬ μ±λ₯μ μΈ‘μ νκΈ° μν΄ νμ 1300λͺ
μ λ°μ΄ν°λ₯Ό ν΅ν΄ κ° κ΅¬μ‘°λ¬Όμ μ§κ²½κ³Ό κ΄λ ¨λ μ¬μ₯μ§νμ΄ μΌλ§λ μ κ²μΆλλμ§ νμΈνμλ€. μ¬μ€μ€κ²©, μ’μ¬μ€ νλ²½, λλλ§₯κ³Ό κ΄λ ¨λ λ³μ μ견μ μ μΈνκ³ λ€λ₯Έκ΅¬μ‘°λ¬Όμ λ―Όκ°λμ νΉμ΄μ±μ λͺ¨λ 90% μ΄μμ΄λ€. μμΆκΈ° λ§κΈ° λ° νμ₯κΈ° λ§κΈ° μμ κ²μΆλ μ ννλλ°, μ¬μ₯μ λ¬Έμμ μν΄ μ νλ νλ μμ λΉνμ¬ μμΆκΈ° λ§κΈ°μ κ²½μ° νκ· 0.52 νλ μ, νμ₯κΈ° λ§κΈ°μ κ²½μ° 0.9 νλ μμ μ°¨μ΄λ₯Ό 보μλ€.
μ μ±λΆμμ μν λ€νΈμν¬μ κ²½μ°, 첫 λ²μ§Έ λ€νΈμν¬λ‘λΆν° μ νλ μμμ 보λ₯Ό λ°νμΌλ‘ 10κ°μ μ
λ ₯λ°μ΄ν°λ₯Ό κ²°μ νμκ³ , 무μμλ‘ μ νλ 10κ°μ κ²°κ³Όλ₯Ό λΉκ΅νμλ€. κ·Έ κ²°κ³Ό μ νλκ° κ°κ° 90.33%, 81.16%λ‘ λνλ¬μΌλ©°, 1μ°¨ μ λμ λ€νΈμν¬ μμ μΆμΆλ μμΆκΈ°λ§, μ΄μκΈ°λ§ νλ μ μ 보λ νμλ₯Ό νλ³νλ λ€νΈμν¬μ μ±λ₯ ν₯μμ κΈ°μ¬νμμ μ μ μλ€. λν Grad-CAM κ²°κ³Όλ 첫 λ²μ§Έ λ€νΈμν¬μ νλ μ μ 보λ₯Ό κΈ°λ°μΌλ‘ λ°μ΄ν°μμ μΆμΆλ10 μ₯μ μ΄λ―Έμ§κ° μ
λ ₯λ°μ΄ν°λ‘ μ°μμ λκ° λ¬΄μμλ‘ μΆμΆλ 10μ₯μ μ΄λ―Έμ§λ‘ νλ ¨λ λ€νΈμν¬ λ³΄λ€ λ³λ³μ μμΉλ₯Ό λ μ ννκ² νμνλ κ²μ νμΈνμλ€.
κ²°λ‘ μ μΌλ‘ λ³Έ μ°κ΅¬λ μ λμ , μ μ±μ λΆμμ μν AI κΈ°λ° μ¬μ₯ μ΄μν μμμμ¬ κ²°μ μ§μ μμ€ν
μ κ°λ°νμμΌλ©°, μ΄ μμ€ν
μ΄ μ€ν κ°λ₯ν κ²μΌλ‘ κ²μ¦λμλ€.Echocardiography is an indispensable tool for cardiologists in the diagnosis of heart diseases. By echocardiography, various structural abnormalities in the heart can be quantitatively or qualitatively diagnosed. Due to its non-invasiveness, the usage of echocardiography in the diagnosis of heart disease has continuously increased.
Despite the increasing role in the cardiology practice, echocardiography requires experience in capturing and knowledge in interpreting images. . Moreover, in contrast to CT or MRI images, important information can be missed once not obtained at the time of examination. Therefore, obtaining and interpreting images should be done simultaneously, or, at least, all obtained images should be audited by the experienced cardiologist before releasing the patient from the examination booth. Because of the peculiar characteristics of echocardiography compared to CT or MRI, there have been incessant demands for the clinical decision support system(CDSS) for echocardiography.
With the advance of Artificial Intelligence (AI), there have been several studies regarding decision support systems for echocardiography. The flow of these studies is divided into two approaches: One is the quantitative approach to segment the images and detects an abnormality in size and function. The other is the qualitative approach to detect abnormality in morphology. Unfortunately, most of these two studies have been conducted separately. However, since cardiologists perform quantitative and qualitative analysis simultaneously in analyzing echocardiography, an optimal CDSS needs to be a combination of these two approaches. From this point of view, this study aims to develop and validate an AI-based CDSS for echocardiograms through a large-scale retrospective cohort. Echocardiographic data of 2,600 patients who visited Seoul National University Hospital (1300 cardiac patients and 1300 non-cardiac patients with normal echocardiogram) between 2016 and 2021. Two networks were developed for the quantitative and qualitative analysis, and their usefulnesses were verified with the patient data.
First, a U-net based deep learning network was developed for segmentation in the quantitative analysis. Annotated images by the experienced cardiologist with the left ventricle, interventricular septum, left ventricular posterior wall, right ventricle, aorta, and left atrium, were used for training. The diameters and areas of the six structures were obtained and vectorized from the segmentation images, and the frame information at the end-systolic and end-diastolic phases was extracted from the vector.
The second network for the qualitative diagnosis was developed using a convolutional neural network (CNN) based on Resnet 152. The input data of this network was extracted from 10 frames of each patient based on end-diastolic and end-systolic phase information extracted from the quantitative network. The network not only distinguished the input data between normal and abnormal but also visualized the location of the abnormality on the image through the Gradient-weighted Class Activation Mapping (Grad-CAM) at the last layer.
The performance of the quantitative network in the chamber size and function measurements was assessed in 1300 patients. Sensitivity and specificity were both over 90% except for pathologies related to the left ventricular posterior wall, interventricular septum, and aorta. The end-systolic and end-diastolic phase detection was also accurate, with an average difference of 0.52 frames for the end-systolic and 0.9 frames for the end-diastolic phases.
In the case of the network for qualitative analysis, 10 input data were selected based on the phase information determined from the first network, and the results of 10 randomly selected images were compared. As a result, the accuracy was 90.3% and 81.2%, respectively, and the phase information selected from the first network contributed to the improvement of the performance of the network. Also, the results of Grad-CAM confirmed that the network trained with 10 images of data extracted based on the phase information from the first network displays the location of the lesion more accurately than the network trained with 10 randomly selected data.
In conclusion, this study proposed an AI-based CDSS for echocardiography in the quantitative and qualitative analysis.ABSTRACT οΌ
CONTENTS v
LIST OF TABLES vii
LIST OF FIGURES viii
CHAPTER 1 1
Introduction 1
1. Introduction 2
1.1. Echocardiogram 2
1.1.1. Diagnosis using Echocardiogram 2
1.1.2. Limitation in Echocardiogram 3
1.1.3. Artificial Intelligence in Echocardiogram 6
1.2. Clinical Background 7
1.2.1. Diagnostic Flow 8
1.2.2. Previous studies and clinical implication of this study 11
1.3. Technical Background 16
1.3.1. Convolutional Neural Network (CNN) 16
1.3.1.1. U-net 18
1.3.1.2. Residual Network 20
1.3.1.3. Gradient-weighted Class Activation Mapping (Grad-CAM) 22
1.4. Unmet Clinical Needs 26
1.5. Objective 27
CHAPTER 2 28
Materials & Methods 28
2. Materials & Methods 29
2.1. Data Description 29
2.2. Annotated Data 32
2.3. Overall Architecture 33
2.3.1. Quantitative Network 35
2.3.2. Qualitative Network 37
2.4. Dice Similarity Score 39
2.5. Intersection over Union 40
CHAPTER 3 41
3. Results & Discussion 42
3.1. Quantitative Network Result 42
3.1.1. Diagnostic results 47
3.1.2. Phase Detection Result 49
3.2. Qualitative Network Results 51
3.2.1. Grad-CAM Result 56
3.3. Limitation 58
3.3.1. Need for external dataset for generalizable network 58
3.3.2. Futurework of the system 59
CHAPTER 4 60
4. Conclusion 61
Abstract in Korean 62
Bibliography 65λ°