치과용 파노라마 방사선 사진에서 골다공증 선별을 위한 심층 합성곱 신경망(deep CNN)의 전이학습 전략

학위논문 (박사) -- 서울대학교 대학원 : 의과대학 의학과, 2020. 8. 최진욱.Osteoporosis is a metabolic bone disease characterized by low bone mass and disruption in bone micro-architecture. Clinical diagnostic methods for osteoporosis are expensive and therefore have limited availability in population. Recent studies have shown that Dental Panoramic Radiographs (DPRs) can provide the bone density change clues in bone structure analysis. This study aims to evaluate the discriminating performance of deep convolutional neural networks (CNNs), employed with various transfer learning strategies, on the classification of specific features of osteoporosis in DPRs. For objective labeling, we collected a dataset containing 680 images from different patients who underwent both skeletal bone mineral density and digital panoramic radiographic examinations at the Korea University Ansan Hospital between 2009 and 2018. In order to select the backbone convolutional neural network which is the basis for applying the transfer learning, we conducted preliminary experiments on the three convolutional neural networks, VGG-16, Resnet50, and Xception networks, which were frequently used in image classification. Since VGG-16 showed the best AUC value in the classification experiment conducted without transfer learning, the transfer learning using the fine-tunning technique was tested using VGG-16 as the backbone network. In order to find the optimal fine-tuning degree in the VGG-16 network, a total of six fine-tuning applied transfer learning groups were set according to the number of fine-tuning blocks in the VGG-16 with five blocks as follows: A group that does not perform fine-tuning at all (VGG-16-TF0), a group that fine-tunes the last 1 block (VGG-16-TF1), a group that fine-tuning the last 2 blocks (VGG-16-TF2), a group that fine-tuning the last 3 blocks (VGG-16-TF3), a group that fine-tuning the last 4 blocks (VGG-16-TF4), and a group that performs fine-tuning all 5 blocks (VGG-16-SCR).The best performing model (VGG-16-TF2) achieved an overall area under the receiver operating characteristic of 0.858. In this study, transfer learning and optimal fine-tuning improved the performance of a deep CNN for screening osteoporosis in DPR images. In addition, using the gradient-weighted class activation mapping technique, a visual interpretation of the best performing deep CNN model indicated that the model relied on image features in the lower left and right border of the mandibular. This result suggests that deep learning-based assessment of DPR images could be useful and reliable in the automated screening of osteoporosis patients.골다공증은 골밀도가 낮고 골 미세 구조의 붕괴가 특징 인 대사성 골 질환입니다. 그러나, 골다공증에 대한 임상 진단 방법중에 하나인 DXA 검사는 대형의 검사용 엑스레이 장비가 별도로 필요하고 검사비용이 높아, 해당 검사의 이용성에 제한성이 있습니다. 최근 연구에 따르면 치과 파노라마 방사선 사진 (DPR) 또한 골 밀도 변화를 예측 할 수 있다고 연구되었습니다. 이에 본 연구는 DPR에서 골다공증에 의한 골 밀도 변화에 따른 엑스레이 영상 특이성 분류에 다양한 전이 학습전략을 적용한 심층 합성곱 신경망 (CNN)의 분류 성능을 평가하는 것에 목표로 두었습니다. 합습 및 검증용 데이터의 객관적인 라벨링을 위해 2009년부터 2018년까지 고려 대학교 안산 병원에서 골밀도 검사와 디지털 파노라마 방사선 촬영을 6개월 이내에 동시에 시행한 환자들로부터 680개의 데이터 세트를 수집했습니다. 전이 학습 전 기본이 되는 합성곱 신경망을 선택하기 위해 이미지 분류에 자주 사용되는 3개의 합성곱 신경망 인 VGG-16, Resnet-50 및 Xception 네트워크에 대해 전이학습이 없는 상태로 사전 분류성능 평가를 수행했습니다. VGG-16은 전이 학습 없이 수행 된 분류 성능 평가에서 다른 2개의 네트워크에 비해 높은 AUC 값을 보여 주었기에, 해당 네트워크를 백본(back-bone) 네트워크로 사용하여 전이학습 효과를 비교 분석하였습니다. 백본 네트워크에서 최적의 fine-tuning 정도를 찾기 위해 VGG-16에 fine-tuning이 적용 가능한 블록 수에 따라 총 6 개의 fine-tuning 적용 전이 학습 그룹이 다음과 같이 설정 하였습니다. fine-tuning을 전혀 하지 않는 그룹 (VGG16-TR0), 마지막 1 블록을 fine-tuning 하는 그룹 (VGG-16-TF1), 마지막 2 블록을 fine-tuning 하는 그룹 (VGG-16-TF2), 마지막 3 개 블록을 fine-tuning하는 그룹 (VGG-16-TF3), 마지막 4 개 블록을 fine-tuning하는 그룹 (VGG-16-TF4) 및 5 개 블록 모두를 fine-tuning하는 그룹 (VGG16-TR5). 실험 결과 최고 성능 모델 은 VGG-16-TF2 였으며, 분류 성능 값의 하나인 AUC 값이 0.858를 달성했습니다. 본 연구를 통하여 학습용 데이터 수에 제한이 있더라도, 전이 학습 및 fine-tuning을 통하여 DPR 이미지를 이용한 골다공증 스크리닝 성능의 개선이 가능함을 보여주었습니다. 또한 gradiant-CAM 기법을 이용하여 성능이 가장 우수한 CNN 모델의 시각적 해석을 통하여, DPR 이미지 상에서 적절한 골다공증의 분류성능은 하악골의 왼쪽 및 오른쪽 하연 경계에있는 이미지에 의존한다는 것을 확인 할 수 있었습니다. 본 결과는 DPR 이미지의 딥 러닝 기반 평가가 골다공증 환자의 자동 선별에 유용하고 신뢰할 수 있음을 시사 하였습니다.Chapter 1. Introduction 1 Chapter 2. Materials and Methods 8 2.1 Dataset Collection 8 2.2 Image Preprocessing 9 2.3 Cross validation 11 2.4 Back-bone Convolutional Neural Networks 14 2.5 Evaluation 16 2.6 Visualizing Model Decisions 18 Chapter 3. Results 19 3.1 Clinical and Demographic Characteristics 19 3.2 Back-bone Convolutional Neural Networks 20 3.3 Fine-Tuning of Transferred deep CNN 22 3.4 Evaluation 27 3.5 Visualizing Model Decisions 30 Chapter 4. Discussion 33 Chapter 5. Conclusion 42 References 42 Abstract 51Docto

GABUNGAN METODE GRAY LEVEL CO-OCCURRENCE MATRIX DAN GRAY LEVEL RUN LENGTH MATRIX PADA ANALISIS CITRA RADIOGRAFI DENTAL PANORAMIC UNTUK DETEKSI DINI OSTEOPOROSIS

ABSTRAKOsteoporosis merupakan salah satu masalah kesehatan utama. Osteoporosis dianggap sebagai penyakit metabolik yang umum, dan sering diabaikan. Penyakit ini kebanyakan menyerang wanita dewasa yang dapat menyebabkan  kekurusan dan kerapuhan tulang, dan memicu patah tulang. Osteoporosis didiagnosis dengan mengukur Densitas Mineral Tulang menggunakan DXA (dual energy X-ray absorptiometry). Perawatan dengan alat ini membutuhkan biaya yang mahal, dan alat ini tidak tersedia secara luas. Sampel penelitian ini mengambil 19 orang dengan kriteria inklusi perempuan telah menopause, dinyatakan sehat, tidak mengalami patah tulang dan tidak memiliki kelainan tulang sejak lahir. Sampel diukur nilai bone mineral density (BMD) atau derajat osteoporosis dengan menggunakan DXA. Kemudian dilakukan pemotretan radiografi untuk mendapatkan citra dental panoramic. Tahapan penelitian adalah: 1) melakukan pre-processing terhadap citra radiografi panoramic tulang mandibular; 2) menentukan nilai tekstur citra metode  gray level co-occurrence matrix 3) menentukan nilai tekstur citra metode  gray level run length matrix 4) mengkalisifikasikan menggunakan metode k means kluster. Hasil Klasifikasi dengan menggunakan k means Kluster menunjukkan ketepatan klasifikasi sebesar 89,47% Kata kunci: radiografi; citra tulang rahang; BMD; analisis tekstur. ABSTRACTOsteoporosis is one of the major health problems. Osteoporosis is considered a common metabolic disease, and is often overlooked. This disease mostly affects adult women which can cause thin and brittle bones, and trigger fractures. Osteoporosis is diagnosed by measuring Bone Mineral Density using DXA (dual energy X-ray absorptiometry). Treatment with this device is expensive, and it is not widely available. The sample of this study took 19 people with the inclusion criteria of women having menopause, declared healthy, had no fractures and had no bone abnormalities since birth. The sample was measured the value of bone mineral density (BMD) or the degree of osteoporosis using DXA. Then, radiography was taken to obtain a panoramic dental image. The stages of the research are: 1) pre-processing the panoramic radiographic image of the mandible; 2) determine the texture value of the image using the gray level co-occurrence matrix method 3) determine the texture value of the image using the gray level run length matrix method 4) classify it using the k means cluster method.Classification results using k means clusters show the classification accuracy of 89.47% Keywords:. Radiography; dental panoramic; BMD; texture analysi

Identification of osteoporosis using ensemble deep learning model with panoramic radiographs and clinical covariates

Osteoporosis is becoming a global health issue due to increased life expectancy. However, it is difficult to detect in its early stages owing to a lack of discernible symptoms. Hence, screening for osteoporosis with widely used dental panoramic radiographs would be very cost-effective and useful. In this study, we investigate the use of deep learning to classify osteoporosis from dental panoramic radiographs. In addition, the effect of adding clinical covariate data to the radiographic images on the identification performance was assessed. For objective labeling, a dataset containing 778 images was collected from patients who underwent both skeletal-bone-mineral density measurement and dental panoramic radiography at a single general hospital between 2014 and 2020. Osteoporosis was assessed from the dental panoramic radiographs using convolutional neural network (CNN) models, including EfficientNet-b0, -b3, and -b7 and ResNet-18, -50, and -152. An ensemble model was also constructed with clinical covariates added to each CNN. The ensemble model exhibited improved performance on all metrics for all CNNs, especially accuracy and AUC. The results show that deep learning using CNN can accurately classify osteoporosis from dental panoramic radiographs. Furthermore, it was shown that the accuracy can be improved using an ensemble model with patient covariates

Multi-Task Deep Learning Model for Classification of Dental Implant Brand and Treatment Stage Using Dental Panoramic Radiograph Images

It is necessary to accurately identify dental implant brands and the stage of treatment to ensure efficient care. Thus, the purpose of this study was to use multi-task deep learning to investigate a classifier that categorizes implant brands and treatment stages from dental panoramic radiographic images. For objective labeling, 9767 dental implant images of 12 implant brands and treatment stages were obtained from the digital panoramic radiographs of patients who underwent procedures at Kagawa Prefectural Central Hospital, Japan, between 2005 and 2020. Five deep convolutional neural network (CNN) models (ResNet18, 34, 50, 101 and 152) were evaluated. The accuracy, precision, recall, specificity, F1 score, and area under the curve score were calculated for each CNN. We also compared the multi-task and single-task accuracies of brand classification and implant treatment stage classification. Our analysis revealed that the larger the number of parameters and the deeper the network, the better the performance for both classifications. Multi-tasking significantly improved brand classification on all performance indicators, except recall, and significantly improved all metrics in treatment phase classification. Using CNNs conferred high validity in the classification of dental implant brands and treatment stages. Furthermore, multi-task learning facilitated analysis accuracy

Parieto-Occipital Alpha and Low-Beta EEG Power Reflect Sense of Agency

The sense of agency (SoA) is part of psychophysiological modules related to the self. Disturbed SoA is found in several clinical conditions, hence understanding the neural correlates of the SoA is useful for the diagnosis and determining the proper treatment strategies. Although there are several neuroimaging studies on SoA, it is desirable to translate the knowledge to more accessible and inexpensive EEG-based biomarkers for the sake of applicability. However, SoA has not been widely investigated using EEG. To address this issue, we designed an EEG experiment on healthy adults (n = 15) to determine the sensitivity of EEG on the SoA paradigm using hand movement with parametrically delayed visual feedback. We calculated the power spectral density over the traditional EEG frequency bands for ten delay conditions relative to no delay condition. Independent component analysis and equivalent current dipole modeling were applied to address artifact rejection, volume conduction, and source localization to determine the effect of interest. The results revealed that the alpha and low-beta EEG power increased in the parieto-occipital regions in proportion to the reduced SoA reported by the subjects. We conclude that the parieto-occipital alpha and low-beta EEG power reflect the sense of agency

Machine Learning in Dentistry: A Scoping Review

Machine learning (ML) is being increasingly employed in dental research and application. We aimed to systematically compile studies using ML in dentistry and assess their methodological quality, including the risk of bias and reporting standards. We evaluated studies employing ML in dentistry published from 1 January 2015 to 31 May 2021 on MEDLINE, IEEE Xplore, and arXiv. We assessed publication trends and the distribution of ML tasks (classification, object detection, semantic segmentation, instance segmentation, and generation) in different clinical fields. We appraised the risk of bias and adherence to reporting standards, using the QUADAS-2 and TRIPOD checklists, respectively. Out of 183 identified studies, 168 were included, focusing on various ML tasks and employing a broad range of ML models, input data, data sources, strategies to generate reference tests, and performance metrics. Classification tasks were most common. Forty-two different metrics were used to evaluate model performances, with accuracy, sensitivity, precision, and intersection-over-union being the most common. We observed considerable risk of bias and moderate adherence to reporting standards which hampers replication of results. A minimum (core) set of outcome and outcome metrics is necessary to facilitate comparisons across studies

Evaluation of PD-L1 expression in various formalin-fixed paraffin embedded tumour tissue samples using SP263, SP142 and QR1 antibody clones

Background & objectives: Cancer cells can avoid immune destruction through the inhibitory ligand PD-L1. PD-1 is a surface cell receptor, part of the immunoglobulin family. Its ligand PD-L1 is expressed by tumour cells and stromal tumour infltrating lymphocytes (TIL). Methods: Forty-four cancer cases were included in this study (24 triple-negative breast cancers (TNBC), 10 non-small cell lung cancer (NSCLC) and 10 malignant melanoma cases). Three clones of monoclonal primary antibodies were compared: QR1 (Quartett), SP 142 and SP263 (Ventana). For visualization, ultraView Universal DAB Detection Kit from Ventana was used on an automated platform for immunohistochemical staining Ventana BenchMark GX. Results: Comparing the sensitivity of two different clones on same tissue samples from TNBC, we found that the QR1 clone gave higher percentage of positive cells than clone SP142, but there was no statistically significant difference. Comparing the sensitivity of two different clones on same tissue samples from malignant melanoma, the SP263 clone gave higher percentage of positive cells than the QR1 clone, but again the difference was not statistically significant. Comparing the sensitivity of two different clones on same tissue samples from NSCLC, we found higher percentage of positive cells using the QR1 clone in comparison with the SP142 clone, but once again, the difference was not statistically significant. Conclusion: The three different antibody clones from two manufacturers Ventana and Quartett, gave comparable results with no statistically significant difference in staining intensity/ percentage of positive tumour and/or immune cells. Therefore, different PD-L1 clones from different manufacturers can potentially be used to evaluate the PD- L1 status in different tumour tissues. Due to the serious implications of the PD-L1 analysis in further treatment decisions for cancer patients, every antibody clone, staining protocol and evaluation process should be carefully and meticulously validated