Pose-Aware Instance Segmentation Framework from Cone Beam CT Images for Tooth Segmentation

Individual tooth segmentation from cone beam computed tomography (CBCT) images is an essential prerequisite for an anatomical understanding of orthodontic structures in several applications, such as tooth reformation planning and implant guide simulations. However, the presence of severe metal artifacts in CBCT images hinders the accurate segmentation of each individual tooth. In this study, we propose a neural network for pixel-wise labeling to exploit an instance segmentation framework that is robust to metal artifacts. Our method comprises of three steps: 1) image cropping and realignment by pose regressions, 2) metal-robust individual tooth detection, and 3) segmentation. We first extract the alignment information of the patient by pose regression neural networks to attain a volume-of-interest (VOI) region and realign the input image, which reduces the inter-overlapping area between tooth bounding boxes. Then, individual tooth regions are localized within a VOI realigned image using a convolutional detector. We improved the accuracy of the detector by employing non-maximum suppression and multiclass classification metrics in the region proposal network. Finally, we apply a convolutional neural network (CNN) to perform individual tooth segmentation by converting the pixel-wise labeling task to a distance regression task. Metal-intensive image augmentation is also employed for a robust segmentation of metal artifacts. The result shows that our proposed method outperforms other state-of-the-art methods, especially for teeth with metal artifacts. The primary significance of the proposed method is two-fold: 1) an introduction of pose-aware VOI realignment followed by a robust tooth detection and 2) a metal-robust CNN framework for accurate tooth segmentation.Comment: 10 pages, 10 figure

Machine learning methods as an aid in planning orthodontic treatment on the example of Cone-Beam Computed Tomography analysis: a literature review

Convolutional neural networks (CNNs) are used in many areas of computer vision, such as object tracking and recognition, security, military, and biomedical image analysis. In this work, we describe the current methods, the architectures of deep convolutional neural networks used in CBCT. Literature from 2000-2020 from the PubMed database, Google Scholar, was analyzed. Account has been taken of publications in English that describe architectures of deep convolutional neural networks used in CBCT. The results of the reviewed studies indicate that deep learning methods employed in orthodontics can be far superior in comparison to other high-performing algorithms

Fully automatic integration of dental CBCT images and full-arch intraoral impressions with stitching error correction via individual tooth segmentation and identification

We present a fully automated method of integrating intraoral scan (IOS) and dental cone-beam computerized tomography (CBCT) images into one image by complementing each image's weaknesses. Dental CBCT alone may not be able to delineate precise details of the tooth surface due to limited image resolution and various CBCT artifacts, including metal-induced artifacts. IOS is very accurate for the scanning of narrow areas, but it produces cumulative stitching errors during full-arch scanning. The proposed method is intended not only to compensate the low-quality of CBCT-derived tooth surfaces with IOS, but also to correct the cumulative stitching errors of IOS across the entire dental arch. Moreover, the integration provide both gingival structure of IOS and tooth roots of CBCT in one image. The proposed fully automated method consists of four parts; (i) individual tooth segmentation and identification module for IOS data (TSIM-IOS); (ii) individual tooth segmentation and identification module for CBCT data (TSIM-CBCT); (iii) global-to-local tooth registration between IOS and CBCT; and (iv) stitching error correction of full-arch IOS. The experimental results show that the proposed method achieved landmark and surface distance errors of 112.4 $\mu$m and 301.7 $\mu$m, respectively

Multiclass CBCT image segmentation for orthodontics with deep learning

Accurate segmentation of the jaw (i.e., mandible and maxilla) and the teeth in cone beam computed tomography (CBCT) scans is essential for orthodontic diagnosis and treatment planning. Although various (semi)automated methods have been proposed to segment the jaw or the teeth, there is still a lack of fully automated segmentation methods that can simultaneously segment both anatomic structures in CBCT scans (i.e., multiclass segmentation). In this study, we aimed to train and validate a mixed-scale dense (MS-D) convolutional neural network for multiclass segmentation of the jaw, the teeth, and the background in CBCT scans. Thirty CBCT scans were obtained from patients who had undergone orthodontic treatment. Gold standard segmentation labels were manually created by 4 dentists. As a benchmark, we also evaluated MS-D networks that segmented the jaw or the teeth (i.e., binary segmentation). All segmented CBCT scans were converted to virtual 3-dimensional (3D) models. The segmentation performance of all trained MS-D networks was assessed by the Dice similarity coefficient and surface deviation. The CBCT scans segmented by the MS-D network demonstrated a large overlap with the gold standard segmentations (Dice similarity coefficient: 0.934 ± 0.019, jaw; 0.945 ± 0.021, teeth). The MS-D network–based 3D models of the jaw and the teeth showed minor surface deviations when compared with the corresponding gold standard 3D models (0.390 ± 0.093 mm, jaw; 0.204 ± 0.061 mm, teeth). The MS-D network took approximately 25 s to segment 1 CBCT scan, whereas manual segmentation took about 5 h. This study showed that multiclass segmentation of jaw and teeth was accurate and its performance was comparable to binary segmentation. The MS-D network trained for multiclass segmentation would therefore make patient-specific orthodontic treatment more feasible by strongly reducing the time required to segment multiple anatomic structures in CBCT scans

Automatic mandibular canal detection using a deep convolutional neural network

The practicability of deep learning techniques has been demonstrated by their successful implementation in varied fields, including diagnostic imaging for clinicians. In accordance with the increasing demands in the healthcare industry, techniques for automatic prediction and detection are being widely researched. Particularly in dentistry, for various reasons, automated mandibular canal detection has become highly desirable. The positioning of the inferior alveolar nerve (IAN), which is one of the major structures in the mandible, is crucial to prevent nerve injury during surgical procedures. However, automatic segmentation using Cone beam computed tomography (CBCT) poses certain difficulties, such as the complex appearance of the human skull, limited number of datasets, unclear edges, and noisy images. Using work-in-progress automation software, experiments were conducted with models based on 2D SegNet, 2D and 3D U-Nets as preliminary research for a dental segmentation automation tool. The 2D U-Net with adjacent images demonstrates higher global accuracy of 0.82 than naïve U-Net variants. The 2D SegNet showed the second highest global accuracy of 0.96, and the 3D U-Net showed the best global accuracy of 0.99. The automated canal detection system through deep learning will contribute significantly to efficient treatment planning and to reducing patients’ discomfort by a dentist. This study will be a preliminary report and an opportunity to explore the application of deep learning to other dental fields.Peer reviewe

Review on Region-Based Segmentation Using Watershed and Region Growing Techniques and their Applications in Different Fields

In digital image processing and computer vision, segmentation operation for an image refers to dividing an image into multiple image segments, and the significant purpose of segmentation operation is to depict an image in a way so that the analysis process of the objects of interest is easier and more accurate. The region-based segmentation scheme act for finding similarities between adjacent pixels to detect each region that constructs the image. Similarity scales have based on different features, in a grayscale image, the scale may be referred to as textures and other spatial appearances, and also the variance in intensity of a region and so on. Significantly, many applications in different fields involved region-based segmentation for instance remote sensing, medical application, and others for recognizing interesting objects in an image. In this paper, two techniques for segmentation operation in region-based which are region growing and watershed are reviewed

Künstliche Intelligenz in der Zahnheilkunde: Scoping-Review und Schließung beobachteter Wissenslücken durch eine methodische und eine klinische Studie

Objectives: The aims of this dissertation were to (1) conduct a scoping review of stud-ies on machine learning (ML) in dentistry and appraise their robustness, (2) perform a benchmarking study to systematically compare various ML algorithms for a specific dental task, and (3) evaluate the influence of a ML-based caries detection software on diagnostic accuracy and decision-making in a randomized controlled trial. Methods: The scoping review included studies using ML in dentistry published between 1st January 2015 and 31st May 2021 on MEDLINE, IEEE Xplore, and arXiv. The risk of bias and reporting quality were assessed with the QUADAS‐2 and TRIPOD checklists, respectively. In the benchmarking study, 216 ML models were built using permutations of six ML model architectures (U-Net, U-Net++, Feature Pyramid Networks, LinkNet, Pyramid Scene Parsing Network, and Mask Attention Network), 12 model backbones of varying complexities (ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, VGG13, VGG16, VGG19, DenseNet121, DenseNet161, DenseNet169, and Dense-Net201), and three initialization strategies (random, ImageNet, and CheXpert weights). 1,625 dental bitewing radiographs were used for training and testing. Five-fold cross-validation was carried out and model performance assessed using F1-score. In the clin-ical trial, each one of 22 dentists examined 20 randomly selected bitewing images for proximal caries; 10 images were evaluated with ML and 10 images without ML. Accura-cy in lesion detection and the suggested treatment were evaluated. Results: The scoping review included 168 studies, describing different ML tasks, mod-els, input data, methods to generate reference tests, and performance metrics, imped-ing comparison across studies. The studies showed considerable risk of bias and mod-erate adherence to reporting standards. In the benchmarking study, more complex models only minimally outperformed their simpler counterparts, if at all. Models initial-ized by ImageNet or CheXpert weights outperformed those using random weights (p<0.05). The clinical trial demonstrated that dentists using ML showed increased accu-racy (area under the receiver operating characteristic [mean (95% confidence interval): 0.89 (0.87–0.90)]) compared with those not using ML [0.85 (0.83–0.86); p<0.05], pri-marily due to their higher sensitivity [0.81 (0.74–0.87) compared to 0.72 (0.64–0.79); p<0.05]. Notably, dentists using ML also showed a higher frequency of invasive treat-ment decisions than those not using it (p<0.05). Conclusion: To facilitate comparisons across ML studies in dentistry, a minimum (core) set of outcomes and metrics should be developed, and researchers should strive to improve robustness and reporting quality of their studies. ML model choice should be performed on an informed basis, and simpler models may often be similarly capable as more complex ones. ML can increase dentists’ diagnostic accuracy but also lead to more invasive treatment.Ziele: Die Ziele dieser Dissertation waren, (1) ein Scoping-Review von Studien über maschinelles Lernen (ML) in der Zahnmedizin, (2) eine Benchmarking-Studie zum systematischen Vergleich verschiedener ML-Algorithmen für eine bestimmte zahnmedizinische Aufgabe, und (3) eine randomisierte kontrollierte Studie zur Bewertung einer ML-basierten Karies-Erkennungssoftware bezüglich diagnostischer Genauigkeit und Einfluss auf den Entscheidungsprozess durchzuführen. Methoden: Das Scoping-Review umfasste Studien über ML in der Zahnmedizin, veröffentlicht vom 1. Januar 2015 bis 31. Mai 2021 auf MEDLINE, IEEE Xplore und arXiv. Bias-Risiko und Berichtsqualität wurden mit den Checklisten QUADAS-2 beziehungsweise TRIPOD bewertet. In der Benchmarking-Studie wurden 216 ML-Modelle durch Permutationen von sechs Architekturen (U-Net, U-Net++, Feature Pyramid Networks, LinkNet, Pyramid Scene Parsing Network und Mask Attention Network), 12 Backbones (Res-Net18, ResNet34, ResNet50, ResNet101, ResNet152, VGG13, VGG16, VGG19, DenseNet121, DenseNet161, DenseNet169 und DenseNet201) und drei Initialisierungsstrategien (zufällige-, ImageNet- und CheXpert-Gewichtungen) erstellt. Zum Training und Testen wurden 1.625 Bissflügel-Röntgenaufnahmen genutzt. Es wurde eine fünffache Kreuzvalidierung durchgeführt und die Modellleistung anhand des F1-Scores bewertet. In der klinischen Studie untersuchten 22 Zahnärzte jeweils 20 zufällig ausgewählte Bissflügelbilder auf Approximalkaries; 10 Bilder wurden mit und 10 Bilder ohne ML ausgewertet. Die Genauigkeit in der Erkennung von Läsionen sowie die abgeleitete Therapieempfehlung wurden bewertet. Ergebnisse: Das Scoping-Review schloss 168 Studien ein, in denen verschiedene ML-Aufgaben, Modelle, Eingabedaten, Methoden zur Generierung von Referenztests und Leistungsmetriken beschrieben wurden. Die Studien zeigten ein erhebliches Bias-Risiko und eine mäßige Einhaltung der Berichtsstandards. In der Benchmarking-Studie hatten komplexere Modelle gegenüber einfachen Modellen allenfalls geringe Vorteile. Mit ImageNet- oder CheXpert-Gewichtungen initialisierte Modelle übertrafen solche mit Zufallsgewichtungen (p<0,05). In der klinischen Studie erreichten Zahnärzte mit ML eine höhere Genauigkeit bei der Kariesdetektion (Receiver-Operating-Charakteristik [Mittelwert (95 % Konfidenzintervall) 0,89 (0,87–0,90)]) als ohne ML [0,85 (0,83–0,86); p<0,05], hauptsächlich aufgrund höherer Sensitivität [0,81 (0,74–0,87) verglichen mit 0,72 (0,64–0,79); p<0,05]. Zahnärzte mit ML wählten auffallend häufiger invasive Behandlungen als ohne ML (p<0,05). Schlussfolgerung: Zur besseren Vergleichbarkeit von ML-Studien in der Zahnmedizin, sollten Core Outcomes und Metriken definiert sowie Robustheit und Berichtsqualität verbessert werden. Die Entwicklung von ML-Modellen sollte auf informierter Basis erfolgen, bei oft ähnlicher Leistung von einfacheren und komplexeren Modellen. ML kann die diagnostische Genauigkeit erhöhen, aber auch zu mehr invasiven Behandlungen führen