6 research outputs found
LatentSwap3D: Semantic Edits on 3D Image GANs
3D GANs have the ability to generate latent codes for entire 3D volumes
rather than only 2D images. These models offer desirable features like
high-quality geometry and multi-view consistency, but, unlike their 2D
counterparts, complex semantic image editing tasks for 3D GANs have only been
partially explored. To address this problem, we propose LatentSwap3D, a
semantic edit approach based on latent space discovery that can be used with
any off-the-shelf 3D or 2D GAN model and on any dataset. LatentSwap3D relies on
identifying the latent code dimensions corresponding to specific attributes by
feature ranking using a random forest classifier. It then performs the edit by
swapping the selected dimensions of the image being edited with the ones from
an automatically selected reference image. Compared to other latent space
control-based edit methods, which were mainly designed for 2D GANs, our method
on 3D GANs provides remarkably consistent semantic edits in a disentangled
manner and outperforms others both qualitatively and quantitatively. We show
results on seven 3D GANs (pi-GAN, GIRAFFE, StyleSDF, MVCGAN, EG3D, StyleNeRF,
and VolumeGAN) and on five datasets (FFHQ, AFHQ, Cats, MetFaces, and CompCars).Comment: The paper has been accepted by ICCV'23 AI3DC
Diffusion-Based Hierarchical Multi-Label Object Detection to Analyze Panoramic Dental X-rays
Due to the necessity for precise treatment planning, the use of panoramic
X-rays to identify different dental diseases has tremendously increased.
Although numerous ML models have been developed for the interpretation of
panoramic X-rays, there has not been an end-to-end model developed that can
identify problematic teeth with dental enumeration and associated diagnoses at
the same time. To develop such a model, we structure the three distinct types
of annotated data hierarchically following the FDI system, the first labeled
with only quadrant, the second labeled with quadrant-enumeration, and the third
fully labeled with quadrant-enumeration-diagnosis. To learn from all three
hierarchies jointly, we introduce a novel diffusion-based hierarchical
multi-label object detection framework by adapting a diffusion-based method
that formulates object detection as a denoising diffusion process from noisy
boxes to object boxes. Specifically, to take advantage of the hierarchically
annotated data, our method utilizes a novel noisy box manipulation technique by
adapting the denoising process in the diffusion network with the inference from
the previously trained model in hierarchical order. We also utilize a
multi-label object detection method to learn efficiently from partial
annotations and to give all the needed information about each abnormal tooth
for treatment planning. Experimental results show that our method significantly
outperforms state-of-the-art object detection methods, including RetinaNet,
Faster R-CNN, DETR, and DiffusionDet for the analysis of panoramic X-rays,
demonstrating the great potential of our method for hierarchically and
partially annotated datasets. The code and the data are available at:
https://github.com/ibrahimethemhamamci/HierarchicalDet.Comment: MICCAI 202
DENTEX: An Abnormal Tooth Detection with Dental Enumeration and Diagnosis Benchmark for Panoramic X-rays
Panoramic X-rays are frequently used in dentistry for treatment planning, but
their interpretation can be both time-consuming and prone to error. Artificial
intelligence (AI) has the potential to aid in the analysis of these X-rays,
thereby improving the accuracy of dental diagnoses and treatment plans.
Nevertheless, designing automated algorithms for this purpose poses significant
challenges, mainly due to the scarcity of annotated data and variations in
anatomical structure. To address these issues, the Dental Enumeration and
Diagnosis on Panoramic X-rays Challenge (DENTEX) has been organized in
association with the International Conference on Medical Image Computing and
Computer-Assisted Intervention (MICCAI) in 2023. This challenge aims to promote
the development of algorithms for multi-label detection of abnormal teeth,
using three types of hierarchically annotated data: partially annotated
quadrant data, partially annotated quadrant-enumeration data, and fully
annotated quadrant-enumeration-diagnosis data, inclusive of four different
diagnoses. In this paper, we present the results of evaluating participant
algorithms on the fully annotated data, additionally investigating performance
variation for quadrant, enumeration, and diagnosis labels in the detection of
abnormal teeth. The provision of this annotated dataset, alongside the results
of this challenge, may lay the groundwork for the creation of AI-powered tools
that can offer more precise and efficient diagnosis and treatment planning in
the field of dentistry. The evaluation code and datasets can be accessed at
https://github.com/ibrahimethemhamamci/DENTEXComment: MICCAI 2023 Challeng
Diffusion-based hierarchical multi-label object detection to analyze panoramic dental X-rays
Due to the necessity for precise treatment planning, the use of panoramic X-rays to identify different dental diseases has tremendously increased. Although numerous ML models have been developed for the interpretation of panoramic X-rays, there has not been an end-to-end model developed that can identify problematic teeth with dental enumeration and associated diagnoses at the same time. To develop such a model, we structure the three distinct types of annotated data hierarchically following the FDI system, the first labeled with only quadrant, the second labeled with quadrant-enumeration, and the third fully labeled with quadrant-enumeration-diagnosis. To learn from all three hierarchies jointly, we introduce a novel diffusion-based hierarchical multi-label object detection framework by adapting a diffusion-based method that formulates object detection as a denoising diffusion process from noisy boxes to object boxes. Specifically, to take advantage of the hierarchically annotated data, our method utilizes a novel noisy box manipulation technique by adapting the denoising process in the diffusion network with the inference from the previously trained model in hierarchical order. We also utilize a multi-label object detection method to learn efficiently from partial annotations and to give all the needed information about each abnormal tooth for treatment planning. Experimental results show that our method significantly outperforms state-of-the-art object detection methods, including RetinaNet, Faster R-CNN, DETR, and DiffusionDet for the analysis of panoramic X-rays, demonstrating the great potential of our method for hierarchically and partially annotated datasets. The code and the datasets are available at https://github.com/ibrahimethemhamamci/HierarchicalDet.Helmut Horten Stiftun
Dental enumeration and multiple treatment detection on panoramic X-rays using deep learning
Abstract In this paper, a new powerful deep learning framework, named as DENTECT, is developed in order to instantly detect five different dental treatment approaches and simultaneously number the dentition based on the FDI notation on panoramic X-ray images. This makes DENTECT the first system that focuses on identification of multiple dental treatments; namely periapical lesion therapy, fillings, root canal treatment (RCT), surgical extraction, and conventional extraction all of which are accurately located within their corresponding borders and tooth numbers. Although DENTECT is trained on only 1005 images, the annotations supplied by experts provide satisfactory results for both treatment and enumeration detection. This framework carries out enumeration with an average precision (AP) score of 89.4% and performs treatment identification with a 59.0% AP score. Clinically, DENTECT is a practical and adoptable tool that accelerates the process of treatment planning with a level of accuracy which could compete with that of dental clinicians