9,887 research outputs found

    Focused Decoding Enables 3D Anatomical Detection by Transformers

    Full text link
    Detection Transformers represent end-to-end object detection approaches based on a Transformer encoder-decoder architecture, exploiting the attention mechanism for global relation modeling. Although Detection Transformers deliver results on par with or even superior to their highly optimized CNN-based counterparts operating on 2D natural images, their success is closely coupled to access to a vast amount of training data. This, however, restricts the feasibility of employing Detection Transformers in the medical domain, as access to annotated data is typically limited. To tackle this issue and facilitate the advent of medical Detection Transformers, we propose a novel Detection Transformer for 3D anatomical structure detection, dubbed Focused Decoder. Focused Decoder leverages information from an anatomical region atlas to simultaneously deploy query anchors and restrict the cross-attention's field of view to regions of interest, which allows for a precise focus on relevant anatomical structures. We evaluate our proposed approach on two publicly available CT datasets and demonstrate that Focused Decoder not only provides strong detection results and thus alleviates the need for a vast amount of annotated data but also exhibits exceptional and highly intuitive explainability of results via attention weights. Our code is available at https://github.com/bwittmann/transoar.Comment: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:00

    An advanced deep learning models-based plant disease detection: A review of recent research

    Get PDF
    Plants play a crucial role in supplying food globally. Various environmental factors lead to plant diseases which results in significant production losses. However, manual detection of plant diseases is a time-consuming and error-prone process. It can be an unreliable method of identifying and preventing the spread of plant diseases. Adopting advanced technologies such as Machine Learning (ML) and Deep Learning (DL) can help to overcome these challenges by enabling early identification of plant diseases. In this paper, the recent advancements in the use of ML and DL techniques for the identification of plant diseases are explored. The research focuses on publications between 2015 and 2022, and the experiments discussed in this study demonstrate the effectiveness of using these techniques in improving the accuracy and efficiency of plant disease detection. This study also addresses the challenges and limitations associated with using ML and DL for plant disease identification, such as issues with data availability, imaging quality, and the differentiation between healthy and diseased plants. The research provides valuable insights for plant disease detection researchers, practitioners, and industry professionals by offering solutions to these challenges and limitations, providing a comprehensive understanding of the current state of research in this field, highlighting the benefits and limitations of these methods, and proposing potential solutions to overcome the challenges of their implementation

    Segment Anything Model (SAM) for Radiation Oncology

    Full text link
    In this study, we evaluate the performance of the Segment Anything Model (SAM) model in clinical radiotherapy. We collected real clinical cases from four regions at the Mayo Clinic: prostate, lung, gastrointestinal, and head \& neck, which are typical treatment sites in radiation oncology. For each case, we selected the OARs of concern in radiotherapy planning and compared the Dice and Jaccard outcomes between clinical manual delineation, automatic segmentation using SAM's "segment anything" mode, and automatic segmentation using SAM with box prompt. Our results indicate that SAM performs better in automatic segmentation for the prostate and lung regions, while its performance in the gastrointestinal and head \& neck regions was relatively inferior. When considering the size of the organ and the clarity of its boundary, SAM displays better performance for larger organs with clear boundaries, such as the lung and liver, and worse for smaller organs with unclear boundaries, like the parotid and cochlea. These findings align with the generally accepted variations in difficulty level associated with manual delineation of different organs at different sites in clinical radiotherapy. Given that SAM, a single trained model, could handle the delineation of OARs in four regions, these results also demonstrate SAM's robust generalization capabilities in automatic segmentation for radiotherapy, i.e., achieving delineation of different radiotherapy OARs using a generic automatic segmentation model. SAM's generalization capabilities across different regions make it technically feasible to develop a generic model for automatic segmentation in radiotherapy

    Research progress on deep learning in magnetic resonance imaging–based diagnosis and treatment of prostate cancer: a review on the current status and perspectives

    Get PDF
    Multiparametric magnetic resonance imaging (mpMRI) has emerged as a first-line screening and diagnostic tool for prostate cancer, aiding in treatment selection and noninvasive radiotherapy guidance. However, the manual interpretation of MRI data is challenging and time-consuming, which may impact sensitivity and specificity. With recent technological advances, artificial intelligence (AI) in the form of computer-aided diagnosis (CAD) based on MRI data has been applied to prostate cancer diagnosis and treatment. Among AI techniques, deep learning involving convolutional neural networks contributes to detection, segmentation, scoring, grading, and prognostic evaluation of prostate cancer. CAD systems have automatic operation, rapid processing, and accuracy, incorporating multiple sequences of multiparametric MRI data of the prostate gland into the deep learning model. Thus, they have become a research direction of great interest, especially in smart healthcare. This review highlights the current progress of deep learning technology in MRI-based diagnosis and treatment of prostate cancer. The key elements of deep learning-based MRI image processing in CAD systems and radiotherapy of prostate cancer are briefly described, making it understandable not only for radiologists but also for general physicians without specialized imaging interpretation training. Deep learning technology enables lesion identification, detection, and segmentation, grading and scoring of prostate cancer, and prediction of postoperative recurrence and prognostic outcomes. The diagnostic accuracy of deep learning can be improved by optimizing models and algorithms, expanding medical database resources, and combining multi-omics data and comprehensive analysis of various morphological data. Deep learning has the potential to become the key diagnostic method in prostate cancer diagnosis and treatment in the future

    Challenges for Monocular 6D Object Pose Estimation in Robotics

    Full text link
    Object pose estimation is a core perception task that enables, for example, object grasping and scene understanding. The widely available, inexpensive and high-resolution RGB sensors and CNNs that allow for fast inference based on this modality make monocular approaches especially well suited for robotics applications. We observe that previous surveys on object pose estimation establish the state of the art for varying modalities, single- and multi-view settings, and datasets and metrics that consider a multitude of applications. We argue, however, that those works' broad scope hinders the identification of open challenges that are specific to monocular approaches and the derivation of promising future challenges for their application in robotics. By providing a unified view on recent publications from both robotics and computer vision, we find that occlusion handling, novel pose representations, and formalizing and improving category-level pose estimation are still fundamental challenges that are highly relevant for robotics. Moreover, to further improve robotic performance, large object sets, novel objects, refractive materials, and uncertainty estimates are central, largely unsolved open challenges. In order to address them, ontological reasoning, deformability handling, scene-level reasoning, realistic datasets, and the ecological footprint of algorithms need to be improved.Comment: arXiv admin note: substantial text overlap with arXiv:2302.1182

    A view of colonial life in South Australia: An osteological investigation of the health status among 19th-century migrant settlers

    Get PDF
    Studies of human skeletal remains contribute to understanding the extent to which conditions prevailing in various past communities were detrimental to health. Few of these studies have evaluated the situation in which the first European colonists of South Australia lived. Colonial Australian skeletal collections are scarce, especially for research purposes. This makes the 19th-century skeletal remains of individuals, excavated from St Mary’s Cemetery, South Australia, a rare and valuable collection. The overarching aim of this thesis was to investigate the general and oral health of this specific group of 19th-century settlers, through the examination of their skeletons and dentitions. Four research papers in this thesis address this overarching aim. The first two papers determine the general skeletal health of the settlers, with a focus on pathological manifestations on bones associated with metabolic deficiencies and the demands of establishing an industrial society. Paper 3 investigated whether Large Volume Micro- Computed Tomography (LV Micro-CT) could be used as a single technique for the analysis of the in situ dentoalveolar complex of individuals from St Mary’s. This led to a detailed investigation of the dentitions of the St Mary’s sample, in paper 4, with the aims of determining the oral health status of these individuals, and understanding how oral conditions may have influenced their general health. The skeletal remains of 65 individuals (20 adults and 45 subadults) from St Mary’s sample were available for the four component investigations using non-destructive techniques - macroscopic, radiographic and micro-CT methods. Signs of nutritional deficiencies (vitamin C and iron) were identified in Paper 1. The findings of paper 2 showed joint diseases and traumatic fractures were seen and that gastrointestinal and pulmonary conditions were the leading causes of death in subadults and adults respectively. Paper 3 found that the LV Micro-CT technique was the only method able to generate images that allowed the full range of detailed measurements across all the oral health categories studied. A combination of macroscopic and radiographic techniques covered a number of these categories, but was more time-consuming, and did not provide the same level of accuracy or include all measurements. Results for paper 4 confirmed that extensive carious lesions, antemortem tooth loss and evidence of periodontal disease were present in the St Mary’s sample. Developmental defects of enamel (EH) and areas of interglobular dentine (IGD) were identified. Many individuals with dental defects also had skeletal signs of co-morbidities. St Mary’s individuals had a similar percentage of carious lesions as the British sample, which was more than other historic Australian samples, but less than a contemporary New Zealand sample. The 19th-century migrants to the colony of South Australia were faced with multiple challenges such as adapting to local environmental conditions as well as participating in the development of settlements, infrastructure and new industries. Evidence of joint diseases, traumatic injuries and health insults, seen as pathological changes and/ or abnormalities on the bone and/or teeth, confirmed that the settlers' health had been affected. The number of burials in the ‘free ground’ area between the 1840s -1870s was greater than the number in the leased plots, reflecting the economic problems of the colony during these early years. Validation of the reliability and accuracy of the LV Micro-CT system for the analysis of the dentoalveolar complex, in situ within archaeological human skull samples, provided a microanalytical approach for the in-depth investigations of the St Mary’s dentition. Extensive carious lesions, antemortem tooth loss and periodontal disease seen in this group would have affected their general health status. The presence of developmental defects (EH and IGD) indicated that many of the settlers had suffered health insults in childhood to young adulthood. Contemporaneous Australian, New Zealand and British samples had comparable findings suggesting that little improvement had occurred in their oral health since arriving in South Australia. In conclusion, the findings of this investigation largely fulfilled the initial aims. Our understanding of the extent to which conditions prevailing in the new colony were detrimental to human health has increased, as has our knowledge of why pathological manifestations and/or abnormalities were seen on the bones and teeth of individuals from the St Mary’s sample. A multiple-method approach, to derive enhanced information has been shown to be effective, whilst establishing a new methodology (LV Micro-CT) for the analysis of dentition in situ in human archaeological skulls. Further, this investigation has digitally preserved data relating to this historical group of individuals for future comparisons.Thesis (Ph.D.) -- University of Adelaide, School of Biomedicine, 202

    Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives

    Full text link
    Deep learning has demonstrated remarkable performance across various tasks in medical imaging. However, these approaches primarily focus on supervised learning, assuming that the training and testing data are drawn from the same distribution. Unfortunately, this assumption may not always hold true in practice. To address these issues, unsupervised domain adaptation (UDA) techniques have been developed to transfer knowledge from a labeled domain to a related but unlabeled domain. In recent years, significant advancements have been made in UDA, resulting in a wide range of methodologies, including feature alignment, image translation, self-supervision, and disentangled representation methods, among others. In this paper, we provide a comprehensive literature review of recent deep UDA approaches in medical imaging from a technical perspective. Specifically, we categorize current UDA research in medical imaging into six groups and further divide them into finer subcategories based on the different tasks they perform. We also discuss the respective datasets used in the studies to assess the divergence between the different domains. Finally, we discuss emerging areas and provide insights and discussions on future research directions to conclude this survey.Comment: Under Revie

    CLIP goes 3D: Leveraging Prompt Tuning for Language Grounded 3D Recognition

    Full text link
    Vision-Language models like CLIP have been widely adopted for various tasks due to their impressive zero-shot capabilities. However, CLIP is not suitable for extracting 3D geometric features as it was trained on only images and text by natural language supervision. We work on addressing this limitation and propose a new framework termed CG3D (CLIP Goes 3D) where a 3D encoder is learned to exhibit zero-shot capabilities. CG3D is trained using triplets of pointclouds, corresponding rendered 2D images, and texts using natural language supervision. To align the features in a multimodal embedding space, we utilize contrastive loss on 3D features obtained from the 3D encoder, as well as visual and text features extracted from CLIP. We note that the natural images used to train CLIP and the rendered 2D images in CG3D have a distribution shift. Attempting to train the visual and text encoder to account for this shift results in catastrophic forgetting and a notable decrease in performance. To solve this, we employ prompt tuning and introduce trainable parameters in the input space to shift CLIP towards the 3D pre-training dataset utilized in CG3D. We extensively test our pre-trained CG3D framework and demonstrate its impressive capabilities in zero-shot, open scene understanding, and retrieval tasks. Further, it also serves as strong starting weights for fine-tuning in downstream 3D recognition tasks.Comment: Website: https://jeya-maria-jose.github.io/cg3d-web

    Multi-head attention-based masked sequence model for mapping functional brain networks

    Get PDF
    The investigation of functional brain networks (FBNs) using task-based functional magnetic resonance imaging (tfMRI) has gained significant attention in the field of neuroimaging. Despite the availability of several methods for constructing FBNs, including traditional methods like GLM and deep learning methods such as spatiotemporal self-attention mechanism (STAAE), these methods have design and training limitations. Specifically, they do not consider the intrinsic characteristics of fMRI data, such as the possibility that the same signal value at different time points could represent different brain states and meanings. Furthermore, they overlook prior knowledge, such as task designs, during training. This study aims to overcome these limitations and develop a more efficient model by drawing inspiration from techniques in the field of natural language processing (NLP). The proposed model, called the Multi-head Attention-based Masked Sequence Model (MAMSM), uses a multi-headed attention mechanism and mask training approach to learn different states corresponding to the same voxel values. Additionally, it combines cosine similarity and task design curves to construct a novel loss function. The MAMSM was applied to seven task state datasets from the Human Connectome Project (HCP) tfMRI dataset. Experimental results showed that the features acquired by the MAMSM model exhibit a Pearson correlation coefficient with the task design curves above 0.95 on average. Moreover, the model can extract more meaningful networks beyond the known task-related brain networks. The experimental results demonstrated that MAMSM has great potential in advancing the understanding of functional brain networks
    • …
    corecore