1,394 research outputs found
Medical Image Analysis using Deep Relational Learning
In the past ten years, with the help of deep learning, especially the rapid
development of deep neural networks, medical image analysis has made remarkable
progress. However, how to effectively use the relational information between
various tissues or organs in medical images is still a very challenging
problem, and it has not been fully studied. In this thesis, we propose two
novel solutions to this problem based on deep relational learning. First, we
propose a context-aware fully convolutional network that effectively models
implicit relation information between features to perform medical image
segmentation. The network achieves the state-of-the-art segmentation results on
the Multi Modal Brain Tumor Segmentation 2017 (BraTS2017) and Multi Modal Brain
Tumor Segmentation 2018 (BraTS2018) data sets. Subsequently, we propose a new
hierarchical homography estimation network to achieve accurate medical image
mosaicing by learning the explicit spatial relationship between adjacent
frames. We use the UCL Fetoscopy Placenta dataset to conduct experiments and
our hierarchical homography estimation network outperforms the other
state-of-the-art mosaicing methods while generating robust and meaningful
mosaicing result on unseen frames.Comment: arXiv admin note: substantial text overlap with arXiv:2007.0778
A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks
Transformer is a deep neural network that employs a self-attention mechanism
to comprehend the contextual relationships within sequential data. Unlike
conventional neural networks or updated versions of Recurrent Neural Networks
(RNNs) such as Long Short-Term Memory (LSTM), transformer models excel in
handling long dependencies between input sequence elements and enable parallel
processing. As a result, transformer-based models have attracted substantial
interest among researchers in the field of artificial intelligence. This can be
attributed to their immense potential and remarkable achievements, not only in
Natural Language Processing (NLP) tasks but also in a wide range of domains,
including computer vision, audio and speech processing, healthcare, and the
Internet of Things (IoT). Although several survey papers have been published
highlighting the transformer's contributions in specific fields, architectural
differences, or performance evaluations, there is still a significant absence
of a comprehensive survey paper encompassing its major applications across
various domains. Therefore, we undertook the task of filling this gap by
conducting an extensive survey of proposed transformer models from 2017 to
2022. Our survey encompasses the identification of the top five application
domains for transformer-based models, namely: NLP, Computer Vision,
Multi-Modality, Audio and Speech Processing, and Signal Processing. We analyze
the impact of highly influential transformer-based models in these domains and
subsequently classify them based on their respective tasks using a proposed
taxonomy. Our aim is to shed light on the existing potential and future
possibilities of transformers for enthusiastic researchers, thus contributing
to the broader understanding of this groundbreaking technology
Immune contexture monitoring in solid tumors focusing on Head and Neck Cancer
Forti evidenze dimostrano una stretta interazione tra il sistema immunitario e lo sviluppo biologico e la progressione clinica dei tumori solidi. L'effetto che il microambiente immunitario del tumore può avere sul comportamento clinico della malattia è indicato come "immunecontexture". Nonostante ciò, l'attuale gestione clinica dei pazienti affetti da cancro non tiene conto di alcuna caratteristica immunologica né per la stadiazione né per le scelte terapeutiche. Il tumore della testa e del collo (HNSCC) rappresenta il 7° tumore più comune al mondo ed è caratterizzato da una prognosi relativamente sfavorevole e dall'effetto negativo dei trattamenti sulla qualità della vita dei pazienti. Oltre alla chirurgia e alla radioterapia, sono disponibili pochi trattamenti sistemici, rappresentati principalmente dalla chemioterapia a base di platino-derivati o dal cetuximab. L'immunoterapia è una nuova strategia terapeutica ancora limitata al setting palliativo (malattia ricorrente non resecabile o metastatica). La ricerca di nuovi biomarcatori o possibili nuovi meccanismi target è molto rilevante quindi nel contesto clinico dell'HNSCC. In questa tesi ci si concentrerà sullo studio di tre possibili popolazioni immunitarie pro-tumorali studiate nell'HNSCC: i neutrofili tumore-associati (TAN), le cellule B intratumorali con fenotipo immunosoppressivo e i T-reg CD8+. Particolare attenzione è data all'applicazione di moderne tecniche biostatistiche e bioinformatiche per riassumere informazioni complesse derivate da variabili cliniche e immunologiche multiparametriche e per validare risultati derivati ​​in situ, attraverso dati di espressione genica derivati da dataset pubblici. Infine, la seconda parte della tesi prenderà in considerazione progetti di ricerca clinica rilevanti, volti a migliorare l'oncologia di precisione nell'HNSCC, sviluppando modelli predittivi di sopravvivenza, confrontando procedure oncologiche alternative, validando nuovi classificatori o testando l'uso di nuovi protocolli clinici come l'uso dell'immunonutrizione.Strong evidences demonstrate a close interplay between the immune system and the biological development and clinical progression of solid tumors. The effect that the tumor immune microenvironment can have on the clinical behavior of the disease is referred as the immuno contexture. Nevertheless, the current clinical management of patients affected by cancer does not take into account any immunological features either for the staging or for the treatment choices. Head and Neck Cancer (HNSCC) represents the 7th most common cancer worldwide and it is characterized by a relatively poor prognosis and detrimental effect of treatments on the quality of life of patients. Beyond surgery and radiotherapy, few systemic treatments are available, mainly represented by platinum-based chemotherapy or cetuximab. Immunotherapy is a new therapeutical strategy still limited to the palliative setting (recurrent not resectable or metastatic disease). The search for new biomarkers or possible new targetable mechanisms is meaningful especially in the clinical setting of HNSCC. In this thesis a focus will be given on the study of three possible pro-tumoral immune populations studied in HNSCC: the tumor associated neutrophils (TAN), intratumoral B-cells with a immunosuppressive phenotype and the CD8+ T-regs. Biostatistical and bioinformatical techniques are applied to summarize complex information derived from multiparametric clinical and immunological variables and to validate in-situ derived findings through gene expression data of public available datasets. Lastly, the second part of the thesis will take into account relevant clinical research projects, aimed at improving the precision oncology in HNSCC developing survival prediction models, comparing alternative oncological procedures, validating new classifiers or testing the use of novel clinical protocols as the use of immunnutrition
Animal models and their role in imaging-assisted co-clinical trials
The availability of high-fidelity animal models for oncology research has grown enormously in recent years, enabling preclinical studies relevant to prevention, diagnosis, and treatment of cancer to be undertaken. This has led to increased opportunities to conduct co-clinical trials, which are studies on patients that are carried out parallel to or sequentially with animal models of cancer that mirror the biology of the patients\u27 tumors. Patient-derived xenografts (PDX) and genetically engineered mouse models (GEMM) are considered to be the models that best represent human disease and have high translational value. Notably, one element of co-clinical trials that still needs significant optimization is quantitative imaging. The National Cancer Institute has organized a Co-Clinical Imaging Resource Program (CIRP) network to establish best practices for co-clinical imaging and to optimize translational quantitative imaging methodologies. This overview describes the ten co-clinical trials of investigators from eleven institutions who are currently supported by the CIRP initiative and are members of the Animal Models and Co-clinical Trials (AMCT) Working Group. Each team describes their corresponding clinical trial, type of cancer targeted, rationale for choice of animal models, therapy, and imaging modalities. The strengths and weaknesses of the co-clinical trial design and the challenges encountered are considered. The rich research resources generated by the members of the AMCT Working Group will benefit the broad research community and improve the quality and translational impact of imaging in co-clinical trials
Analytical validation of innovative magneto-inertial outcomes: a controlled environment study.
peer reviewe
Image-based Decision Support Systems: Technical Concepts, Design Knowledge, and Applications for Sustainability
Unstructured data accounts for 80-90% of all data generated, with image data contributing its largest portion. In recent years, the field of computer vision, fueled by deep learning techniques, has made significant advances in exploiting this data to generate value. However, often computer vision models are not sufficient for value creation. In these cases, image-based decision support systems (IB-DSSs), i.e., decision support systems that rely on images and computer vision, can be used to create value by combining human and artificial intelligence. Despite its potential, there is only little work on IB-DSSs so far.
In this thesis, we develop technical foundations and design knowledge for IBDSSs and demonstrate the possible positive effect of IB-DSSs on environmental sustainability. The theoretical contributions of this work are based on and evaluated in a series of artifacts in practical use cases: First, we use technical experiments to demonstrate the feasibility of innovative approaches to exploit images for IBDSSs.
We show the feasibility of deep-learning-based computer vision and identify future research opportunities based on one of our practical use cases. Building on this, we develop and evaluate a novel approach for combining human and artificial intelligence for value creation from image data. Second, we develop design knowledge that can serve as a blueprint for future IB-DSSs. We perform two design science research studies to formulate generalizable principles for purposeful design — one for IB-DSSs and one for the subclass of image-mining-based decision support systems (IM-DSSs). While IB-DSSs can provide decision support based on single images, IM-DSSs are suitable when large amounts of image data are available and required for decision-making. Third, we demonstrate the viability of applying IBDSSs to enhance environmental sustainability by performing life cycle assessments for two practical use cases — one in which the IB-DSS enables a prolonged product lifetime and one in which the IB-DSS facilitates an improvement of manufacturing processes.
We hope this thesis will contribute to expand the use and effectiveness of imagebased decision support systems in practice and will provide directions for future research
SAM-Med3D
Although the Segment Anything Model (SAM) has demonstrated impressive
performance in 2D natural image segmentation, its application to 3D volumetric
medical images reveals significant shortcomings, namely suboptimal performance
and unstable prediction, necessitating an excessive number of prompt points to
attain the desired outcomes. These issues can hardly be addressed by
fine-tuning SAM on medical data because the original 2D structure of SAM
neglects 3D spatial information. In this paper, we introduce SAM-Med3D, the
most comprehensive study to modify SAM for 3D medical images. Our approach is
characterized by its comprehensiveness in two primary aspects: firstly, by
comprehensively reformulating SAM to a thorough 3D architecture trained on a
comprehensively processed large-scale volumetric medical dataset; and secondly,
by providing a comprehensive evaluation of its performance. Specifically, we
train SAM-Med3D with over 131K 3D masks and 247 categories. Our SAM-Med3D
excels at capturing 3D spatial information, exhibiting competitive performance
with significantly fewer prompt points than the top-performing fine-tuned SAM
in the medical domain. We then evaluate its capabilities across 15 datasets and
analyze it from multiple perspectives, including anatomical structures,
modalities, targets, and generalization abilities. Our approach, compared with
SAM, showcases pronouncedly enhanced efficiency and broad segmentation
capabilities for 3D volumetric medical images. Our code is released at
https://github.com/uni-medical/SAM-Med3D
Segmentation of Pathology Images: A Deep Learning Strategy with Annotated Data
Cancer has significantly threatened human life and health for many years. In the clinic, histopathology image segmentation is the golden stand for evaluating the prediction of patient prognosis and treatment outcome. Generally, manually labelling tumour regions in hundreds of high-resolution histopathological images is time-consuming and expensive for pathologists. Recently, the advancements in hardware and computer vision have allowed deep-learning-based methods to become mainstream to segment tumours automatically, significantly reducing the workload of pathologists. However, most current methods rely on large-scale labelled histopathological images. Therefore, this research studies label-effective tumour segmentation methods using deep-learning paradigms to relieve the annotation limitations. Chapter 3 proposes an ensemble framework for fully-supervised tumour segmentation. Usually, the performance of an individual-trained network is limited by significant morphological variances in histopathological images. We propose a fully-supervised learning ensemble fusion model that uses both shallow and deep U-Nets, trained with images of different resolutions and subsets of images, for robust predictions of tumour regions. Noise elimination is achieved with Convolutional Conditional Random Fields. Two open datasets are used to evaluate the proposed method: the ACDC@LungHP challenge at ISBI2019 and the DigestPath challenge at MICCAI2019. With a dice coefficient of 79.7 %, the proposed method takes third place in ACDC@LungHP. In DigestPath 2019, the proposed method achieves a dice coefficient 77.3 %. Well-annotated images are an indispensable part of training fully-supervised segmentation strategies. However, large-scale histopathology images are hardly annotated finely in clinical practice. It is common for labels to be of poor quality or for only a few images to be manually marked by experts. Consequently, fully-supervised methods cannot perform well in these cases. Chapter 4 proposes a self-supervised contrast learning for tumour segmentation. A self-supervised cancer segmentation framework is proposed to reduce label dependency. An innovative contrastive learning scheme is developed to represent tumour features based on unlabelled images. Unlike a normal U-Net, the backbone is a patch-based segmentation network. Additionally, data augmentation and contrastive losses are applied to improve the discriminability of tumour features. A convolutional Conditional Random Field is used to smooth and eliminate noise. Three labelled, and fourteen unlabelled images are collected from a private skin cancer dataset called BSS. Experimental results show that the proposed method achieves better tumour segmentation performance than other popular self-supervised methods. However, by evaluated on the same public dataset as chapter 3, the proposed self-supervised method is hard to handle fine-grained segmentation around tumour boundaries compared to the supervised method we proposed. Chapter 5 proposes a sketch-based weakly-supervised tumour segmentation method. To segment tumour regions precisely with coarse annotations, a sketch-supervised method is proposed, containing a dual CNN-Transformer network and a global normalised class activation map. CNN-Transformer networks simultaneously model global and local tumour features. With the global normalised class activation map, a gradient-based tumour representation can be obtained from the dual network predictions. We invited experts to mark fine and coarse annotations in the private BSS and the public PAIP2019 datasets to facilitate reproducible performance comparisons. Using the BSS dataset, the proposed method achieves 76.686 % IOU and 86.6 % Dice scores, outperforming state-of-the-art methods. Additionally, the proposed method achieves a Dice gain of 8.372 % compared with U-Net on the PAIP2019 dataset. The thesis presents three approaches to segmenting cancers from histology images: fully-supervised, unsupervised, and weakly supervised methods. This research effectively segments tumour regions based on histopathological annotations and well-designed modules. Our studies comprehensively demonstrate label-effective automatic histopathological image segmentation. Experimental results prove that our works achieve state-of-the-art segmentation performances on private and public datasets. In the future, we plan to integrate more tumour feature representation technologies with other medical modalities and apply them to clinical research
Beam scanning by liquid-crystal biasing in a modified SIW structure
A fixed-frequency beam-scanning 1D antenna based on Liquid Crystals (LCs) is designed for application in 2D scanning with lateral alignment. The 2D array environment imposes full decoupling of adjacent 1D antennas, which often conflicts with the LC requirement of DC biasing: the proposed design accommodates both. The LC medium is placed inside a Substrate Integrated Waveguide (SIW) modified to work as a Groove Gap Waveguide, with radiating slots etched on the upper broad wall, that radiates as a Leaky-Wave Antenna (LWA). This allows effective application of the DC bias voltage needed for tuning the LCs. At the same time, the RF field remains laterally confined, enabling the possibility to lay several antennas in parallel and achieve 2D beam scanning. The design is validated by simulation employing the actual properties of a commercial LC medium
Is attention all you need in medical image analysis? A review
Medical imaging is a key component in clinical diagnosis, treatment planning
and clinical trial design, accounting for almost 90% of all healthcare data.
CNNs achieved performance gains in medical image analysis (MIA) over the last
years. CNNs can efficiently model local pixel interactions and be trained on
small-scale MI data. The main disadvantage of typical CNN models is that they
ignore global pixel relationships within images, which limits their
generalisation ability to understand out-of-distribution data with different
'global' information. The recent progress of Artificial Intelligence gave rise
to Transformers, which can learn global relationships from data. However, full
Transformer models need to be trained on large-scale data and involve
tremendous computational complexity. Attention and Transformer compartments
(Transf/Attention) which can well maintain properties for modelling global
relationships, have been proposed as lighter alternatives of full Transformers.
Recently, there is an increasing trend to co-pollinate complementary
local-global properties from CNN and Transf/Attention architectures, which led
to a new era of hybrid models. The past years have witnessed substantial growth
in hybrid CNN-Transf/Attention models across diverse MIA problems. In this
systematic review, we survey existing hybrid CNN-Transf/Attention models,
review and unravel key architectural designs, analyse breakthroughs, and
evaluate current and future opportunities as well as challenges. We also
introduced a comprehensive analysis framework on generalisation opportunities
of scientific and clinical impact, based on which new data-driven domain
generalisation and adaptation methods can be stimulated
- …