162 research outputs found
Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification
Computer-aided pathology diagnosis based on the classification of Whole Slide
Image (WSI) plays an important role in clinical practice, and it is often
formulated as a weakly-supervised Multiple Instance Learning (MIL) problem.
Existing methods solve this problem from either a bag classification or an
instance classification perspective. In this paper, we propose an end-to-end
weakly supervised knowledge distillation framework (WENO) for WSI
classification, which integrates a bag classifier and an instance classifier in
a knowledge distillation framework to mutually improve the performance of both
classifiers. Specifically, an attention-based bag classifier is used as the
teacher network, which is trained with weak bag labels, and an instance
classifier is used as the student network, which is trained using the
normalized attention scores obtained from the teacher network as soft pseudo
labels for the instances in positive bags. An instance feature extractor is
shared between the teacher and the student to further enhance the knowledge
exchange between them. In addition, we propose a hard positive instance mining
strategy based on the output of the student network to force the teacher
network to keep mining hard positive instances. WENO is a plug-and-play
framework that can be easily applied to any existing attention-based bag
classification methods. Extensive experiments on five datasets demonstrate the
efficiency of WENO. Code is available at https://github.com/miccaiif/WENO.Comment: Accepted by NeurIPS 202
Reducing Domain Gap in Frequency and Spatial domain for Cross-modality Domain Adaptation on Medical Image Segmentation
Unsupervised domain adaptation (UDA) aims to learn a model trained on source
domain and performs well on unlabeled target domain. In medical image
segmentation field, most existing UDA methods depend on adversarial learning to
address the domain gap between different image modalities, which is ineffective
due to its complicated training process. In this paper, we propose a simple yet
effective UDA method based on frequency and spatial domain transfer uner
multi-teacher distillation framework. In the frequency domain, we first
introduce non-subsampled contourlet transform for identifying domain-invariant
and domain-variant frequency components (DIFs and DVFs), and then keep the DIFs
unchanged while replacing the DVFs of the source domain images with that of the
target domain images to narrow the domain gap. In the spatial domain, we
propose a batch momentum update-based histogram matching strategy to reduce the
domain-variant image style bias. Experiments on two cross-modality medical
image segmentation datasets (cardiac, abdominal) show that our proposed method
achieves superior performance compared to state-of-the-art methods.Comment: accepted at Thirty-Seventh AAAI Conference on Artificial Intelligence
(AAAI-23
Towards Arbitrary-scale Histopathology Image Super-resolution: An Efficient Dual-branch Framework based on Implicit Self-texture Enhancement
Existing super-resolution models for pathology images can only work in fixed
integer magnifications and have limited performance. Though implicit neural
network-based methods have shown promising results in arbitrary-scale
super-resolution of natural images, it is not effective to directly apply them
in pathology images, because pathology images have special fine-grained image
textures different from natural images. To address this challenge, we propose a
dual-branch framework with an efficient self-texture enhancement mechanism for
arbitrary-scale super-resolution of pathology images. Extensive experiments on
two public datasets show that our method outperforms both existing fixed-scale
and arbitrary-scale algorithms. To the best of our knowledge, this is the first
work to achieve arbitrary-scale super-resolution in the field of pathology
images. Codes will be available
The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification
This paper introduces the novel concept of few-shot weakly supervised
learning for pathology Whole Slide Image (WSI) classification, denoted as FSWC.
A solution is proposed based on prompt learning and the utilization of a large
language model, GPT-4. Since a WSI is too large and needs to be divided into
patches for processing, WSI classification is commonly approached as a Multiple
Instance Learning (MIL) problem. In this context, each WSI is considered a bag,
and the obtained patches are treated as instances. The objective of FSWC is to
classify both bags and instances with only a limited number of labeled bags.
Unlike conventional few-shot learning problems, FSWC poses additional
challenges due to its weak bag labels within the MIL framework. Drawing
inspiration from the recent achievements of vision-language models (V-L models)
in downstream few-shot classification tasks, we propose a two-level prompt
learning MIL framework tailored for pathology, incorporating language prior
knowledge. Specifically, we leverage CLIP to extract instance features for each
patch, and introduce a prompt-guided pooling strategy to aggregate these
instance features into a bag feature. Subsequently, we employ a small number of
labeled bags to facilitate few-shot prompt learning based on the bag features.
Our approach incorporates the utilization of GPT-4 in a question-and-answer
mode to obtain language prior knowledge at both the instance and bag levels,
which are then integrated into the instance and bag level language prompts.
Additionally, a learnable component of the language prompts is trained using
the available few-shot labeled data. We conduct extensive experiments on three
real WSI datasets encompassing breast cancer, lung cancer, and cervical cancer,
demonstrating the notable performance of the proposed method in bag and
instance classification. All codes will be made publicly accessible
Unifying Large Language Models and Knowledge Graphs: A Roadmap
Large language models (LLMs), such as ChatGPT and GPT4, are making new waves
in the field of natural language processing and artificial intelligence, due to
their emergent ability and generalizability. However, LLMs are black-box
models, which often fall short of capturing and accessing factual knowledge. In
contrast, Knowledge Graphs (KGs), Wikipedia and Huapu for example, are
structured knowledge models that explicitly store rich factual knowledge. KGs
can enhance LLMs by providing external knowledge for inference and
interpretability. Meanwhile, KGs are difficult to construct and evolving by
nature, which challenges the existing methods in KGs to generate new facts and
represent unseen knowledge. Therefore, it is complementary to unify LLMs and
KGs together and simultaneously leverage their advantages. In this article, we
present a forward-looking roadmap for the unification of LLMs and KGs. Our
roadmap consists of three general frameworks, namely, 1) KG-enhanced LLMs,
which incorporate KGs during the pre-training and inference phases of LLMs, or
for the purpose of enhancing understanding of the knowledge learned by LLMs; 2)
LLM-augmented KGs, that leverage LLMs for different KG tasks such as embedding,
completion, construction, graph-to-text generation, and question answering; and
3) Synergized LLMs + KGs, in which LLMs and KGs play equal roles and work in a
mutually beneficial way to enhance both LLMs and KGs for bidirectional
reasoning driven by both data and knowledge. We review and summarize existing
efforts within these three frameworks in our roadmap and pinpoint their future
research directions.Comment: 29 pages, 25 figure
Recommended from our members
Arabidopsis SWR1-associated protein methyl-CpG-binding domain 9 is required for histone H2A.Z deposition.
Deposition of the histone variant H2A.Z by the SWI2/SNF2-Related 1 chromatin remodeling complex (SWR1-C) is important for gene regulation in eukaryotes, but the composition of the Arabidopsis SWR1-C has not been thoroughly characterized. Here, we aim to identify interacting partners of a conserved Arabidopsis SWR1 subunit ACTIN-RELATED PROTEIN 6 (ARP6). We isolate nine predicted components and identify additional interactors implicated in histone acetylation and chromatin biology. One of the interacting partners, methyl-CpG-binding domain 9 (MBD9), also strongly interacts with the Imitation SWItch (ISWI) chromatin remodeling complex. MBD9 is required for deposition of H2A.Z at a distinct subset of ARP6-dependent loci. MBD9 is preferentially bound to nucleosome-depleted regions at the 5' ends of genes containing high levels of activating histone marks. These data suggest that MBD9 is a SWR1-C interacting protein required for H2A.Z deposition at a subset of actively transcribing genes
A Survey on Temporal Knowledge Graph Completion: Taxonomy, Progress, and Prospects
Temporal characteristics are prominently evident in a substantial volume of
knowledge, which underscores the pivotal role of Temporal Knowledge Graphs
(TKGs) in both academia and industry. However, TKGs often suffer from
incompleteness for three main reasons: the continuous emergence of new
knowledge, the weakness of the algorithm for extracting structured information
from unstructured data, and the lack of information in the source dataset.
Thus, the task of Temporal Knowledge Graph Completion (TKGC) has attracted
increasing attention, aiming to predict missing items based on the available
information. In this paper, we provide a comprehensive review of TKGC methods
and their details. Specifically, this paper mainly consists of three
components, namely, 1)Background, which covers the preliminaries of TKGC
methods, loss functions required for training, as well as the dataset and
evaluation protocol; 2)Interpolation, that estimates and predicts the missing
elements or set of elements through the relevant available information. It
further categorizes related TKGC methods based on how to process temporal
information; 3)Extrapolation, which typically focuses on continuous TKGs and
predicts future events, and then classifies all extrapolation methods based on
the algorithms they utilize. We further pinpoint the challenges and discuss
future research directions of TKGC
Cold sintering of microwave dielectric ceramics and devices
Microwave (MW) dielectric ceramics are used in numerous electronic components for modern wireless communication systems, including antennas, resonators, capacitors and filters. However, to date, MW ceramics are manufactured by an energy-intensive, conventional high-temperature (> 1000 °C) sintering technology and thus cannot be co-sintered with low melting point and base electrodes (Ag, Al, etc., < 1000 °C), nor directly integrated with polymers (< 200 °C). Cold sintering is able to densify ceramics at < 200 °C via a combination of external pressure and a transient liquid phase, reducing the energy consumed and facilitating greater integration with dissimilar materials. This review outlines the basics of MW ceramics alongside the mechanism of cold sintering. Recent developments in cold sintering of MW ceramics, composites and devices are described, emphasizing new materials and progress towards component/device fabrication. Future prospects and critical issues for advancing cold-sintered MW materials and devices, such as unclear mechanism, low Q × f values and poor mechanical properties, are discussed
- …