The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification

This paper introduces the novel concept of few-shot weakly supervised learning for pathology Whole Slide Image (WSI) classification, denoted as FSWC. A solution is proposed based on prompt learning and the utilization of a large language model, GPT-4. Since a WSI is too large and needs to be divided into patches for processing, WSI classification is commonly approached as a Multiple Instance Learning (MIL) problem. In this context, each WSI is considered a bag, and the obtained patches are treated as instances. The objective of FSWC is to classify both bags and instances with only a limited number of labeled bags. Unlike conventional few-shot learning problems, FSWC poses additional challenges due to its weak bag labels within the MIL framework. Drawing inspiration from the recent achievements of vision-language models (V-L models) in downstream few-shot classification tasks, we propose a two-level prompt learning MIL framework tailored for pathology, incorporating language prior knowledge. Specifically, we leverage CLIP to extract instance features for each patch, and introduce a prompt-guided pooling strategy to aggregate these instance features into a bag feature. Subsequently, we employ a small number of labeled bags to facilitate few-shot prompt learning based on the bag features. Our approach incorporates the utilization of GPT-4 in a question-and-answer mode to obtain language prior knowledge at both the instance and bag levels, which are then integrated into the instance and bag level language prompts. Additionally, a learnable component of the language prompts is trained using the available few-shot labeled data. We conduct extensive experiments on three real WSI datasets encompassing breast cancer, lung cancer, and cervical cancer, demonstrating the notable performance of the proposed method in bag and instance classification. All codes will be made publicly accessible

Robust Point Cloud Registration Framework Based on Deep Graph Matching(TPAMI Version)

3D point cloud registration is a fundamental problem in computer vision and robotics. Recently, learning-based point cloud registration methods have made great progress. However, these methods are sensitive to outliers, which lead to more incorrect correspondences. In this paper, we propose a novel deep graph matching-based framework for point cloud registration. Specifically, we first transform point clouds into graphs and extract deep features for each point. Then, we develop a module based on deep graph matching to calculate a soft correspondence matrix. By using graph matching, not only the local geometry of each point but also its structure and topology in a larger range are considered in establishing correspondences, so that more correct correspondences are found. We train the network with a loss directly defined on the correspondences, and in the test stage the soft correspondences are transformed into hard one-to-one correspondences so that registration can be performed by a correspondence-based solver. Furthermore, we introduce a transformer-based method to generate edges for graph construction, which further improves the quality of the correspondences. Extensive experiments on object-level and scene-level benchmark datasets show that the proposed method achieves state-of-the-art performance. The code is available at: \href{https://github.com/fukexue/RGM}{https://github.com/fukexue/RGM}.Comment: accepted by TPAMI 2022. arXiv admin note: substantial text overlap with arXiv:2103.0425

Group-in-Group Relation-Based Transformer for 3D Point Cloud Learning

Deep point cloud neural networks have achieved promising performance in remote sensing applications, and the prevalence of Transformer in natural language processing and computer vision is in stark contrast to underexplored point-based methods. In this paper, we propose an effective transformer-based network for point cloud learning. To better learn global and local information, we propose a group-in-group relation-based transformer architecture to learn the relationships between point groups to model global information and between points within each group to model local semantic information. To further enhance the local feature representation, we propose a Radius Feature Abstraction (RFA) module to extract radius-based density features characterizing the sparsity of local point clouds. Extensive evaluation on public benchmark datasets demonstrate the effectiveness and competitive performance of our proposed method on point cloud classification and part segmentation

Group-in-Group Relation-Based Transformer for 3D Point Cloud Learning

Deep point cloud neural networks have achieved promising performance in remote sensing applications, and the prevalence of Transformer in natural language processing and computer vision is in stark contrast to underexplored point-based methods. In this paper, we propose an effective transformer-based network for point cloud learning. To better learn global and local information, we propose a group-in-group relation-based transformer architecture to learn the relationships between point groups to model global information and between points within each group to model local semantic information. To further enhance the local feature representation, we propose a Radius Feature Abstraction (RFA) module to extract radius-based density features characterizing the sparsity of local point clouds. Extensive evaluation on public benchmark datasets demonstrate the effectiveness and competitive performance of our proposed method on point cloud classification and part segmentation

DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion

Point cloud processing is a challenging task due to its sparsity and irregularity. Prior works introduce delicate designs on either local feature aggregator or global geometric architecture, but few combine both advantages. We propose Dual-Scale Point Cloud Recognition with High-frequency Fusion (DSPoint) to extract local-global features by concurrently operating on voxels and points. We reverse the conventional design of applying convolution on voxels and attention to points. Specifically, we disentangle point features through channel dimension for dual-scale processing: one by point-wise convolution for fine-grained geometry parsing, the other by voxel-wise global attention for long-range structural exploration. We design a co-attention fusion module for feature alignment to blend local-global modalities, which conducts inter-scale cross-modality interaction by communicating high-frequency coordinates information. Experiments and ablations on widely-adopted ModelNet40, ShapeNet, and S3DIS demonstrate the state-of-the-art performance of our DSPoint

Estrogen regulation of cardiac cAMP-L-type Ca2+ channel pathway modulates sex differences in basal contraction and responses to β2AR-mediated stress in left ventricular apical myocytes

Abstract Backgrounds/Aim Male and female hearts have many structural and functional differences. Here, we investigated the role of estrogen (E2) in the mechanisms of sex differences in contraction through the cAMP-L-type Ca2+channel pathway in adult mice left ventricular (LV) apical myocytes at basal and stress state. Methods Isolated LV apical myocytes from male, female (Sham) and ovariectomised mice (OVX) were used to investigate contractility, Ca2+ transients and L-type Ca2+ channel (LTCC) function. The levels of β2AR, intracellular cAMP, phosphodiesterase (PDE 3 and PDE 4), RyR2, PLB, SLN, and SERCA2a were compared among the experimental groups. Results We found that (1) intracellular cAMP, I CaL density, contraction and Ca2+ transient amplitudes were larger in Sham and OVX + E2 myocytes compared to male and OVX. (2) The mRNA expression of PDE 3 and 4 were lower in Sham and OVX + E2 groups compared with male and OVX groups. Treatment of myocytes with IBMX (100 μM) increased contraction and Ca2+ transient amplitude in both sexes and canceled differences between them. (3) β2AR-mediated stress decreased cAMP concentration and peak contraction and Ca2+ transient amplitude only in male and OVX groups but not in Sham or OVX + E2 groups suggesting a cardioprotective role of E2 in female mice. (4) Pretreatment of OVX myocytes with GPR30 antagonist G15 (100 nM) abolished the effects of E2, but ERα and ERβ antagonist ICI 182,780 (1 μM) did not. Moreover, activation of GPR30 with G1 (100 nM) replicated the effects of E2 on cAMP, contraction and Ca2+ transient amplitudes suggesting that the acute effects of E2 were mediated by GPR30 via non-genomic signaling. (5) mRNA expression of RyR2 was higher in myocytes from Sham than those of male while PLB and SLN were higher in male than Sham but no sex differences were observed in the mRNA of SERCA2a. Conclusion Collectively, these results demonstrate that E2 modulates the expression of genes related to the cAMP-LTCC pathway and contributes to sex differences in cardiac contraction and responses to stress. We also show that estrogen confers cardioprotection against cardiac stress by non-genomic acute signaling via GPR30