75 research outputs found
Beyond Empirical Risk Minimization: Local Structure Preserving Regularization for Improving Adversarial Robustness
It is broadly known that deep neural networks are susceptible to being fooled
by adversarial examples with perturbations imperceptible by humans. Various
defenses have been proposed to improve adversarial robustness, among which
adversarial training methods are most effective. However, most of these methods
treat the training samples independently and demand a tremendous amount of
samples to train a robust network, while ignoring the latent structural
information among these samples. In this work, we propose a novel Local
Structure Preserving (LSP) regularization, which aims to preserve the local
structure of the input space in the learned embedding space. In this manner,
the attacking effect of adversarial samples lying in the vicinity of clean
samples can be alleviated. We show strong empirical evidence that with or
without adversarial training, our method consistently improves the performance
of adversarial robustness on several image classification datasets compared to
the baselines and some state-of-the-art approaches, thus providing promising
direction for future research.Comment: 13 pages, 4 figure
ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation
It is critical that the models pay attention not only to accuracy but also to
the certainty of prediction. Uncertain predictions of deep models caused by
noisy data raise significant concerns in trustworthy AI areas. To explore and
handle uncertainty due to intrinsic data noise, we propose a novel method
called ALUM to simultaneously handle the model uncertainty and data uncertainty
in a unified scheme. Rather than solely modeling data uncertainty in the
ultimate layer of a deep model based on randomly selected training data, we
propose to explore mined adversarial triplets to facilitate data uncertainty
modeling and non-parametric uncertainty estimations to compensate for the
insufficiently trained latent model layers. Thus, the critical data uncertainty
and model uncertainty caused by noisy data can be readily quantified for
improving model robustness. Our proposed ALUM is model-agnostic which can be
easily implemented into any existing deep model with little extra computation
overhead. Extensive experiments on various noisy learning tasks validate the
superior robustness and generalization ability of our method. The code is
released at https://github.com/wwzjer/ALUM.Comment: 10 pages, 5 figure
Adaptive Spot-Guided Transformer for Consistent Local Feature Matching
Local feature matching aims at finding correspondences between a pair of
images. Although current detector-free methods leverage Transformer
architecture to obtain an impressive performance, few works consider
maintaining local consistency. Meanwhile, most methods struggle with large
scale variations. To deal with the above issues, we propose Adaptive
Spot-Guided Transformer (ASTR) for local feature matching, which jointly models
the local consistency and scale variations in a unified coarse-to-fine
architecture. The proposed ASTR enjoys several merits. First, we design a
spot-guided aggregation module to avoid interfering with irrelevant areas
during feature aggregation. Second, we design an adaptive scaling module to
adjust the size of grids according to the calculated depth information at fine
stage. Extensive experimental results on five standard benchmarks demonstrate
that our ASTR performs favorably against state-of-the-art methods. Our code
will be released on https://astr2023.github.io.Comment: Accepted to CVPR 2023. Project page: https://astr2023.github.io
GCL: Gradient-Guided Contrastive Learning for Medical Image Segmentation with Multi-Perspective Meta Labels
Since annotating medical images for segmentation tasks commonly incurs
expensive costs, it is highly desirable to design an annotation-efficient
method to alleviate the annotation burden. Recently, contrastive learning has
exhibited a great potential in learning robust representations to boost
downstream tasks with limited labels. In medical imaging scenarios, ready-made
meta labels (i.e., specific attribute information of medical images) inherently
reveal semantic relationships among images, which have been used to define
positive pairs in previous work. However, the multi-perspective semantics
revealed by various meta labels are usually incompatible and can incur
intractable "semantic contradiction" when combining different meta labels. In
this paper, we tackle the issue of "semantic contradiction" in a
gradient-guided manner using our proposed Gradient Mitigator method, which
systematically unifies multi-perspective meta labels to enable a pre-trained
model to attain a better high-level semantic recognition ability. Moreover, we
emphasize that the fine-grained discrimination ability is vital for
segmentation-oriented pre-training, and develop a novel method called Gradient
Filter to dynamically screen pixel pairs with the most discriminating power
based on the magnitude of gradients. Comprehensive experiments on four medical
image segmentation datasets verify that our new method GCL: (1) learns
informative image representations and considerably boosts segmentation
performance with limited labels, and (2) shows promising generalizability on
out-of-distribution datasets
Sample-efficient Multi-objective Molecular Optimization with GFlowNets
Many crucial scientific problems involve designing novel molecules with
desired properties, which can be formulated as a black-box optimization problem
over the discrete chemical space. In practice, multiple conflicting objectives
and costly evaluations (e.g., wet-lab experiments) make the diversity of
candidates paramount. Computational methods have achieved initial success but
still struggle with considering diversity in both objective and search space.
To fill this gap, we propose a multi-objective Bayesian optimization (MOBO)
algorithm leveraging the hypernetwork-based GFlowNets (HN-GFN) as an
acquisition function optimizer, with the purpose of sampling a diverse batch of
candidate molecular graphs from an approximate Pareto front. Using a single
preference-conditioned hypernetwork, HN-GFN learns to explore various
trade-offs between objectives. We further propose a hindsight-like off-policy
strategy to share high-performing molecules among different preferences in
order to speed up learning for HN-GFN. We empirically illustrate that HN-GFN
has adequate capacity to generalize over preferences. Moreover, experiments in
various real-world MOBO settings demonstrate that our framework predominantly
outperforms existing methods in terms of candidate quality and sample
efficiency. The code is available at https://github.com/violet-sto/HN-GFN.Comment: NeurIPS 202
Geniposide Alleviates Glucocorticoid-Induced Inhibition of Osteogenic Differentiation in MC3T3-E1 Cells by ERK Pathway
Glucocorticoid (GC) therapy is the leading cause of secondary osteoporosis and the therapeutic and preventative drugs for GC-induced osteoporosis are limited. In this study, we investigated the protective effects of geniposide on dexamethasone (DEX)-induced osteogenic inhibition in MC3T3-E1 cells. The results showed that there was no obvious toxicity on MC3T3-E1 cells when geniposide was used at the doses ranging from 1 to 75 μM. In DEX-treated MC3T3-E1 cells, geniposide promoted the alkaline phosphatase (ALP) activity and the mineralization. In addition, geniposide also significantly increased the mRNA and protein expression of osteopontin (OPN), Runt-related transcription factor 2 (Runx2), and Osterix (Osx) in DEX-treated MC3T3-E1 cells. Furthermore, geniposide activated ERK pathway in DEX-treated MC3T3-E1 cells. The ERK activation inhibitor U0126 and glucagon-like peptide-1 (GLP-1) receptor antagonist exendin 9-39 abolished the geniposide-induced activation of ERK and inhibited the protective effect of geniposide. Taken together, our study revealed that geniposide alleviated GC-induced osteogenic suppression in MC3T3-E1 cells. The effect of geniposide was at least partially associated with activating ERK signaling pathway via GLP-1 receptor. Geniposide might be a potential therapeutic agent for GC-induced osteoporosis
Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification
Deep learning approaches exhibit promising performances on various text
tasks. However, they are still struggling on medical text classification since
samples are often extremely imbalanced and scarce. Different from existing
mainstream approaches that focus on supplementary semantics with external
medical information, this paper aims to rethink the data challenges in medical
texts and present a novel framework-agnostic algorithm called Text2Tree that
only utilizes internal label hierarchy in training deep learning models. We
embed the ICD code tree structure of labels into cascade attention modules for
learning hierarchy-aware label representations. Two new learning schemes,
Similarity Surrogate Learning (SSL) and Dissimilarity Mixup Learning (DML), are
devised to boost text classification by reusing and distinguishing samples of
other labels following the label representation hierarchy, respectively.
Experiments on authoritative public datasets and real-world medical records
show that our approach stably achieves superior performances over classical and
advanced imbalanced classification methods.Comment: EMNLP 2023 Findings. Code: https://github.com/jyansir/Text2Tre
Empowering AI drug discovery with explicit and implicit knowledge
Motivation: Recently, research on independently utilizing either explicit
knowledge from knowledge graphs or implicit knowledge from biomedical
literature for AI drug discovery has been growing rapidly. These approaches
have greatly improved the prediction accuracy of AI models on multiple
downstream tasks. However, integrating explicit and implicit knowledge
independently hinders their understanding of molecules. Results: We propose
DeepEIK, a unified deep learning framework that incorporates both explicit and
implicit knowledge for AI drug discovery. We adopt feature fusion to process
the multi-modal inputs, and leverage the attention mechanism to denoise the
text information. Experiments show that DeepEIK significantly outperforms
state-of-the-art methods on crucial tasks in AI drug discovery including
drug-target interaction prediction, drug property prediction and
protein-protein interaction prediction. Further studies show that benefiting
from explicit and implicit knowledge, our framework achieves a deeper
understanding of molecules and shows promising potential in facilitating drug
discovery applications.Comment: Bioinformatic
- …