9 research outputs found
Relational Self-Supervised Learning on Graphs
Over the past few years, graph representation learning (GRL) has been a
powerful strategy for analyzing graph-structured data. Recently, GRL methods
have shown promising results by adopting self-supervised learning methods
developed for learning representations of images. Despite their success,
existing GRL methods tend to overlook an inherent distinction between images
and graphs, i.e., images are assumed to be independently and identically
distributed, whereas graphs exhibit relational information among data
instances, i.e., nodes. To fully benefit from the relational information
inherent in the graph-structured data, we propose a novel GRL method, called
RGRL, that learns from the relational information generated from the graph
itself. RGRL learns node representations such that the relationship among nodes
is invariant to augmentations, i.e., augmentation-invariant relationship, which
allows the node representations to vary as long as the relationship among the
nodes is preserved. By considering the relationship among nodes in both global
and local perspectives, RGRL overcomes limitations of previous contrastive and
non-contrastive methods, and achieves the best of both worlds. Extensive
experiments on fourteen benchmark datasets over various downstream tasks
demonstrate the superiority of RGRL over state-of-the-art baselines. The source
code for RGRL is available at https://github.com/Namkyeong/RGRL.Comment: CIKM 202
Heterogeneous Graph Learning for Multi-modal Medical Data Analysis
Routine clinical visits of a patient produce not only image data, but also
non-image data containing clinical information regarding the patient, i.e.,
medical data is multi-modal in nature. Such heterogeneous modalities offer
different and complementary perspectives on the same patient, resulting in more
accurate clinical decisions when they are properly combined. However, despite
its significance, how to effectively fuse the multi-modal medical data into a
unified framework has received relatively little attention. In this paper, we
propose an effective graph-based framework called HetMed (Heterogeneous Graph
Learning for Multi-modal Medical Data Analysis) for fusing the multi-modal
medical data. Specifically, we construct a multiplex network that incorporates
multiple types of non-image features of patients to capture the complex
relationship between patients in a systematic way, which leads to more accurate
clinical decisions. Extensive experiments on various real-world datasets
demonstrate the superiority and practicality of HetMed. The source code for
HetMed is available at https://github.com/Sein-Kim/Multimodal-Medical.Comment: AAAI 202
Task Relation-aware Continual User Representation Learning
User modeling, which learns to represent users into a low-dimensional
representation space based on their past behaviors, got a surge of interest
from the industry for providing personalized services to users. Previous
efforts in user modeling mainly focus on learning a task-specific user
representation that is designed for a single task. However, since learning
task-specific user representations for every task is infeasible, recent studies
introduce the concept of universal user representation, which is a more
generalized representation of a user that is relevant to a variety of tasks.
Despite their effectiveness, existing approaches for learning universal user
representations are impractical in real-world applications due to the data
requirement, catastrophic forgetting and the limited learning capability for
continually added tasks. In this paper, we propose a novel continual user
representation learning method, called TERACON, whose learning capability is
not limited as the number of learned tasks increases while capturing the
relationship between the tasks. The main idea is to introduce an embedding for
each task, i.e., task embedding, which is utilized to generate task-specific
soft masks that not only allow the entire model parameters to be updated until
the end of training sequence, but also facilitate the relationship between the
tasks to be captured. Moreover, we introduce a novel knowledge retention module
with pseudo-labeling strategy that successfully alleviates the long-standing
problem of continual learning, i.e., catastrophic forgetting. Extensive
experiments on public and proprietary real-world datasets demonstrate the
superiority and practicality of TERACON. Our code is available at
https://github.com/Sein-Kim/TERACON.Comment: KDD 202
Conditional Graph Information Bottleneck for Molecular Relational Learning
Molecular relational learning, whose goal is to learn the interaction
behavior between molecular pairs, got a surge of interest in molecular sciences
due to its wide range of applications. Recently, graph neural networks have
recently shown great success in molecular relational learning by modeling a
molecule as a graph structure, and considering atom-level interactions between
two molecules. Despite their success, existing molecular relational learning
methods tend to overlook the nature of chemistry, i.e., a chemical compound is
composed of multiple substructures such as functional groups that cause
distinctive chemical reactions. In this work, we propose a novel relational
learning framework, called CGIB, that predicts the interaction behavior between
a pair of graphs by detecting core subgraphs therein. The main idea is, given a
pair of graphs, to find a subgraph from a graph that contains the minimal
sufficient information regarding the task at hand conditioned on the paired
graph based on the principle of conditional graph information bottleneck. We
argue that our proposed method mimics the nature of chemical reactions, i.e.,
the core substructure of a molecule varies depending on which other molecule it
interacts with. Extensive experiments on various tasks with real-world datasets
demonstrate the superiority of CGIB over state-of-the-art baselines. Our code
is available at https://github.com/Namkyeong/CGIB.Comment: ICML 202
Shift-Robust Molecular Relational Learning with Causal Substructure
Recently, molecular relational learning, whose goal is to predict the
interaction behavior between molecular pairs, got a surge of interest in
molecular sciences due to its wide range of applications. In this work, we
propose CMRL that is robust to the distributional shift in molecular relational
learning by detecting the core substructure that is causally related to
chemical reactions. To do so, we first assume a causal relationship based on
the domain knowledge of molecular sciences and construct a structural causal
model (SCM) that reveals the relationship between variables. Based on the SCM,
we introduce a novel conditional intervention framework whose intervention is
conditioned on the paired molecule. With the conditional intervention
framework, our model successfully learns from the causal substructure and
alleviates the confounding effect of shortcut substructures that are spuriously
correlated to chemical reactions. Extensive experiments on various tasks with
real-world and synthetic datasets demonstrate the superiority of CMRL over
state-of-the-art baseline models. Our code is available at
https://github.com/Namkyeong/CMRL.Comment: KDD 202
Predicting Density of States via Multi-modal Transformer
The density of states (DOS) is a spectral property of materials, which
provides fundamental insights on various characteristics of materials. In this
paper, we propose a model to predict the DOS by reflecting the nature of DOS:
DOS determines the general distribution of states as a function of energy.
Specifically, we integrate the heterogeneous information obtained from the
crystal structure and the energies via multi-modal transformer, thereby
modeling the complex relationships between the atoms in the crystal structure,
and various energy levels. Extensive experiments on two types of DOS, i.e.,
Phonon DOS and Electron DOS, with various real-world scenarios demonstrate the
superiority of DOSTransformer. The source code for DOSTransformer is available
at https://github.com/HeewoongNoh/DOSTransformer.Comment: ICLR 2023 Workshop on Machine Learning for Materials (ML4Materials
Augmentation-Free Self-Supervised Learning on Graphs
Inspired by the recent success of self-supervised methods applied on images, self-supervised learning on graph structured data has seen rapid growth especially centered on augmentation-based contrastive methods. However, we argue that without carefully designed augmentation techniques, augmentations on graphs may behave arbitrarily in that the underlying semantics of graphs can drastically change. As a consequence, the performance of existing augmentation-based methods is highly dependent on the choice of augmentation scheme, i.e., augmentation hyperparameters and combinations of augmentation. In this paper, we propose a novel augmentation-free self-supervised learning framework for graphs, named AFGRL. Specifically, we generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics with the graph. Extensive experiments towards various node-level tasks, i.e., node classification, clustering, and similarity search on various real-world datasets demonstrate the superiority of AFGRL. The source code for AFGRL is available at https://github.com/Namkyeong/AFGRL
Maximal Differentiability for a General Class of Quasilinear Elliptic Equations with Right-hand Side Measures
Byun S-S, Cho N, Lee H-S. Maximal Differentiability for a General Class of Quasilinear Elliptic Equations with Right-hand Side Measures. International Mathematics Research Notices. 2022;2022(13):9722-9754.**Abstract**
We prove maximal differentiability for the gradient of solutions to a certain type of nonlinear elliptic equations with a measure on the right-hand side. Our results generalize the limiting case of Calderón–Zygmund theory for an elliptic equation with -growth to the case of Orlicz growth