113,865 research outputs found
Molecular Joint Representation Learning via Multi-modal Information
In recent years, artificial intelligence has played an important role on
accelerating the whole process of drug discovery. Various of molecular
representation schemes of different modals (e.g. textual sequence or graph) are
developed. By digitally encoding them, different chemical information can be
learned through corresponding network structures. Molecular graphs and
Simplified Molecular Input Line Entry System (SMILES) are popular means for
molecular representation learning in current. Previous works have done attempts
by combining both of them to solve the problem of specific information loss in
single-modal representation on various tasks. To further fusing such
multi-modal imformation, the correspondence between learned chemical feature
from different representation should be considered. To realize this, we propose
a novel framework of molecular joint representation learning via Multi-Modal
information of SMILES and molecular Graphs, called MMSG. We improve the
self-attention mechanism by introducing bond level graph representation as
attention bias in Transformer to reinforce feature correspondence between
multi-modal information. We further propose a Bidirectional Message
Communication Graph Neural Network (BMC GNN) to strengthen the information flow
aggregated from graphs for further combination. Numerous experiments on public
property prediction datasets have demonstrated the effectiveness of our model
Learning Robust Node Representations on Graphs
Graph neural networks (GNN), as a popular methodology for node representation
learning on graphs, currently mainly focus on preserving the smoothness and
identifiability of node representations. A robust node representation on graphs
should further hold the stability property which means a node representation is
resistant to slight perturbations on the input. In this paper, we introduce the
stability of node representations in addition to the smoothness and
identifiability, and develop a novel method called contrastive graph neural
networks (CGNN) that learns robust node representations in an unsupervised
manner. Specifically, CGNN maintains the stability and identifiability by a
contrastive learning objective, while preserving the smoothness with existing
GNN models. Furthermore, the proposed method is a generic framework that can be
equipped with many other backbone models (e.g. GCN, GraphSage and GAT).
Extensive experiments on four benchmarks under both transductive and inductive
learning setups demonstrate the effectiveness of our method in comparison with
recent supervised and unsupervised models.Comment: 16 page
Learning Laplacian Matrix in Smooth Graph Signal Representations
The construction of a meaningful graph plays a crucial role in the success of
many graph-based representations and algorithms for handling structured data,
especially in the emerging field of graph signal processing. However, a
meaningful graph is not always readily available from the data, nor easy to
define depending on the application domain. In particular, it is often
desirable in graph signal processing applications that a graph is chosen such
that the data admit certain regularity or smoothness on the graph. In this
paper, we address the problem of learning graph Laplacians, which is equivalent
to learning graph topologies, such that the input data form graph signals with
smooth variations on the resulting topology. To this end, we adopt a factor
analysis model for the graph signals and impose a Gaussian probabilistic prior
on the latent variables that control these signals. We show that the Gaussian
prior leads to an efficient representation that favors the smoothness property
of the graph signals. We then propose an algorithm for learning graphs that
enforces such property and is based on minimizing the variations of the signals
on the learned graph. Experiments on both synthetic and real world data
demonstrate that the proposed graph learning framework can efficiently infer
meaningful graph topologies from signal observations under the smoothness
prior
Leveraging Orbital Information and Atomic Feature in Deep Learning Model
Predicting material properties base on micro structure of materials has long
been a challenging problem. Recently many deep learning methods have been
developed for material property prediction. In this study, we propose a crystal
representation learning framework, Orbital CrystalNet, OCrystalNet, which
consists of two parts: atomic descriptor generation and graph representation
learning. In OCrystalNet, we first incorporate orbital field matrix (OFM) and
atomic features to construct OFM-feature atomic descriptor, and then the atomic
descriptor is used as atom embedding in the atom-bond message passing module
which takes advantage of the topological structure of crystal graphs to learn
crystal representation. To demonstrate the capabilities of OCrystalNet we
performed a number of prediction tasks on Material Project dataset and JARVIS
dataset and compared our model with other baselines and state of art methods.
To further present the effectiveness of OCrystalNet, we conducted ablation
study and case study of our model. The results show that our model have various
advantages over other state of art models
Hyperbolic Graph Representation Learning: A Tutorial
Graph-structured data are widespread in real-world applications, such as
social networks, recommender systems, knowledge graphs, chemical molecules etc.
Despite the success of Euclidean space for graph-related learning tasks, its
ability to model complex patterns is essentially constrained by its
polynomially growing capacity. Recently, hyperbolic spaces have emerged as a
promising alternative for processing graph data with tree-like structure or
power-law distribution, owing to the exponential growth property. Different
from Euclidean space, which expands polynomially, the hyperbolic space grows
exponentially which makes it gains natural advantages in abstracting tree-like
or scale-free graphs with hierarchical organizations.
In this tutorial, we aim to give an introduction to this emerging field of
graph representation learning with the express purpose of being accessible to
all audiences. We first give a brief introduction to graph representation
learning as well as some preliminary Riemannian and hyperbolic geometry. We
then comprehensively revisit the hyperbolic embedding techniques, including
hyperbolic shallow models and hyperbolic neural networks. In addition, we
introduce the technical details of the current hyperbolic graph neural networks
by unifying them into a general framework and summarizing the variants of each
component. Moreover, we further introduce a series of related applications in a
variety of fields. In the last part, we discuss several advanced topics about
hyperbolic geometry for graph representation learning, which potentially serve
as guidelines for further flourishing the non-Euclidean graph learning
community.Comment: Accepted as ECML-PKDD 2022 Tutoria
Laplacian Matrix Learning for Smooth Graph Signal Representation
The construction of a meaningful graph plays a crucial role in the emerging field of signal processing on graphs. In this paper, we address the problem of learning graph Laplacians, which is similar to learning graph topologies, such that the input data form graph signals with smooth variations on the resulting topology. We adopt a factor analysis model for the graph signals and impose a Gaussian probabilistic prior on the latent variables that control these graph signals. We show that the Gaussian prior leads to an efficient representation that favours the smoothness property of the graph signals, and propose an algorithm for learning graphs that enforce such property. Experiments demonstrate that the proposed framework can efficiently infer meaningful graph topologies from only the signal observations
A Universal Knowledge Model and Cognitive Architecture for Prototyping AGI
The article identified 42 cognitive architectures for creating general
artificial intelligence (AGI) and proposed a set of interrelated functional
blocks that an agent approaching AGI in its capabilities should possess. Since
the required set of blocks is not found in any of the existing architectures,
the article proposes a new cognitive architecture for intelligent systems
approaching AGI in their capabilities. As one of the key solutions within the
framework of the architecture, a universal method of knowledge representation
is proposed, which allows combining various non-formalized, partially and fully
formalized methods of knowledge representation in a single knowledge base, such
as texts in natural languages, images, audio and video recordings, graphs,
algorithms, databases, neural networks, knowledge graphs, ontologies, frames,
essence-property-relation models, production systems, predicate calculus
models, conceptual models, and others. To combine and structure various
fragments of knowledge, archigraph models are used, constructed as a
development of annotated metagraphs. As components, the cognitive architecture
being developed includes machine consciousness, machine subconsciousness,
blocks of interaction with the external environment, a goal management block,
an emotional control system, a block of social interaction, a block of
reflection, an ethics block and a worldview block, a learning block, a
monitoring block, blocks of statement and solving problems, self-organization
and meta learning block
Causal discovery with ancestral graphs
Graphical models serve as a visual representation that captures the underlying conditional independence relationships within distributions, employing either directed or undirected graphs. In this thesis, we explore maximal ancestral graphs (MAGs), which is an extension to the conventionaldirected acyclic graphs (DAGs). While DAGs excel in illustrating causal relationships, they fail to capture all the conditional independences on the margin in the absence of latent confounders and selection bias. MAGs provide a more comprehensive depiction of complex dependencies by encompassing both direct causal connections and indirect influences stemming from latent variables and selection bias.
The scalability and accuracy of MAG learning algorithms have been some problems due to the complexity of the space of Markov equivalence classes (MECs) of MAGs and instability of scoring criteria. We first use the concept of heads, tails and parametrizing sets to characterize Markov equivalent MAGs. Then we study imsets of MAGs to address the above issues.
The framework of imsets (Studeny, 2006) is an algebraic approach to represent conditional independences. Given the remarkable success of standard imsets within DAGs, where they efficiently represent MECs and offer reliable scoring criteria, we endeavor to extend this framework to MAGs. Through an exploration of 0-1 imsets defined by parametrizing sets, we show under which conditions does this extended `standard imset' of MAGs define the correct model. Consequently, we refine the ordered local Markov property of MAGs (Richardson, 2003), demonstrating that the newly proposed refined Markov property can be constructed in polynomial time if we bound maximal head size.
Finally, we apply the above results to develop novel score-based learning algorithms for MAGs. To efficiently traverse between MECs of MAGs, we identify some important graphical features within MAGs whose independence models are subsets of others. Leveraging the imsets derived from the refined Markov property, we establish a consistent scoring criterion, offering an alternative to BIC by relying solely on estimates of entropy over subsets of variables. Empirical experiments show promising results when compared to state-of-the-art algorithms
- …