3,022 research outputs found
Disentangled Variational Auto-Encoder for Semi-supervised Learning
Semi-supervised learning is attracting increasing attention due to the fact
that datasets of many domains lack enough labeled data. Variational
Auto-Encoder (VAE), in particular, has demonstrated the benefits of
semi-supervised learning. The majority of existing semi-supervised VAEs utilize
a classifier to exploit label information, where the parameters of the
classifier are introduced to the VAE. Given the limited labeled data, learning
the parameters for the classifiers may not be an optimal solution for
exploiting label information. Therefore, in this paper, we develop a novel
approach for semi-supervised VAE without classifier. Specifically, we propose a
new model called Semi-supervised Disentangled VAE (SDVAE), which encodes the
input data into disentangled representation and non-interpretable
representation, then the category information is directly utilized to
regularize the disentangled representation via the equality constraint. To
further enhance the feature learning ability of the proposed VAE, we
incorporate reinforcement learning to relieve the lack of data. The dynamic
framework is capable of dealing with both image and text data with its
corresponding encoder and decoder networks. Extensive experiments on image and
text datasets demonstrate the effectiveness of the proposed framework.Comment: 6 figures, 10 pages, Information Sciences 201
Graph-based Semi-Supervised & Active Learning for Edge Flows
We present a graph-based semi-supervised learning (SSL) method for learning
edge flows defined on a graph. Specifically, given flow measurements on a
subset of edges, we want to predict the flows on the remaining edges. To this
end, we develop a computational framework that imposes certain constraints on
the overall flows, such as (approximate) flow conservation. These constraints
render our approach different from classical graph-based SSL for vertex labels,
which posits that tightly connected nodes share similar labels and leverages
the graph structure accordingly to extrapolate from a few vertex labels to the
unlabeled vertices. We derive bounds for our method's reconstruction error and
demonstrate its strong performance on synthetic and real-world flow networks
from transportation, physical infrastructure, and the Web. Furthermore, we
provide two active learning algorithms for selecting informative edges on which
to measure flow, which has applications for optimal sensor deployment. The
first strategy selects edges to minimize the reconstruction error bound and
works well on flows that are approximately divergence-free. The second approach
clusters the graph and selects bottleneck edges that cross cluster-boundaries,
which works well on flows with global trends
Machine Learning for Fluid Mechanics
The field of fluid mechanics is rapidly advancing, driven by unprecedented
volumes of data from field measurements, experiments and large-scale
simulations at multiple spatiotemporal scales. Machine learning offers a wealth
of techniques to extract information from data that could be translated into
knowledge about the underlying fluid mechanics. Moreover, machine learning
algorithms can augment domain knowledge and automate tasks related to flow
control and optimization. This article presents an overview of past history,
current developments, and emerging opportunities of machine learning for fluid
mechanics. It outlines fundamental machine learning methodologies and discusses
their uses for understanding, modeling, optimizing, and controlling fluid
flows. The strengths and limitations of these methods are addressed from the
perspective of scientific inquiry that considers data as an inherent part of
modeling, experimentation, and simulation. Machine learning provides a powerful
information processing framework that can enrich, and possibly even transform,
current lines of fluid mechanics research and industrial applications.Comment: To appear in the Annual Reviews of Fluid Mechanics, 202
Hodge-Aware Contrastive Learning
Simplicial complexes prove effective in modeling data with multiway
dependencies, such as data defined along the edges of networks or within other
higher-order structures. Their spectrum can be decomposed into three
interpretable subspaces via the Hodge decomposition, resulting foundational in
numerous applications. We leverage this decomposition to develop a contrastive
self-supervised learning approach for processing simplicial data and generating
embeddings that encapsulate specific spectral information.Specifically, we
encode the pertinent data invariances through simplicial neural networks and
devise augmentations that yield positive contrastive examples with suitable
spectral properties for downstream tasks. Additionally, we reweight the
significance of negative examples in the contrastive loss, considering the
similarity of their Hodge components to the anchor. By encouraging a stronger
separation among less similar instances, we obtain an embedding space that
reflects the spectral properties of the data. The numerical results on two
standard edge flow classification tasks show a superior performance even when
compared to supervised learning techniques. Our findings underscore the
importance of adopting a spectral perspective for contrastive learning with
higher-order data.Comment: 4 pages, 2 figure
Residual Correlation in Graph Neural Network Regression
A graph neural network transforms features in each vertex's neighborhood into
a vector representation of the vertex. Afterward, each vertex's representation
is used independently for predicting its label. This standard pipeline
implicitly assumes that vertex labels are conditionally independent given their
neighborhood features. However, this is a strong assumption, and we show that
it is far from true on many real-world graph datasets. Focusing on regression
tasks, we find that this conditional independence assumption severely limits
predictive power. This should not be that surprising, given that traditional
graph-based semi-supervised learning methods such as label propagation work in
the opposite fashion by explicitly modeling the correlation in predicted
outcomes.
Here, we address this problem with an interpretable and efficient framework
that can improve any graph neural network architecture simply by exploiting
correlation structure in the regression residuals. In particular, we model the
joint distribution of residuals on vertices with a parameterized multivariate
Gaussian, and estimate the parameters by maximizing the marginal likelihood of
the observed labels. Our framework achieves substantially higher accuracy than
competing baselines, and the learned parameters can be interpreted as the
strength of correlation among connected vertices. Furthermore, we develop
linear time algorithms for low-variance, unbiased model parameter estimates,
allowing us to scale to large networks. We also provide a basic version of our
method that makes stronger assumptions on correlation structure but is painless
to implement, often leading to great practical performance with minimal
overhead
- …