120,364 research outputs found
Combining Models of Approximation with Partial Learning
In Gold's framework of inductive inference, the model of partial learning
requires the learner to output exactly one correct index for the target object
and only the target object infinitely often. Since infinitely many of the
learner's hypotheses may be incorrect, it is not obvious whether a partial
learner can be modifed to "approximate" the target object.
Fulk and Jain (Approximate inference and scientific method. Information and
Computation 114(2):179--191, 1994) introduced a model of approximate learning
of recursive functions. The present work extends their research and solves an
open problem of Fulk and Jain by showing that there is a learner which
approximates and partially identifies every recursive function by outputting a
sequence of hypotheses which, in addition, are also almost all finite variants
of the target function.
The subsequent study is dedicated to the question how these findings
generalise to the learning of r.e. languages from positive data. Here three
variants of approximate learning will be introduced and investigated with
respect to the question whether they can be combined with partial learning.
Following the line of Fulk and Jain's research, further investigations provide
conditions under which partial language learners can eventually output only
finite variants of the target language. The combinabilities of other partial
learning criteria will also be briefly studied.Comment: 28 page
Operator inference for non-intrusive model reduction with quadratic manifolds
This paper proposes a novel approach for learning a data-driven quadratic
manifold from high-dimensional data, then employing this quadratic manifold to
derive efficient physics-based reduced-order models. The key ingredient of the
approach is a polynomial mapping between high-dimensional states and a
low-dimensional embedding. This mapping consists of two parts: a representation
in a linear subspace (computed in this work using the proper orthogonal
decomposition) and a quadratic component. The approach can be viewed as a form
of data-driven closure modeling, since the quadratic component introduces
directions into the approximation that lie in the orthogonal complement of the
linear subspace, but without introducing any additional degrees of freedom to
the low-dimensional representation. Combining the quadratic manifold
approximation with the operator inference method for projection-based model
reduction leads to a scalable non-intrusive approach for learning reduced-order
models of dynamical systems. Applying the new approach to transport-dominated
systems of partial differential equations illustrates the gains in efficiency
that can be achieved over approximation in a linear subspace
Learning Graphical Models Using Multiplicative Weights
We give a simple, multiplicative-weight update algorithm for learning
undirected graphical models or Markov random fields (MRFs). The approach is
new, and for the well-studied case of Ising models or Boltzmann machines, we
obtain an algorithm that uses a nearly optimal number of samples and has
quadratic running time (up to logarithmic factors), subsuming and improving on
all prior work. Additionally, we give the first efficient algorithm for
learning Ising models over general alphabets.
Our main application is an algorithm for learning the structure of t-wise
MRFs with nearly-optimal sample complexity (up to polynomial losses in
necessary terms that depend on the weights) and running time that is
. In addition, given samples, we can also learn the
parameters of the model and generate a hypothesis that is close in statistical
distance to the true MRF. All prior work runs in time for
graphs of bounded degree d and does not generate a hypothesis close in
statistical distance even for t=3. We observe that our runtime has the correct
dependence on n and t assuming the hardness of learning sparse parities with
noise.
Our algorithm--the Sparsitron-- is easy to implement (has only one parameter)
and holds in the on-line setting. Its analysis applies a regret bound from
Freund and Schapire's classic Hedge algorithm. It also gives the first solution
to the problem of learning sparse Generalized Linear Models (GLMs)
Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP)
We introduce a new structured kernel interpolation (SKI) framework, which
generalises and unifies inducing point methods for scalable Gaussian processes
(GPs). SKI methods produce kernel approximations for fast computations through
kernel interpolation. The SKI framework clarifies how the quality of an
inducing point approach depends on the number of inducing (aka interpolation)
points, interpolation strategy, and GP covariance kernel. SKI also provides a
mechanism to create new scalable kernel methods, through choosing different
kernel interpolation strategies. Using SKI, with local cubic kernel
interpolation, we introduce KISS-GP, which is 1) more scalable than inducing
point alternatives, 2) naturally enables Kronecker and Toeplitz algebra for
substantial additional gains in scalability, without requiring any grid data,
and 3) can be used for fast and expressive kernel learning. KISS-GP costs O(n)
time and storage for GP inference. We evaluate KISS-GP for kernel matrix
approximation, kernel learning, and natural sound modelling.Comment: 19 pages, 4 figure
Covariate dimension reduction for survival data via the Gaussian process latent variable model
The analysis of high dimensional survival data is challenging, primarily due
to the problem of overfitting which occurs when spurious relationships are
inferred from data that subsequently fail to exist in test data. Here we
propose a novel method of extracting a low dimensional representation of
covariates in survival data by combining the popular Gaussian Process Latent
Variable Model (GPLVM) with a Weibull Proportional Hazards Model (WPHM). The
combined model offers a flexible non-linear probabilistic method of detecting
and extracting any intrinsic low dimensional structure from high dimensional
data. By reducing the covariate dimension we aim to diminish the risk of
overfitting and increase the robustness and accuracy with which we infer
relationships between covariates and survival outcomes. In addition, we can
simultaneously combine information from multiple data sources by expressing
multiple datasets in terms of the same low dimensional space. We present
results from several simulation studies that illustrate a reduction in
overfitting and an increase in predictive performance, as well as successful
detection of intrinsic dimensionality. We provide evidence that it is
advantageous to combine dimensionality reduction with survival outcomes rather
than performing unsupervised dimensionality reduction on its own. Finally, we
use our model to analyse experimental gene expression data and detect and
extract a low dimensional representation that allows us to distinguish high and
low risk groups with superior accuracy compared to doing regression on the
original high dimensional data
- …