3,860 research outputs found
Low-Rank Approximation of Weighted Tree Automata
We describe a technique to minimize weighted tree automata (WTA), a powerful
formalisms that subsumes probabilistic context-free grammars (PCFGs) and
latent-variable PCFGs. Our method relies on a singular value decomposition of
the underlying Hankel matrix defined by the WTA. Our main theoretical result is
an efficient algorithm for computing the SVD of an infinite Hankel matrix
implicitly represented as a WTA. We provide an analysis of the approximation
error induced by the minimization, and we evaluate our method on real-world
data originating in newswire treebank. We show that the model achieves lower
perplexity than previous methods for PCFG minimization, and also is much more
stable due to the absence of local optima.Comment: To appear in AISTATS 201
Learning probability distributions generated by finite-state machines
We review methods for inference of probability distributions generated by probabilistic automata and related models for sequence generation. We focus on methods that can be proved to learn in the inference
in the limit and PAC formal models. The methods we review are state merging and state splitting methods for probabilistic deterministic automata and the recently developed spectral method for nondeterministic probabilistic automata. In both cases, we derive them from a high-level algorithm described in terms of the Hankel matrix of the distribution to be learned, given as an oracle, and then describe how to adapt that algorithm to account for the error introduced by a finite sample.Peer ReviewedPostprint (author's final draft
Program Synthesis using Natural Language
Interacting with computers is a ubiquitous activity for millions of people.
Repetitive or specialized tasks often require creation of small, often one-off,
programs. End-users struggle with learning and using the myriad of
domain-specific languages (DSLs) to effectively accomplish these tasks.
We present a general framework for constructing program synthesizers that
take natural language (NL) inputs and produce expressions in a target DSL. The
framework takes as input a DSL definition and training data consisting of
NL/DSL pairs. From these it constructs a synthesizer by learning optimal
weights and classifiers (using NLP features) that rank the outputs of a
keyword-programming based translation. We applied our framework to three
domains: repetitive text editing, an intelligent tutoring system, and flight
information queries. On 1200+ English descriptions, the respective synthesizers
rank the desired program as the top-1 and top-3 for 80% and 90% descriptions
respectively
Graph Kernels
We present a unified framework to study graph kernels, special cases of which include the random
walk (Gärtner et al., 2003; Borgwardt et al., 2005) and marginalized (Kashima et al., 2003, 2004;
Mahé et al., 2004) graph kernels. Through reduction to a Sylvester equation we improve the time
complexity of kernel computation between unlabeled graphs with n vertices from O(n^6) to O(n^3).
We find a spectral decomposition approach even more efficient when computing entire kernel matrices.
For labeled graphs we develop conjugate gradient and fixed-point methods that take O(dn^3)
time per iteration, where d is the size of the label set. By extending the necessary linear algebra to
Reproducing Kernel Hilbert Spaces (RKHS) we obtain the same result for d-dimensional edge kernels,
and O(n^4) in the infinite-dimensional case; on sparse graphs these algorithms only take O(n^2)
time per iteration in all cases. Experiments on graphs from bioinformatics and other application
domains show that these techniques can speed up computation of the kernel by an order of magnitude
or more. We also show that certain rational kernels (Cortes et al., 2002, 2003, 2004) when
specialized to graphs reduce to our random walk graph kernel. Finally, we relate our framework to
R-convolution kernels (Haussler, 1999) and provide a kernel that is close to the optimal assignment
kernel of Fröhlich et al. (2006) yet provably positive semi-definite
Optimal Approximate Minimization of One-Letter Weighted Finite Automata
In this paper, we study the approximate minimization problem of weighted
finite automata (WFAs): to compute the best possible approximation of a WFA
given a bound on the number of states. By reformulating the problem in terms of
Hankel matrices, we leverage classical results on the approximation of Hankel
operators, namely the celebrated Adamyan-Arov-Krein (AAK) theory.
We solve the optimal spectral-norm approximate minimization problem for
irredundant WFAs with real weights, defined over a one-letter alphabet. We
present a theoretical analysis based on AAK theory, and bounds on the quality
of the approximation in the spectral norm and norm. Moreover, we
provide a closed-form solution, and an algorithm, to compute the optimal
approximation of a given size in polynomial time.Comment: 32 pages. arXiv admin note: substantial text overlap with
arXiv:2102.0686
- …