1,246 research outputs found
Unsupervised Statistical Learning of Context-free Grammar
In this paper, we address the problem of inducing (weighted) context-free grammar (WCFG) on data given.
The induction is performed by using a new model of grammatical inference, i.e., weighted Grammar-based
Classifier System (wGCS). wGCS derives from learning classifier systems and searches grammar structure
using a genetic algorithm and covering. Weights of rules are estimated by using a novelty Inside-Outside
Contrastive Estimation algorithm. The proposed method employs direct negative evidence and learns WCFG
both form positive and negative samples. Results of experiments on three synthetic context-free languages
show that wGCS is competitive with other statistical-based method for unsupervised CFG learning
Derivation of Context-free Stochastic L-Grammar Rules for Promoter Sequence Modeling Using Support Vector Machine
Formal grammars can used for describing complex repeatable structures such as DNA sequences. In
this paper, we describe the structural composition of DNA sequences using a context-free stochastic L-grammar.
L-grammars are a special class of parallel grammars that can model the growth of living organisms, e.g. plant
development, and model the morphology of a variety of organisms. We believe that parallel grammars also can
be used for modeling genetic mechanisms and sequences such as promoters. Promoters are short regulatory
DNA sequences located upstream of a gene. Detection of promoters in DNA sequences is important for
successful gene prediction. Promoters can be recognized by certain patterns that are conserved within a species,
but there are many exceptions which makes the promoter recognition a complex problem. We replace the
problem of promoter recognition by induction of context-free stochastic L-grammar rules, which are later used for
the structural analysis of promoter sequences. L-grammar rules are derived automatically from the drosophila and
vertebrate promoter datasets using a genetic programming technique and their fitness is evaluated using a
Support Vector Machine (SVM) classifier. The artificial promoter sequences generated using the derived L-
grammar rules are analyzed and compared with natural promoter sequences
Growing Graphs with Hyperedge Replacement Graph Grammars
Discovering the underlying structures present in large real world graphs is a
fundamental scientific problem. In this paper we show that a graph's clique
tree can be used to extract a hyperedge replacement grammar. If we store an
ordering from the extraction process, the extracted graph grammar is guaranteed
to generate an isomorphic copy of the original graph. Or, a stochastic
application of the graph grammar rules can be used to quickly create random
graphs. In experiments on large real world networks, we show that random
graphs, generated from extracted graph grammars, exhibit a wide range of
properties that are very similar to the original graphs. In addition to graph
properties like degree or eigenvector centrality, what a graph "looks like"
ultimately depends on small details in local graph substructures that are
difficult to define at a global level. We show that our generative graph model
is able to preserve these local substructures when generating new graphs and
performs well on new and difficult tests of model robustness.Comment: 18 pages, 19 figures, accepted to CIKM 2016 in Indianapolis, I
Maintaining regularity and generalization in data using the minimum description length principle and genetic algorithm: Case of grammatical inference
In this paper, a genetic algorithm with minimum description length (GAWMDL) is proposed for grammatical inference. The primary challenge of identifying a language of infinite cardinality from a finite set of examples should know when to generalize and specialize the training data. The minimum description length principle that has been incorporated addresses this issue is discussed in this paper. Previously, the e-GRIDS learning model was proposed, which enjoyed the merits of the minimum description length principle, but it is limited to positive examples only. The proposed GAWMDL, which incorporates a traditional genetic algorithm and has a powerful global exploration capability that can exploit an optimum offspring. This is an effective approach to handle a problem which has a large search space such the grammatical inference problem. The computational capability, the genetic algorithm poses is not questionable, but it still suffers from premature convergence mainly arising due to lack of population diversity. The proposed GAWMDL incorporates a bit mask oriented data structure that performs the reproduction operations, creating the mask, then Boolean based procedure is applied to create an offspring in a generative manner. The Boolean based procedure is capable of introducing diversity into the population, hence alleviating premature convergence. The proposed GAWMDL is applied in the context free as well as regular languages of varying complexities. The computational experiments show that the GAWMDL finds an optimal or close-to-optimal grammar. Two fold performance analysis have been performed. First, the GAWMDL has been evaluated against the elite mating pool genetic algorithm which was proposed to introduce diversity and to address premature convergence. GAWMDL is also tested against the improved tabular representation algorithm. In addition, the authors evaluate the performance of the GAWMDL against a genetic algorithm not using the minimum description length principle. Statistical tests demonstrate the superiority of the proposed algorithm. Overall, the proposed GAWMDL algorithm greatly improves the performance in three main aspects: maintains regularity of the data, alleviates premature convergence and is capable in grammatical inference from both positive and negative corpora
Data Compression Concepts and Algorithms and Their Applications to Bioinformatics
Data compression at its base is concerned with how information is organized in data. Understanding this organization can lead to efficient ways of representing the information and hence data compression. In this paper we review the ways in which ideas and approaches fundamental to the theory and practice of data compression have been used in the area of bioinformatics. We look at how basic theoretical ideas from data compression, such as the notions of entropy, mutual information, and complexity have been used for analyzing biological sequences in order to discover hidden patterns, infer phylogenetic relationships between organisms and study viral populations. Finally, we look at how inferred grammars for biological sequences have been used to uncover structure in biological sequences
Grammar Variational Autoencoder
Deep generative models have been wildly successful at learning coherent
latent representations for continuous data such as video and audio. However,
generative modeling of discrete data such as arithmetic expressions and
molecular structures still poses significant challenges. Crucially,
state-of-the-art methods often produce outputs that are not valid. We make the
key observation that frequently, discrete data can be represented as a parse
tree from a context-free grammar. We propose a variational autoencoder which
encodes and decodes directly to and from these parse trees, ensuring the
generated outputs are always valid. Surprisingly, we show that not only does
our model more often generate valid outputs, it also learns a more coherent
latent space in which nearby points decode to similar discrete outputs. We
demonstrate the effectiveness of our learned models by showing their improved
performance in Bayesian optimization for symbolic regression and molecular
synthesis
- …