A Method of Incorporating Bigram Constraints into an LR Table and Its Effectiveness in Natural Language Processing


In this paper, we propose a method for constructing bigram LR tables by way of incorporating bigram constraints into an LR table. Using a bigram LR table, it is possible for a GLR parser to make use of both bigram and CFG constraints in natural language processing. Applying bigram LR tables to our GLR method has the following advantages: (1) Language models utilizing bigram LR tables have lower perplexity than simple bigram language models, since local constraints (bigram) and global constraints (CFG) are combined in a single bigram LR table. (2) Bigram constraints are easily acquired from a given corpus. Therefore data sparseness is not likely to arise. (3) Separation of local and global constraints keeps down the number of CFG rules. The first advantage leads to a reduction in complexity, and as the result, better performance in GLR parsing. Our experiments demonstrate the effectiveness of our method. 1 Introduction In natural language processing, stochastic language models are com..

Similar works

Full text

oaioai:CiteSeerX.psu:10.1...Last time updated on 10/22/2014

This paper was published in CiteSeerX.

Having an issue?

Is data on this page outdated, violates copyrights or anything else? Report the problem now and we will take corresponding actions after reviewing your request.