Example-based sentence reduction using the hidden markov model

Sentence reduction is the problem of removing redundant words or phrases from an input sentence by creating a new sentence, in which the gist of the meaning of the original sentence is unchanged. All most previous methods required a syntax parser before reducing sentence. However, these methods were difficult to apply to a language in which there was not a reliable parser. In this paper, we propose two new sentence reduction algorithms without using syntactic parsing for the input sentence. In the first algorithm, we present an novel application of using one of Example-Based Machine Translation method, the template translation learning algorithm. This algorithm works well in reduction, but the problem of using it is the computational calculation problem. To solve this problem, we extend the template translation algorithm by making an innovative use of Hidden Markov Model based on a set of template rules that obtained by learning from the examples. Experiments on applying the proposed algorithms shows a promising result without complex processing

Example-based sentence reduction using the hidden markov model

Sentence reduction is the removal of redundant words or phrases from an input sentence by creating a new sentence in which the gist of the original meaning of the sentence remains unchanged. All previous methods required a syntax parser before sentences could be reduced; hence it was difficult to apply them to a language with no reliable parser. In this article we propose two new sentence-reduction algorithms that do not use syntactic parsing for the input sentence. The first algorithm, based on the template-translation learning algorithm, one of example-based machine-translation methods, works quite well in reducing sentences, but its computational complexity can be exponential in certain cases. The second algorithm, an extension of the template-translation algorithm via innovative employment of the Hidden Markov model, which uses the set of template rules learned from examples, can overcome this computation problem. Experiments show that the proposed algorithms achieve acceptable results in comparison to sentence reduction done by humans