2,448 research outputs found
End-to-end Recurrent Neural Network Models for Vietnamese Named Entity Recognition: Word-level vs. Character-level
This paper demonstrates end-to-end neural network architectures for
Vietnamese named entity recognition. Our best model is a combination of
bidirectional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network
(CNN), Conditional Random Field (CRF), using pre-trained word embeddings as
input, which achieves an F1 score of 88.59% on a standard test set. Our system
is able to achieve a comparable performance to the first-rank system of the
VLSP campaign without using any syntactic or hand-crafted features. We also
give an extensive empirical study on using common deep learning models for
Vietnamese NER, at both word and character level.Comment: 14 pages, 5 figures, 7 tables, accepted to PACLING 2017, fix CRF
formula
On the Use of Machine Translation-Based Approaches for Vietnamese Diacritic Restoration
This paper presents an empirical study of two machine translation-based
approaches for Vietnamese diacritic restoration problem, including phrase-based
and neural-based machine translation models. This is the first work that
applies neural-based machine translation method to this problem and gives a
thorough comparison to the phrase-based machine translation method which is the
current state-of-the-art method for this problem. On a large dataset, the
phrase-based approach has an accuracy of 97.32% while that of the neural-based
approach is 96.15%. While the neural-based method has a slightly lower
accuracy, it is about twice faster than the phrase-based method in terms of
inference speed. Moreover, neural-based machine translation method has much
room for future improvement such as incorporating pre-trained word embeddings
and collecting more training data.Comment: 4 pages, 2 figures, 4 tables, accepted to IALP 201
Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation
Attention-based sequence-to-sequence model has proved successful in Neural
Machine Translation (NMT). However, the attention without consideration of
decoding history, which includes the past information in the decoder and the
attention mechanism, often causes much repetition. To address this problem, we
propose the decoding-history-based Adaptive Control of Attention (ACA) for the
NMT model. ACA learns to control the attention by keeping track of the decoding
history and the current information with a memory vector, so that the model can
take the translated contents and the current information into consideration.
Experiments on Chinese-English translation and the English-Vietnamese
translation have demonstrated that our model significantly outperforms the
strong baselines. The analysis shows that our model is capable of generating
translation with less repetition and higher accuracy. The code will be
available at https://github.com/lancopk
An Empirical Study of Discriminative Sequence Labeling Models for Vietnamese Text Processing
This paper presents an empirical study of two widely-used sequence prediction
models, Conditional Random Fields (CRFs) and Long Short-Term Memory Networks
(LSTMs), on two fundamental tasks for Vietnamese text processing, including
part-of-speech tagging and named entity recognition. We show that a strong
lower bound for labeling accuracy can be obtained by relying only on simple
word-based features with minimal hand-crafted feature engineering, of 90.65\%
and 86.03\% performance scores on the standard test sets for the two tasks
respectively. In particular, we demonstrate empirically the surprising
efficiency of word embeddings in both of the two tasks, with both of the two
models. We point out that the state-of-the-art LSTMs model does not always
outperform significantly the traditional CRFs model, especially on
moderate-sized data sets. Finally, we give some suggestions and discussions for
efficient use of sequence labeling models in practical applications.Comment: To appear in the Proceedings of the 9th International Conference on
Knowledge and Systems Engineering (KSE) 201
Combining Advanced Methods in Japanese-Vietnamese Neural Machine Translation
Neural machine translation (NMT) systems have recently obtained state-of-the
art in many machine translation systems between popular language pairs because
of the availability of data. For low-resourced language pairs, there are few
researches in this field due to the lack of bilingual data. In this paper, we
attempt to build the first NMT systems for a low-resourced language
pairs:Japanese-Vietnamese. We have also shown significant improvements when
combining advanced methods to reduce the adverse impacts of data sparsity and
improve the quality of NMT systems. In addition, we proposed a variant of
Byte-Pair Encoding algorithm to perform effective word segmentation for
Vietnamese texts and alleviate the rare-word problem that persists in NMT
systems
Machine Translation between Vietnamese and English: an Empirical Study
Machine translation is shifting to an end-to-end approach based on deep
neural networks. The state of the art achieves impressive results for popular
language pairs such as English - French or English - Chinese. However for
English - Vietnamese the shortage of parallel corpora and expensive
hyper-parameter search present practical challenges to neural-based approaches.
This paper highlights our efforts on improving English-Vietnamese translations
in two directions: (1) Building the largest open Vietnamese - English corpus to
date, and (2) Extensive experiments with the latest neural models to achieve
the highest BLEU scores. Our experiments provide practical examples of
effectively employing different neural machine translation models with
low-resource language pairs
A Comparative Study of Neural Network Models for Sentence Classification
This paper presents an extensive comparative study of four neural network
models, including feed-forward networks, convolutional networks, recurrent
networks and long short-term memory networks, on two sentence classification
datasets of English and Vietnamese text. We show that on the English dataset,
the convolutional network models without any feature engineering outperform
some competitive sentence classifiers with rich hand-crafted linguistic
features. We demonstrate that the GloVe word embeddings are consistently better
than both Skip-gram word embeddings and word count vectors. We also show the
superiority of convolutional neural network models on a Vietnamese newspaper
sentence dataset over strong baseline models. Our experimental results suggest
some good practices for applying neural network models in sentence
classification.Comment: To appear in the 5th NAFOSTED Conference on Information and Computer
Scienc
Attentive Neural Network for Named Entity Recognition in Vietnamese
We propose an attentive neural network for the task of named entity
recognition in Vietnamese. The proposed attentive neural model makes use of
character-based language models and word embeddings to encode words as vector
representations. A neural network architecture of encoder, attention, and
decoder layers is then utilized to encode knowledge of input sentences and to
label entity tags. The experimental results show that the proposed attentive
neural network achieves the state-of-the-art results on the benchmark named
entity recognition datasets in Vietnamese in comparison to both hand-crafted
features based models and neural models
A comparison of Vietnamese Statistical Parametric Speech Synthesis Systems
In recent years, statistical parametric speech synthesis (SPSS) systems have
been widely utilized in many interactive speech-based systems (e.g.~Amazon's
Alexa, Bose's headphones). To select a suitable SPSS system, both speech
quality and performance efficiency (e.g.~decoding time) must be taken into
account. In the paper, we compared four popular Vietnamese SPSS techniques
using: 1) hidden Markov models (HMM), 2) deep neural networks (DNN), 3)
generative adversarial networks (GAN), and 4) end-to-end (E2E) architectures,
which consists of Tacontron~2 and WaveGlow vocoder in terms of speech quality
and performance efficiency. We showed that the E2E systems accomplished the
best quality, but required the power of GPU to achieve real-time performance.
We also showed that the HMM-based system had inferior speech quality, but it
was the most efficient system. Surprisingly, the E2E systems were more
efficient than the DNN and GAN in inference on GPU. Surprisingly, the GAN-based
system did not outperform the DNN in term of quality.Comment: 9 pages, submitted to KSE 202
End to End Recognition System for Recognizing Offline Unconstrained Vietnamese Handwriting
Inspired by recent successes in neural machine translation and image caption
generation, we present an attention based encoder decoder model (AED) to
recognize Vietnamese Handwritten Text. The model composes of two parts: a
DenseNet for extracting invariant features, and a Long Short-Term Memory
network (LSTM) with an attention model incorporated for generating output text
(LSTM decoder), which are connected from the CNN part to the attention model.
The input of the CNN part is a handwritten text image and the target of the
LSTM decoder is the corresponding text of the input image. Our model is trained
end-to-end to predict the text from a given input image since all the parts are
differential components. In the experiment section, we evaluate our proposed
AED model on the VNOnDB-Word and VNOnDB-Line datasets to verify its efficiency.
The experiential results show that our model achieves 12.30% of word error rate
without using any language model. This result is competitive with the
handwriting recognition system provided by Google in the Vietnamese Online
Handwritten Text Recognition competition
- …