5 research outputs found
Enhancing Chinese Intent Classification by Dynamically Integrating Character Features into Word Embeddings with Ensemble Techniques
Intent classification has been widely researched on English data with deep
learning approaches that are based on neural networks and word embeddings. The
challenge for Chinese intent classification stems from the fact that, unlike
English where most words are made up of 26 phonologic alphabet letters, Chinese
is logographic, where a Chinese character is a more basic semantic unit that
can be informative and its meaning does not vary too much in contexts. Chinese
word embeddings alone can be inadequate for representing words, and pre-trained
embeddings can suffer from not aligning well with the task at hand. To account
for the inadequacy and leverage Chinese character information, we propose a
low-effort and generic way to dynamically integrate character embedding based
feature maps with word embedding based inputs, whose resulting word-character
embeddings are stacked with a contextual information extraction module to
further incorporate context information for predictions. On top of the proposed
model, we employ an ensemble method to combine single models and obtain the
final result. The approach is data-independent without relying on external
sources like pre-trained word embeddings. The proposed model outperforms
baseline models and existing methods
Text Classification based on Multi-granularity Attention Hybrid Neural Network
Neural network-based approaches have become the driven forces for Natural
Language Processing (NLP) tasks. Conventionally, there are two mainstream
neural architectures for NLP tasks: the recurrent neural network (RNN) and the
convolution neural network (ConvNet). RNNs are good at modeling long-term
dependencies over input texts, but preclude parallel computation. ConvNets do
not have memory capability and it has to model sequential data as un-ordered
features. Therefore, ConvNets fail to learn sequential dependencies over the
input texts, but it is able to carry out high-efficient parallel computation.
As each neural architecture, such as RNN and ConvNets, has its own pro and con,
integration of different architectures is assumed to be able to enrich the
semantic representation of texts, thus enhance the performance of NLP tasks.
However, few investigation explores the reconciliation of these seemingly
incompatible architectures. To address this issue, we propose a hybrid
architecture based on a novel hierarchical multi-granularity attention
mechanism, named Multi-granularity Attention-based Hybrid Neural Network
(MahNN). The attention mechanism is to assign different weights to different
parts of the input sequence to increase the computation efficiency and
performance of neural models. In MahNN, two types of attentions are introduced:
the syntactical attention and the semantical attention. The syntactical
attention computes the importance of the syntactic elements (such as words or
sentence) at the lower symbolic level and the semantical attention is used to
compute the importance of the embedded space dimension corresponding to the
upper latent semantics. We adopt the text classification as an exemplifying way
to illustrate the ability of MahNN to understand texts
Combine Convolution with Recurrent Networks for Text Classification
Convolutional neural network (CNN) and recurrent neural network (RNN) are two
popular architectures used in text classification. Traditional methods to
combine the strengths of the two networks rely on streamlining them or
concatenating features extracted from them. In this paper, we propose a novel
method to keep the strengths of the two networks to a great extent. In the
proposed model, a convolutional neural network is applied to learn a 2D weight
matrix where each row reflects the importance of each word from different
aspects. Meanwhile, we use a bi-directional RNN to process each word and employ
a neural tensor layer that fuses forward and backward hidden states to get word
representations. In the end, the weight matrix and word representations are
combined to obtain the representation in a 2D matrix form for the text. We
carry out experiments on a number of datasets for text classification. The
experimental results confirm the effectiveness of the proposed method
Multichannel CNN with Attention for Text Classification
Recent years, the approaches based on neural networks have shown remarkable
potential for sentence modeling. There are two main neural network structures:
recurrent neural network (RNN) and convolution neural network (CNN). RNN can
capture long term dependencies and store the semantics of the previous
information in a fixed-sized vector. However, RNN is a biased model and its
ability to extract global semantics is restricted by the fixed-sized vector.
Alternatively, CNN is able to capture n-gram features of texts by utilizing
convolutional filters. But the width of convolutional filters restricts its
performance. In order to combine the strengths of the two kinds of networks and
alleviate their shortcomings, this paper proposes Attention-based Multichannel
Convolutional Neural Network (AMCNN) for text classification. AMCNN utilizes a
bi-directional long short-term memory to encode the history and future
information of words into high dimensional representations, so that the
information of both the front and back of the sentence can be fully expressed.
Then the scalar attention and vectorial attention are applied to obtain
multichannel representations. The scalar attention can calculate the word-level
importance and the vectorial attention can calculate the feature-level
importance. In the classification task, AMCNN uses a CNN structure to cpture
word relations on the representations generated by the scalar and vectorial
attention mechanism instead of calculating the weighted sums. It can
effectively extract the n-gram features of the text. The experimental results
on the benchmark datasets demonstrate that AMCNN achieves better performance
than state-of-the-art methods. In addition, the visualization results verify
the semantic richness of multichannel representations
Brain-inspired Search Engine Assistant based on Knowledge Graph
Search engines can quickly response a hyperlink list according to query
keywords. However, when a query is complex, developers need to repeatedly
refine the search keywords and open a large number of web pages to find and
summarize answers. Many research works of question and answering (Q and A)
system attempt to assist search engines by providing simple, accurate and
understandable answers. However, without original semantic contexts, these
answers lack explainability, making them difficult for users to trust and
adopt. In this paper, a brain-inspired search engine assistant named
DeveloperBot based on knowledge graph is proposed, which aligns to the
cognitive process of human and has the capacity to answer complex queries with
explainability. Specifically, DeveloperBot firstly constructs a multi-layer
query graph by splitting a complex multi-constraint query into several ordered
constraints. Then it models the constraint reasoning process as subgraph search
process inspired by the spreading activation model of cognitive science. In the
end, novel features of the subgraph will be extracted for decision-making. The
corresponding reasoning subgraph and answer confidence will be derived as
explanations. The results of the decision-making demonstrate that DeveloperBot
can estimate the answers and answer confidences with high accuracy. We
implement a prototype and conduct a user study to evaluate whether and how the
direct answers and the explanations provided by DeveloperBot can assist
developers' information needs.Comment: 12 page