1,271 research outputs found
Do Convolutional Networks need to be Deep for Text Classification ?
We study in this work the importance of depth in convolutional models for
text classification, either when character or word inputs are considered. We
show on 5 standard text classification and sentiment analysis tasks that deep
models indeed give better performances than shallow networks when the text
input is represented as a sequence of characters. However, a simple
shallow-and-wide network outperforms deep models such as DenseNet with word
inputs. Our shallow word model further establishes new state-of-the-art
performances on two datasets: Yelp Binary (95.9\%) and Yelp Full (64.9\%)
On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis
Text preprocessing is often the first step in the pipeline of a Natural
Language Processing (NLP) system, with potential impact in its final
performance. Despite its importance, text preprocessing has not received much
attention in the deep learning literature. In this paper we investigate the
impact of simple text preprocessing decisions (particularly tokenizing,
lemmatizing, lowercasing and multiword grouping) on the performance of a
standard neural text classifier. We perform an extensive evaluation on standard
benchmarks from text categorization and sentiment analysis. While our
experiments show that a simple tokenization of input text is generally
adequate, they also highlight significant degrees of variability across
preprocessing techniques. This reveals the importance of paying attention to
this usually-overlooked step in the pipeline, particularly when comparing
different models. Finally, our evaluation provides insights into the best
preprocessing practices for training word embeddings.Comment: Blackbox EMNLP 2018. 7 page
- …