Neural Reranking for Named Entity Recognition

We propose a neural reranking system for named entity recognition (NER). The basic idea is to leverage recurrent neural network models to learn sentence-level patterns that involve named entity mentions. In particular, given an output sentence produced by a baseline NER model, we replace all entity mentions, such as \textit{Barack Obama}, into their entity types, such as \textit{PER}. The resulting sentence patterns contain direct output information, yet is less sparse without specific named entities. For example, "PER was born in LOC" can be such a pattern. LSTM and CNN structures are utilised for learning deep representations of such sentences for reranking. Results show that our system can significantly improve the NER accuracies over two different baselines, giving the best reported results on a standard benchmark.Comment: Accepted as regular paper by RANLP 201

Conditional Random Field Autoencoders for Unsupervised Structured Prediction

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines

Semi-supervised sequence tagging with bidirectional language models

Pre-trained word embeddings learned from unlabeled text have become a standard component of neural network architectures for NLP tasks. However, in most cases, the recurrent network that operates on word-level representations to produce context sensitive representations is trained on relatively little labeled data. In this paper, we demonstrate a general semi-supervised approach for adding pre- trained context embeddings from bidirectional language models to NLP systems and apply it to sequence labeling tasks. We evaluate our model on two standard datasets for named entity recognition (NER) and chunking, and in both cases achieve state of the art results, surpassing previous systems that use other forms of transfer or joint learning with additional labeled data and task specific gazetteers.Comment: To appear in ACL 201

Robust Multilingual Part-of-Speech Tagging via Adversarial Training

Adversarial training (AT) is a powerful regularization method for neural networks, aiming to achieve robustness to input perturbations. Yet, the specific effects of the robustness obtained from AT are still unclear in the context of natural language processing. In this paper, we propose and analyze a neural POS tagging model that exploits AT. In our experiments on the Penn Treebank WSJ corpus and the Universal Dependencies (UD) dataset (27 languages), we find that AT not only improves the overall tagging accuracy, but also 1) prevents over-fitting well in low resource languages and 2) boosts tagging accuracy for rare / unseen words. We also demonstrate that 3) the improved tagging performance by AT contributes to the downstream task of dependency parsing, and that 4) AT helps the model to learn cleaner word representations. 5) The proposed AT model is generally effective in different sequence labeling tasks. These positive results motivate further use of AT for natural language tasks.Comment: NAACL 201

Aplicação de resinas compostas em situações de severo envolvimento estético : a propósito de um caso clínico

hipodontia é uma anomalia de desenvolvimento dentário rara e que caracteriza pela ausência de dentes, condicionando um impacto funcional e estético relevante. Afeta com maior frequência indivíduos de raça branca e os dentes permanentes. A sua etiologia permanece por compreender completamente, podendo resultar de fatores ambientais ou distúrbios genéticos. A sua apresentação pode ser isolada ou parte de um síndrome. O tratamento atual pode implicar recurso a próteses dentárias, implantes dentários, correção ortodôntica e restauração direta estética. O propósito desta tese é apresentar uma revisão da literatura sobre as técnicas de restauração diretas atuais da Dentisteria Operatória no tratamento da hipodontia, a partir do estudo de um caso clínico de hipodontia grave. Perante a abordagem tradicional protético-ortodôntica apresenta-se um caso clínico de hipodontia com elevado comprometimento estético que demonstra idêntico grau de sucesso terapêutico com o recurso a técnicas de restauração direta da Dentisteria Operatória modernaThe condition known as hypodontia is a rare anomaly of teeth development characterized by the absence of teeth, with functional and aesthetic relevance. Most often affects Caucasians and permanent teeth. Its etiology remains to fully understand and may result from environmental factors or genetic disorders. The presentation may be isolated or part of a syndrome. Current treatment may involve use of dentures, dental implants, orthodontic correction and direct aesthetic restoration. The purpose of this thesis is to present a literature review on direct restoring techniques of modern Dentistry: it is presented a clinical case of severe non syndromic oligodontia undergone successful treatment of direct aesthetic restoration with adhesive techniques of contemporary Operative Dentistry. A Discussion of the case emphatises the operative dentistry treatment of application of composites and it was based on a non systematic literature review, about hypodontia. Given the traditional prosthetic-orthodontic approach it is presented a clinical case of hypodontia with high aesthetic commitment that demonstrates the same degree of success with the use of therapeutic direct adhesive restoration techniques of Modern Operative Dentistry

Natural Language Processing (Almost) from Scratch

We propose a unified neural network architecture and learning algorithm that can be applied to various natural language processing tasks including part-of-speech tagging, chunking, named entity recognition, and semantic role labeling. This versatility is achieved by trying to avoid task-specific engineering and therefore disregarding a lot of prior knowledge. Instead of exploiting man-made input features carefully optimized for each task, our system learns internal representations on the basis of vast amounts of mostly unlabeled training data. This work is then used as a basis for building a freely available tagging system with good performance and minimal computational requirements

Investigations into the value of labeled and unlabeled data in biomedical entity recognition and word sense disambiguation

Human annotations, especially in highly technical domains, are expensive and time consuming togather, and can also be erroneous. As a result, we never have sufficiently accurate data to train andevaluate supervised methods. In this thesis, we address this problem by taking a semi-supervised approach to biomedical namedentity recognition (NER), and by proposing an inventory-independent evaluation framework for supervised and unsupervised word sense disambiguation. Our contributions are as follows: We introduce a novel graph-based semi-supervised approach to named entity recognition(NER) and exploit pre-trained contextualized word embeddings in several biomedical NER tasks. We propose a new evaluation framework for word sense disambiguation that permits a fair comparison between supervised methods trained on different sense inventories as well as unsupervised methods without a fixed sense inventory