Multi-space Variational Encoder-Decoders for Semi-supervised Labeled Sequence Transduction

Labeled sequence transduction is a task of transforming one sequence into another sequence that satisfies desiderata specified by a set of labels. In this paper we propose multi-space variational encoder-decoders, a new model for labeled sequence transduction with semi-supervised learning. The generative model can use neural networks to handle both discrete and continuous latent variables to exploit various features of data. Experiments show that our model provides not only a powerful supervised framework but also can effectively take advantage of the unlabeled data. On the SIGMORPHON morphological inflection benchmark, our model outperforms single-model state-of-art results by a large margin for the majority of languages.Comment: Accepted by ACL 201

Learning Character-level Compositionality with Visual Features

Previous work has modeled the compositionality of words by creating character-level models of meaning, reducing problems of sparsity for rare words. However, in many writing systems compositionality has an effect even on the character-level: the meaning of a character is derived by the sum of its parts. In this paper, we model this effect by creating embeddings for characters based on their visual characteristics, creating an image for the character and running it through a convolutional neural network to produce a visual character embedding. Experiments on a text classification task demonstrate that such model allows for better processing of instances with rare characters in languages such as Chinese, Japanese, and Korean. Additionally, qualitative analyses demonstrate that our proposed model learns to focus on the parts of characters that carry semantic content, resulting in embeddings that are coherent in visual space.Comment: Accepted to ACL 201

Making AI Meaningful Again

Artificial intelligence (AI) research enjoyed an initial period of enthusiasm in the 1970s and 80s. But this enthusiasm was tempered by a long interlude of frustration when genuinely useful AI applications failed to be forthcoming. Today, we are experiencing once again a period of enthusiasm, fired above all by the successes of the technology of deep neural networks or deep machine learning. In this paper we draw attention to what we take to be serious problems underlying current views of artificial intelligence encouraged by these successes, especially in the domain of language processing. We then show an alternative approach to language-centric AI, in which we identify a role for philosophy

Probabilities and Quantum Reality: Are There Correlata?

Any attempt to introduce probabilities into quantum mechanics faces difficulties due to the mathematical structure of Hilbert space, as reflected in Birkhoff and von Neumann's proposal for a quantum logic. The (consistent or decoherent) histories solution is provided by its single framework rule, an approach that includes conventional (Copenhagen) quantum theory as a special case. Mermin's Ithaca interpretation addresses the same problem by defining probabilities which make no reference to a sample space or event algebra (``correlations without correlata''). But this leads to severe conceptual difficulties, which almost inevitably couple quantum theory to unresolved problems of human consciousness. Using histories allows a sharper quantum description than is possible with a density matrix, suggesting that the latter provides an ensemble rather than an irreducible single-system description as claimed by Mermin. The histories approach satisfies the first five of Mermin's desiderata for a good interpretation of quantum mechanics, including Einstein locality, but the Ithaca interpretation seems to have difficulty with the first (independence of observers) and the third (describing individual systems).Comment: Latex 31 pages, 3 figures in text using PSTrick