Sequence Transduction with Recurrent Neural Networks

Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to represent both the input and output sequences in a way that is invariant to sequential distortions such as shrinking, stretching and translating. Recurrent neural networks (RNNs) are a powerful sequence learning architecture that has proven capable of learning such representations. However RNNs traditionally require a pre-defined alignment between the input and output sequences to perform transduction. This is a severe limitation since \emph{finding} the alignment is the most difficult aspect of many sequence transduction problems. Indeed, even determining the length of the output sequence is often challenging. This paper introduces an end-to-end, probabilistic sequence transduction system, based entirely on RNNs, that is in principle able to transform any input sequence into any finite, discrete output sequence. Experimental results for phoneme recognition are provided on the TIMIT speech corpus.Comment: First published in the International Conference of Machine Learning (ICML) 2012 Workshop on Representation Learnin

Pattern Recognition In Non-Kolmogorovian Structures

We present a generalization of the problem of pattern recognition to arbitrary probabilistic models. This version deals with the problem of recognizing an individual pattern among a family of different species or classes of objects which obey probabilistic laws which do not comply with Kolmogorov's axioms. We show that such a scenario accommodates many important examples, and in particular, we provide a rigorous definition of the classical and the quantum pattern recognition problems, respectively. Our framework allows for the introduction of non-trivial correlations (as entanglement or discord) between the different species involved, opening the door to a new way of harnessing these physical resources for solving pattern recognition problems. Finally, we present some examples and discuss the computational complexity of the quantum pattern recognition problem, showing that the most important quantum computation algorithms can be described as non-Kolmogorovian pattern recognition problems