8 research outputs found
Unsupervised Generative Modeling Using Matrix Product States
Generative modeling, which learns joint probability distribution from data
and generates samples according to it, is an important task in machine learning
and artificial intelligence. Inspired by probabilistic interpretation of
quantum physics, we propose a generative model using matrix product states,
which is a tensor network originally proposed for describing (particularly
one-dimensional) entangled quantum states. Our model enjoys efficient learning
analogous to the density matrix renormalization group method, which allows
dynamically adjusting dimensions of the tensors and offers an efficient direct
sampling approach for generative tasks. We apply our method to generative
modeling of several standard datasets including the Bars and Stripes, random
binary patterns and the MNIST handwritten digits to illustrate the abilities,
features and drawbacks of our model over popular generative models such as
Hopfield model, Boltzmann machines and generative adversarial networks. Our
work sheds light on many interesting directions of future exploration on the
development of quantum-inspired algorithms for unsupervised machine learning,
which are promisingly possible to be realized on quantum devices.Comment: 11 pages, 12 figures (not including the TNs) GitHub Page:
https://congzlwag.github.io/UnsupGenModbyMPS
Quantum theory in finite dimension cannot explain every general process with finite memory
Arguably, the largest class of stochastic processes generated by means of a
finite memory consists of those that are sequences of observations produced by
sequential measurements in a suitable generalized probabilistic theory (GPT).
These are constructed from a finite-dimensional memory evolving under a set of
possible linear maps, and with probabilities of outcomes determined by linear
functions of the memory state. Examples of such models are given by classical
hidden Markov processes, where the memory state is a probability distribution,
and at each step it evolves according to a non-negative matrix, and hidden
quantum Markov processes, where the memory state is a finite dimensional
quantum state, and at each step it evolves according to a completely positive
map. Here we show that the set of processes admitting a finite-dimensional
explanation do not need to be explainable in terms of either classical
probability or quantum mechanics. To wit, we exhibit families of processes that
have a finite-dimensional explanation, defined manifestly by the dynamics of
explicitly given GPT, but that do not admit a quantum, and therefore not even
classical, explanation in finite dimension. Furthermore, we present a family of
quantum processes on qubits and qutrits that do not admit a classical
finite-dimensional realization, which includes examples introduced earlier by
Fox, Rubin, Dharmadikari and Nadkarni as functions of infinite dimensional
Markov chains, and lower bound the size of the memory of a classical model
realizing a noisy version of the qubit processes.Comment: 18 pages, 0 figure
Méthodes des moments pour l'inférence de systèmes séquentiels linéaires rationnels
Learning stochastic models generating sequences has many applications in natural language processing, speech recognitions or bioinformatics. Multiplicity Automata (MA) are graphical latent variable models that encompass a wide variety of linear systems. In particular, they can model stochastic languages, stochastic processes and controlled processes. Traditional learning algorithms such as the one of Baum-Welch are iterative, slow and may converge to local optima. A recent alternative is to use the Method of Moments (MoM) to design consistent and fast algorithms with pseudo-PAC guarantees.However, MoM-based algorithms have two main disadvantages. First, the PAC guarantees hold only if the size of the learned model corresponds to the size of the target model. Second, although these algorithms learn a function close to the target distribution, most do not ensure it will be a distribution. Thus, a model learned from a finite number of examples may return negative values or values that do not sum to one.This thesis addresses both problems. First, we extend the theoretical guarantees for compressed models, and propose a regularized spectral algorithm that adjusts the size of the model to the data. Then, an application in electronic warfare is proposed to sequence of the dwells of a superheterodyne receiver. Finally, we design new learning algorithms based on the MoM that do not suffer the problem of negative probabilities. We show for one of them pseudo-PAC guarantees.L’apprentissage de modèles stochastiques générant des séquences a de nombreuses applications comme en traitement de la parole, du langage ou bien encore en bio-informatique. Les Automates à Multiplicité (MA) sont des modèles graphiques à variables latentes qui englobent une grande variété de systèmes linéaires pouvant représenter entre autres des langues stochastiques, des processus stochastiques ainsi que des processus contrôlés. Les algorithmes traditionnels d’apprentissage comme celui de Baum-Welch sont itératifs, lent et peuvent converger vers des optima locaux. Une alternative récente consiste à utiliser la méthode des moments (MoM) pour concevoir des algorithmes rapides et consistent avec des garanties pseudo-PAC.Cependant, les algorithmes basés sur la MoM ont deux inconvénients principaux. Tout d'abord, les garanties PAC ne sont valides que si la dimension du modèle appris correspond à la dimension du modèle cible. Deuxièmement, bien que les algorithmes basés sur la MoM apprennent une fonction proche de la distribution cible, la plupart ne contraignent pas celle-ci à être une distribution. Ainsi, un modèle appris à partir d’un nombre fini d’exemples peut renvoyer des valeurs négatives et qui ne somment pas à un.Ainsi, cette thèse s’adresse à ces deux problèmes en proposant 1) un élargissement des garanties théoriques pour les modèles compressés et 2) de nouveaux algorithmes d’apprentissage ne souffrant pas du problème des probabilités négatives et dont certains bénéficient de garanties PAC. Une application en guerre électronique est aussi proposée pour le séquencement des écoutes du récepteur superhétéordyne