GraphLab: A New Framework for Parallel Machine Learning

Designing and implementing efficient, provably correct parallel machine learning (ML) algorithms is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the GraphLab framework by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing. We show that using GraphLab we can achieve excellent parallel performance on large scale real-world problems

On sampling nodes in a network

Random walk is an important tool in many graph mining applications including estimating graph parameters, sampling portions of the graph, and extracting dense communities. In this paper we consider the problem of sampling nodes from a large graph according to a prescribed distribution by using random walk as the basic primitive. Our goal is to obtain algorithms that make a small number of queries to the graph but output a node that is sampled according to the prescribed distribution. Focusing on the uniform distribution case, we study the query complexity of three algorithms and show a near-tight bound expressed in terms of the parameters of the graph such as average degree and the mixing time. Both theoretically and empirically, we show that some algorithms are preferable in practice than the others. We also extend our study to the problem of sampling nodes according to some polynomial function of their degrees; this has implications for designing efficient algorithms for applications such as triangle counting

Designing experience replay algorithms for off-policy reinforcement learning by studying their sampling distributions

Els algorismes off-policy d’aprenentatge per reforç fan ús dels mecanismes de repetició de la memòria per a aprendre de l’experiència viscuda prèviament per altres agents. Existeixen diversos algorismes per a obtenir mostres de la memòria, i tots tenen la intenció de fer el procés d’aprenentatge més ràpid i eficient en el nombre de mostres visualitzades. Tot i això, no existeix cap marc comú que permeti comparar-los i explicar les diferències dels seus rendiments. A aquesta tesi presentarem una eina per a estudiar aquests algorismes: les distribucions de mostreig de transicions i estats. Aquestes eines fan possible establir comparacions entre diferents algorismes de repetició de memòria. Una anàlisi des del punt de vista de les distribucions evidencia que les memòries dels agents no són equilibrades: hi ha parts de l’espai d’estats que estan sobrerepresentades, mentre que d’altres amb prou feines són a la memòria. Aquesta troballa es repeteix en diversos entorns diferents, i de manera marcada a entorns on les recompenses són disperses. Acabarem proposant dos nous algorismes que solucionen el problema del desequilibri i obtenen millor rendiment a tasques amb recompenses disperses, mentre que funcionen igual de bé que els algorismes de referència en situacions més equilibrades.Los algoritmos off-policy de aprendizaje por refuerzo usan los mecanismos de repetición de la memoria para aprender de la experiencia previamente obtenida por otros agentes. Existen varios algoritmos que permiten obtener muestras de la memoria, con la intención de acelerar el proceso de aprendizaje y hacerlo más eficiente en el número de muestras visualizadas. Sin embargo, no existe un marco común que permita compararlos y explicar diferencias en su rendimiento. En esta tesis presentamos una herramienta para estudiar estos algoritmos: las distribuciones de muestreo de transiciones y de estados. Estas herramientas permiten comparar diferentes algoritmos de repetición de la memoria. Un análisis desde el punto de vista de las distribuciones muestra que las memorias de los agentes no están equilibradas: hay partes del espacio de estados que están sobrerrepresentadas, y hay otras que prácticamente no están. Este descubrimiento se repite en distintos entornos, y de manera destacada en entornos con recompensas dispersas. Finalizamos proponiendo dos nuevos algoritmos que solucionan el problema del desequilibrio y consiguen mejor rendimiento en situaciones con recompensas dispersas, mientras que funcionan tan bien como los algoritmos de referencia en situaciones donde el desequilibrio no es un problema.Off-policy reinforcement learning algorithms make use of experience replay mechanisms to learn from experience gathered by earlier policies. Several algorithms have been proposed to sample transitions from the replay buffer to make training more sample efficient. However, a general framework to compare them and explain their performance differences is missing. In this thesis we propose the transition and state sampling distributions as tools to study these algorithms, allowing to draw comparisons across sampling strategies. An analysis from the distribution point of view reveals that replay buffers are imbalanced, with parts of the state space being underrepresented, while other sections are massively overrepresented. These findings happen across several environments, especially in sparse reward settings. We finish by proposing two algorithms that address the imbalance problem and show that they lead to better performance in sparse reward tasks while matching our baselines in low-imbalance situations.Outgoin

Quantum rejection sampling

Rejection sampling is a well-known method to sample from a target distribution, given the ability to sample from a given distribution. The method has been first formalized by von Neumann (1951) and has many applications in classical computing. We define a quantum analogue of rejection sampling: given a black box producing a coherent superposition of (possibly unknown) quantum states with some amplitudes, the problem is to prepare a coherent superposition of the same states, albeit with different target amplitudes. The main result of this paper is a tight characterization of the query complexity of this quantum state generation problem. We exhibit an algorithm, which we call quantum rejection sampling, and analyze its cost using semidefinite programming. Our proof of a matching lower bound is based on the automorphism principle which allows to symmetrize any algorithm over the automorphism group of the problem. Our main technical innovation is an extension of the automorphism principle to continuous groups that arise for quantum state generation problems where the oracle encodes unknown quantum states, instead of just classical data. Furthermore, we illustrate how quantum rejection sampling may be used as a primitive in designing quantum algorithms, by providing three different applications. We first show that it was implicitly used in the quantum algorithm for linear systems of equations by Harrow, Hassidim and Lloyd. Secondly, we show that it can be used to speed up the main step in the quantum Metropolis sampling algorithm by Temme et al.. Finally, we derive a new quantum algorithm for the hidden shift problem of an arbitrary Boolean function and relate its query complexity to "water-filling" of the Fourier spectrum.Comment: 19 pages, 5 figures, minor changes and a more compact style (to appear in proceedings of ITCS 2012