Differentiable Programming Tensor Networks

Differentiable programming is a fresh programming paradigm which composes parameterized algorithmic components and trains them using automatic differentiation (AD). The concept emerges from deep learning but is not only limited to training neural networks. We present theory and practice of programming tensor network algorithms in a fully differentiable way. By formulating the tensor network algorithm as a computation graph, one can compute higher order derivatives of the program accurately and efficiently using AD. We present essential techniques to differentiate through the tensor networks contractions, including stable AD for tensor decomposition and efficient backpropagation through fixed point iterations. As a demonstration, we compute the specific heat of the Ising model directly by taking the second order derivative of the free energy obtained in the tensor renormalization group calculation. Next, we perform gradient based variational optimization of infinite projected entangled pair states for quantum antiferromagnetic Heisenberg model and obtain start-of-the-art variational energy and magnetization with moderate efforts. Differentiable programming removes laborious human efforts in deriving and implementing analytical gradients for tensor network programs, which opens the door to more innovations in tensor network algorithms and applications.Comment: Typos corrected, discussion and refs added; revised version accepted for publication in PRX. Source code available at https://github.com/wangleiphy/tensorgra

Revealing networks from dynamics: an introduction

What can we learn from the collective dynamics of a complex network about its interaction topology? Taking the perspective from nonlinear dynamics, we briefly review recent progress on how to infer structural connectivity (direct interactions) from accessing the dynamics of the units. Potential applications range from interaction networks in physics, to chemical and metabolic reactions, protein and gene regulatory networks as well as neural circuits in biology and electric power grids or wireless sensor networks in engineering. Moreover, we briefly mention some standard ways of inferring effective or functional connectivity.Comment: Topical review, 48 pages, 7 figure

Computer Aided Aroma Design. II. Quantitative structure-odour relationship

Computer Aided Aroma Design (CAAD) is likely to become a hot issue as the REACH EC document targets many aroma compounds to require substitution. The two crucial steps in CAMD are the generation of candidate molecules and the estimation of properties, which can be difficult when complex molecular structures like odours are sought and their odour quality are definitely subjective or their odour intensity are partly subjective as stated in Rossitier’s review (1996). The CAAD methodology and a novel molecular framework were presented in part I. Part II focuses on a classification methodology to characterize the odour quality of molecules based on Structure – Odour Relation (SOR). Using 2D and 3D molecular descriptors, Linear Discriminant Analysis (LDA) and Artificial Neural Network are compared in favour of LDA. The classification into balsamic / non balsamic quality was satisfactorily solved. The classification among five sub notes of the balsamic quality was less successful, partly due to the selection of the Aldrich’s Catalog as the reference classification. For the second case, it is shown that the sweet sub note considered in Aldrich’s Catalog is not a relevant sub note, confirming the alternative and popular classification of Jaubert et al., (1995), the field of odours