Intrinsically motivated exploration has proven useful for reinforcement
learning, even without additional extrinsic rewards. When the environment is
naturally represented as a graph, how to guide exploration best remains an open
question. In this work, we propose a novel approach for exploring
graph-structured data motivated by two theories of human curiosity: the
information gap theory and the compression progress theory. The theories view
curiosity as an intrinsic motivation to optimize for topological features of
subgraphs induced by the visited nodes in the environment. We use these
proposed features as rewards for graph neural-network-based reinforcement
learning. On multiple classes of synthetically generated graphs, we find that
trained agents generalize to larger environments and to longer exploratory
walks than are seen during training. Our method computes more efficiently than
the greedy evaluation of the relevant topological properties. The proposed
intrinsic motivations bear particular relevance for recommender systems. We
demonstrate that curiosity-based recommendations are more predictive of human
behavior than PageRank centrality for several real-world graph datasets,
including MovieLens, Amazon Books, and Wikispeedia.Comment: 14 pages, 5 figures in main text, and 15 pages, 8 figures in
supplemen