Parameter Sharing in Coagent Networks

In this paper, we aim to prove the theorem that generalizes the Coagent Network Policy Gradient Theorem (Kostas et. al., 2019) to the context where parameters are shared among the function approximators involved. This provides the theoretical foundation to use any pattern of parameter sharing and leverage the freedom in the graph structure of the network to possibility exploit relational bias in a given task. As another application, we will apply our result to give a more intuitive proof for the Hierarchical Option Critic Policy Gradient Theorem, first shown in (Riemer et. al., 2019)

Better bounds for low-energy product formulas

Product formulas are one of the main approaches for quantum simulation of the Hamiltonian dynamics of a quantum system. Their implementation cost is computed based on error bounds which are often pessimistic, resulting in overestimating the total runtime. In this work, we rigorously consider the error induced by product formulas when the state undergoing time evolution lies in the low-energy sector with respect to the Hamiltonian of the system. We show that in such a setting, the usual error bounds based on the operator norm of nested commutators can be replaced by those restricted to suitably chosen low-energy subspaces, yielding tighter error bounds. Furthermore, under some locality and positivity assumptions, we show that the simulation of generic product formulas acting on low-energy states can be done asymptotically more efficiently when compared with previous results

Quantum computing with Octonions

There are two schools of "measurement-only quantum computation". The first ([11]) using prepared entanglement (cluster states) and the second ([4]) using collections of anyons, which according to how they were produced, also have an entanglement pattern. We abstract the common principle behind both approaches and find the notion of a graph or even continuous family of equiangular projections. This notion is the leading character in the paper. The largest continuous family, in a sense made precise in Corollary 4.2, is associated with the octonions and this example leads to a universal computational scheme. Adiabatic quantum computation also fits into this rubric as a limiting case: nearby projections are nearly equiangular, so as a gapped ground state space is slowly varied the corrections to unitarity are small.Comment: Added some new results in section

The Smallest Interacting Universe

The co-emergence of locality between the Hamiltonian and initial state of the universe is studied in a simple toy model. We hypothesize a fundamental loss functional for the combined Hamiltonian and quantum state and minimize it by gradient descent. This minimization yields a tensor product structure simultaneously respected by both the Hamiltonian and the state, suggesting that locality can emerge by a process analogous to spontaneous symmetry breaking. We discuss the relevance of this program to the arrow of time problem. In our toy model, we interpret the emergence of a tensor factorization as the appearance of individual degrees of freedom within a previously undifferentiated (raw) Hilbert space. Earlier work [5, 6] looked at the emergence of locality in Hamiltonians only, and found strong numerical confirmation of that raw Hilbert spaces of $\dim = n$ are unstable and prefer to settle on tensor factorization when $n=pq$ is not prime, and in [6] even primes were seen to "factor" after first shedding a small summand, e.g. $7=1+2\cdot 3$. This was found in the context of a rather general potential functional $F$ on the space of metrics $\{g_{ij}\}$ on $\mathfrak{su}(n)$, the Lie algebra of symmetries. This emergence of qunits through operator-level spontaneous symmetry breaking (SSB) may help us understand why the world seems to consist of myriad interacting degrees of freedom. But understanding why the universe has an initial Hamiltonian $H_0$ with a many-body structure is of limited conceptual value unless the initial state, $|\psi_0\rangle$, is also structured by this tensor decomposition. Here we adapt $F$ to become a functional on $\{g,|\psi_0\rangle\}=(\text{metrics})\times (\text{initial states})$, and find SSB now produces a conspiracy between $g$ and $|\psi_0\rangle$, where they simultaneously attain low entropy by settling on the same qubit decomposition

Improved Bayesian Regret Bounds for Thompson Sampling in Reinforcement Learning

In this paper, we prove the first Bayesian regret bounds for Thompson Sampling in reinforcement learning in a multitude of settings. We simplify the learning problem using a discrete set of surrogate environments, and present a refined analysis of the information ratio using posterior consistency. This leads to an upper bound of order $\widetilde{O}(H\sqrt{d_{l_1}T})$ in the time inhomogeneous reinforcement learning problem where $H$ is the episode length and $d_{l_1}$ is the Kolmogorov $l_1-$dimension of the space of environments. We then find concrete bounds of $d_{l_1}$ in a variety of settings, such as tabular, linear and finite mixtures, and discuss how how our results are either the first of their kind or improve the state-of-the-art.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS 2023