22,733 research outputs found
Optimistic Agents are Asymptotically Optimal
We use optimism to introduce generic asymptotically optimal reinforcement
learning agents. They achieve, with an arbitrary finite or compact class of
environments, asymptotically optimal behavior. Furthermore, in the finite
deterministic case we provide finite error bounds.Comment: 13 LaTeX page
Private Pareto Optimal Exchange
We consider the problem of implementing an individually rational,
asymptotically Pareto optimal allocation in a barter-exchange economy where
agents are endowed with goods and have preferences over the goods of others,
but may not use money as a medium of exchange. Because one of the most
important instantiations of such economies is kidney exchange -- where the
"input"to the problem consists of sensitive patient medical records -- we ask
to what extent such exchanges can be carried out while providing formal privacy
guarantees to the participants. We show that individually rational allocations
cannot achieve any non-trivial approximation to Pareto optimality if carried
out under the constraint of differential privacy -- or even the relaxation of
\emph{joint} differential privacy, under which it is known that asymptotically
optimal allocations can be computed in two-sided markets, where there is a
distinction between buyers and sellers and we are concerned only with privacy
of the buyers~\citep{Matching}. We therefore consider a further relaxation that
we call \emph{marginal} differential privacy -- which promises, informally,
that the privacy of every agent is protected from every other agent so long as does not collude or share allocation information with other
agents. We show that, under marginal differential privacy, it is possible to
compute an individually rational and asymptotically Pareto optimal allocation
in such exchange economies
On the Computability of Solomonoff Induction and Knowledge-Seeking
Solomonoff induction is held as a gold standard for learning, but it is known
to be incomputable. We quantify its incomputability by placing various flavors
of Solomonoff's prior M in the arithmetical hierarchy. We also derive
computability bounds for knowledge-seeking agents, and give a limit-computable
weakly asymptotically optimal reinforcement learning agent.Comment: ALT 201
Convergence of Least Squares Learning in Self-Referential Discontinuous Stochastic Models
We examine the stability of rational expectations equilibria in the class of models in which the decision of the individual agent is discontinuous with respect to the state variables. Instead of rational expectations, each agent learns the unknown parameters through a recursive stochastic algorithm. If the agents the estimated value function ``rapidly'' enough, then each agent learns the true value function associated with the optimal action with probability, and almost always takes the optimal action asymptotically.
- …