Model-based Reinforcement Learning and the Eluder Dimension

We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly as $\tilde{O}(\sqrt{d_K d_E T})$ where $T$ is time elapsed, $d_K$ is the Kolmogorov dimension and $d_E$ is the \emph{eluder dimension}. These represent the first unified regret bounds for model-based reinforcement learning and provide state of the art guarantees in several important settings. Moreover, we present a simple and computationally efficient algorithm \emph{posterior sampling for reinforcement learning} (PSRL) that satisfies these bounds

The Human Development Index as a Criterion for Optimal Planning

Planning strategies that maximize the Human Development Index (HDI) tend towards minimizing consumption and maximizing non-investment expenditures on education and health. Interestingly, such strategies also tend towards equitable outcomes, even though inequality aversion is not modelled in the HDI. A problematic feature of strategies that maximize the HDI is that the income component in the index only role is to distort the allocation between health and education expenditure. Because the income component does not play its intended role of securing resources for a decent standard of living, we argue that it is better to drop income from the index in considering optimal plans. Alternatively, we consider net income, income net of education and health expenditures, as indicator of capabilities not already reflected in the education and life expectancy components of the index. When net income is used in a modified HDI index, optimal plans yield a balance between allocations for consumption, education, and health. Finally, we calculate our modified indexes for OECD countries and compare them with the HDI.Consumption; Human development index; Income; Inequality; Planning

Coherent-Classical Estimation versus Purely-Classical Estimation for Linear Quantum Systems

We consider a coherent-classical estimation scheme for a class of linear quantum systems. It comprises an estimator that is a mixed quantum-classical system without involving coherent feedback. The estimator yields a classical estimate of a variable for the quantum plant. We demonstrate that for a passive plant that can be characterized by annihilation operators only, such coherent-classical estimation provides no improvement over purely-classical estimation. An example is also given which shows that if the plant is not assumed to be an annihilation operator only quantum system, it is possible to get better estimates with such coherent-classical estimation compared with purely-classical estimation.Comment: 7 pages, 5 figures. Minor corrections. Accepted, 2014 Conference on Decision and Contro