Model-based Reinforcement Learning and the Eluder Dimension

We consider the problem of learning to optimize an unknown Markov decision process (MDP). We show that, if the MDP can be parameterized within some known function class, we can obtain regret bounds that scale with the dimensionality, rather than cardinality, of the system. We characterize this dependence explicitly as $\tilde{O}(\sqrt{d_K d_E T})$ where $T$ is time elapsed, $d_K$ is the Kolmogorov dimension and $d_E$ is the \emph{eluder dimension}. These represent the first unified regret bounds for model-based reinforcement learning and provide state of the art guarantees in several important settings. Moreover, we present a simple and computationally efficient algorithm \emph{posterior sampling for reinforcement learning} (PSRL) that satisfies these bounds

The Human Development Index as a Criterion for Optimal Planning

Planning strategies that maximize the Human Development Index (HDI) tend towards minimizing consumption and maximizing non-investment expenditures on education and health. Interestingly, such strategies also tend towards equitable outcomes, even though inequality aversion is not modelled in the HDI. A problematic feature of strategies that maximize the HDI is that the income component in the index only role is to distort the allocation between health and education expenditure. Because the income component does not play its intended role of securing resources for a decent standard of living, we argue that it is better to drop income from the index in considering optimal plans. Alternatively, we consider net income, income net of education and health expenditures, as indicator of capabilities not already reflected in the education and life expectancy components of the index. When net income is used in a modified HDI index, optimal plans yield a balance between allocations for consumption, education, and health. Finally, we calculate our modified indexes for OECD countries and compare them with the HDI.Consumption; Human development index; Income; Inequality; Planning

Data-Intensive Computing in the 21st Century

The deluge of data that future applications must process—in domains ranging from science to business informatics—creates a compelling argument for substantially increased R&D targeted at discovering scalable hardware and software solutions for data-intensive problems