Bayesian Reinforcement Learning via Deep, Sparse Sampling

We address the problem of Bayesian reinforcement learning using efficient model-based online planning. We propose an optimism-free Bayes-adaptive algorithm to induce deeper and sparser exploration with a theoretical bound on its performance relative to the Bayes optimal policy, with a lower computational complexity. The main novelty is the use of a candidate policy generator, to generate long-term options in the planning tree (over beliefs), which allows us to create much sparser and deeper trees. Experimental results on different environments show that in comparison to the state-of-the-art, our algorithm is both computationally more efficient, and obtains significantly higher reward in discrete environments.Comment: Published in AISTATS 202

Sparsely Aggregated Convolutional Networks

We explore a key architectural aspect of deep convolutional neural networks: the pattern of internal skip connections used to aggregate outputs of earlier layers for consumption by deeper layers. Such aggregation is critical to facilitate training of very deep networks in an end-to-end manner. This is a primary reason for the widespread adoption of residual networks, which aggregate outputs via cumulative summation. While subsequent works investigate alternative aggregation operations (e.g. concatenation), we focus on an orthogonal question: which outputs to aggregate at a particular point in the network. We propose a new internal connection structure which aggregates only a sparse set of previous outputs at any given depth. Our experiments demonstrate this simple design change offers superior performance with fewer parameters and lower computational requirements. Moreover, we show that sparse aggregation allows networks to scale more robustly to 1000+ layers, thereby opening future avenues for training long-running visual processes.Comment: Accepted to ECCV 201

Systematic revision of the american taxa belonging to the genera Alloblackburneus Bordat, 2009, and Blackburneus Schmidt, 1913, with description of seven new species (Coleoptera: Scarabaeidae: Aphodiinae)

The American species belonging to the genera Alloblackburneus Bordat and Blackburneus Schmidt (Coleoptera: Scarabaeidae: Aphodiinae) are redescribed and figured. Seven new species are described: Alloblackburneus guadalajarae, A. ibanezbernali; Blackburneus amazonicus, B. sanfilippoi, B. surinamensis, B. teposcolulaensis, B. thomasi. The neotype of Scarabaeus rubeolus Palisot de Beauvois, 1809 is designated. The lectotype of Blackburneus argentinensis (Schmidt, 1909) and of Blackburneus laxepunctatus (Schmidt, 1910) are designated. The following new combinations are proposed: Alloblackburneus aegrotus (Horn, 1870); Alloblackburneus cavidomus (Brown, 1927); Alloblackburneus cynomysi (Brown, 1927); Alloblackburneus fordi (Gordon, 1974); Alloblackburneus geomysi (Cartwright, 1939); Alloblackburneus lentus (Horn, 1870); Alloblackburneus rubeolus (Palisot de Beauvois, 1805); Alloblackburneus saylori (Hinton, 1934); Alloblackburneus tenuistriatus (Horn, 1887); Alloblackburneus troglodytes (Hubbard, 1894)

Sample Efficient Bayesian Reinforcement Learning

Artificial Intelligence (AI) has been an active field of research for over a century now. The research field of AI may be grouped into various tasks that are expected from an intelligent agent; two major ones being learning & inference and planning. The act of storing new knowledge is known as learning while inference refers to the act to extracting conclusions given agent’s limited knowledge base. They are tightly knit by the design of its knowledge base. The process of deciding long-term actions or plans given its current knowledge is called planning.Reinforcement Learning (RL) brings together these two tasks by posing a seemingly benign question “How to act optimally in an unknown environment?”. This requires the agent to learn about its environment as well as plan actions given its current knowledge about it. In RL, the environment can be represented by a mathematical model and we associate an intrinsic value to the actions that the agent may choose.In this thesis, we present a novel Bayesian algorithm for the problem of RL. Bayesian RL is a widely explored area of research but is constrained by scalability and performance issues. We provide first steps towards rigorous analysis of these types of algorithms. Bayesian algorithms are characterized by the belief that they maintain over their unknowns; which is updated based on the collected evidence. This is different from the traditional approach in RL in terms of problem formulation and formal guarantees. Our novel algorithm combines aspects of planning and learning due to its inherent Bayesian formulation. It does so in a more scalable fashion, with formal PAC guarantees. We also give insights on the application of Bayesian framework for the estimation of model and value, in a joint work on Bayesian backward induction for RL