ASPiRe:Adaptive Skill Priors for Reinforcement Learning

We introduce ASPiRe (Adaptive Skill Prior for RL), a new approach that leverages prior experience to accelerate reinforcement learning. Unlike existing methods that learn a single skill prior from a large and diverse dataset, our framework learns a library of different distinction skill priors (i.e., behavior priors) from a collection of specialized datasets, and learns how to combine them to solve a new task. This formulation allows the algorithm to acquire a set of specialized skill priors that are more reusable for downstream tasks; however, it also brings up additional challenges of how to effectively combine these unstructured sets of skill priors to form a new prior for new tasks. Specifically, it requires the agent not only to identify which skill prior(s) to use but also how to combine them (either sequentially or concurrently) to form a new prior. To achieve this goal, ASPiRe includes Adaptive Weight Module (AWM) that learns to infer an adaptive weight assignment between different skill priors and uses them to guide policy learning for downstream tasks via weighted Kullback-Leibler divergences. Our experiments demonstrate that ASPiRe can significantly accelerate the learning of new downstream tasks in the presence of multiple priors and show improvement on competitive baselines.Comment: 36th Conference on Neural Information Processing Systems (NeurIPS 2022

Of evolution, information, vitalism and entropy: reflections of the history of science and epistemology in the works of Balzac, Zola, Queneau, and Houellebecq

This dissertation proposes the application of rarely-used epistemological and scientific lenses to the works of four authors spanning two centuries: Honoré de Balzac, Émile Zola, Raymond Queneau, and Michel Houellebecq. Each of these novelists engaged closely with questions of science and epistemology, yet each approached that engagement from a different scientific perspective and epistemological moment. In Balzac’s La Peau de chagrin, limits of determinism and experimental method tend to demonstrate that there remains an inscrutable yet guided excess in the interactions between the protagonist Raphaël and his enchanted skin. This speaks to an embodiment of the esprit préscientifique, a framework that minimizes the utility of scientific practice in favor of the unresolved mystery of vitalism. With Zola comes a move away from undefinable mystery to a construction of the novel consistent with Claude Bernard’s deterministic experimental medicine. Yet Zola’s Roman expérimental project is only partially executed, in that the Newtonian framework that underlies Bernard’s method yields to contrary evidence in Zola’s text of entropy, error, and loss of information consistent with the field of thermodynamics. In Queneau’s texts, Zola’s interest in current science not only remains, but is updated to reflect the massive upheaval in scientific thought that took place in the last half of the nineteenth and early part of the twentieth centuries. If Queneau’s texts explicitly mention advances like relativity, however, they often do so in a humorously dismissive manner that values pre-entropic and even early geometric constructs like perpetual motion machines and squared circles. Queneau’s apparent return to the pre-scientific ultimately yields to Houellebecq’s textual abyss. For Houellebecq, science is not only to be embraced in its entropic and relativistic constructs; it is these very constructs - and the style typically used to present them – that serve as a reminder of the abjection, decay, and hopelessness of human existence. Gone is the mystery of life in its totality. In its place remain humans acting as a series of particles mechanically obeying deterministic laws. The parenthesis that opened with Balzac’s positive coding of pre-scientific thought closes with Houellebecq’s negative coding of modern scientific theory