Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

Deep learning as closure for irreversible processes: A data-driven generalized Langevin equation

The ultimate goal of physics is finding a unique equation capable of describing the evolution of any observable quantity in a self-consistent way. Within the field of statistical physics, such an equation is known as the generalized Langevin equation (GLE). Nevertheless, the formal and exact GLE is not particularly useful, since it depends on the complete history of the observable at hand, and on hidden degrees of freedom typically inaccessible from a theoretical point of view. In this work, we propose the use of deep neural networks as a new avenue for learning the intricacies of the unknowns mentioned above. By using machine learning to eliminate the unknowns from GLEs, our methodology outperforms previous approaches (in terms of efficiency and robustness) where general fitting functions were postulated. Finally, our work is tested against several prototypical examples, from a colloidal systems and particle chains immersed in a thermal bath, to climatology and financial models. In all cases, our methodology exhibits an excellent agreement with the actual dynamics of the observables under consideration