Designing agent that can autonomously discover and learn a diversity of
structures and skills in unknown changing environments is key for lifelong
machine learning. A central challenge is how to learn incrementally
representations in order to progressively build a map of the discovered
structures and re-use it to further explore. To address this challenge, we
identify and target several key functionalities. First, we aim to build lasting
representations and avoid catastrophic forgetting throughout the exploration
process. Secondly we aim to learn a diversity of representations allowing to
discover a "diversity of diversity" of structures (and associated skills) in
complex high-dimensional environments. Thirdly, we target representations that
can structure the agent discoveries in a coarse-to-fine manner. Finally, we
target the reuse of such representations to drive exploration toward an
"interesting" type of diversity, for instance leveraging human guidance.
Current approaches in state representation learning rely generally on
monolithic architectures which do not enable all these functionalities.
Therefore, we present a novel technique to progressively construct a Hierarchy
of Observation Latent Models for Exploration Stratification, called HOLMES.
This technique couples the use of a dynamic modular model architecture for
representation learning with intrinsically-motivated goal exploration processes
(IMGEPs). The paper shows results in the domain of automated discovery of
diverse self-organized patterns, considering as testbed the experimental
framework from Reinke et al. (2019)
