From motor babbling to hierarchical learning by imitation: a robot developmental pathway

How does an individual use the knowledge acquired through self exploration as a manipulable model through which to understand others and benefit from their knowledge? How can developmental and social learning be combined for their mutual benefit? In this paper we review a hierarchical architecture (HAMMER) which allows a principled way for combining knowledge through exploration and knowledge from others, through the creation and use of multiple inverse and forward models. We describe how Bayesian Belief Networks can be used to learn the association between a robot’s motor commands and sensory consequences (forward models), and how the inverse association can be used for imitation. Inverse models created through self exploration, as well as those from observing others can coexist and compete in a principled unified framework, that utilises the simulation theory of mind approach to mentally rehearse and understand the actions of others

Developmental learning of internal models for robotics

Abstract: Robots that operate in human environments can learn motor skills asocially, from selfexploration, or socially, from imitating their peers. A robot capable of doing both can be more ~daptiveand autonomous. Learning by imitation, however, requires the ability to understand the actions ofothers in terms ofyour own motor system: this information can come from a robot's own exploration. This thesis investigates the minimal requirements for a robotic system than learns from both self-exploration and imitation of others. .Through self.exploration and computer vision techniques, a robot can develop forward 'models: internal mo'dels of its own motor system that enable it to predict the consequences of its actions. Multiple forward models are learnt that give the robot a distributed, causal representation of its motor system. It is demon~trated how a controlled increase in the complexity of these forward models speeds up the robot's learning. The robot can determine the uncertainty of its forward models, enabling it to explore so as to improve the accuracy of its???????predictions. Paying attention fO the forward models according to how their uncertainty is changing leads to a development in the robot's exploration: its interventions focus on increasingly difficult situations, adapting to the complexity of its motor system. A robot can invert forward models, creating inverse models, in order to estimate the actions that will achieve a desired goal. Switching to socialleaming. the robot uses these inverse model~ to imitate both a demonstrator's gestures and the underlying goals of their movement.Imperial Users onl

RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learning

Whole-slide images (WSI) in computational pathology have high resolution with gigapixel size, but are generally with sparse regions of interest, which leads to weak diagnostic relevance and data inefficiency for each area in the slide. Most of the existing methods rely on a multiple instance learning framework that requires densely sampling local patches at high magnification. The limitation is evident in the application stage as the heavy computation for extracting patch-level features is inevitable. In this paper, we develop RLogist, a benchmarking deep reinforcement learning (DRL) method for fast observation strategy on WSIs. Imitating the diagnostic logic of human pathologists, our RL agent learns how to find regions of observation value and obtain representative features across multiple resolution levels, without having to analyze each part of the WSI at the high magnification. We benchmark our method on two whole-slide level classification tasks, including detection of metastases in WSIs of lymph node sections, and subtyping of lung cancer. Experimental results demonstrate that RLogist achieves competitive classification performance compared to typical multiple instance learning algorithms, while having a significantly short observation path. In addition, the observation path given by RLogist provides good decision-making interpretability, and its ability of reading path navigation can potentially be used by pathologists for educational/assistive purposes. Our code is available at: \url{https://github.com/tencent-ailab/RLogist}.Comment: accepted by AAAI 202

A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VISis needed. In this paper, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: "what visualization processes can be assisted by ML?" and "how ML techniques can be used to solve visualization problems?" This survey reveals seven main processes where the employment of ML techniques can benefit visualizations:Data Processing4VIS, Data-VIS Mapping, InsightCommunication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling. The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations.Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this paper can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.ioComment: 19 pages, 12 figures, 4 table