Generating Adaptive Behaviour within a Memory-Prediction Framework

The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex

Continual few-shot learning with Hippocampal-inspired replay

Continual learning and few-shot learning are important frontiers in the quest to improve Machine Learning. There is a growing body of work in each frontier, but very little combining the two. Recently however, Antoniou et al. arXiv:2004.11967 introduced a Continual Few-shot Learning framework, CFSL, that combines both. In this study, we extended CFSL to make it more comparable to standard continual learning experiments, where usually a much larger number of classes are presented. We also introduced an `instance test' to classify very similar specific instances - a capability of animal cognition that is usually neglected in ML. We selected representative baseline models from the original CFSL work and compared to a model with Hippocampal-inspired replay, as the Hippocampus is considered to be vital to this type of learning in animals. As expected, learning more classes is more difficult than the original CFSL experiments, and interestingly, the way in which they are presented makes a difference to performance. Accuracy in the instance test is comparable to the classification tasks. The use of replay for consolidation improves performance substantially for both types of tasks, particularly the instance test.Comment: 11 pages, 5 figure

Deep learning in a bilateral brain with hemispheric specialization

The brains of all bilaterally symmetric animals on Earth are are divided into left and right hemispheres. The anatomy and functionality of the hemispheres have a large degree of overlap, but they specialize to possess different attributes. The left hemisphere is believed to specialize in specificity and routine, the right in generalities and novelty. In this study, we propose an artificial neural network that imitates that bilateral architecture using two convolutional neural networks with different training objectives and test it on an image classification task. The bilateral architecture outperforms architectures of similar representational capacity that don't exploit differential specialization. It demonstrates the efficacy of bilateralism and constitutes a new principle that could be incorporated into other computational neuroscientific models and used as an inductive bias when designing new ML systems. An analysis of the model can help us to understand the human brain.Comment: 11 pages, 10 figure

AHA! an 'Artificial Hippocampal Algorithm' for Episodic Machine Learning

The majority of ML research concerns slow, statistical learning of i.i.d. samples from large, labelled datasets. Animals do not learn this way. An enviable characteristic of animal learning is `episodic' learning - the ability to memorise a specific experience as a composition of existing concepts, after just one experience, without provided labels. The new knowledge can then be used to distinguish between similar experiences, to generalise between classes, and to selectively consolidate to long-term memory. The Hippocampus is known to be vital to these abilities. AHA is a biologically-plausible computational model of the Hippocampus. Unlike most machine learning models, AHA is trained without external labels and uses only local credit assignment. We demonstrate AHA in a superset of the Omniglot one-shot classification benchmark. The extended benchmark covers a wider range of known hippocampal functions by testing pattern separation, completion, and recall of original input. These functions are all performed within a single configuration of the computational model. Despite these constraints, image classification results are comparable to conventional deep convolutional ANNs

Nave physics for effective odour localisation

This paper describes current progress of a project that uses naïve physics to enable a robot to perform efficient odour localisation. Odour localisation is the problem of finding the source of an odour or other volatile chemical. Performing this effectively could lead to many humanitarian and other valuable applications. Current techniques utilise reactive control schemes requiring the robot to follow the plume along its entire length, which is slow and may be especially difficult in a cluttered environment. This research is concerned with creating a more ‘intelligent ’ system to overcome these limitations. A map of the robot’s environment was used, together with a naïve physics model of airflow to predict the pattern of air movement. The robot used the airflow pattern to reason about the probable location of the odour source. A prototype system was successful in a simplified cluttered environment, locating the source comparatively quickly. This demonstrates that naïve physics can be used for effective odour localisation, and has the potential to allow a robots operating in unstructured environments to reason about their surroundings. This paper presents details of the naïve physical model of airflow, reasoning system, experimental work, and results of practical odour source localisation experiments. 1

Robot odour localisation in enclosed and cluttered environments using naïve physics

Odour localisation is the problem of finding the source of an odour or other volatile chemical. It promises many valuable practical and humanitarian applications. Most localisation methods require a robot to reactively track an odour plume along its entire length. This approach is time consuming and may be not be possible in a cluttered indoor environment, where airflow tends to form sectors of circulating airflow. Such environments may be encountered in crawl-ways under floors, roof cavities, mines, caves, tree-canopies, air-ducts, sewers or tunnel systems. Operation in these places is important for such applications as search and rescue and locating the sources of toxic chemicals in an industrial setting. This thesis addresses odour localisation in this class of environments. The solution consists of a sense-map-plan-act style control scheme (and low level behaviour based controller) with two main stages. Firstly, the airflow in the environment is modelled using naive physics rules which are encapsulated into an algorithm named a Naive Reasoning Machine. It was used in preference to conventional methods as it is fast, does not require boundary conditions, and most importantly, provides approximate solutions to the degree of accuracy required for the task, with analogical data structures that are readily useful to a reasoning algorithm. Secondly, a reasoning algorithm navigates the robot to specific target locations that are determined with a physical map, the airflow map, and knowledge of odour dispersal. Sensor measurements at the target positions provide information regarding the likelihood that odour was emitted from potential odour source locations. The target positions and their traversal are determined so that all the potential odour source sites are accounted for. The core method provides values corresponding to the confidence that the odour source is located in a given region. A second search stage exploiting vision is then used to locate the specific location of the odour source within the predicted region. This comprises the second part of a bi-modal, two-stage search, with each stage exploiting complementary sensing modalities. Single hypothesis airflow modelling faces limitations due to the fact that large differences between airflow topologies are predicted for only small variations in a physical map. This is due to uncertainties in the map and approximations in the modelling process. Furthermore, there are uncertainties regarding the flow direction through inlet/outlet ducts. A method is presented for dealing with these uncertainties, by generating multiple airflow hypotheses. As the robot performs odour localisation, airflow in the environment is measured and used to adjust the confidences of the hypotheses using Bayesian inference. The best hypothesis is then selected, which allows the completion of the localisation task. This method improves the robustness of odour localisation in the presence of uncertainties, making it possible where the single hypothesis method would fail. It also demonstrates the potential for integrating naive physics into a statistical framework. Extensive experimental results are presented to support the methods described above

Robot odour localisation in enclosed and cluttered environments using naïve physics

Odour localisation is the problem of finding the source of an odour or other volatile chemical. It promises many valuable practical and humanitarian applications. Most localisation methods require a robot to reactively track an odour plume along its entire length. This approach is time consuming and may be not be possible in a cluttered indoor environment, where airflow tends to form sectors of circulating airflow. Such environments may be encountered in crawl-ways under floors, roof cavities, mines, caves, tree-canopies, air-ducts, sewers or tunnel systems. Operation in these places is important for such applications as search and rescue and locating the sources of toxic chemicals in an industrial setting. This thesis addresses odour localisation in this class of environments. The solution consists of a sense-map-plan-act style control scheme (and low level behaviour based controller) with two main stages. Firstly, the airflow in the environment is modelled using naive physics rules which are encapsulated into an algorithm named a Naive Reasoning Machine. It was used in preference to conventional methods as it is fast, does not require boundary conditions, and most importantly, provides approximate solutions to the degree of accuracy required for the task, with analogical data structures that are readily useful to a reasoning algorithm. Secondly, a reasoning algorithm navigates the robot to specific target locations that are determined with a physical map, the airflow map, and knowledge of odour dispersal. Sensor measurements at the target positions provide information regarding the likelihood that odour was emitted from potential odour source locations. The target positions and their traversal are determined so that all the potential odour source sites are accounted for. The core method provides values corresponding to the confidence that the odour source is located in a given region. A second search stage exploiting vision is then used to locate the specific location of the odour source within the predicted region. This comprises the second part of a bi-modal, two-stage search, with each stage exploiting complementary sensing modalities. Single hypothesis airflow modelling faces limitations due to the fact that large differences between airflow topologies are predicted for only small variations in a physical map. This is due to uncertainties in the map and approximations in the modelling process. Furthermore, there are uncertainties regarding the flow direction through inlet/outlet ducts. A method is presented for dealing with these uncertainties, by generating multiple airflow hypotheses. As the robot performs odour localisation, airflow in the environment is measured and used to adjust the confidences of the hypotheses using Bayesian inference. The best hypothesis is then selected, which allows the completion of the localisation task. This method improves the robustness of odour localisation in the presence of uncertainties, making it possible where the single hypothesis method would fail. It also demonstrates the potential for integrating naive physics into a statistical framework. Extensive experimental results are presented to support the methods described above

Formulation of an adaptive-MPF hierarchy.

<p>Messages between units (U) in different layers are relayed via “reward correlator” components (RC). FF messages (blue arrows) represent classifications of the current state of the agent in the world; these are correlated with objective internal measures of agent state (reward). The same reward value is provided to every RC; the hierarchy is tasked with modelling the separate external causes of changes in reward. FB messages are “predictions” of future agent-world state (red arrows). Biased messages are produced by RC components, making the hierarchy more likely to “predict” states in which it performs actions correlated with high reward. Sensor data is concatenated with motor output to form the interface to the MPF hierarchy. The FB output of an MPF unit is of the same form as its FF input. Different data may be presented to each unit at the bottom of the hierarchy. Sensor inputs and motor outputs may be mixed within one unit or interfaced to different units.</p