Projective simulation for artificial intelligence

We propose a model of a learning agent whose interaction with the environment is governed by a simulation-based projection, which allows the agent to project itself into future situations before it takes real action. Projective simulation is based on a random walk through a network of clips, which are elementary patches of episodic memory. The network of clips changes dynamically, both due to new perceptual input and due to certain compositional principles of the simulation process. During simulation, the clips are screened for specific features which trigger factual action of the agent. The scheme is different from other, computational, notions of simulation, and it provides a new element in an embodied cognitive science approach to intelligent action and learning. Our model provides a natural route for generalization to quantum-mechanical operation and connects the fields of reinforcement learning and quantum computation.Comment: 22 pages, 18 figures. Close to published version, with footnotes retaine

Quantum speedup for active learning agents

Can quantum mechanics help us in building intelligent robots and agents? One of the defining characteristics of intelligent behavior is the capacity to learn from experience. However, a major bottleneck for agents to learn in any real-life situation is the size and complexity of the corresponding task environment. Owing to, e.g., a large space of possible strategies, learning is typically slow. Even for a moderate task environment, it may simply take too long to rationally respond to a given situation. If the environment is impatient, allowing only a certain time for a response, an agent may then be unable to cope with the situation and to learn at all. Here we show that quantum physics can help and provide a significant speed-up for active learning as a genuine problem of artificial intelligence. We introduce a large class of quantum learning agents for which we show a quadratic boost in their active learning efficiency over their classical analogues. This result will be particularly relevant for applications involving complex task environments.Comment: Minor updates, 14 pages, 3 figure

A model of the emergence and evolution of integrated worldviews

It \ud is proposed that the ability of humans to flourish in diverse \ud environments and evolve complex cultures reflects the following two \ud underlying cognitive transitions. The transition from the \ud coarse-grained associative memory of Homo habilis to the \ud fine-grained memory of Homo erectus enabled limited \ud representational redescription of perceptually similar episodes, \ud abstraction, and analytic thought, the last of which is modeled as \ud the formation of states and of lattices of properties and contexts \ud for concepts. The transition to the modern mind of Homo \ud sapiens is proposed to have resulted from onset of the capacity to \ud spontaneously and temporarily shift to an associative mode of thought \ud conducive to interaction amongst seemingly disparate concepts, \ud modeled as the forging of conjunctions resulting in states of \ud entanglement. The fruits of associative thought became ingredients \ud for analytic thought, and vice versa. The ratio of \ud associative pathways to concepts surpassed a percolation threshold \ud resulting in the emergence of a self-modifying, integrated internal \ud model of the world, or worldview

A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems

We introduce a kind of partial observability to the projective simulation (PS) learning method. It is done by adding a belief projection operator and an observability parameter to the original framework of the efficiency of the PS model. I provide theoretical formulations, network representations, and situated scenarios derived from the invasion toy problem as a starting point for some multi-agent PS models.Comment: 28 pages, 21 figure

Episodic Source Memory over Distribution by Quantum-Like Dynamics – A Model Exploration

In source memory studies, a decision-maker is concerned with identifying the context in which a given episodic experience occurred. A common paradigm for studying source memory is the ‘three-list’ experimental paradigm, where a subject studies three lists of words and is later asked whether a given word appeared on one or more of the studied lists. Surprisingly, the sum total of the acceptance probabilities generated by asking for the source of a word separately for each list (‘list 1?’, ‘list 2?’, ‘list 3?’) exceeds the acceptance probability generated by asking whether that word occurred on the union of the lists (‘list 1 or 2 or 3?’). The episodic memory for a given word therefore appears over distributed on the disjoint contexts of the lists. A quantum episodic memory model [QEM] was proposed by Brainerd, Wang and Reyna [8] to explain this type of result. In this paper, we apply a Hamiltonian dynamical extension of QEM for over distribution of source memory. The Hamiltonian operators are simultaneously driven by parameters for re-allocation of gist-based and verbatim-based acceptance support as subjects are exposed to the cue word in the first temporal stage, and are attenuated for description-dependence by the querying probe in the second temporal stage. Overall, the model predicts well the choice proportions in both separate list and union list queries and the over distribution effect, suggesting that a Hamiltonian dynamics for QEM can provide a good account of the acceptance processes involved in episodic memory tasks