DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics

Robots are still limited to controlled conditions, that the robot designer knows with enough details to endow the robot with the appropriate models or behaviors. Learning algorithms add some flexibility with the ability to discover the appropriate behavior given either some demonstrations or a reward to guide its exploration with a reinforcement learning algorithm. Reinforcement learning algorithms rely on the definition of state and action spaces that define reachable behaviors. Their adaptation capability critically depends on the representations of these spaces: small and discrete spaces result in fast learning while large and continuous spaces are challenging and either require a long training period or prevent the robot from converging to an appropriate behavior. Beside the operational cycle of policy execution and the learning cycle, which works at a slower time scale to acquire new policies, we introduce the redescription cycle, a third cycle working at an even slower time scale to generate or adapt the required representations to the robot, its environment and the task. We introduce the challenges raised by this cycle and we present DREAM (Deferred Restructuring of Experience in Autonomous Machines), a developmental cognitive architecture to bootstrap this redescription process stage by stage, build new state representations with appropriate motivations, and transfer the acquired knowledge across domains or tasks or even across robots. We describe results obtained so far with this approach and end up with a discussion of the questions it raises in Neuroscience

Mental Imagery in Humanoid Robots

Mental imagery presents humans with the opportunity to predict prospective happenings based on own intended actions, to reminisce occurrences from the past and reproduce the perceptual experience. This cognitive capability is mandatory for human survival in this folding and changing world. By means of internal representation, mental imagery offers other cognitive functions (e.g., decision making, planning) the possibility to assess information on objects or events that are not being perceived. Furthermore, there is evidence to suggest that humans are able to employ this ability in the early stages of infancy. Although materialisation of humanoid robot employment in the future appears to be promising, comprehensive research on mental imagery in these robots is lacking. Working within a human environment required more than a set of pre-programmed actions. This thesis aims to investigate the use of mental imagery in humanoid robots, which could be used to serve the demands of their cognitive skills as in humans. Based on empirical data and neuro-imaging studies on mental imagery, the thesis proposes a novel neurorobotic framework which proposes to facilitate humanoid robots to exploit mental imagery. Through conduction of a series of experiments on mental rotation and tool use, the results from this study confirm this potential. Chapters 5 and 6 detail experiments on mental rotation that investigate a bio-constrained neural network framework accounting for mental rotation processes. They are based on neural mechanisms involving not only visual imagery, but also affordance encoding, motor simulation, and the anticipation of the visual consequences of actions. The proposed model is in agreement with the theoretical and empirical research on mental rotation. The models were validated with both a simulated and physical humanoid robot (iCub), engaged in solving a typical mental rotation task. The results show that the model is able to solve a typical mental rotation task and in agreement with data from psychology experiments, they also show response times linearly dependent on the angular disparity between the objects. Furthermore, the experiments in chapter 6 propose a novel neurorobotic model that has a macro-architecture constrained by knowledge on brain, which encompasses a rather general mental rotation mechanism and incorporates a biologically plausible decision making mechanism. The new model is tested within the humanoid robot iCub in tasks requiring to mentally rotate 2D geometrical images appearing on a computer screen. The results show that the robot has an enhanced capacity to generalize mental rotation of new objects and shows the possible effects of overt movements of the wrist on mental rotation. These results indicate that the model represents a further step in the identification of the embodied neural mechanisms that might underlie mental rotation in humans and might also give hints to enhance robots' planning capabilities. In Chapter 7, the primary purpose for conducting the experiment on tool use development through computational modelling refers to the demonstration that developmental characteristics of tool use identified in human infants can be attributed to intrinsic motivations. Through the processes of sensorimotor learning and rewarding mechanisms, intrinsic motivations play a key role as a driving force that drives infants to exhibit exploratory behaviours, i.e., play. Sensorimotor learning permits an emergence of other cognitive functions, i.e., affordances, mental imagery and problem-solving. Two hypotheses on tool use development are also conducted thoroughly. Secondly, the experiment tests two candidate mechanisms that might underlie an ability to use a tool in infants: overt movements and mental imagery. By means of reinforcement learning and sensorimotor learning, knowledge of how to use a tool might emerge through random movements or trial-and-error which might reveal a solution (sequence of actions) of solving a given tool use task accidentally. On the other hand, mental imagery was used to replace the outcome of overt movements in the processes of self-determined rewards. Instead of determining a reward from physical interactions, mental imagery allows the robots to evaluate a consequence of actions, in mind, before performing movements to solve a given tool use task. Therefore, collectively, the case of mental imagery in humanoid robots was systematically addressed by means of a number of neurorobotic models and, furthermore, two categories of spatial problem solving tasks: mental rotation and tool use. Mental rotation evidently involves the employment of mental imagery and this thesis confirms the potential for its exploitation by humanoid robots. Additionally, the studies on tool use demonstrate that the key components assumed and included in the experiments on mental rotation, namely affordances and mental imagery, can be acquired by robots through the processes of sensorimotor learning.Ministry of Science and Technology, the Thai Governmen

Artificial cognitive architecture with self-learning and self-optimization capabilities. Case studies in micromachining processes

Tesis doctoral inédita leída en la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de lectura : 22-09-201

The psychological and human brain effects of music in combination with psychedelic drugs

This research investigated how psychedelics and music work together in the brain and modulate subjective experience. Chapter 1 highlighted the prominent role of music in psychedelic therapy in the 1950s and 1960s, and how music continues to be used in modern psychotherapeutic trials with psychedelics. Although ‘psychedelic therapy’ shows promising findings for mental health care, little is known empirically about the therapeutic functions of music. The primary objective of this thesis was to address this knowledge gap, via studying the effects of psychedelics and music on human brain function in healthy volunteers, and via studying the subjective experience of music, both in healthy volunteers and in patients undergoing psychedelic therapy. Study 1 (Chapter 3) demonstrated intensified music-evoked emotions under the classic psychedelic LSD, including emotions of ‘wonder’ and ‘transcendence’. In subsequent work (study 2, Chapter 4), increased activation in the inferior frontal gyrus and the precuneus to the timbre features in the music, was associated with increased music-evoked emotions of wonder. Study 3 (Chapter 5) demonstrated that LSD and music interact to enhance information flow from the parahippocampus to the visual cortex, and that this effect correlated with increased complex mental imagery and autobiographical memories. Study 4 (Chapter 6), showed that music has a substantial influence on the therapeutic experience with psilocybin in patients with depression, and the quality of the music-experience predicted peak experiences and insightfulness during sessions, and reductions in depression after sessions. These findings support the hypothesis that the music-experience is intensified under psychedelics, and the widely-held view that this effect may be therapeutically significant. Possible brain mechanisms and therapeutic mechanisms are discussed in Chapter 7, but further research is warranted to better understand these mechanisms, and to learn how music can be best used in psychedelic therapy.Open Acces

Retrieval-, Distributed-, and Interleaved Practice in the Classroom:A Systematic Review

Three of the most effective learning strategies identified are retrieval practice, distributed practice, and interleaved practice, also referred to as desirable difficulties. However, it is yet unknown to what extent these three practices foster learning in primary and secondary education classrooms (as opposed to the laboratory and/or tertiary education classrooms, where most research is conducted) and whether these strategies affect different students differently. To address these gaps, we conducted a systematic review. Initial and detailed screening of 869 documents found in a threefold search resulted in a pool of 29 journal articles published from 2006 through June 2020. Seventy-five effect sizes nested in 47 experiments nested in 29 documents were included in the review. Retrieval- and interleaved practice appeared to benefit students’ learning outcomes quite consistently; distributed practice less so. Furthermore, only cognitive Student*Task characteristics (i.e., features of the student’s cognition regarding the task, such as initial success) appeared to be significant moderators. We conclude that future research further conceptualising and operationalising initial effort is required, as is a differentiated approach to implementing desirable difficulties