Procedural-Reasoning Architecture for Applied Behavior Analysis-based Instructions

Autism Spectrum Disorder (ASD) is a complex developmental disability affecting as many as 1 in every 88 children. While there is no known cure for ASD, there are known behavioral and developmental interventions, based on demonstrated efficacy, that have become the predominant treatments for improving social, adaptive, and behavioral functions in children. Applied Behavioral Analysis (ABA)-based early childhood interventions are evidence based, efficacious therapies for autism that are widely recognized as effective approaches to remediation of the symptoms of ASD. They are, however, labor intensive and consequently often inaccessible at the recommended levels. Recent advancements in socially assistive robotics and applications of virtual intelligent agents have shown that children with ASD accept intelligent agents as effective and often preferred substitutes for human therapists. This research is nascent and highly experimental with no unifying, interdisciplinary, and integral approach to development of intelligent agents based therapies, especially not in the area of behavioral interventions. Motivated by the absence of the unifying framework, we developed a conceptual procedural-reasoning agent architecture (PRA-ABA) that, we propose, could serve as a foundation for ABA-based assistive technologies involving virtual, mixed or embodied agents, including robots. This architecture and related research presented in this disser- tation encompass two main areas: (a) knowledge representation and computational model of the behavioral aspects of ABA as applicable to autism intervention practices, and (b) abstract architecture for multi-modal, agent-mediated implementation of these practices

When robots weep : a computational approach to affective learning

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.Includes bibliographical references (p. 245-262).This thesis presents a unified computational framework for the study of emotion that integrates several concepts and mechanisms which have been traditionally deemed to be integral components of intelligent behavior. We introduce the notion of affect programs as the primary theoretical constructs for investigating the function and the mechanisms of emotion, and instantiate these in a variety of embodied agents, including physical and simulated robots. Each of these affect programs establishes a functionally distinct mode of operation for the robots, that is activated when specific environmental contingencies are appraised. These modes involve the coordinated adjustment and entrainment of several different systems-including those governing perception, attention, motivation regulation, action selection, learning, and motor control-as part of the implementation of specialized solutions that take advantage of the regularities found in highly recurrent and prototypical environmental contingencies. We demonstrate this framework through multiple experimental scenarios that explore important features of the affect program abstraction and its function, including the demonstration of affective behavior, evaluative conditioning, incentive salience, and affective learning.by Juan David Velásquez.Ph.D

Why do I lose time playing? Game features reported by players as being more satisfying

This dissertation had the objetive to relate elements from cognitive psychology with game design. For this purpose, it was discussed some features that constitutes a game, game design in order to create a good playing experience for players, presented persuasion strategies and thus how rewards given to players may keep them interested in the game and influence their behavior. To conclude, a list has been created with elements that were discussed and identified as being relevant in creating a good game experience. Besides, data collected from an online form done with players, with a total of 182 answers, was analysed for the sake of testing some hypotheses and concluding that challenge difficulty, player choices and playing with friends are important features.Este trabalho teve como objetivo tentar encontrar elementos da psicologia cognitiva e os associar com design de jogos. Para isso foi demonstrado o que são jogos, seu game design para criação de uma boa experiência de jogo aos jogadores, demonstração de elementos de persuasão e então como as recompensas fornecidas aos jogadores podem mantê-los interessados no jogo e influenciar seu comportamento. Por fim, foi criada uma tabela com elementos que foram discutidos e identificados como importantes para possibilitar uma boa experiência de jogo. Além disso, dados coletados de um questionário online, realizado com jogadores e obtidas 182 respostas, foram analisados afim de testar algumas hipóteses e concluímos que dificuldade do desafio, escolhas e jogar com amigos são fatores importantes

A neurocomputational model of reward-based motor learning

2015 - 2016The following thesis deals with computational models of nervous system employed in motor reinforcement learning. The novel contribution of this work is that it includes a methodology of experiments for evaluating learning rates for human which we compared with the results coming from a computational model we derived from a deep analysis of literature. Rewards or punishments are particular stimuli able to drive for good or for worse the performance of the action to learn. This happens because they can strengthen or weaken the connections among a combination of sensory input stimuli and a combination of motor activation outputs, attributing them some kind of value. A reward/ punisher can originate from innate needs(hunger, thirst, etc), coming from hardwired structures in the brain (hypothalamus), yet it could also come from an initially neutral cue (from cortex or sensory inputs) that acquires the ability to produce value after learning(for example money value, approval).We called the formers primary value, while the latter learned values. The efficacy of a stimulus as a reinforcer/punisher depends on the specific context the action take place (Motivating operation). It is claimed that values drive learning through dopamine firing and that learned values acquire this ability after repetitive pairings with innate primary values, in a Pavlovian classic conditioning paradigm. Under some hypothesis made we propose a computational model made of: A block taking place in Cortex mapping sensory combinations(posterior cortex) and possible actions(motor cortex) . The weights of the net which corresponds to the probability of a movement , given a sensory combination in input. Rewards/punishments alter these probabilities trhought a selection rule we implemented in Basal Ganglia for action selection; A block for the production of values (critic): we evaluated two different scenarios In the first we considered the block only fo innate rewards, made of VTA(Ventral Tegmental Area) and Lateral Hypothalamus(innate rewards) and Lateral Habenula(innate punishments) In the second scenario we added the structures for learning of rewards, Amygdala, which learns to produce a dopamine activation on the onset of an initially neutral stimulus and a Ventral Striatum, which learns to predict the occurrence of the innate reward, cancelling its dopamine activation. Innate reward is fundamental for learning value system: even in a well trained system, if the learned stimulus reward is no more able to expect innate stimulus reward( because is occurring late or not at all ), and if this occurs frequently it could lose its reinforcing/weakening abilities. This phenomenon is called acquisition extinction and is strictly dependent on the context (motivating operation). Validation of the model started from Emergent , which provides a biologically accurate model of neuron networks and learning mechanisms and was ported to Matlab , more versatile, in order to prove the ability of system to learn for a specific task . In this simple task the system has to learn among two possible actions , given a group of stimuli of varying cardinality: 2, 4 and 8. We evaluated the task in the 2 scenarios described, one with innate rewards and one with learned rewards. Finally several experiments were performed to evaluate human learning rate: volunteers had to learn to press the right keyboard buttons when visual stimuli appeared on monitor, in order to get an auditory and visual reward. The experiments were carefully designed in a way such to make comparable the result of simple artificial neural network with those of human performers. The strategy was to select a reduced set of responses and a set of visual stimuli as simple as possibles (edges), thus bypassing the problem of a hierarchical complex information representation, by collapsing them in one layer . The result were then fitted with an exponential and a hyperbolical function. Both fitting showed that human learning rate is slow compared to artificial network and decreases with the number of stimuli it has to learn. [edited by author]XV n.s

Exploring the psychobiology of emotions and motivations through computational models

This thesis investigates emotions and motivations on the basis of an operational approach. This approach has both computational and psychobiological roots. Three main directions of research are followed: (1) investigation on the neural substrates of emotional systems though the exploration of the literature about comparative functional anatomy and physiology; (2) definition the relationship between emotion, cognition and behaviour through the exploration of the psychobiological literature about animal models; (3) building of computational models constrained by the sources of information 1 and 2; (4) testing the behaviour of such models within simulated robots acting in simulated environments. The main focus will be on the interaction between the emotional and motivational systems and high level cognitive processes behind adaptive behaviour. The whole study will be informed by the current psychobiological knowledge about the functioning of the neural systems pivoting on amygdala, given that this is considered to be one of the major nodes of interaction between the processing of internal values and the processing about the past, current and future world outside the organism in superior vertebrates

Levels of naturalism in social neuroscience research

In order to understand ecologically meaningful social behaviors and their neural substrates in humans and other animals, researchers have been using a variety of social stimuli in the laboratory with a goal of extracting specific processes in real-life scenarios. However, certain stimuli may not be sufficiently effective at evoking typical social behaviors and neural responses. Here, we review empirical research employing different types of social stimuli by classifying them into five levels of naturalism. We describe the advantages and limitations while providing selected example studies for each level. We emphasize the important trade-off between experimental control and ecological validity across the five levels of naturalism. Taking advantage of newly emerging tools, such as real-time videos, virtual avatars, and wireless neural sampling techniques, researchers are now more than ever able to adopt social stimuli at a higher level of naturalism to better capture the dynamics and contingency of real-life social interaction