From Robot-Assisted Intervention to New Generation of Autism Screening: an Engineering Implementation Beyond the Technical Approach

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects people from birth, whose symptoms are found in the early developmental period. The ASD diagnosis is usually performed through several sessions of behavioral observation, exhaustive screening, and manual coding behavior. The early detection of ASD signs in naturalistic behavioral observation may be improved through Social Assistive Robotics (SAR) and technological-based tools for an automated behavior assessment. Robot-assisted tools using Child-Robot Interaction (CRI) theories have been of interest in intervention for children with Autism Spectrum Disorder (CwASD), elucidating faster and more significant gains from the diagnosis and therapeutic intervention when compared with classical methods. Additionally, using computer vision to analyze the childs behaviors and automated video coding to summarize the responses would help clinicians to reduce the delay of ASD diagnosis. Despite the increment of researches related to SAR, achieving a plausible Robot-Assisted Diagnosis (RAD) for CwASD remains a considerable challenge to the clinical and robotics community. The work of specialists regarding ASD diagnosis is hard and labor-intensive, as the conditions manifestations are inherently heterogeneous and make the process more difficult. In addition, the aforementioned complexity may be the main reason for the slow progress in the development of SAR with diagnostic purpose. Also, there still is a lack of guidelines on how to select the appropriate robotic features, such as appearance, morphology, autonomy level, and how to design and implement the robots role in the CRI. Thus, this Ph.D. Thesis provides a comprehensive Robot-Assisted intervention for CwASD to assess autism risk factors for an autism diagnostic purpose. More specifically, two studies were conducted to analyze and validate the system performance. Through statistical data analysis, different behavior pattern of the CwASD group were identified, which suggest that these patterns can be used to detect autism risk factors through robot-based interventions. To increase the scope of this research, a theoretical conceptualization of the pervasive version of the multimodal environment was described as well as a participatory design methodology was designed and implemented on the Colombian autism community, providing, a set of guidelines regarding the design of a social robot-device suitable to be applied for robot-assisted intervention for CwASD

Toward a second-person neuroscience

LS & BT : equal contributions (shared first-authorship)Peer reviewedPreprin

Emotional recognition in computing

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University 8/4/2010.Emotions are fundamental to human lives and decision-making. Understanding and expression of emotional feeling between people forms an intricate web. This complex interactional phenomena, is a hot topic for research, as new techniques such as brain imaging give us insights about how emotions are tied to human functions. Communication of emotions is mixed with communication of other types of information (such as factual details) and emotions can be consciously or unconsciously displayed. Affective computer systems, using sensors for emotion recognition and able to make emotive responses are under development. The increased potential for emotional interaction with products and services, in many domains, is generating much interest. Emotionally enhanced systems have potential to improve human computer interaction and so to improve how systems are used and what they can deliver. They may also have adverse implications such as creating systems capable of emotional manipulation of users. Affective systems are in their infancy and lack human complexity and capability. This makes it difficult to assess whether human interaction with such systems will actually prove beneficial or desirable to users. By using experimental design, a Wizard of Oz methodology and a game that appeared to respond to the user’s emotional signals with human-like capability, I tested user experience and reactions to a system that appeared affective. To assess users’ behaviour, I developed a novel affective behaviour coding system called ‘affectemes’. I found significant gains in user satisfaction and performance when using an affective system. Those believing the system responded to emotional signals blinked more frequently. If the machine failed to respond to their emotional signals, they increased their efforts to convey emotion, which might be an attempt to ‘repair’ the interaction. This work highlights how very complex and difficult it is to design and evaluate affective systems. I identify many issues for future work, including the unconscious nature of emotions and how they are recognised and displayed with affective systems; issues about the power of emotionally interactive systems and their evaluation; and critical ethical issues. These are important considerations for future design of systems that use emotion recognition in computing.EPSRC project grant (R81374/01

UNOBTRUSIVE Technique Based On Infrared Thermal Imaging For Emotion Recognition In Children- With-asd- Robot Interaction

Emoções são relevantes para as relações sociais, e indivíduos com Transtorno do Espectro Autista (TEA) possuem compreensão e expressão de emoções prejudicadas. Esta tese consiste em estudos sobre a análise de emoções em crianças com desenvolvimento típico e crianças com TEA (idade entre 7 e 12 anos), por meio do imageamento térmico infravermelho (ITIV), uma técnica segura e não obtrusiva (isenta de contato), usada para registrar variações de temperatura em regiões de interesse (RIs) da face, tais como testa, nariz, bochechas, queixo e regiões periorbital e perinasal. Um robô social chamado N-MARIA (Novo-Robô Autônomo Móvel para Interação com Autistas) foi usado como estímulo emocional e mediador de tarefas sociais e pedagógicas. O primeiro estudo avaliou a variação térmica facial para cinco emoções (alegria, tristeza, medo, nojo e surpresa), desencadeadas por estímulos audiovisuais afetivos, em crianças com desenvolvimento típico. O segundo estudo avaliou a variação térmica facial para três emoções (alegria, surpresa e medo), desencadeadas pelo robô social N-MARIA, em crianças com desenvolvimento típico. No terceiro estudo, duas sessões foram realizadas com crianças com TEA, nas quais tarefas sociais e pedagógicas foram avaliadas tendo o robô N-MARIA como ferramenta e mediador da interação com as crianças. Uma análise emocional por variação térmica da face foi possível na segunda sessão, na qual o robô foi o estímulo para desencadear alegria, surpresa ou medo. Além disso, profissionais (professores, terapeuta ocupacional e psicóloga) avaliaram a usabilidade do robô social. Em geral, os resultados mostraram que o ITIV foi uma técnica eficiente para avaliar as emoções por meio de variações térmicas. No primeiro estudo, predominantes decréscimos térmicos foram observados na maioria das RIs, com as maiores variações de emissividade induzidas pelo nojo, felicidade e surpresa, e uma precisão maior que 85% para a classificação das cinco emoções. No segundo estudo, as maiores probabilidades de emoções detectadas pelo sistema de classificação foram para surpresa e alegria, e um aumento significativo de temperatura foi predominante no queixo e nariz. O terceiro estudo realizado com crianças com TEA encontrou aumentos térmicos significativos em todas as RIs e uma classificação com a maior probabilidade para surpresa. N-MARIA foi um estímulo promissor capaz de desencadear emoções positivas em crianças. A interação criança-com-TEA-e-robô foi positiva, com habilidades sociais e tarefas pedagógicas desempenhadas com sucesso pelas crianças. Além disso, a usabilidade do robô avaliada por profissionais alcançou pontuação satisfatória, indicando a N-MARIA como uma potencial ferramenta para terapias

New approaches to the emerging social neuroscience of human-robot interaction

Prehistoric art, like the Venus of Willendorf sculpture, shows that we have always looked for ways to distil fundamental human characteristics and capture them in physically embodied representations of the self. Recently, this undertaking has gained new momentum through the introduction of robots that resemble humans in their shape and their behaviour. These social robots are envisioned to take on important roles: alleviate loneliness, support vulnerable children and serve as helpful companions for the elderly. However, to date, few commercially available social robots are living up to these expectations. Given their importance for an ever older and more socially isolated society, rigorous research at the intersection of psychology, social neuroscience and human-robot interaction is needed to determine to which extent mechanisms active during human-human interaction can be co-opted when we encounter social robots. This thesis takes an anthropocentric approach to answering the question how socially motivated we are to interact with humanoid robots. Across three empirical and one theoretical chapter, I use self-report, behavioural and neural measures relevant to the study of interactions with robots to address this question. With the Social Motivation Theory of Autism as a point of departure, the first empirical chapter (Chapter 3) investigates the relevance of interpersonal synchrony for human-robot interaction. This chapter reports a null effect: participants did not find a robot that synchronised its movement with them on a drawing task more likeable, nor were they more motivated to ask it more questions in a semi-structured interaction scenario. As this chapter heavily relies on self-report as a main outcome measure, Chapter 4 addresses this limitation by adapting an established behavioural paradigm for the study of human-robot interaction. This chapter shows that a failure to conceptually extend an effect in the field of social attentional capture calls for a different approach when seeking to adapt paradigms for HRI. Chapter 5 serves as a moment of reflection on the current state-of-the-art research at the intersection of neuroscience and human-robot interaction. Here, I argue that the future of HRI research will rely on interaction studies with mobile brain imaging systems (like functional near-infrared spectroscopy) that allow data collection during embodied encounters with social robots. However, going forward, the field should slowly and carefully move outside of the lab and into real situations with robots. As the previous chapters have established, well-known effects have to be replicated before they are implemented for robots, and before they are taken out of the lab, into real life. The final empirical chapter (Chapter 6), takes the first step of this proposed slow approach: in addition to establishing the detection rate of a mobile fNIRS system in comparison to fMRI, this chapter contributes a novel way to digitising optode positions by means of photogrammetry. In the final chapter of this thesis, I highlight the main lessons learned conducting studies with social robots. I propose an updated roadmap which takes into account the problems raised in this thesis and emphasise the importance of incorporating more open science practices going forward. Various tools that emerged out of the open science movement will be invaluable for researchers working on this exciting, interdisciplinary endeavour

Actor & Avatar: A Scientific and Artistic Catalog

What kind of relationship do we have with artificial beings (avatars, puppets, robots, etc.)? What does it mean to mirror ourselves in them, to perform them or to play trial identity games with them? Actor & Avatar addresses these questions from artistic and scholarly angles. Contributions on the making of "technical others" and philosophical reflections on artificial alterity are flanked by neuroscientific studies on different ways of perceiving living persons and artificial counterparts. The contributors have achieved a successful artistic-scientific collaboration with extensive visual material

Moral cognition: An interdisciplinary investigation of judgment versus action

Parts of this thesis have been published or submitted for publication: Chapter 1 Terbeck, S., & Francis, K. B. (2017). We should if we could, but we can’t. Experimental problems with moral enhancement. In: M. Hauskeller (Ed). Moral enhancement. Cambridge, England: Cambridge University Press. Chapter 2 Francis, K. B., Howard, C., Howard, I. S., Gummerum, M., Ganis, G., Anderson, G., & Terbeck, S. (2016). Virtual morality: Transitioning from moral judgment to moral action? PLoS One. doi: 10.1371/journal.pone.0164374 Chapter 3 Francis, K. B., Terbeck, S., Briazu, R. A., Haines, A., Gummerum, M., Ganis, G., & Howard, I. S. (2017). Moral muscles: Simulating harm in virtual moral dilemmas. Under Review. Chapter 5 Francis, K. B., Gummerum, M., Ganis, G., Howard, I. S., & Terbeck, S., (2017). Virtual morality in the helping professions: Simulated action and resilience. Submitted.In the past, experiments on human morality have predominantly utilised theoretical moral dilemmas to shed light on the nature of moral judgment. However, little attention has been given to determining how these judgments might translate into moral actions. In this thesis, I utilised novel and state-of-the-art Virtual Reality environments and combined approaches from social psychology, experimental philosophy, computer science, robotics, and speculative design. Over the course of six experiments with more than 200 participants, simulated moral actions made in Virtual Reality were found to be dissociated from moral judgments made in conventional paradigms. The results suggest that moral judgment and action may be driven by distinct mechanisms. The association between personality traits and moral judgments versus actions, was also investigated. In two experiments, psychopathic and associated traits predicted moral actions and the power with which these were simulated, but failed to predict moral judgments. With research suggesting a mediating role for empathy in this relationship, two further experiments examined empathic and affective processing in moral judgment versus action. In the first of these, alcohol consumption successfully lowered affective empathy and arousal in virtual dilemmas, but moral judgment and action remained unaffected. In the second, an investigation of professionally trained paramedics and fire service incident commanders, revealed distinct differences in empathic and related personality traits, reduced emotional arousal, and less regret following moral action. Taken together, this research suggests that novel virtual technologies can provide insights into self-referent actions, which sit in contrast to judgments motivated by social norms. Ethically, incorporating Virtual Reality in investigations of morality of harm offers a balanced approach; protecting participant wellbeing while increasing the ecological validity of moral investigations. The roles of personality traits and associated emotional processes in moral judgment and action remain multifaceted and as such, I outline the necessity of considering both the characteristics of the decision-maker and the context in which the decision is undertaken, within an interactionist model of morality

Sit, Stand, Speak: Examining the Perceptions of the Basic Public Speaking Student on Normative Forensic Practices and their Effect on Competitor Credibility in Oratory

This paper examines basic public speaking students\u27 perceptions forensic competitor credibility based on normative factors present within the forensic community. Anecdotal and experiential evidence provided this researcher with reason to believe that the unwritten rules and normative expectations of forensics were so far-removed from what students were used to seeing in their classrooms and in the media, that they could have a negative impact on a competitor\u27s ethos, from the basic public speaking students\u27 perspective. This research was performed in an attempt to determine whether these anecdotal and experiential assumptions were accurate and also to gain insight into the how students were conceptualizing ethos in public speaking. Students were recruited from Communication Studies 100 & 102 classrooms to participate in focus groups, in which they were shown three persuasive speaking national finalists from the 2013 American Forensics Association national tournament. Students were then asked a series of discussion questions based on the normative expectations connected to persuasive speaking (i.e. business-professional attire, formal posture/gestures, language use, topic choice, memorization, and speech structure) to determine whether these normative factors had an effect on how students perceived the competitors\u27 credibility (ethos). Students\u27 responses were analyzed to determine that the normative expectations, except in the areas of memorization and nonverbal communication (i.e. posture and gesturing), were found to positively impact a competitor\u27s credibility. However, further inductive critical discourse analysis revealed three intriguing themes regarding the students\u27 conceptualization of ethos: professionalism as assumed competence, high credibility/low identification, and gendered expectations of appropriateness. These findings indicate numerous critical implications regarding how teaching and coaching practices alike may perpetuate capitalistic assumptions of professionalism, power, and the meaning of success