52 research outputs found

    A pilot study using imitation and storytelling scenarios as activities for labelling emotions by children with autism using a humanoid robot

    Get PDF
    In this paper we present a child-robot interaction pilot study, focusing on recognizing and labelling emotions displayed by a humanoid robot. ZECA (Zeno Engaging Children with Autism) has a special skin covering its face which allows the display of facial expressions representing five emotions: joy, sadness, fear, anger, and surprise. These facial expressions were used in two different game scenarios, involving imitation and storytelling activities. The goal of these scenarios is to help the child acquire knowledge about different emotions and to improve their skill in recognizing them. The results show that these scenarios are appropriate for the goal established for this study, and positive behaviours concerning non-verbal communication were observed. This exploratory study demonstrated the possible positive outcomes this child-robot interaction can produce and highlighted the issues regarding data collection and their analysis that will inform future studies.This work has been supported by FCT Fundação para a Ciência e Tecnologia in the scope of the project: PEst-OE/EEI/UI0319/2014. The authors are grateful to the Portuguese Foundation (FCT) for the R&D project RIPD/ADA/109407/2009 and SFRH/BD/71600/2010 scholarship.info:eu-repo/semantics/publishedVersio

    Affective robotics for socio-emotional development in children with autism spectrum disorders

    Get PDF
    Tese de doutoramento do Programa Doutoral em Engenharia Eletrónica e de ComputadoresAutism Spectrum Disorders (ASD) are a group of complex developmental disorders of the brain. Individuals affected by this disorder are characterized by repetitive patterns of behaviour, restricted activities or interests, and impairments in social communication. The use of robots had already been proven to encourage the promotion of social interaction and skills lacking in children with ASD. The main goal of this thesis is to study the influence of humanoid robots to develop socio-emotional skills in children with ASD. The investigation demonstrates the potential benefits a robotic tool provides to attract the attention of children with ASD, and therefore use that focus to develop further skills. The main focus of this thesis is divided into three topics. The first topic concerns the use of a robot to encourage learning appropriate physical social engagement, and to facilitate the ability to acquire knowledge about human body parts. The results show that the robot proved to be a useful tool, attracting the children’s attention and improving their knowledge about human body parts. The second topic regards the process of designing game scenarios to be used with children with ASD, targeting the promotion of emotion recognition skills. Three game scenarios were developed based on the expertise of professionals and they were successfully tested in pilot studies. Finally, the last topic presents two child-robot interaction studies with a large sample. They examine the use of a humanoid robot as a tool to teach recognition and labelling of emotions. The first study focuses on verbal and non-verbal communicative behaviours as measures to evaluate the social interaction and children interacting with the robot displayed more non-verbal behaviours indicating social engagement. The second study analyses the children’s attention patterns, and the children’s performance in the game scenarios previously designed. Along the sessions, the children increased their eye contact with the experimenter and in the study comparing the use of the robot with a traditional intervention, children who performed the game scenarios with the robot and the experimenter had a significantly better performance than the children who performed the game scenarios without the robot. The main conclusions of this research support that a humanoid robot is a useful tool to develop socio-emotional skills in the intervention of children with ASD, due to the engagement and positive learning outcome observed.As Perturbações do Espectro do Autismo (PEA) são um distúrbio complexo do desenvolvimento do cérebro. Os indivíduos afetados por esse transtorno são caracterizados por padrões repetitivos do comportamento, atividades ou interesses restritos e dificuldades na comunicação social. A utilização de robôs já provou ser um estímulo promovendo a interação social e competências em falta nestes indivíduos. O objetivo principal desta tese é estudar a influência de robôs humanoides para desenvolver competências sócio emocionais em crianças com PEA. A investigação demonstra os potenciais benefícios de uma ferramenta robótica para atrair a atenção de crianças com PEA e utilizar esta atenção para desenvolver outras competências. O foco principal desta tese está dividido em três tópicos. O primeiro tópico consiste na utilização de um robô para incentivar a aprendizagem sobre a interação físico-social apropriada e para facilitar a aquisição de conhecimento sobre partes do corpo. Os resultados mostram que o robô provou ser uma ferramenta útil, atraindo a atenção das crianças e melhorando o seu conhecimento sobre partes do corpo. A segunda parte refere-se ao processo de construção de atividades para serem utilizadas com crianças com PEA, promovendo competências de reconhecimento de emoções. Três atividades foram desenvolvidas com base na opinião de profissionais e foram testadas em estudo piloto com sucesso. Finalmente, o último tópico apresenta dois estudos de interação criança-robô examinando a utilização de um robô humanoide como ferramenta para ensinar reconhecimento e identificação de emoções. O primeiro estudo foca a comunicação verbal e não-verbal como medidas de avaliação da interação social e as crianças que interagiram com o robô mostraram mais comportamentos não-verbais que indicam interação social. O segundo estudo analisa os padrões de atenção e o desempenho das crianças nas atividades concebidas anteriormente. Ao longo das sessões, as crianças aumentaram o contacto ocular com o experimentador e no estudo que comparou a utilização do robô com intervenção tradicional, as crianças que realizaram as atividades com o robô e o experimentador tiveram um desempenho significativamente melhor do que as crianças que realizaram as ativdades sem o robô. As conclusões principais desta investigação suportam que um robô humanoide foi uma ferramenta útil para desenvolver competências sócio emocionais na intervenção de crianças com PEA, devido à interação e resultados positivos de aprendizagem observados.Fundação para a Ciência e Tecnologia (FCT) in the scope of the project: PEst-OE/EEI/UI0319/2014.This work was performed in part under the R&D project RIPD/ADA/109407/2009.SFRH/BD/71600/2010 scholarship

    Serious games assisted by playware as a way to improve socio-emotional skills in children with autism spectrum disorder

    Get PDF
    This paper presents a project developed with the aim of promoting emotional skills in children with Autism Spectrum Disorders (ASD). The project involves a serious game and a playware object, which is a physical component that allows the user to interactively play the serious game. The playware object has six buttons, each one showing an emoji with a specific facial expression and communicates via Bluetooth with the serious game app installed in an Android device. The facial expressions used are: happiness, sadness, fear, anger, surprise and neutral/normal. They were applied to the three game activities (imitation, recognition and storytelling). The chain of tests started with an online questionnaire to validate the avatars created to represent the previously mentioned facial expressions in the game, which was followed by a usability test of the application (serious game and playware object) with six typically developing children. Finally, the three game activities were tested with six children with ASD in three/four sessions. Due to the small test group and reduced number of sessions, the primary objective was to assess if the target group accepted the application. In fact, it had a high level of approval regarding both the serious game and the playware object. had a high level of approval regarding both the serious game and the playware object.COMPETE: POCI-01-0145-FEDER-007043 and FCT –Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2013. Vinicius Silva also thanks FCT for the PhD scholarship SFRH/BD/SFRH/BD/133314/2017. The authors thank the teachers and students of the Elementary School of Gualtar (EB1/JI Gualtar) in Braga for the participation in the testsinfo:eu-repo/semantics/publishedVersio

    Dialogue Design for a Robot-Based Face-Mirroring Game to Engage Autistic Children with Emotional Expressions

    Get PDF
    We present design strategies for Human Robot Interaction for school-aged autistic children with limited receptive language. Applying these strategies to the DE-ENIGMA project (large EU project addressing emotion recognition in autistic children) supported development of a new activity for in facial expression imitation whereby the robot imitates the child’s face to encourage the child to notice facial expressions in a play-based game. A usability case study with 15 typically-developing children aged 4–6 at an English-language school in the Netherlands was performed to observe the feasibility of the setup and make design revisions before exposing the robot to autistic children

    ENGAGEMENT RECOGNITION WITHIN ROBOT-ASSISTED AUTISM THERAPY

    Get PDF
    Autism is a neurodevelopmental condition typically diagnosed in early childhood, which is characterized by challenges in using language and understanding abstract concepts, effective communication, and building social relationships. The utilization of social robots in autism therapy represents a significant area of research. An increasing number of studies explore the use of social robots as mediators between therapists and children diagnosed with autism. Assessing a child’s engagement can enhance the effectiveness of robot-assisted interventions while also providing an objective metric for later analysis. The thesis begins with a comprehensive multiple-session study involving 11 children diagnosed with autism and Attention Deficit Hyperactivity Disorder (ADHD). This study employs multi-purposeful robot activities designed to target various aspects of autism. The study yields both quantitative and qualitative findings based on four behavioural measures that were obtained from video recordings of the sessions. Statistical analysis reveals that adaptive therapy provides a longer engagement duration as compared to non-adaptive therapy sessions. Engagement is a key element in evaluating autism therapy sessions that are needed for acquiring knowledge and practising new skills necessary for social and cognitive development. With the aim to create an engagement recognition model, this research work also involves the manual labelling of collected videos to generate a QAMQOR dataset. This dataset comprises 194 therapy sessions, spanning over 48 hours of video recordings. Additionally, it includes demographic information for 34 children diagnosed with ASD. It is important to note that videos of 23 children with autism were collected from previous records. The QAMQOR dataset was evaluated using standard machine learning and deep learning approaches. However, the development of an accurate engagement recognition model remains challenging due to the unique personal characteristics of each individual with autism. In order to address this challenge and improve recognition accuracy, this PhD work also explores a data-driven model using transfer learning techniques. Our study contributes to addressing the challenges faced by machine learning in recognizing engagement among children with autism, such as diverse engagement activities, multimodal raw data, and the resources and time required for data collection. This research work contributes to the growing field of using social robots in autism therapy by illuminating an understanding of the importance of adaptive therapy and providing valuable insights into engagement recognition. The findings serve as a foundation for further advancements in personalized and effective robot-assisted interventions for individuals with autism

    An application to improve emotional skills in children with Autism Spectrum Disorder

    Get PDF
    Dissertação de Mestrado Integrado em Engenharia de Eletrónica Industrial e ComputadoresThis dissertation presents a project developed with the aim of promoting emotional skills in children with Autism Spectrum Disorders (ASD). The project involves a serious game and a playware object, which is a physical component that acts as the game controller and allows the user to interactively play the serious game. The playware object has six pressure buttons, each one showing an emoji with a specific facial expression and communicates wirelessly via Bluetooth with the serious game app installed in an Android device. The facial expressions used are: happiness, sadness, fear, anger, surprise and neutral/normal. They were applied to the three game activities (imitation, recognition and storytelling). The chain of tests started with an online questionnaire to validate the avatars created to represent the previously mentioned facial expressions in the game (with 114 answers and a mean success rate of 96.2%), which was followed by a usability test of the application (serious game and playware object) with six typically developing children (with 94.4% answer accuracy). Finally, the three game activities were tested with six children with ASD in three/four sessions. Due to the small group test and the short number of sessions, the goal was to test the acceptance of the game rather than the users´ improvement in the activity. It is worth referring that both the serious game and the playware object had a high level of approval from the children and they expressed their interest during the activities. With this project it was intended to contribute to the development of pedagogical resources to be used by professionals and families in the support of children with ASD.Esta dissertação apresenta um projeto desenvolvido com o objetivo de promover capacidades emocionais em crianças com Perturbação do Espectro do Autismo. Este projeto envolve um jogo sério e um objeto playware, que é um componente físico que funciona como controlador de jogo e permite que o utilizador jogue o jogo sério de uma forma interativa. O objeto playware tem seis botões de pressão, cada um com um emoji com uma expressão facial específica, e comunica sem fios por Bluetooth com a aplicação do jogo sério instalada no dispositivo Android. As expressões faciais usadas são: felicidade, tristeza, medo, raiva, surpresa e neutro/normal. Estas foram aplicadas às três diferentes atividades de jogo (imitar, reconhecer e contar histórias). A cadeia de testes começou com um questionário online para validar os avatares criados para representar as expressões faciais previamente mencionadas no jogo (com 114 submissões e uma taxa média de sucesso de 96,2%), seguido de um teste de usabilidade da aplicação (jogo sério e objeto playware) com seis crianças tipicamente desenvolvidas (com 94,4% de respostas corretas). Por fim, as três atividades de jogo foram testadas com seis crianças com Perturbação do Espectro do Autismo durante 3 a 4 sessões. Devido à pequena dimensão do grupo de teste e ao baixo número de sessões, o objetivo foi testar a aceitação do jogo em vez da evolução das capacidades dos utilizadores na atividade. É importante referir que tanto o jogo sério como o objeto playware tiveram um alto nível de aprovação por parte das crianças que expressaram o seu interesse durante as atividades. Este projeto pretende contribuir para o desenvolvimento de recursos pedagógicos a serem usados por profissionais e famílias no apoio a crianças com Perturbação do Espectro do Autismo

    Affective Computing

    Get PDF
    This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

    TouchStory: Interactive Software Designed to Assist Children with Autism to Understand Narrative

    Get PDF
    The work described in this thesis falls under the umbrella of the Aurora project (Aurora 2000). Aurora is a long-term research project which, through diverse studies, investigates the potential enhancement of the everyday lives of children with autism through the use of robots, and other interactive systems, in playful contexts. Autism is a lifelong pervasive disability which affects social interaction and communication. Importantly for this thesis, children with autism exhibit a deficit in narrative comprehension which adversely impacts their social world. The research agenda addressed by this thesis was to develop an interactive software system which promotes an understanding of narrative structure (and thus the social world) while addressing the needs of individual children. The conceptual approach developed was to break down narrative into proto-narrative components and address these components individually through the introduction of simple game-like tasks, called t-stories, presented in a human-computer interaction context. The overarching hypothesis addressed was that it is possible to help children with autism to improve their narrative skills by addressing proto-narrative components independently. An interactive software system called TouchStory was developed to present t-stories to children with autism. Following knowledge of the characteristics and preferences of this group of learners TouchStory maintained strong analogies with the concrete, physical world. The design approach was to keep things simple, introducing features only if necessary to provide a focussed and enjoyable game. TouchStory uses a touch-sensitive screen as the interaction device as it affords immediate direct manipulation of the t-story components. Socially mediated methods of requirements elicitation and software evaluation (such as focus groups, thinking aloud protocols, or intergenerational design teams) are not appropriate for use with children with autism who are not socially oriented and, in the case of children with ‗lower functioning‘ autism, may have very few words or no productive language. Therefore a new strategy was developed to achieve an inclusive, child-centred design; this was to interleave prototype development with evaluation over several long-term trials. The trials were carried out in the participants‘ own school environments to provide an ecologically valid contextual enquiry. In the first trial 18 participants were each seen individually once. The second and third trials were extended studies of 12 and 20 school visits with 12 and 6 participants respectively; each participant was seen individually on each school visit, provided that the participant was at school on the day of the visit. Evaluation was carried out on the basis of video recordings of the sessions and software logs of the on-screen interactions. Individual learning needs were addressed by adapting the set of t-stories presented to the participant on the basis of success during recent sessions. No ordering of difficulty among the proto-narrative categories could be known a priori for any individual child, and may vary from child to child. Therefore the intention was to gradually, over multiple sessions, increase the proportion of t-stories from proto-narrative categories which the individual participant found challenging, while retaining sufficient scope for the expression of skills already mastered for the session to be enjoyable and rewarding. The adaptation of the software was achieved by introducing a simple adaptive formula, evaluating it over successive long terms trials, and increasing the complexity of the formula only where necessary. Results indicate that individual participants found the interactive presentation of the simple game-like tasks engaging, even after repeated exposures on as many as 20 occasions. The adaptive formula developed in this study did, for engaged participants, focus on the proto-narrative categories which the participant needed to practice but was likely to succeed; that is it did target an effective learning zone. While little evidence was seen of learning with respect to the fully developed narratives encountered in everyday life, results strongly suggest that some participants were actively engaged in self-directed, curiosity-driven activity that functioned as learning in that they were able to transfer knowledge about the appropriateness of particular responses to previously unseen t-stories. This thesis was driven by the needs of children with autism; contributions are made in a number of cognate areas. A conceptual contribution was made by the introduction of the proto-narrative concept which was shown to identify narrative deficits in children with autism and to form a basis for learning. A contribution was made to computational adaptation by the development of a novel adaptive formula which was shown to present a challenging experience while maintaining sufficient predictability and opportunities for the expression of skills already mastered to provide a comfortable experience for children with autism. A contribution was made to software development by showing that children with autism may be included in the design process through iterative development combined with long term trials. A contribution was made to assistive technology by demonstrating that simplicity together with evaluation over long term trials engages children with autism and is a route to inclusion. We cannot expect any magic fixes for children with autism, progress will be made by small steps; this thesis forms a small but significant contribution

    An emotion and memory model for social robots : a long-term interaction

    Get PDF
    In this thesis, we investigate the role of emotions and memory in social robotic companions. In particular, our aim is to study the effect of an emotion and memory model towards sustaining engagement and promoting learning in a long-term interaction. Our Emotion and Memory model was based on how humans create memory under various emotional events/states. The model enabled the robot to create a memory account of user's emotional events during a long-term child-robot interaction. The robot later adapted its behaviour through employing the developed memory in the following interactions with the users. The model also had an autonomous decision-making mechanism based on reinforcement learning to select behaviour according to the user preference measured through user's engagement and learning during the task. The model was implemented on the NAO robot in two different educational setups. Firstly, to promote user's vocabulary learning and secondly, to inform how to calculate area and perimeter of regular and irregular shapes. We also conducted multiple long-term evaluations of our model with children at the primary schools to verify its impact on their social engagement and learning. Our results showed that the behaviour generated based on our model was able to sustain social engagement. Additionally, it also helped children to improve their learning. Overall, the results highlighted the benefits of incorporating memory during child-Robot Interaction for extended periods of time. It promoted personalisation and reflected towards creating a child-robot social relationship in a long-term interaction
    corecore