An Intelligent Robot and Augmented Reality Instruction System

Human-Centered Robotics (HCR) is a research area that focuses on how robots can empower people to live safer, simpler, and more independent lives. In this dissertation, I present a combination of two technologies to deliver human-centric solutions to an important population. The first nascent area that I investigate is the creation of an Intelligent Robot Instructor (IRI) as a learning and instruction tool for human pupils. The second technology is the use of augmented reality (AR) to create an Augmented Reality Instruction (ARI) system to provide instruction via a wearable interface. To function in an intelligent and context-aware manner, both systems require the ability to reason about their perception of the environment and make appropriate decisions. In this work, I construct a novel formulation of several education methodologies, particularly those known as response prompting, as part of a cognitive framework to create a system for intelligent instruction, and compare these methodologies in the context of intelligent decision making using both technologies. The IRI system is demonstrated through experiments with a humanoid robot that uses object recognition and localization for perception and interacts with students through speech, gestures, and object interaction. The ARI system uses augmented reality, computer vision, and machine learning methods to create an intelligent, contextually aware instructional system. By using AR to teach prerequisite skills that lend themselves well to visual, augmented reality instruction prior to a robot instructor teaching skills that lend themselves to embodied interaction, I am able to demonstrate the potential of each system independently as well as in combination to facilitate students\u27 learning. I identify people with intellectual and developmental disabilities (I/DD) as a particularly significant use case and show that IRI and ARI systems can help fulfill the compelling need to develop tools and strategies for people with I/DD. I present results that demonstrate both systems can be used independently by students with I/DD to quickly and easily acquire the skills required for performance of relevant vocational tasks. This is the first successful real-world application of response-prompting for decision making in a robotic and augmented reality intelligent instruction system

Xylo-Bot: A Therapeutic Robot-Based Music Platform for Children with Autism

Children with Autism Spectrum Disorder (ASD) experience deficits in verbal and nonverbal communication skills, including motor control, emotional facial expressions, and eye gaze / joint attention. This Ph.D. dissertation focuses on studying the feasibility and effectiveness of using a social robot, called NAO, and a toy music instrument, xylophone, at modeling and improving the social responses and behaviors of children with ASD. In our investigation, we designed an autonomous social interactive music teaching system to fulfill this mission. A novel modular robot-music teaching system consisting of three modules is presented. Module 1 provides an autonomous self-awareness positioning system for the robot to localize the instrument and make a micro adjustment for the arm joints to play the note bars properly. Module 2 allows the robot to be able to play any customized song per user’s request. This design provides an opportunity to translate songs into C-major or a-minor with a set of hexadecimal numbers without music experience. After the music score converted robot should be able to play it immediately. Module 3 is designed for providing real-life music teaching experience for the users. Two key features of this module are a) music detection and b) smart scoring and feedback . Short-time Fourier transform and Levenshtein distance are adapted to fulfill the design requirements, which allow the robot to understand music and provide a proper dosage of practice and oral feedback to users. A new instrument has designed to present better emotions from music due to the limitation of the original xylophone. This new programmable xylophone can provide a more extensive frequency range of notes, easily switch between the Major and Minor keys, extensively easy to control, and have fun with it as an advanced music instrument. Because our initial intention has been to study emotion in children with autism, an automated method for emotion classification in children using electrodermal activity (EDA) signals. The time-frequency analysis of the acquired raw EDAs provides a feature space based on which different emotions can be recognized. To this end, the complex Morlet (C-Morlet) wavelet function is applied to the recorded EDA signals. The dataset used in this research includes a set of multimodal recordings of social and communicative behavior as well as EDA recordings of 100 children younger than 30 months old. The dataset is annotated by two experts to extract the time sequence corresponding to three primary emotions, including “Joy”, “Boredom”, and “Acceptance”. Various experiments are conducted on the annotated EDA signals to classify emotions using a support vector machine (SVM) classifier. The quantitative results show that emotion classification performance remarkably improves compared to other methods when the proposed wavelet-based features are used. By using this emotion classification, emotion engagement during sessions, and feelings between different music can be detected after data analysis. NAO music education platform will be thought-about as a decent tool to facilitate improving fine motor control, turn-taking skills, and social activities engagement. Most of the ASD youngsters began to develop the strike movement within the two initial intervention sessions; some even mastered the motor ability throughout the early events. More than half of the subjects could dominate proper turn-taking after few sessions. Music teaching is a good example for accomplishing social skill tasks by taking advantage of customized songs selected by individuals. According to researcher and video annotator, majority of the subjects showed high level of engagement for all music game activities, especially with the free play mode. Based on the conversation and music performance with NAO, subjects showed strong interest in challenging the robot with a friendly way

Psychology, Learning, Technology

This open access book constitutes the refereed proceedings of 1st International Workshop on Psychology, Learning, Technology, PLT 2022, Foggia, Italy, during January 2022. The 8 full papers presented here were carefully reviewed and selected from 23 submissions. In addition, one invited paper is also included. Psychology, Learning, ad Technology Conference (PLT2022) aims to explore learning paths that incorporate digital technologies in innovative and transformative ways and the improvement of the psychological and relational life. The conference includes topics about the methodology of application of the ICT tools in psychology and education: from blended learning to the application of artificial intelligence in education; from the teaching, learning, and assessment strategies and practices to the new frontiers on Human-Computer Interaction

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

Proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET 2013)

"This book contains the proceedings of the International Workshop on EuroPLOT Persuasive Technology for Learning, Education and Teaching (IWEPLET) 2013 which was held on 16.-17.September 2013 in Paphos (Cyprus) in conjunction with the EC-TEL conference. The workshop and hence the proceedings are divided in two parts: on Day 1 the EuroPLOT project and its results are introduced, with papers about the specific case studies and their evaluation. On Day 2, peer-reviewed papers are presented which address specific topics and issues going beyond the EuroPLOT scope. This workshop is one of the deliverables (D 2.6) of the EuroPLOT project, which has been funded from November 2010 – October 2013 by the Education, Audiovisual and Culture Executive Agency (EACEA) of the European Commission through the Lifelong Learning Programme (LLL) by grant #511633. The purpose of this project was to develop and evaluate Persuasive Learning Objects and Technologies (PLOTS), based on ideas of BJ Fogg. The purpose of this workshop is to summarize the findings obtained during this project and disseminate them to an interested audience. Furthermore, it shall foster discussions about the future of persuasive technology and design in the context of learning, education and teaching. The international community working in this area of research is relatively small. Nevertheless, we have received a number of high-quality submissions which went through a peer-review process before being selected for presentation and publication. We hope that the information found in this book is useful to the reader and that more interest in this novel approach of persuasive design for teaching/education/learning is stimulated. We are very grateful to the organisers of EC-TEL 2013 for allowing to host IWEPLET 2013 within their organisational facilities which helped us a lot in preparing this event. I am also very grateful to everyone in the EuroPLOT team for collaborating so effectively in these three years towards creating excellent outputs, and for being such a nice group with a very positive spirit also beyond work. And finally I would like to thank the EACEA for providing the financial resources for the EuroPLOT project and for being very helpful when needed. This funding made it possible to organise the IWEPLET workshop without charging a fee from the participants.

Psychology, Learning, Technology

This open access book constitutes the refereed proceedings of 1st International Workshop on Psychology, Learning, Technology, PLT 2022, Foggia, Italy, during January 2022. The 8 full papers presented here were carefully reviewed and selected from 23 submissions. In addition, one invited paper is also included. Psychology, Learning, ad Technology Conference (PLT2022) aims to explore learning paths that incorporate digital technologies in innovative and transformative ways and the improvement of the psychological and relational life. The conference includes topics about the methodology of application of the ICT tools in psychology and education: from blended learning to the application of artificial intelligence in education; from the teaching, learning, and assessment strategies and practices to the new frontiers on Human-Computer Interaction

Standardising the design of educational computer reading programs for children.

Educationalists working in the sphere of special education, psychologists and software engineers continue to debate the efficacy of technology interfaces and the merits of information technology with regard to supporting learning in children with Autistic Spectrum Disorder (ASD), and the need to standardise software design for this group. This research argues that, for these children to optimise the use of this technology, it must be designed to meet the learning needs and characteristics of this condition, and so a design/development standard is needed. There is currently no instruction to aid educational professionals in choosing suitable computer programs that can be employed to support learning to read in children with ASD. The present research offers a rigorous comparative analysis of the multimedia conditions. A selection instruction (SI) was developed to facilitate the choice of appropriate computer programs for children with ASD, and forms part of the pilot study for this investigation, which was later modified and developed into an educational computer programs design standard. This SI serves as a set of guidelines that is intended to assist professionals and the parents/guardians of children with ASD in their search for good, useable programs that will assist in the acquisition of early reading skills by this group. This research advocates the development of effective computer programs based on individualistic considerations and the stringent application of Human Computer Interaction (HCI) principles in the design of multimedia computer technology for children with ASD. Two educational programs were employed in the investigation, and the data collection method included quantitative (pre-tests; a comparative study of children with ASD and typically developing children in videorecorded sessions, and post-tests) and qualitative (interviews, and an attitude questionnaire) methods. This approach was triangulated, thereby promoting the validity and rigour of the investigation. The present research concludes that, although there were recorded gains in the application of computer technology to teach new words to children with ASD, there were problems relating to the appropriateness and suitability of the programs for the children employed in this research, as detected during the interviews and from the attitude questionnaires, noting the need for autistic preferences in the design and development of these educational computer programs. This investigation offers a broader approach to the theoretical understanding and explanation of autistic learning styles, reading methodologies and issues relating to the design, development and usability of multimedia computer technology. Attention is drawn to the inadequacy of the existing technology and research into ASD and how the disorder affects learning in these children

Utilizing social media as an assistive technology tool for children with autism spectrum disorder

The number of children diagnosed with autism has continued to rise since 2000 at startling rates, leading to an increased need for schools to discover innovative ways to meet the unique learning needs of these students. Technology has proven to be adaptable and successful in teaching social skills to children with autism. Assistive technology and the continuing development of online platforms offer unique opportunities for children with special needs to access the general education curriculum. This combined with the increased usage of social media among school-aged children, make it a potential tool for learning. Understanding how social media can be used as a learning tool for Autism Spectrum Disorder (ASD) students is crucial in order to meet the needs of the growing ASD students in classrooms. This exploratory qualitative study examined the necessary considerations for schools when implementing social media as a learning tool in the classroom. Eight experts from the special education field were interviewed to gain knowledge from their experience working with students with autism in the classroom setting. These eight professionals were composed of special education teachers, school psychologists, assistive technology specialists, and Board Certified Behavior Analysts. Themes discussed by the experts focused on considerations for students, staff, budgets, safety, training, and technology. Thematic analysis resulted in four study conclusions. First, rapidly changing learning environments, especially during the pandemic, are forcing schools to innovate with virtual instructional tools. Second, lack of knowledge, training, and known practices impede integration of social media as an instructional tool. Third, there are increased safety risks for children with ASD with using social media as a learning tool and fourth, the adoption of social media as a learning tool needs stakeholder buy-in. Specific recommendations for schools include formally integrating technology into their learning programs, establishing best practices for teachers to use for implementation, formulating digital safety measures for students, and collaborating with parents to ensure beneficial utilization. Additional research around the safety of social media use for the autism population, adoption strategies for new and ever-changing technology in the classroom, technology use for the ASD population, and gathering of student and parent perspectives is recommended