Engaging students with profound and multiple disabilities using humanoid robots

Engagement is the single best predictor of successful learning for children with intellectual disabilities yet achieving engagement with pupils who have profound or multiple disabilities (PMD) presents a challenge to educators. Robots have been used to engage children with autism but are they effective with pupils whose disabilities limit their ability to control other technology? Learning objectives were identified for eleven pupils with PMD and a humanoid robot was programmed to enable teachers to use it to help pupils achieve these objectives. These changes were evaluated with a series of eleven case studies where teacher-pupil dyads were observed during four planned video recorded sessions. Engagement was rated in a classroom setting and during the last session with the robot. Video recordings were analysed for duration of engagement and teacher assistance and number of goals achieved. Rated engagement was significantly higher with the robot than in the classroom. Observations of engagement, assistance and goal achievement remained at the same level throughout the sessions suggesting no reduction in the novelty factor

Engaging students with profound and multiple disabilities using humanoid robots

Engagement is the single best predictor of successful learning for children with intellectual disabilities yet achieving engagement with pupils who have profound or multiple disabilities (PMD) presents a challenge to educators. Robots have been used to engage children with autism but are they effective with pupils whose disabilities limit their ability to control other technology? Learning objectives were identified for eleven pupils with PMD and a humanoid robot was programmed to enable teachers to use it to help pupils achieve these objectives. These changes were evaluated with a series of eleven case studies where teacher-pupil dyads were observed during four planned video recorded sessions. Engagement was rated in a classroom setting and during the last session with the robot. Video recordings were analysed for duration of engagement and teacher assistance and number of goals achieved. Rated engagement was significantly higher with the robot than in the classroom. Observations of engagement, assistance and goal achievement remained at the same level throughout the sessions suggesting no reduction in the novelty factor

Efectos de la Robótica Social en la Memoria Episódica de Niños con Discapacidad Intelectual

Episodic memory is crucial to develop complex cognitive abilities like learning or reasoning, and non-complex cognitive abilities like calling the name of someone or remembering an appointment. It is known that an intellectual disability implies a deficit over tasks related to episodic memory, however, in the literature, there is no approach to stimulate episodic memory in children with intellectual disabilities. Because interactions with social robots have generated positive effects in children with intellectual disabilities, we propose an approach composed of three training sessions based on social robotics. In this paper, we present an exploratory study to know the effects of our approach on episodic memory in children with intellectual disabilities. The results have shown that our approach can enhance episodic memory in these children when they interact with interest and improve their performance in session

Robots in special education: reasons for low uptake

Purpose: This paper identifies the main reasons for low uptake of robots in Special Education, obtained from an analysis of previous studies that used robots in the area, and from interviewing Special Education teachers about the topic. Design/methodology/approach: An analysis of 18 studies that used robots in Special Education was performed, and the conclusions were complemented and compared with the feedback from interviewing 13 Special Education teachers from Spain and UK about the reasons they believed caused the low uptake of robots in Special Education classrooms. Findings: Five main reasons why Special Education schools do not normally use robots in their classrooms were identified: the inability to acquire the system due to its price or availability; its difficulty of use; the low range of activities offered; the limited ways of interaction offered; and the inability to use different robots with the same software. Originality/value: Previous studies focused on exploring the advantages of using robots to help children with Autistic Spectrum Conditions and Learning Disabilities. This study takes a step further and looks into the reasons why, despite the benefits shown, robots are rarely used in real-life settings after the relevant study ends. The authors also present a potential solution to the issues found: involving end users in the design and development of new systems using a user-centred design approach for all the components, including methods of interaction, learning activities, and the most suitable type of robots

Examining engagement and achievement in learners with individual needs through robotic-based teaching sessions

Research suggests that robotics can provide an engaging learning experience for learners with special educational needs. However, further work is required to explore the impact of robots within the classroom, particularly for learners with intellectual disabilities (ID). This paper seeks to further explore the potential effects of robots on such learners through examining engagement and goal achievement within teaching sessions. Eleven participants with ID were recruited from two countries to take part in the study using an ABAB design where the participants acted as their own controls. An appropriate learning goal for each participant was selected by the teacher and equivalent control sessions designed seeking to achieve the same learning goal but without the robot. Engagement, using eye‐gaze, learning goal achievement with and without help and goals not achieved provided the outcome measures from the sessions. This study found no significant difference between the robot and the control sessions for any of the outcome measures utilized suggesting robots are as effective as teaching tools as traditional methods. Through an increased sample size and a rigorously applied experimental protocol, this study provides new data and methodological considerations for further work based on the techniques applied in this study

Modeling engagement with multimodal multisensor data: the continuous performance test as an objective tool to track flow

Engagement is one of the most important factors in determining successful outcomes and deep learning in students. Existing approaches to detect student engagement involve periodic human observations that are subject to inter-rater reliability. Our solution uses real-time multimodal multisensor data labeled by objective performance outcomes to infer the engagement of students. The study involves four students with a combined diagnosis of cerebral palsy and a learning disability who took part in a 3-month trial over 59 sessions. Multimodal multisensor data were collected while they participated in a continuous performance test. Eye gaze, electroencephalogram, body pose, and interaction data were used to create a model of student engagement through objective labeling from the continuous performance test outcomes. In order to achieve this, a type of continuous performance test is introduced, the Seek-X type. Nine features were extracted including high-level handpicked compound features. Using leaveone-out cross-validation, a series of different machine learning approaches were evaluated. Overall, the random forest classification approach achieved the best classification results. Using random forest, 93.3% classification for engagement and 42.9% accuracy for disengagement were achieved. We compared these results to outcomes from different models: AdaBoost, decision tree, k-Nearest Neighbor, naïve Bayes, neural network, and support vector machine. We showed that using a multisensor approach achieved higher accuracy than using features from any reduced set of sensors. We found that using high-level handpicked features can improve the classification accuracy in every sensor mode. Our approach is robust to both sensor fallout and occlusions. The single most important sensor feature to the classification of engagement and distraction was shown to be eye gaze. It has been shown that we can accurately predict the level of engagement of students with learning disabilities in a real-time approach that is not subject to inter-rater reliability, human observation or reliant on a single mode of sensor input. This will help teachers design interventions for a heterogeneous group of students, where teachers cannot possibly attend to each of their individual needs. Our approach can be used to identify those with the greatest learning challenges so that all students are supported to reach their full potential

Nonlinear Storytelling Approach to Developing Computational Thinking Skills

Current methods for developing computational thinking skills usually have a technical and programming-centric approach and are not suitable for all people. In this research, the use of nonlinear storytelling as an educational method was examined. The specific interest was to analyze its relationship with the concept of computational thinking and to investigate if nonlinear storytelling can be used as a low-threshold method for teaching fundamental computational thinking skills. This research situates itself in computer science education. It consists of four independent studies. Study I investigates how nonlinear storytelling can be integrated into an adult education course for developing basic information technology skills. Special attention was given to understanding the role of storytelling in the process. The result of this study was a method that integrates nonlinear storytelling into educational game development. Study II studied the relationship between nonlinear stories and computational thinking by examining how typical computer programs are implemented using stories. The study shows that nonlinear stories are best suited for implementing finite state machine programs and programs that include interaction. The natural character of applicability indicates that nonlinear storytelling can improve students’ readiness for learning programming skills. In study III, experiences and observations made at the end of the aforementioned adult education course are reported. The technical quality of the stories collected (N = 14) was investigated and common challenges in the storytelling process such as understanding hyperlinking and its purpose in gamification were identified. In this study, a practical classification for storytelling software and metrics for analyzing stories were developed. Finally, study IV focused on investigating whether the concept of computational thinking allows broader interpretations compared to how it is traditionally used. The concept of computational thinking was explored by using the Extended Mind thesis by Clark and Chalmers. Analysis showed that it is reasonable to expand the concept beyond the traditional computer programming-based interpretation.Nykyiset menetelmät algoritmisen ajattelun opetuksessa ovat usein teknisiä ja ohjelmointikeskeisiä eivätkä ne sovi kaikille kohderyhmille. Tässä työssä selvitettiin epälineaaristen tarinoiden käyttöä opetuksessa. Erityinen kiinnostuksen kohde oli se, mikä on epälineaaristen tarinoiden suhde algoritmiseen ajatteluun ja voiko epälineaarisia tarinoita käyttää matalan kynnyksen menetelmänä algoritmisen ajattelun harjoittamiseen. Tämä tutkimus sijoittuu tietotekniikan opetuksen alalle. Työ koostuu neljästä osatutkimuksesta. Osatutkimuksessa I tutkittiin, miten epälineaarinen tarinankerronta voidaan ottaa osaksi tietotekniikkavalmiuksia kehittävää aikuiskoulutusta. Tutkimuksen tuloksena syntyi menetelmä, joka yhdistää epälineaarisen tarinankerronnan pelinkehitykseen. Osatutkimuksessa II tutkittiin epälineaaristen tarinoiden suhdetta algoritmiseen ajatteluun selvittämällä tyypillisten tietokoneohjelmien toteutuksia. Toteutuksista kävi ilmi, että epälineaariset tarinat sopivat erityisesti äärellisillä automaateilla esitettävissä olevien ohjelmien sekä interaktiivisten ohjelmien toteuttamiseen. Tarinallisten toteutusten luontevuus osoitti, että tarinoiden avulla voidaan harjoittaa opiskelijoiden ohjelmointivalmiuksia. Osatutkimuksessa III raportoitiin edellä mainitun tietotekniikkavalmiuksia kehittävän aikuiskoulutuksen aikana tehtyjä havaintoja ja saatuja kokemuksia hankkeen loputtua. Erityisesti selvitettiin hankkeessa kerättyjen tarinoiden (N = 14) teknistä laatua sekä yleisimpiä ongelmia hyperlinkkien toiminnan sekä pelillisen merkityksen ymmärtäminen kanssa. Osatutkimuksessa IV selvitettiin, miten algoritmisen ajattelun käsitettä voidaan tulkita sen perinteistä tulkintaa laajemmin. Problematiikkaa lähestyttiin Clarkin ja Chalmersin laajennetun mielen hypoteesia käyttäen. Epälineaaristen tarinoiden yhteyttä algoritmiseen ajatteluun ei ole tutkittu aiemmin, joten aihe on uusi. Tämän tutkimuksen perusteella epälineaarista tarinankerrontaa voidaan soveltaa algoritmisen ajattelun harjoittamiseen. Uudenlainen lähestymistapa kuitenkin haastaa algoritmisen ajattelun käsitteen, joka on perinteisesti ymmärretty ohjelmoinnin kautta

Multimodal Multisensor attention modelling

Introduction: Sustaining attention is one of the most important factors in determining successful outcomes and deep learning in students. Existing approaches to track student engagement involve periodic human observations that are subject to inter-rater reliability. Our solution uses real-time Multimodal Multisensor data labeled by objective performance outcomes to track the attention of students. Method: The study involves four students with a combined diagnosis of cerebral palsy and a learning disability who took part in a 3-month trial over 59 sessions. Multimodal Multisensor data were collected while they participated in a Continuous Performance Test (CPT). Eyegaze, electroencephalogram, body pose, and interaction data were used to create a model of student attention through objective labeling from the Continuous Performance Test outcomes. To achieve this, a type of continuous performance test is introduced, the Seek-X type. Nine features were extracted including High-Level handpicked Compound Features (HLCF). Using leave-one-out cross-validation, a series of different machine learning approaches were evaluated. Research questions: RQ1: Can we create a model of attention for PMLD/CP students using the CPT? RQ2: What are the main correlations found in the CPT outcomes and the Multimodal Multisensor data? Results: Overall, the random forest classification approach achieved the best classification results. Using random forest, 84.8% classification for attention and 65.4% accuracy for inattention were achieved. We compared these results to outcomes from different models: AdaBoost, decision tree, k-Nearest Neighbor, naïve Bayes, neural network, and support vector machine. We showed that using a multisensor approach achieved higher accuracy than using features from any reduced set of sensors. Incorporating person-specific data improved the classification outcome, compared to being participant neutral. We found that using HighLevel handpicked Compound Features (HLCF) can improve the classification accuracy in every sensor mode. Our approach is robust to both sensor fallout and occlusions. The single most important sensor feature to the classification of attention and inattention was shown to be eye-gaze. We have shown that we can accurately predict the level of attention of students with learning disabilities in a real-time approach that is not subject to inter-rater reliability, human observation, or reliant on a single mode of sensor input. In total, 2475 separate correlation tests were carried over 55 data points using Pearson’s correlation coefficient. Data points from the SDT, CPT outcomes measures, Multimodal Multisensor features, and participant characteristics were assessed longitudinally for cross-correlation significance. A strong positive correlation was found between participant ability to maintain sustained and selective attention in the CPT to their academic progress in school (d′), P < .01. Participants who showed more inhibition in tests had progressed further in their academic assessments P < .01. The Seek-X type CPT also showed specific physiological characteristics, including body movement range and eye-gaze that were significant in P scales such as ‘Reading’ and ‘Listening’ P < .05. We found that participant bias was overall liberal B″D < 0. Participants iii showed no significant bias change during the sessions, and we found no significant correlation between bias (B″D) and sensitivity (d′). Conclusion: An approach to labeling Multimodal Multisensor data to train machine-learning algorithms to track the attention of students with profound and multiple disabilities has been presented. We posit that this approach can overcome the variation in observer inter-rater reliability when using standardized scales in tracking the emotional expression of students with such profound disabilities. The accuracy of our approach increases with multiple modes of sensor input, and our method is robust to sensor occlusion and fall-out. Multiple sources of sensor input are provided, to accommodate a wide variety of users and their needs. Our model can reliably track the attention of students with profound disabilities, regardless of the sensors available. A system incorporating this model can help teachers design personalized interventions for a very heterogeneous group of students, where teachers cannot possibly attend to each of their individual needs. This approach could be used to identify those with the greatest learning challenges, to guarantee that all students are supported to reach their full potential. Keywords—Affective computing in education, affect detection, attention, continuous performance test, engagement, flow, HCI, interaction, learning disabilities, machine learning, multimodal, multisensor, physiological sensors, Signal Detection Theory, selective attention, sustained attention, student engagement

On accessible Visual Programming Tools for children with Autism Spectrum Condition and additional learning disabilities

Visual Programming Tools (VPTs) provide a visual programming and execution environment, in addition to other visual resources and tools appropriate for creating visual programs for a particular domain. Several VPTs have been created for teaching children to program at an early age. Research on the use of these tools to teach programming, academic and non-academic skills has reported positive results. However, children with learning disabilities including those also diagnosed with Autism Spectrum Condition (ASC) are left out of research in this area. Therefore, this research aims to contribute to existing knowledge in this area by exploring the accessibility of existing VPTs for this group of users and creating design tools and recommendations for the design of accessible VPTs for this target group. This research began with the evaluation of the accessibility of the most popular VPT, Scratch. A user evaluation was conducted with seven children with learning disabilities, five of them were also diagnosed with ASC; three special education needs teachers were also interviewed as part of the evaluation. Analysis of the findings from this evaluation showed that the children faced several difficulties while using Scratch to create stories; and also identified the causes of the difficulties. Accessibility heuristics were derived from the identified 'causes of difficulties' and were used to evaluate the accessibility of three additional VPTs. The findings of this second evaluation showed that the assessed VPTs have features similar to those of Scratch that caused accessibility difficulties to the target group. In creating tools and recommendations for designing accessible VPTs, the research focused on children with ASC (with learning disabilities) due to the match between their reported preferences and the features of VPTs. A method of creating personae to represent their requirements and goals was created and used to create three data-grounded personae. Experts were then interviewed to propose a set of recommendations for designing accessible VPTs for the target group. Therefore, this research contributed methods for conducting accessibility evaluation of VPTs for children with learning disabilities and for creating personae for children with ASC; a theoretical model for the use of VPTs by children with learning disabilities in a class setting to achieve a learning goal; findings on the accessibility of existing VPTs for children with learning disabilities; and recommendations for designing accessible VPTs for children with ASC