Bringing letters to life: handwriting with haptic-enabled tangible robots

In this paper, we present a robotic approach to improve the teaching of handwriting using the tangible, haptic-enabled and classroom-friendly Cellulo robots. Our efforts presented here are in line with the philosophy of the Cellulo platform: we aim to create a ready-to-use tool (i.e. a set of robot-assisted activities) to be used for teaching handwriting, one that is to coexist harmoniously with traditional tools and will contribute new added values to the learning process, complementing existing teaching practices. To maximize our potential contributions to this learning process, we focus on two promising aspects of handwriting: the visual perception and the visual-motor coordination. These two aspects enhance in particular two sides of the representation of letters in the mind of the learner: the shape of the letter (the grapheme) and the way it is drawn, namely the dynamics of the letter (the ductus). With these two aspects in mind, we do a detailed content analysis for the process of learning the representation of letters, which leads us to discriminate the specific skills involved in letter representation. We then compare our robotic method with traditional methods as well as with the combination of the two methods, in order to discover which of these skills can benefit from the use of Cellulo. As handwriting is taught from age 5, we conducted our experiments with 17 five-year-old children in a public school. Results show a clear potential of our robot-assisted learning activities, with a visible improvement in certain skills of handwriting, most notably in creating the ductus of the letters, discriminating a letter among others and in the average handwriting speed. Moreover, we show that the benefit of our learning activities to the handwriting process increases when it is used after traditional learning methods. These results lead to the initial insights into how such a tangible robotic learning technology may be used to create cost-effective collaborative scenarios for the learning of handwriting

Teacher's Perception on Social Robots to Promote the Integration of Children with Migration Background

This paper describes the first steps of an ongoing participatory design with teachers from Switzerland to co-create Human-Robot Interaction setups for integrating children with migration history. The herein presented phase had two main goals: (i) initially mapping the current issues and the teachers’ strategy when integrating these children, and (ii) understanding teachers’ perceptions regarding social robots for this goal. Results show that teachers we interviewed are already using technology to communicate with immigrant children, not necessarily for inclusion or promote socialisation with their peers, but simply to understand them. Findings also point to a well-defined application of social robots in inclusion activities, even when never seeing or using them, which contradicts previous results in the literature and which gives potential ways to unfold the next steps of the participatory design.QC 20231206</p

Design of Dynamic Tangible Workspaces for Games: Application on Robot-Assisted Upper Limb Rehabilitation

A key element for the success of any game is its ability to produce a different experience at each round, thus keeping the player engagement high. This is particularly important for those games that also have a serious objective, such as gamified rehabilitation systems, aiming at encouraging patients in performing home rehabilitation exercises. In all cases, a game element which is typically static is the workspace, i.e. the "floor" upon which the game takes place. This is especially true for robot-assisted rehabilitation games, where the workspace must satisfy the requirements given by the robot’s locomotion and localization systems, as well as the patient’s exercise motion requirements. In this article, we present a simple yet effective solution for designing dynamic and customizable tangible workspaces, which relies on hexagonal tiles and our previously proposed Cellulo localization system. These "hextiles" can be easily tangibly rearranged at each game round to yield a desired workspace shape and configuration, allowing tabletop mobile robots to move continuously within each new workspace. We ground our solution in the context of robot-assisted rehabilitation, where high adaptability is crucial for the efficacy of the solution, and propose a dynamic extension of our "tangible Pacman" rehabilitation game. Experiments show that the proposed solution allows for adaptation in range of motions, exercise types, physical and cognitive difficulty, besides reducing repetitiveness

Designing Configurable Arm Rehabilitation Games: How Do Different Game Elements Affect User Motion Trajectories?

For successful rehabilitation of a patient after a stroke or traumatic brain injury, it is crucial that rehabilitation activities are motivating, provide feedback and have a high rate of repetitions. Advancements in recent technologies provide solutions to address these aspects where needed. Additionally, through the use of gamification, we are able to increase the motivation for participants. However, many of these systems require complex set-ups, which can be a big challenge when conducting rehabilitation in a home-based setting. To address the lack of simple rehabilitation tools for arm function for a home-based application, we previously developed a system, Cellulo for rehabilitation, that is comprised of paper-supported tangible robots that are orchestrated by applications deployed on consumer tablets. These components enable different features that allow for gamification, easy setup, portability, and scalability. To support the configuration of game elements to patients’ level of motor skills and strategies, their motor trajectories need to be classified. In this paper, we investigate the classification of different motor trajectories and how game elements impact these in unimpaired, healthy participants. We show that the manipulation of certain game elements do have an impact on motor trajectories, which might indicate that it is possible to adapt the arm remediation of patients by configuring game elements. These results provide a first step towards providing adaptive rehabilitation based upon patients’ measured trajectories

Predictive Analysis of Errors During Robot-Mediated Gamified Training

This paper presents our approach to predicting future error-related events in a robot-mediated gamified phys- ical training activity for stroke patients. The ability to predict future error under such conditions suggests the existence of distinguishable features and separated class characteristics between the casual gameplay state and error prune state in the data. Identifying such features provides valuable insight to creating individually tailored, adaptive games as well as possible ways to increase rehabilitation success by patients. Considering the time-series nature of sensory data created by motor actions of patients we employed a predictive analysis strategy on carefully engineered features of sequenced data. We split the data into fixed time windows and explored logistic regression models, decision trees, and recurrent neural networks to predict the likelihood of a patient making an error based on the features from the time window before the error. We achieved an 84.4% F1-score with a 0.76 ROC value in our best model for predicting motion accuracy related errors. Moreover, we computed the permutation importance of the features to explain which ones are more indicative of future errors.CHIL

Towards an Adaptive Upper Limb Rehabilitation Game with Tangible Robots

A key feature of a successful game is its ability to provide the player with an adequate level of challenge. However, the objective of difficulty adaptation in serious games is not only to maintain the player’s motivation by challenging, but also to ensure the completion of training objectives. This paper describes our proposed upper-limb rehabilitation game with tangible robots and investigates the effect of game elements and gameplay on the amount of the performed motion in several planes and percentage of failure by using the data from 33 unimpaired subjects who played 53 games within two consecutive days. In order to provide a more generic adaptation strategy in the future, we discretize the game area to circular zones. We then show the effect of changing these zones during gameplay on the activation of different muscles through EMG data in a pilot study. The study shows that it is possible to increase the challenge level by adding more active agents chasing the player and increasing the speed of these agents. However, only the increase in number of agents significantly increases the users’ motion on both planes. Analysis of player behaviors leads us to suggest that by adapting the behaviour of these active agents in specific zones, it is possible to change the trajectory of the user, and to provide a focus on the activation of specific muscles