Haptic guidance improves the visuo-manual tracking of trajectories

BACKGROUND: Learning to perform new movements is usually achieved by following visual demonstrations. Haptic guidance by a force feedback device is a recent and original technology which provides additional proprioceptive cues during visuo-motor learning tasks. The effects of two types of haptic guidances-control in position (HGP) or in force (HGF)-on visuo-manual tracking ("following") of trajectories are still under debate. METHODOLOGY/PRINCIPALS FINDINGS: Three training techniques of haptic guidance (HGP, HGF or control condition, NHG, without haptic guidance) were evaluated in two experiments. Movements produced by adults were assessed in terms of shapes (dynamic time warping) and kinematics criteria (number of velocity peaks and mean velocity) before and after the training sessions. CONCLUSION/SIGNIFICANCE: These results show that the addition of haptic information, probably encoded in force coordinates, play a crucial role on the visuo-manual tracking of new trajectories

Dysgraphia detection based on convolutional neural networks and child-robot interaction

Dysgraphia is a disorder of expression with the writing of letters, words, and numbers. Dysgraphia is one of the learning disabilities attributed to the educational sector, which has a strong impact on the academic, motor, and emotional aspects of the individual. The purpose of this study is to identify dysgraphia in children by creating an engaging robot-mediated activity, to collect a new dataset of Latin digits written exclusively by children aged 6 to 12 years. An interactive scenario that explains and demonstrates the steps involved in handwriting digits is created using the verbal and non-verbal behaviors of the social humanoid robot Nao. Therefore, we have collected a dataset that contains 11,347 characters written by 174 participants with and without dysgraphia. And through the advent of deep learning technologies and their success in various fields, we have developed an approach based on these methods. The proposed approach was tested on the generated database. We performed a classification with a convolutional neural network (CNN) to identify dysgraphia in children. The results show that the performance of our model is promising, reaching an accuracy of 91%

Use of Automatic Chinese Character Decomposition and Human Gestures for Chinese Calligraphy Robots

Conventional Chinese calligraphy robots often suffer from the limited sizes of predefined font databases, which prevent the robots from writing new characters. This paper presents a robotic handwriting system to address such limitations, which extracts Chinese characters from textbooks and uses a robot’s manipulator to write the characters in a different style. The key technologies of the proposed approach include the following: (1) automatically decomposing Chinese characters into strokes using Harris corner detection technology and (2) matching the decomposed strokes to robotic writing trajectories learned from human gestures. Briefly, the system first decomposes a given Chinese character into a set of strokes and obtains the stroke trajectory writing ability by following the gestures performed by a human demonstrator. Then, it applies a stroke classification method that recognizes the decomposed strokes as robotic writing trajectories. Finally, the robot arm is driven to follow the trajectories and thus write the Chinese character. Seven common Chinese characters have been used in an experiment for system validation and evaluation. The experimental results demonstrate the power of the proposed system, given that the robot successfully wrote all the testing characters in the given Chinese calligraphic style

Integration of an actor-critic model and generative adversarial networks for a Chinese calligraphy robot

As a combination of robotic motion planning and Chinese calligraphy culture, robotic calligraphy plays a significant role in the inheritance and education of Chinese calligraphy culture. Most existing calligraphy robots focus on enabling the robots to learn writing through human participation, such as human–robot interactions and manually designed evaluation functions. However, because of the subjectivity of art aesthetics, these existing methods require a large amount of implementation work from human engineers. In addition, the written results cannot be accurately evaluated. To overcome these limitations, in this paper, we propose a robotic calligraphy model that combines a generative adversarial network (GAN) and deep reinforcement learning to enable a calligraphy robot to learn to write Chinese character strokes directly from images captured from Chinese calligraphic textbooks. In our proposed model, to automatically establish an aesthetic evaluation system for Chinese calligraphy, a GAN is first trained to understand and reconstruct stroke images. Then, the discriminator network is independently extracted from the trained GAN and embedded into a variant of the reinforcement learning method, the “actor-critic model”, as a reward function. Thus, a calligraphy robot adopts the improved actor-critic model to learn to write multiple character strokes. The experimental results demonstrate that the proposed model successfully allows a calligraphy robot to write Chinese character strokes based on input stroke images. The performance of our model, compared with the state-of-the-art deep reinforcement learning method, shows the efficacy of the combination approach. In addition, the key technology in this work shows promise as a solution for robotic autonomous assembly

Robot-assisted intermanual transfer of handwriting skills

n/

Handwriting Correction System using Wearable Sleeve with Optimal Tactor Configuration

Handwriting remains an elusive skill with practice worksheets being the common method of learning. Since these worksheets provide only visual feedback and no quantitative feedback, it can often be a challenge to improve. For children with learning disabilities, learning handwriting skills is one of the most difficult tasks. We propose a handwriting training system that uses off-the-shelf webcam, a pen tracking software and a haptic sleeve which provides active feedback to the user based on their deviation from the original pattern. The sleeve has 4 individual motors that vibrate at different intensities based on the direction (right, left, up or down) and severity of the deviation (\u3c 1cm, 1cm – 3cm, \u3e 3cm). Different motor placements around the forearm are evaluated for vibro-tactile feedback accuracy and time response, and a novel spaced-ring configuration is proposed. This paper provides details on the system architecture and sleeve characterization, and the results show promise in utilizing the system for self-correction and visual-motor skills development. The results from sleeve characterization suggest the applicability of the spaced-ring configuration (perceived feedback accuracy \u3e 98%, time response \u3c 1s) in other vibrotactile hand guidance systems, in addition to handwriting correction. Recommendations on tactor placements around the forearm are provided

Basic and supplementary sensory feedback in handwriting

International audienceThe mastering of handwriting is so essential in our society that it is important to try to find new methods for facilitating its learning and rehabilitation. The ability to control the graphic movements clearly impacts on the quality of the writing. This control allows both the programming of letter formation before movement execution and the online adjustments during execution, thanks to diverse sensory feedback (FB). New technologies improve existing techniques or enable new methods to supply the writer with real-time computer-assisted FB. The possibilities are numerous and various. Therefore, two main questions arise: (1) What aspect of the movement is concerned and (2) How can we best inform the writer to help them correct their handwriting? In a first step, we report studies on FB naturally used by the writer. The purpose is to determine which information is carried by each sensory modality, how it is used in handwriting control and how this control changes with practice and learning. In a second step, we report studies on supplementary FB provided to the writer to help them to better control and learn how to write. We suggest that, depending on their contents, certain sensory modalities will be more appropriate than others to assist handwriting motor control. We emphasize particularly the relevance of auditory modality as online supplementary FB on handwriting movements. Using real-time supplementary FB to assist in the handwriting process is probably destined for a brilliant future with the growing availability and rapid development of tablets

SoundScript - Supporting the acquisition of character writing by multisensory integration

This work is introducing a new movement sonification method called 'SoundScript' to support the acquisition of character writing by children. SoundScript creates 'sound traces' from the writing trace in real-time during the process of handwriting. The structural correlation of both - optic and acoustic - traces leads to an integrated audio-visual perception of writing with the expected stimulation of multisensory integration sites of the CNS. Data of a pilot study are introduced indicating that the writing kinematics is reproduced more adequately if additional sound traces are available during writing. In the future SoundScript shall be applied to verify if the establishment of internal character representations can be accelerated, if the conciseness of the specific shape of the particular characters can be made stronger and if thereby the efficiency of the handwriting learning process can be enhanced

Towards Modelling Trust in Voice at Zero Acquaintance

Trust is essential in many human relationships, especially where there is an element of inter-dependency. However, humans tend to make quick judgements about trusting other individuals, even those met at zero acquaintance. Past studies have shown the significance of voice in perceived trustworthiness, but research associating trustworthiness and different vocal features such as speech rate and fundamental frequency (f0) has yet to yield consistent results. Therefore, this paper proposes a method to investigate 1) the association between trustworthiness and different vocal features, 2) the vocal characteristics that Malaysian ethnic groups base their judgement of trustworthiness on and 3) building a neural network model that predicts the degree of trustworthiness in a human voice. In the method proposed, a reliable set of audio clips will be obtained and analyzed with SoundGen to determine the acoustical characteristics. Then the audio clips will be distributed to a large group of untrained respondents to rate their degree of trust in the speakers of each audio clip. The participants will be able to choose from 30 sets of audio clips which will consist of 6 audio clips each. The acoustic characteristics will be analyzed and com-pared with the ratings to determine if there are any correlations between the acoustic characteristic and the trustworthiness ratings. After that, a neural network model will be built based on the collected data. The neural network model will be able to predict the trustworthiness of a person’s voice. Keywords—prosody, trust, voice, vocal cues, zero acquaintance

Scalability of the Size of Patterns Drawn Using Tactile Hand Guidance

Haptic feedback for handwriting training has been extensively studied, but with primary focus on kinematic feedback. We provide vibrotactile feedback through a wrist worn sleeve to guide the user to recreate unknown patterns and study the impact of vibrational duration (1, 2, 3 seconds) on pattern scaling. User traces a line at 90° angles, while attempting to maintain a constant speed, in the direction of the motor activated till a different motor activation is perceived. Shape and size are two features of good letter formation. Study performed on three subjects showed the ability to utilize four vibrotactile motors to guide the hand towards correct shape formation with high accuracy (\u3e 95%). The overall size of the letter was observed to scale linearly with the vibrational duration. Implications for utilizing the vibrational feedback for handwriting correction are discussed