Augmented learning for sports using wearable head-worn and wrist-worn devices

Novices can learn sports in a variety of ways ranging from guidance from an instructor to watching video tutorials. In each case, subsequent and repeated self-directed practice sessions are an essential step. However, during such self-directed practice, constant guidance and feedback is absent. As a result, the novices do not know if they are making mistake or if there are any areas for improvement. In this position paper, we propose using wearable devices to augment such self-directed practice sessions by providing augmented guidance and feedback. In particular, a head-worn display can provide real-time guidance whilst wrist-worn devices can provide real-time tracking and monitoring of various states. We envision this approach being applied to various sports, and in particular this is suitable for sports that utilize precise hand motion such as snooker, billiards, golf, archery, cricket, tennis and table tennis.Postprin

Loudness affects motion: asymmetric volume of auditory feedback results in asymmetric gait in healthy young adults

Background: The potential of auditory feedback for motor learning in the rehabilitation of various diseases has become apparent in recent years. However, since the volume of auditory feedback has played a minor role so far and its influence has hardly been considered, we investigate the volume effect of auditory feedback on gait pattern and gait direction and its interaction with pitch. Methods: Thirty-two healthy young participants were randomly divided into two groups: Group 1 (n = 16) received a high pitch (150-250 Hz) auditory feedback; group 2 (n = 16) received a lower pitch (95-112 Hz) auditory feedback. The feedback consisted of a real-time sonification of the right and left foot ground contact. After an initial condition (no auditory feedback and full vision), both groups realized a 30-minute habituation period followed by a 30-minute asymmetry period. At any condition, the participants were asked to walk blindfolded and with auditory feedback towards a target at 15 m distance and were stopped 5 m before the target. Three different volume conditions were applied in random order during the habituation period: loud, normal, and quiet. In the subsequent asymmetry period, the three volume conditions baseline, right quiet and left quiet were applied in random order. Results: In the habituation phase, the step width from the loud to the quiet condition showed a significant interaction of volume*pitch with a decrease at high pitch (group 1) and an increase at lower pitch (group 2) (group 1: loud 1.02 ± 0.310, quiet 0.98 ± 0.301; group 2: loud 0.95 ± 0.229, quiet 1.11 ± 0.298). In the asymmetry period, a significantly increased ground contact time on the side with reduced volume could be found (right quiet: left foot 0.988 ± 0.033, right foot 1.003 ± 0.040, left quiet: left foot 1.004 ± 0.036, right foot 1.002 ± 0.033). Conclusions: Our results suggest that modifying the volume of auditory feedback can be an effective way to improve gait symmetry. This could facilitate gait therapy and rehabilitation of hemiparetic and arthroplasty patients, in particular if gait improvement based on verbal corrections and conscious motor control is limited. © 2022, The Author(s)

Analysis and enhancement of interpersonal coordination using inertial measurement unit solutions

Die heutigen mobilen Kommunikationstechnologien haben den Umfang der verbalen und textbasierten Kommunikation mit anderen Menschen, sozialen Robotern und künstlicher Intelligenz erhöht. Auf der anderen Seite reduzieren diese Technologien die nonverbale und die direkte persönliche Kommunikation, was zu einer gesellschaftlichen Thematik geworden ist, weil die Verringerung der direkten persönlichen Interaktionen eine angemessene Wahrnehmung sozialer und umgebungsbedingter Reizmuster erschweren und die Entwicklung allgemeiner sozialer Fähigkeiten bremsen könnte. Wissenschaftler haben aktuell die Bedeutung nonverbaler zwischenmenschlicher Aktivitäten als soziale Fähigkeiten untersucht, indem sie menschliche Verhaltensmuster in Zusammenhang mit den jeweilgen neurophysiologischen Aktivierungsmustern analzsiert haben. Solche Querschnittsansätze werden auch im Forschungsprojekt der Europäischen Union "Socializing sensori-motor contingencies" (socSMCs) verfolgt, das darauf abzielt, die Leistungsfähigkeit sozialer Roboter zu verbessern und Autismus-Spektrumsstörungen (ASD) adäquat zu behandeln. In diesem Zusammenhang ist die Modellierung und das Benchmarking des Sozialverhaltens gesunder Menschen eine Grundlage für theorieorientierte und experimentelle Studien zum weiterführenden Verständnis und zur Unterstützung interpersoneller Koordination. In diesem Zusammenhang wurden zwei verschiedene empirische Kategorien in Abhängigkeit von der Entfernung der Interagierenden zueinander vorgeschlagen: distale vs. proximale Interaktionssettings, da sich die Struktur der beteiligten kognitiven Systeme zwischen den Kategorien ändert und sich die Ebene der erwachsenden socSMCs verschiebt. Da diese Dissertation im Rahmen des socSMCs-Projekts entstanden ist, wurden Interaktionssettings für beide Kategorien (distal und proximal) entwickelt. Zudem wurden Ein-Sensor-Lösungen zur Reduzierung des Messaufwands (und auch der Kosten) entwickelt, um eine Messung ausgesuchter Verhaltensparameter bei einer Vielzahl von Menschen und sozialen Interaktionen zu ermöglichen. Zunächst wurden Algorithmen für eine kopfgetragene Trägheitsmesseinheit (H-IMU) zur Messung der menschlichen Kinematik als eine Ein-Sensor-Lösung entwickelt. Die Ergebnisse bestätigten, dass die H-IMU die eigenen Gangparameter unabhängig voneinander allein auf Basis der Kopfkinematik messen kann. Zweitens wurden—als ein distales socSMC-Setting—die interpersonellen Kopplungen mit einem Bezug auf drei interagierende Merkmale von „Übereinstimmung“ (engl.: rapport) behandelt: Positivität, gegenseitige Aufmerksamkeit und Koordination. Die H-IMUs überwachten bestimmte soziale Verhaltensereignisse, die sich auf die Kinematik der Kopforientierung und Oszillation während des Gehens und Sprechens stützen, so dass der Grad der Übereinstimmung geschätzt werden konnte. Schließlich belegten die Ergebnisse einer experimentellen Studie, die zu einer kollaborativen Aufgabe mit der entwickelten IMU-basierten Tablet-Anwendung durchgeführt wurde, unterschiedliche Wirkungen verschiedener audio-motorischer Feedbackformen für eine Unterstützung der interpersonellen Koordination in der Kategorie proximaler sensomotorischer Kontingenzen. Diese Dissertation hat einen intensiven interdisziplinären Charakter: Technologische Anforderungen in den Bereichen der Sensortechnologie und der Softwareentwicklung mussten in direktem Bezug auf vordefinierte verhaltenswissenschaftliche Fragestellungen entwickelt und angewendet bzw. gelöst werden—und dies in zwei unterschiedlichen Domänen (distal, proximal). Der gegebene Bezugsrahmen wurde als eine große Herausforderung bei der Entwicklung der beschriebenen Methoden und Settings wahrgenommen. Die vorgeschlagenen IMU-basierten Lösungen könnten dank der weit verbreiteten IMU-basierten mobilen Geräte zukünftig in verschiedene Anwendungen perspektiv reich integriert werden.Today’s mobile communication technologies have increased verbal and text-based communication with other humans, social robots and intelligent virtual assistants. On the other hand, the technologies reduce face-to-face communication. This social issue is critical because decreasing direct interactions may cause difficulty in reading social and environmental cues, thereby impeding the development of overall social skills. Recently, scientists have studied the importance of nonverbal interpersonal activities to social skills, by measuring human behavioral and neurophysiological patterns. These interdisciplinary approaches are in line with the European Union research project, “Socializing sensorimotor contingencies” (socSMCs), which aims to improve the capability of social robots and properly deal with autism spectrum disorder (ASD). Therefore, modelling and benchmarking healthy humans’ social behavior are fundamental to establish a foundation for research on emergence and enhancement of interpersonal coordination. In this research project, two different experimental settings were categorized depending on interactants’ distance: distal and proximal settings, where the structure of engaged cognitive systems changes, and the level of socSMCs differs. As a part of the project, this dissertation work referred to this spatial framework. Additionally, single-sensor solutions were developed to reduce costs and efforts in measuring human behaviors, recognizing the social behaviors, and enhancing interpersonal coordination. First of all, algorithms using a head worn inertial measurement unit (H-IMU) were developed to measure human kinematics, as a baseline for social behaviors. The results confirmed that the H-IMU can measure individual gait parameters by analyzing only head kinematics. Secondly, as a distal sensorimotor contingency, interpersonal relationship was considered with respect to a dynamic structure of three interacting components: positivity, mutual attentiveness, and coordination. The H-IMUs monitored the social behavioral events relying on kinematics of the head orientation and oscillation during walk and talk, which can contribute to estimate the level of rapport. Finally, in a new collaborative task with the proposed IMU-based tablet application, results verified effects of different auditory-motor feedbacks on the enhancement of interpersonal coordination in a proximal setting. This dissertation has an intensive interdisciplinary character: Technological development, in the areas of sensor and software engineering, was required to apply to or solve issues in direct relation to predefined behavioral scientific questions in two different settings (distal and proximal). The given frame served as a reference in the development of the methods and settings in this dissertation. The proposed IMU-based solutions are also promising for various future applications due to widespread wearable devices with IMUs.European Commission/HORIZON2020-FETPROACT-2014/641321/E

Gait sonification for rehabilitation: adjusting gait patterns by acoustic transformation of kinematic data

To enhance motor learning in both sport and rehabilitation, auditory feedback has emerged as an effective tool. Since it requires less attention than visual feedback and hardly affects the visually dominated orientation in space, it can be used safely and effectively in natural locomotion such as walking. One method for generating acoustic movement feedback is the direct mapping of kinematic data to sound (movement sonification). Using this method in orthopedic gait rehabilitation could make an important contribution to the prevention of falls and secondary diseases. This would not only reduce the individual suffering of the patients, but also medical treatment costs. To determine the possible applications of movement sonification in gait rehabilitation in the context of this work, a new gait sonification method based on inertial sensor technology was developed. Against the background of current scientific findings on sensorimotor function, feedback methods, and gait analysis, three studies published in scientific journals are presented in this thesis: The first study shows the applicability and acceptance of the feedback method in patients undergoing inpatient rehabilitation after unilateral total hip arthroplasty. In addition, the direct effect of gait sonification during ten gait training sessions on the patients’ gait pattern was revealed. In the second study, the immediate follow-up effect of gait sonification on the kinematics of the same patient group is examined at four measurement points after gait training. In this context, a significant influence of sonification on the gait pattern of the patients was shown, which, however, did not meet the previously expected effects. In view of this finding, the effect of the specific sound parameter loudness of gait sonification on the gait of healthy persons was analyzed in a third study. Thus, an impact of asymmetric loudness of gait sonification on the ground contact time could be detected. Considering this cause-effect relationship can be a component in improving gait sonfication in rehabilitation. Overall, the feasibility and effectiveness of movement sonification in gait rehabilitation of patients after unilateral hip arthroplasty becomes evident. The findings thus illustrate the potential of the method to efficiently support orthopedic gait rehabilitation in the future. On the basis of the results presented, this potential can be exploited in particular by an adequate mapping of movement to sound, a systematic modification of selected sound parameters, and a target-group-specific selection of the gait sonification mode. In addition to a detailed investigation of the three factors mentioned above, an optimization and refinement of gait analysis in patients after arthroplasty using inertial sensor technology will be beneficial in the future.Akustisches Feedback kann wirkungsvoll eingesetzt werden, um das Bewegungslernen sowohl im Sport als auch in der Rehabilitation zu erleichtern. Da es weniger Aufmerksamkeit als visuelles Feedback erfordert und die visuell dominierte Orientierung im Raum kaum beeinträchtigt, kann es während einer natürlichen Fortbewegung wie dem Gehen sicher und effektiv genutzt werden. Eine Methode zur Generierung akustischen Bewegungsfeedbacks ist die direkte Abbildung kinematischer Daten auf Sound (Bewegungssonifikation). Ein Einsatz dieser Methode in der orthopädischen Gangrehabilitation könnte einen wichtigen Beitrag zur Prävention von Stürzen und Folgeerkrankungen leisten. Neben dem individuellen Leid der Patienten ließen sich so auch medizinische Behandlungskosten erheblich reduzieren. Um im Rahmen dieser Arbeit die Einsatzmöglichkeiten der Bewegungssonifikation in der Gangrehabilitation zu bestimmen, wurde eine neue Gangsonifikationsmethodik auf Basis von Inertialsensorik entwickelt. Zu der entwickelten Methodik werden, vor dem Hintergrund aktueller wissenschaftlicher Erkenntnisse zur Sensomotorik, zu Feedbackmethoden und zur Ganganalyse, in dieser Thesis drei in Fachzeitschriften publizierte Studien vorgestellt. Die erste Studie beschreibt die Anwendbarkeit und Akzeptanz der Feedbackmethode bei Patienten in stationärer Rehabilitation nach unilateraler Hüftendoprothetik. Darüber hinaus wird der direkte Effekt der Gangsonifikation während eines zehnmaligen Gangtrainings auf das Gangmuster der Patienten deutlich. In der zweiten Studie wird der unmittelbare Nacheffekt der Gangsonifikation auf die Kinematik der gleichen Patientengruppe zu vier Messzeitpunkten nach dem Gangtraining untersucht. In diesem Zusammenhang zeigte sich ein signifikanter Einfluss der Sonifikation auf das Gangbild der Patienten, der allerdings nicht den zuvor erwarteten Effekten entsprach. Aufgrund dieses Ergebnisses wurde in einer dritten Studie die Wirkung des spezifischen Klangparameters Lautstärke der Gangsonifikation auf das Gangbild von gesunden Personen analysiert. Dabei konnte ein Einfluss von asymmetrischer Lautstärke der Gangsonifikation auf die Bodenkontaktzeit nachgewiesen werden. Die Berücksichtigung dieses Ursache-Wirkungs-Zusammenhangs kann einen Baustein bei der Verbesserung der Gangsonifikation in der Rehabilitation darstellen. Insgesamt wird die Anwendbarkeit und Wirksamkeit von Bewegungssonifikation in der Gangrehabilitation bei Patienten nach unilateraler Hüftendoprothetik evident. Die gewonnenen Erkenntnisse verdeutlichen das Potential der Methode, die orthopädische Gangrehabilitation zukünftig effizient zu unterstützen. Ausschöpfen lässt sich dieses Potential auf Grundlage der vorgestellten Ergebnisse insbesondere anhand einer adäquaten Zuordnung von Bewegung zu Sound, einer systematischen Modifikation ausgewählter Soundparameter sowie einer zielgruppenspezifischen Wahl des Modus der Sonifikation. Neben einer differenzierten Untersuchung der genannten Faktoren, erscheint zukünftig eine Optimierung und Verfeinerung der Ganganalyse bei Patienten nach Endoprothetik unter Einsatz von Inertialsensorik notwendig

Auditory space representation: from perception to action

de Castro Campos Velten M. Auditory space representation: from perception to action. Bielefeld: Universitätsbibliothek Bielefeld; 2014

Recent Advances in Motion Analysis

The advances in the technology and methodology for human movement capture and analysis over the last decade have been remarkable. Besides acknowledged approaches for kinematic, dynamic, and electromyographic (EMG) analysis carried out in the laboratory, more recently developed devices, such as wearables, inertial measurement units, ambient sensors, and cameras or depth sensors, have been adopted on a wide scale. Furthermore, computational intelligence (CI) methods, such as artificial neural networks, have recently emerged as promising tools for the development and application of intelligent systems in motion analysis. Thus, the synergy of classic instrumentation and novel smart devices and techniques has created unique capabilities in the continuous monitoring of motor behaviors in different fields, such as clinics, sports, and ergonomics. However, real-time sensing, signal processing, human activity recognition, and characterization and interpretation of motion metrics and behaviors from sensor data still representing a challenging problem not only in laboratories but also at home and in the community. This book addresses open research issues related to the improvement of classic approaches and the development of novel technologies and techniques in the domain of motion analysis in all the various fields of application

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Ultra-high-speed imaging of bubbles interacting with cells and tissue

Ultrasound contrast microbubbles are exploited in molecular imaging, where bubbles are directed to target cells and where their high-scattering cross section to ultrasound allows for the detection of pathologies at a molecular level. In therapeutic applications vibrating bubbles close to cells may alter the permeability of cell membranes, and these systems are therefore highly interesting for drug and gene delivery applications using ultrasound. In a more extreme regime bubbles are driven through shock waves to sonoporate or kill cells through intense stresses or jets following inertial bubble collapse. Here, we elucidate some of the underlying mechanisms using the 25-Mfps camera Brandaris128, resolving the bubble dynamics and its interactions with cells. We quantify acoustic microstreaming around oscillating bubbles close to rigid walls and evaluate the shear stresses on nonadherent cells. In a study on the fluid dynamical interaction of cavitation bubbles with adherent cells, we find that the nonspherical collapse of bubbles is responsible for cell detachment. We also visualized the dynamics of vibrating microbubbles in contact with endothelial cells followed by fluorescent imaging of the transport of propidium iodide, used as a membrane integrity probe, into these cells showing a direct correlation between cell deformation and cell membrane permeability