Control Interface for Hands-free Navigation of Standing Mobility Vehicles based on Upper-Body Natural Movements

In this paper, we propose and evaluate a novel human-machine interface (HMI) for controlling a standing mobility vehicle or person carrier robot, aiming for a hands-free control through upper-body natural postures derived from gaze tracking while walking. We target users with lower-body impairment with remaining upper-body motion capabilities. The developed HMI bases on a sensing array for capturing body postures; an intent recognition algorithm for continuous mapping of body motions to robot control space; and a personalizing system for multiple body sizes and shapes. We performed two user studies: first, an analysis of the required body muscles involved in navigating with the proposed control; and second, an assessment of the HMI compared with a standard joystick through quantitative and qualitative metrics in a narrow circuit task. We concluded that the main user control contribution comes from Rectus Abdominis and Erector Spinae muscle groups at different levels. Finally, the comparative study showed that a joystick still outperforms the proposed HMI in usability perceptions and controllability metrics, however, the smoothness of user control was similar in jerk and fluency. Moreover, users' perceptions showed that hands-free control made it more anthropomorphic, animated, and even safer.Comment: 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other use

Development of a rehabilitation assist robot for stroke paretic patients based on the physiology of initiation of locomotion

科学研究費助成事業(学術研究助成基金助成金)研究成果報告書：若手研究(B)2011-2012課題番号：2370066

Voluntary initiation of movement: multifunctional integration of subjective agency

金沢大学国際基幹教育部This paper investigates subjective agency (SA) as a special type of efficacious action consciousness. Our central claims are, firstly, that SA is a conscious act of voluntarily initiating bodily motion. Secondly, we argue that SA is a case of multifunctional integration of behavioral functions being analogous to multisensory integration of sensory modalities. This is based on new perspectives on the initiation of action opened up by recent advancements in robot assisted neuro-rehabilitation which depends on the active participation of the patient and yields experimental evidence that there is SA in terms of a conscious act of voluntarily initiating bodily motion (phenomenal performance). Conventionally, action consciousness has been considered as a sense of agency (SoA). According to this view, the conscious subject merely echoes motor performance and does not cause bodily motion. Depending on sensory input, SoA is implemented by means of unifunctional integration (binding) and inevitably results in non-efficacious action consciousness. In contrast, SA comes as a phenomenal performance which causes motion and builds on multifunctional integration. Therefore, the common conception of the brain should be shifted toward multifunctional integration in order to allow for efficacious action consciousness. For this purpose, we suggest the heterarchic principle of asymmetric reciprocity and neural operators underlying SA. The general idea is that multifunctional integration allows conscious acts to be simultaneously implemented with motor behavior so that the resulting behavior (SA) comes as efficacious action consciousness. Regarding the neural implementation, multifunctional integration rather relies on operators than on modular functions. A robotic case study and possible experimental setups with testable hypotheses building on SA are presented

Personal Mobility With Synchronous Trunk-Knee Passive Exoskeleton: Optimizing Human-Robot Energy Transfer

We present a personal mobility device for lower-body impaired users through a light-weighted exoskeleton on wheels. On its core, a novel passive exoskeleton provides postural transition leveraging natural body postures with support to the trunk on sit-to-stand and stand-to-sit (STS) transitions by a single gas spring as an energy storage unit. We propose a direction-dependent coupling of knees and hip joints through a double-pulley wire system, transferring energy from the torso motion towards balancing the moment load at the knee joint actuator. Herewith, the exoskeleton maximizes energy transfer and the naturalness of the user's movement. We introduce an embodied user interface for hands-free navigation through a torso pressure sensing with minimal trunk rotations, resulting on average $19^{\circ} \pm 13^{\circ}$ on six unimpaired users. We evaluated the design for STS assistance on 11 unimpaired users observing motions and muscle activity during the transitions. Results comparing assisted and unassisted STS transitions validated a significant reduction (up to $68\%$ $p<0.01$) at the involved muscle groups. Moreover, we showed it feasible through natural torso leaning movements of $+12^{\circ}\pm 6.5^{\circ}$ and $- 13.7^{\circ} \pm 6.1^{\circ}$ for standing and sitting, respectively. Passive postural transition assistance warrants further work on increasing its applicability and broadening the user population.Comment: IEEE/ASME Transactions on Mechatronics. 2022. 11 pages. doi: 10.1109/TMECH.2021.313545

Two-stage video-based convolutional neural networks for adult spinal deformity classification

IntroductionAssessment of human gait posture can be clinically effective in diagnosing human gait deformities early in life. Currently, two methods—static and dynamic—are used to diagnose adult spinal deformity (ASD) and other spinal disorders. Full-spine lateral standing radiographs are used in the standard static method. However, this is a static assessment of joints in the standing position and does not include information on joint changes when the patient walks. Careful observation of long-distance walking can provide a dynamic assessment that reveals an uncompensated posture; however, this increases the workload of medical practitioners. A three-dimensional (3D) motion system is proposed for the dynamic method. Although the motion system successfully detected dynamic posture changes, access to the facilities was limited. Therefore, a diagnostic approach that is facility-independent, has low practice flow, and does not involve patient contact is required.MethodsWe focused on a video-based method to classify patients with spinal disorders either as ASD, or other forms of ASD. To achieve this goal, we present a video-based two-stage machine-learning method. In the first stage, deep learning methods are used to locate the patient and extract the area where the patient is located. In the second stage, a 3D CNN (convolutional neural network) device is used to capture spatial and temporal information (dynamic motion) from the extracted frames. Disease classification is performed by discerning posture and gait from the extracted frames. Model performance was assessed using the mean accuracy, F1 score, and area under the receiver operating characteristic curve (AUROC), with five-fold cross-validation. We also compared the final results with professional observations.ResultsOur experiments were conducted using a gait video dataset comprising 81 patients. The experimental results indicated that our method is effective for classifying ASD and other spinal disorders. The proposed method achieved a mean accuracy of 0.7553, an F1 score of 0.7063, and an AUROC score of 0.7864. Additionally, ablation experiments indicated the importance of the first stage (detection stage) and transfer learning of our proposed method.DiscussionThe observations from the two doctors were compared using the proposed method. The mean accuracies observed by the two doctors were 0.4815 and 0.5247, with AUROC scores of 0.5185 and 0.5463, respectively. We proved that the proposed method can achieve accurate and reliable medical testing results compared with doctors' observations using videos of 1 s duration. All our code, models, and results are available at https://github.com/ChenKaiXuSan/Walk_Video_PyTorch. The proposed framework provides a potential video-based method for improving the clinical diagnosis for ASD and non-ASD. This framework might, in turn, benefit both patients and clinicians to treat the disease quickly and directly and further reduce facility dependency and data-driven systems

Bodily Expression Support for Creative Dance Education by Grasping-Type Musical Interface with Embedded Motion and Grasp Sensors

Dance has been made mandatory as one of the physical education courses in Japan because it can cultivate capacities for expression and communication. Among several types of dance education, creative dance especially contributes to the cultivation of these capacities. However, creative dance requires some level of particular skills, as well as creativity, and it is difficult to presuppose these pre-requisites in beginner-level dancers without experience. We propose a novel supporting device for dance beginners to encourage creative dance performance by continuously generating musical sounds in real-time in accordance with their bodily movements. It has embedded sensors developed for this purpose. Experiments to evaluate the effectiveness of the device were conducted with ten beginner-level dancers. Using the proposed device, the subjects demonstrated enhanced creative dance movements with greater variety, evaluated in terms of Laban dance movement description. Also, using the device, they performed with better accuracy and repeatability in a task where they produced an imagined circular trajectory by hand. The proposed interface is effective in terms of creative dance activity and accuracy of motion generation for beginner-level dancers

Voluntary Elbow Extension-Flexion Using Single Joint Hybrid Assistive Limb (HAL) for Patients of Spastic Cerebral Palsy: Two Cases Report

Cerebral palsy (CP) patients with spastic diplegia struggle to perform activities of daily life (ADL) using their upper arms. The single-joint-type Hybrid Assistive limb (HAL) for upper limbs is a new portable robot that can provide elbow motion support in accordance with bioelectric activation of patient's biceps and triceps brachii muscles. The purpose of this study is to assess the feasibility and efficacy of the use of HAL for CP patients. Two patients were enrolled in this study. (Case 1: a 19-years-old male, at the Gross Motor Function Classification System (GMFCS) level IV, Case 2: a 17-years-old male at GMFCS level III). Both these patients experienced difficulty in voluntary elbow extension in ADLs. The HAL intervention (eight sessions; voluntary extension-flexion training of the elbow with HAL and clinical evaluation) was conducted for both sides in Case 1 and for the right side in Case 2. Clinical assessments were conducted as follows: Surface electromyography was used to evaluate the muscle activities of the biceps, triceps brachii, trapezius, and pectoralis major during elbow extension-flexion. The voluntary extension-flexion angles of the elbow, the coactivation index of the biceps and triceps brachii muscles, synergy analysis, and the Action Research Arm Test (ARAT) scores were assessed before and after the HAL sessions; the FIM score was evaluated before and after the entire intervention. In Case 1, the voluntary extension angle tended to increase after the HAL sessions. In both cases, the ARAT scores improved after the sessions. The FIM scores improved after HAL intervention. The voluntary extension-flexion of the elbow using the HAL may be a feasible option for rehabilitation of CP patients

Robot-assisted voluntary initiation reduces control-related difficulties of initiating joint movement: A phenomenal questionnaire study on shaping and compensation of forward gait

金沢大学国際基幹教育部Humans employ various control strategies to initiate and maintain bodily movement. In case that the normal gait function is impaired, exoskeleton robots provide motor assistance during therapy. While the robotic control system builds on kinematic gait functions, the patient’s voluntary efforts to initiate motion also contribute to the effectiveness of the therapy process. However, it is currently not well understood how voluntary initiation as a subjective capacity affects the physiological level of motor control. In order to understand the functional nexus between voluntary initiation and motor control, we interviewed patients undergoing robotic gait rehabilitation with the HAL exoskeleton robot about their experience and command of voluntarily initiating forward gait while using the HAL system. Their reports provide phenomenal evidence for voluntary initiation as a distinct cognitive act that comes as phenomenal performance. Furthermore, phenomenal evidence about the functional relation of intention and initiation correlates with FIM-M gait scores. Based on the assumption that HAL reduces control-related difficulties of voluntarily initiating joint movement, we identified two cognitive control strategies, shaping and compensation of gait, that imply a heterarchic organization of the human system of action control. © 2018 Grüneberg et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Lateral Symmetry of Synergies in Lower Limb Muscles of Acute Post-stroke Patients After Robotic Intervention

Gait disturbance is commonly associated with stroke, which is a serious neurological disease. With current technology, various exoskeletons have been developed to provide therapy, leading to many studies evaluating the use of such exoskeletons as an intervention tool. Although these studies report improvements in patients who had undergone robotic intervention, they are usually reported with clinical assessment, which are unable to characterize how muscle activations change in patients after robotic intervention. We believe that muscle activations can provide an objective view on gait performance of patients. To quantify improvement of lateral symmetry before and after robotic intervention, muscle synergy analysis with Non-Negative Matrix Factorization was used to evaluate patients\u27 EMG data. Eight stroke patients in their acute phase were evaluated before and after a course of robotic intervention with the Hybrid Assistive Limb (HAL), lasting over 3 weeks. We found a significant increase in similarity between lateral synergies of patients after robotic intervention. This is associated with significant improvements in gait measures like walking speed, step cadence, stance duration percentage of gait cycle. Clinical assessments [Functional Independence Measure-Locomotion (FIM-Locomotion), FIM-Motor (General), and Fugl-Meyer Assessment-Lower Extremity (FMA-LE)] showed significant improvements as well. Our study shows that muscle synergy analysis can be a good tool to quantify the change in neuromuscular coordination of lateral symmetry during walking in stroke patients

Reshaping of Bilateral Gait Coordination in Hemiparetic Stroke Patients After Early Robotic Intervention

Hemiparetic gait is a common condition after stroke which alters importantly the quality of life of stroke survivors. In recent years, several robotic interventions have been developed to support and enhance rehabilitation strategies for such population. The Hybrid Assistive Limb® (HAL) robot suit is a unique device able to collect in real time bioelectric signals from the patient to support and enhance voluntary gait. HAL has been used before in early stages of stroke showing gait improvement after the intervention. However, evaluation of the coordination of gait has not been done yet. Coordination is a key factor for an adequate gait performance; consequently, its changes may be closely related to gait recovery. In this study, we used planar covariation to evaluate coordination changes in hemiparetic stroke patients after early HAL intervention. Before starting, impaired intersegmental coordination for the paretic and non-paretic side was evident. HAL intervention was able to induce recovery of the covariation loop shape and deviation from the covariation plane improving intersegmental coordination. Also, there was a tendency of recovery for movement range evidenced by comparison of peak elevation angles of each limb segment of the patients before and after HAL intervention, and also when compared to healthy volunteers. Our results suggest that early HAL intervention contributed to the improvement of gait coordination in hemiparetic stroke patients by reinforcing central pattern generators and therefore reshaping their gait pattern.Trial registration: UMIN000022410 2016/05/23