Functional Electrical Stimulation (FES) facilitates the motor recovery of the hand function after stroke. The integration of biofeedback and other strategies to actively involve a patient in the therapy is important for the rehabilitation progress. We introduce a combined control approach for a FES-driven neuroprosthesis using volitional electromyo-graphy (vEMG) and motion capturing via a novel inertial sensor network for patients that still possess a residual activity in the paralyzed muscles. A real-time vEMG measurement and signal processing in between stimulation pulses has been realized during active FES. Experiments showed that our system allows for quick adaption to individual users.BMBF, 16SV7069K, Verbundprojekt: Bewegungsfähigkeit und Mobilität wiedererlangen - BeMobil -; Teilvorhaben: Nutzerzentrierte Entwicklung technischer Methoden für eine optimale Mensch-Technik-Interaktion in der Bewegungsrehabilitatio

Dorn, Andreas

Salchow-Hömmen, Christina

Schauer, Thomas

Valtin, Markus

Current Directions in Biomedical Engineering

DepositOnce

 Current Directions in Biomedical Engineering 2017; 3(2): 161–165 Christina Salchow*, Andreas Dorn, Markus Valtin and Thomas Schauer Intention recognition for FES in a grasp-and-release task using volitional EMG and inertial sensors Abstract: Functional Electrical Stimulation (FES) facilitates the motor recovery of the hand function after stroke. The integration of biofeedback and other strategies to actively in-volve a patient in the therapy is important for the rehabili-tation progress. We introduce a combined control approach for a FES-driven neuroprosthesis using volitional electromyo-graphy (vEMG) and motion capturing via a novel inertial sensor network for patients that still possess a residual activity in the paralyzed muscles. A real-time vEMG measurement and signal processing in between stimulation pulses has been realized during active FES. Experiments showed that our system allows for quick adaption to individual users. Keywords: Functional electrical stimulation, volitional EMG, inertial sensors, hand neuroprosthesis, grasping https://doi.org/10.1515/cdbme-2017-0034 1 Introduction Stroke is a major cause for disability in adulthood in developed countries. According to the World Heart Federation (2016), nearly 5 million stroke survivors world-wide are left perma-nently disabled or paralyzed. Many patients still possess a residual activity in the paralyzed muscles. Functional Electrical Stimulation (FES) is a beneficial treatment modality used in therapy to facilitate the motor recovery of disabled limbs after stroke. The application at the forearm with surface electrode arrays allows for generating complex hand movements such as grasping of objects or pointing [1]. Studies revealed that a synchronized biofeedback maximizes the benefits of FES therapy [2]. Several methods have been introduced for detecting the patient’s intention in FES applications. They range from ordinary triggering of the FES via push button, complex motion capturing via optical systems or inertial sensors, to brain computer interfaces [3,4]. Simple methods lack sufficient involvement and instinctiveness, whereas physio-logical approaches usually require high costs regarding adaption and training with the individual patient. A popular physiological approach is the detection and enhancement of the surface electromyogram (EMG) of the remaining muscle activity [5]. Schauer et al. [6] measured the volitional EMG (vEMG) before and during electrical stimulation and realized a vEMG-proportional control of the stimulation intensity for a wrist extension. The restricted control of the remaining volitional muscle activity led to oscillations in the stimulation. Salbert et al. [7] developed an EMG-triggered state machine, which enabled the user to control the motion sequence of hand opening and closing via vEMG-measurements of the hand extensors and flexors. However, first experiments with patients revealed that the assumption of a higher vEMG in the hand extensors compared to flexors during the attempt of hand opening and vice versa did not always hold due to a strong co-activation of muscle groups. Besides, different electrode placement, shifting contact resistances, as well as measure-ment noise led to a varying vEMG quality in each trial.  We present a novel method for the control of a grasp-and-release task with a hand neuroprosthesis. The vEMG of the stimulated muscles and motion capturing via inertial sensors were combined for intention recognition. Patients, who still possess a residual activity in the paralyzed arm, should be able to control the stimulation on- and offset of three stimulation channels. Our goals were (1) to gain a high robustness of intention detection for patients, and (2) to provide a setup with a short adaption time, which can be used in clinical practice. We achieved this by implementing a state machine with adjustable conditions for the transition events that rely on adaptive thresholds for the decision making. In ______ *Corresponding author: Christina Salchow: Control Systems Group, TU Berlin, D-10587 Berlin, Germany, e-mail: salchow@control.tu-berlin.de Andreas Dorn, Markus Valtin, Thomas Schauer: Control Systems Group, TU Berlin, D-10587 Berlin, Germany, e-mail: dorn-/valtin-/schauer@control.tu-berlin.de 162   this paper, we introduce the concept of our method and show preliminary results with healthy volunteers and stroke patients. 2 Methods 2.1 Experimental setup Central elements of our hand neuroprosthesis are the recently introduced RehaMovePro stimulator (Hasomed GmbH, Germany) with science adapter and de-multiplexer [8], the StiMyo II EMG measurement unit (TU Berlin, Control Sys-tems Group), as well as a lately presented hand sensor system [9]. We applied FES via two customized electrode arrays (see Figure 1): one with 35 elements placed above the wrist and finger extensor muscles (array E), and one with 24 elements placed above the finger flexors (array F). A single hydro-gel layer (AG702, Axelgaard Manufacturing Co., Ltd., USA) was used to attach the array electrodes. Bi-phasic pulses with constant pulse width were applied at 25 Hz using the current amplitude   as adjustable stimulation intensity. We measured EMG of the extensors (EMGE) and flexors (EMGF) by the two channels of the StiMyo II unit via separate EMG electrodes (cf. Figure 1) at 4 kHz. The synchronization between the StiMyo II and the RehaMovePro allows the blanking of the stimulation pulse artefacts in the EMG signal. To extract the vEMG (vEMGE, vEMGF) from the measurement in real-time, the EMG signal was digitally high-pass filtered at 200 Hz and smoothened by a moving average filter with a cut-off frequency of ≈ 2.2 Hz. The hand sensor system consists of a wireless IMU loca-ted on the dorsal side of the forearm, a base unit, which is attached to the back of the hand, and two sensor stripes on the index and middle finger. Each sensor stripe contains three single IMUs, one placed on each finger segment (cf. Figure 1) [9]. In total, the system measures ten joint angles at 100 Hz. The extension (negative) and flexion (positive) angle of the metacarpal-phalangael (MCP) joint of the index (α1) and middle (α2) finger, as well as the wrist extension/flexion angle (β) proved to offer the best information regarding intention. 2.2 Control of a grasp-and-release task In general, the extraction of the vEMG via the two EMG-channels allows the distinction of hand closing (grasping) and hand opening (release). However, in stroke patients, high co-activation levels between the extensor and flexor muscle groups might obstruct a classification, e.g. due to the presence of spasticity. Therefore, we combined the vEMG with the hand joint angles to classify the patient’s intention. A grasp-and-release task can be interpreted as a pre-set sequence of movements. Similar to [7], we designed an event-triggered state machine (see Figure 2). It consists of four states: (1) rest, (2) hand opening, (3) grasp, and (4) release. During state (1), the stimulation is turned off. In state (2) and (4), the hand and finger extensor muscles are stimulated above motor threshold (stimE,1), whereas in state (3) FES above motor threshold is applied to the finger flexors (stimF) and the extensors for stabilizing the wrist (stimE,2). Via vEMG, wrist, and finger movements, the patient controls the onset and the duration of each state and thereby the Figure 2: vEMG and motion triggered state machine for a grasp- and-release task. Figure 1: Experimental setup on the left forearm. timing of the stimulation. The stimulation channels are always at least stimulated below motor threshold during state (2) – (4), so that both EMG channels are effected equally by the active electrical stimulation. The stimulation intensity profile as well as the array element configuration for the three stimulation positions (hand opening, wrist stabilization, and grasp) must be preselected by the therapist. Table 1: Possible conditions for the events of the state machine. 2.3 Adaptive classification To provide a flexible system, which can be easily adapted to the patient’s capabilities, we utilized adaptive thresholds and developed a various number of conditions for the state machine as listed in Table 1. By linking the conditions regarding vEMG and angular motion with either ‘AND’ or ‘OR’, as well as by deleting conditions, the events and thereby the behaviour of the state machine can be changed. The event ‘relax’ can be deactivated completely, which might be necessary for patients with spasticity, as they might not able to relax their forearm muscles completely. This flexible framework was realized to enable an adaption to individual requirements for a patient.  The angular velocities of finger and wrist joints were utilized to detect intention in hand motion. In comparison with joint angles, angular velocities offer the benefit to apply thresholds that are independent of the current hand posture. We defined three constant but adjustable thresholds     ,     , and     for the signed angular velocities   1,   2 and   . Default values were obtained heuristically and were set to 0.7 °/ms for      and     , and 1°/ms for    . Three adaptive thresholds for the vEMG were used to classify whether the patient wants to perform a movement or not (relax). sE is designated for the extensors (vEMGE) and sF holds for the flexors (vEMGF), respectively. Additionally, a difference signal   was introduced to distinguish between the intention of hand opening and grasping. The difference is calculated according to eq. 1, where             is the vEMG normalized to the maximum recorded volitional mus-cular activity of each person and channel. To obtain those maxima, the voluntary activity during three hand openings and grasps are recorded initially. In this way,   is usually negative when closing the hand and positive when opening the hand. The threshold sD holds for   during active stimulation.                          (1)                              (2) All thresholds are set automatically when initializing the system. The patient has to relax for 3 s, while the thresholds are calculated according to eq. 2, which holds analogically for sF and sD with ‘std’ as standard deviation. The thresholds are continuously updated when the corresponding signal is below its threshold for at least 3 s. To increase the robustness of the detection, time constraints are applied to each condition. A condition is classified as fulfilled, if it holds for 50 % of the values in the considered time window. A window length of 300 ms showed to be appropriate for vEMG, whereas for the angular velocities a shorter time window of 150 ms was applied. To trigger the ‘relax’ event, 100% of the values need to fulfil condition C1 (see Table 1).  When FES is applied, the measured vEMG of the stimulated muscle group is slightly disturbed by the FES. This effect is crucial when using vEMGF to estimate the patient’s intention. For this reason, sF adapts proportional to the applied stimulation intensity of the extensors (stimE,1). Furthermore, different events were established for utilizing the vEMG condition (C1) during rest (cf. open(r)/close(r)) and during active stimulation (cf. open(a)/close(a) in Table 1). 3 Results The described setup and method were evaluated in three healthy volunteers and one chronic stroke patient (male, age 54, paralysis of the left arm). Individual stimulation sides in the electrode arrays were defined using the approach presented in [10]. The stimulation intensities were manually adjusted.  For each participant, we started the intention recognition by using the state machine with all conditions of Table 1 Event Conditions C1  C2  C3 open (r)                                1          2        open (a)                    1          2        close (r)                               1         2       close (a)                      1         2       relax             164   ‘OR’-linked. The participant was invited to try to reach each state and stay for a while in it. Figure 3 shows the results of one healthy volunteer of the ‘OR’-linked state machine. The participant approached the four states consecutively and complete grasp-and-release motions were generated. If a participant was unable to remain in a desired state, we removed misleading conditions or we ‘AND’-linked con-ditions to increase the robustness of the intention detection. Thereby, we were able to adapt to the specific behaviour of each participant and a successful detection was possible in each case. This procedure took less than 3 min.  The stroke patient appeared to have a good control of his wrist extensors but not of his finger extensors. By removing condition C3 from the events, the patient could perform three complete grasp-and-release motions after another. Afterwards, the stimulation intensity for the extensors needed to be increased manually to gain sufficient hand opening. 4 Discussion and conclusion We introduced a flexible framework for intention recognition utilizing vEMG and motion capture. A state machine was chosen to control the stimulation on- and offset in a grasp-and-release task using adaptive vEMG thresholds. The possibility of adjusting the transition event conditions individually for each person respond to the specific requirements and capabilities of each patient.  Our hand neuroprosthesis utilizes electrode arrays, which complicates the EMG measurement due to shortage of space on the forearm. Only the sum of the muscle groups, extensors and flexors, can be measured. This and the problems of vEMG mentioned previously render a standalone vEMG intention recognition difficult. Our results revealed that the combination of vEMG and inertial motion tracking is a promising approach for an intention detection, because it covers patients with an abnormal activation of hand extensors and flexors.  The stimulation strategy in our approach did not take into account the patient’s habituation to the applied FES. We consider to replace the pre-set stimulation intensity profiles by feedback control of unused finger joint angles. Additional experiments with a larger group of stroke patients are necessary to evaluate our framework. Based on those future results, we plan to develop an automatic, logic-based adaption procedure to configure the state machine.   Acknowledgment: We thank Axelgaard Manufacturing Co. (Ltd., USA) for contributing the electrodes and gel layers.  Author’s Statement Research funding: As part of the research project BeMobil, this work was supported by the German Federal Ministry of Education and Research (FKZ16SV7069K). Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use complies with all the relevant national regulations, institutional policies and was performed in accordance with the tenets of the Helsinki Declaration, and has been approved by the Berlin Chamber of Physicians. References [1] Malešević NM, Popović-Maneski LZ, IIić V, et al. A multipad electrode based functional electrical stimulation system for restoration of grasp. J Neuroeng Rehabil 2012; 9(66). [2] Burridge JH, Ladoceur M. Clinical and therapeutic applications of neuromuscular stimulation: a review of current use and speculation into future developments. Neuromodulation 2001; 4(4): 147–154. [3] Štrbac M, Kočović S, Marković M, Popović DB. Microsoft Kinect-based artificial perception system for control of functional electrical stimulation assisted grasping. BioMed Research International, 2014; 740469. [4] Rupp R, Rohm M, Schneiders M, Kreilinger A, Müller-Putz GR. Functional rehabilitation of paralyzed upper extremity after spinal cord injury by noninvasive hybrid neuro-prostheses. Proc. IEEE 2015; 103(6): 954–968. [5] Saxena S, Nikolić S, Popović DB. An EMG-controlled grasping system for tetraplegics. J Rehab Res Dev 1995; 32(1): 17–24. [6] Schauer T, Hossaini D, Hesse S, Raisch J. EMG-controlled electrical stimulation of the paretic wrist and finger extensors in stroke patients. In Proc. of the European Symp. Technical Aids for Rehabilitation (TAR), Berlin, Germany, 2005. Figure 3: Two successful iterations of the state machine of one healthy volunteer. The states are highlighted as grey bars. [7] Salbert RC, Schauer T, Schmidt S, Raisch J. Funktionelles Handöffnen und -schließen mittels EMG-gesteuerter elektrischer Stimulation. In Proc. of the 4th Symposium on Automatic Control, Wismar, Germany, 2005. [8] Valtin M, Kociemba K, Behling C, Kuberski B, Becker S, Schauer T. RehaMovePro: A versatile mobile stimulation system for transcutaneous FES applications. Eur J Transl Myol 2016; 26(3): 203–208. [9] Valtin M, Salchow C, Seel T, Laidig D, Schauer T. Modular Finger and Hand Motion Capturing System Based on Inertial and Magnetic Sensors. Current Directions in Biomedical Engineering 2017, 3(1): 19–23.  [10] Salchow C, Valtin M, Seel T, Schauer T. A new semi-automatic approach to find suitable virtual electrodes in arrays using an interpolation strategy. Eur J Transl Myol 2016; 26(3): 6029.

Intention recognition for FES in a grasp-and-release task using volitional EMG and inertial sensors

https://depositonce.tu-berlin.de/bitstream/11303/11990/2/Salchow_etal_2017.pdf

Intention recognition for FES in a grasp-and-release task using volitional EMG and inertial sensors

Abstract

Similar works

Full text

Available Versions

DepositOnce