Human-Machine Interfaces in rehabilitation engineering often use activity signals. Examples are electrical wheelchairs or prostheses controlled by means of muscle contractions. Activity signals are user-dependent and often reflect neurological weaknesses. Thus, not all users are able to operate the same control scheme in a robust manner. To avoid under- and overstraining, the interface ideally uses a control scheme which reflects the user’s control ability best. Therefore, we explored typical phenomena of activation signals. We derive criteria to quantify the user’s performance and abilities and present a routine which automatically selects and adapts one of three control schemes being best suited

Doneit, Wolfgang

Liebetanz, David

Mikut, Ralf

Reischl, Markus

Rupp, Rüdiger

English

KITopen

Current Directions in Biomedical Engineering 2016; 2(1): 707–710Open AccessWolfgang Doneit, Ralf Mikut, David Liebetanz, Rüdiger Rupp and Markus Reischl*Control scheme selection in human-machine-interfaces by analysis of activity signalsDOI 10.1515/cdbme-2016-0153Abstract: Human-Machine Interfaces in rehabilitation en-gineering often use activity signals. Examples are elec-trical wheelchairs or prostheses controlled by means ofmuscle contractions. Activity signals are user-dependentand often reflect neurological weaknesses. Thus, not allusers are able to operate the same control scheme in arobust manner. To avoid under- and overstraining, theinterface ideally uses a control scheme which reflects theuser’s control ability best. Therefore, we explored typicalphenomena of activation signals. We derive criteria toquantify the user’s performance and abilities and presenta routine which automatically selects and adapts one ofthree control schemes being best suited.Keywords: calibration; data quality; human-machineinterfaces; rehabilitation engineering.1 IntroductionHuman-machine interfaces (HMI) for controlling technicaldevices in rehabilitation engineering often use electroen-cephalography (EEG) [1] or electromyography (EMG) [2] toobtain bioelectric signals. Normalization procedures [3]andpattern recognition techniques [4] are used to estimatecontrol signals for devices like electrical wheelchairs [5] orprostheses [6].A common feature generated from biosignals is thenormalized activity signal (e.g. amplitude of a low-pass-filter). Activity signals can also be derived from joystick*Corresponding author: Markus Reischl, Institute for AppliedComputer Science, Karlsruhe Institute of Technology, Eggenstein-Leopoldshafen, E-mail: markus.reischl@kit.eduWolfgang Doneit and Ralf Mikut: Institute for Applied ComputerScience, Karlsruhe Institute of Technology, Eggenstein-LeopoldshafenDavid Liebetanz: Department of Clinical Neurophysiology,Georg-August-University, GoettingenRüdiger Rupp: Orthopedic Hospital of Heidelberg University,Heidelbergaxes [7], air pressure with sipping and puffing and posi-tions of shoulders, tongue [8] or head [9].To generate control signals, commercial systems userobust threshold approaches [10], whereas experimentalsystems use pattern recognition techniques [11]. A calibra-tion is necessary to adapt the system to the user. Commer-cial HMIs are adapted mostly by hand.Ref. [12] presents a wheelchair control interface basedon the bilateral recording of myoelectric signals from leftand right ear muscle. The raw signals are rectified, filteredand normalized to receive activity signals that correlatewith the strength of contractions. The ability to contractthe ear muscles is trainable and left and right ear mus-cle can be activated independently. However, not onlytraining progress is user-dependent, but also extrema andcoactivations of the activation signals, dispersion of the in-tended constant activations, difference betweenmeasuredand intended activation.As the control abilities of users vary, users might notonly need adapted parameters but also individual controlschemes. However, there is only little knowledge aboutuser-specific selections of the control scheme.Therefore, we propose a method to select a controlscheme based on activity signals from a calibration rou-tine. We derive a rule set to assign each user one out ofthree control schemes. Using benchmark exampleswe testthe method and discuss effects. We provide a real-worlddataset to prove functionality.2 Material and methods2.1 Calibration routineIf a user is not able to activate two signal channelsindependently, coactivations occur and control schemesbased on difference signals fail. Polynomial regressionsapproximate the nonlinear relationship of intended acti-vation andmeasured coactivated signals. Therefore, bilat-eral calibration routineswere proposed in [13, 14] to gatherdata for model building: The user is asked to simultane-ously hold activation levels in two signal channels. Theactivation levels are the intended activations (intentions)© 2016 Markus Reischl et al., licensee De Gruyter.This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 License.Bereitgestellt von | KIT-Bibliothek | Karlsruher Institut für TechnologieAngemeldetHeruntergeladen am | 17.10.16 15:13708 | W. Doneit et al.: Control scheme selection in HMIs by analysis of activity signalsy = (y1, y2). The calibration routine records the activationof each channel according to Table 1 for– single maximal activations of each channel,– simultaneous maximal activation of both channels,– single activations of each channel at 50% activationlevel and– simultaneous activations of both channels at 50%activation level.Each of the six calibration steps is described by themembership to a class z = {B1, · · · , B6} and a vector ofintentions y = (y1, y2). The user-generated normalized ac-tivation signals are x1 and x2. Table 1 shows the calibrationsteps, the class membership and intended activations.2.2 Selection of a control schemeA control scheme estimates intentions by thresholds orpattern recognition algorithms. We consider three differ-ent control schemes:A threshold-based control using one signal channelto recognize intentions given by short/long activations.Sequences comparable to the Morse code are assignedto actions of the rehabilitation device (e.g. turning leftor right with a wheelchair). The advantage of the controlscheme is its simplicity. Even with coactivations and lowabilities of holding an activation level this control schemeis applicable. Disadvantages are time lags in control.If the user is able to generate better discriminableactivation patterns a classifier-based control can beapplied. A classifier ẑ = f (x1, x2; θ) is trained for the 6classes Bi within a calibration B1, · · · , B6 and classifies thesignals at each time sample. Classes represent intentions,the control scheme maps each intention to an action.No activation in both channels corresponds to a neutralstate. Inputs of the classifier are the normalized activationsignals x1 and x2. The parameter vector θ is estimatedbased on calibration data.If the user is able to hold different activation levelswith both signal channels, a proportional control can beTable 1: Intended activations and classes of the calibration steps.Calibration step y1 y2 z1: single x1 1 0 B12: single x1 (50%) 0.5 0 B23: single x2 0 1 B34: single x2 (50%) 0 0.5 B45: simultaneous x1 and x2 1 1 B56: simultaneous x1 and x2 (50%) 0.5 0.5 B6applied. It determines two independent, time-continuouscontrol signals by the level of simultaneous activationof both channels and the difference of both channels(e.g. translational velocity and rotational velocity of awheelchair). To reduce the influence of unintended coac-tivations, regression models ŷi = f i(x1, x2; θi) are usedto estimate the continuously valued intention of eachchannel [13] with the constraints f 1(0, 0, θ1) = f 2(0, 0,θ2) = 0. f 1 and f 2 are polynomial functions of seconddegree and the parameter vectors θ1 and θ2 are estimatedby a least squares algorithm. Fluctuating activity signalslead to incessantly changing velocities in the proportionalcontrol.To select an appropriate control scheme three criteriaare definedwhich are also useful for amedical supervisor:Q1 describes the ability of the user to generate discrim-inable activation signals for the classifier-based controlscheme. Therefore, we calculate the minimal accuracy ofa class for the used classification algorithmQ1 = minj(︂(︂1Nj· N(ẑ = Bj ∩ z = Bj)︂)︂. (1)Q2 quantifies whether the generated signals match theintended activations and describes the ability of the userto generate predetermined activation signals. For a robustestimation of themean value, themedian operator is used.x̃j, i is the median of the xi values of all datapoints withclass label Bj. yj, i is the intended activation for channel iof all datapoints with class label Bj:Q2 = exp(︂−4(︂maxj,i(︀x̃j,i − yj,i)︀)︂)︂(2)The factor −4 is selected empirically to achieve a smoothcourse of the criterion. Q3 quantifies the dispersion of thegenerated signals. It describes the ability of the user tohold an activation level. For a robust estimation of thedispersion the interquartile range is used. x̃0.75, j, i is theupper quartile und x̃0.25, j, i the lower quartile of the xivalues of all datapoints with class label Bj:Q3 = exp(︂−4(︂maxj,i(︀x̃0.75,j,i − x̃0.25,j,i)︀)︂)︂(3)(4) shows the rules for the selection of an appropriatecontrol scheme.(Q1 > τ1) ∧ (Q2 > τ2) ∧ (Q3 > τ3) → proportional(Q1 > τ1) ∧ ¬ ((Q2 > τ2) ∧ (Q3 > τ3)) → classifier¬ (Q1 > τ1) → threshold (4)The thresholds τ1, τ2 and τ3 are selected empirically.Bereitgestellt von | KIT-Bibliothek | Karlsruher Institut für TechnologieAngemeldetHeruntergeladen am | 17.10.16 15:13W. Doneit et al.: Control scheme selection in HMIs by analysis of activity signals | 7092.3 DatasetsWe simulated three benchmark datasets which representthree states of user performance: Figure 1(A) shows thecalibration data of a user with low performance. The dataof the calibration steps is overlapping and the dispersionof single classes is high. Figure 1(B) shows the calibrationdata of a user with medium performance. The dispersionof the classes is high but the classes are discriminable.Figure 1(C) shows the calibration data of a user with highperformance. The datapoints of the calibration steps arebuilding small clusters near to their intended activations.In [14] real-world datasets in a bilateral calibrationroutine were recorded with antagonistic forearm muscles.Three datasets are generated by one user with varyingsensor placements for 12 intentions. Figure 1(D–F) showsubsets of the datasets. The subsets correspond to theintended activations of Table 1. The data is preprocessed,i.e. outliers of the calibration steps were already deleted.3 ResultsAnalyses were performed with MATLAB and Gait-CAD[15]. Table 2 shows Q1, Q2, Q3 and the control schemeselection for the simulated and the real-world datasetswith τ1 = 0.9, τ2 = 0.15 and τ3 = 0.5. The thresholds areempirically selected.The assessment by the criteria conforms to the de-scription of the benchmark datasets. For dataset 1, Q1and Q3 are below their respective thresholds. Thus, thecalibration steps are not discriminable and the dispersionof at least one calibration step is high. The threshold-basedcontrol scheme is selected. This evaluation coincides withthe visual inspection of dataset 1. With the threshold-based control scheme, the user is at least able to apply sev-eral movements even if it takes time. In a classifier-basedor a proportional control scheme, the estimated intentionsare fluctuating due to the high dispersion in classes andthe overlap of different classes. Also for dataset 2,Q3 is be-low its threshold. Since the calibration steps are discrim-inable, the classifier-based control scheme is selected. Inthis control scheme, the user is able to hold a movement.In the proportional control scheme the velocities are fluc-tuating due to the high dispersion in classes. For dataset 3,all criteria are above their thresholds. The proportionalcontrol scheme is selected.The real-world datasets look similar to dataset 3whichis appropriate for the proportional control scheme. But theproximity of calibration steps (e.g. B4 and B6 in dataset 4or B1 and B5 in dataset 5) leads to a rapid change ofvelocities by small fluctuations of activation signals in0.1 0.2 0.3 0.4 0.5 0.6Dataset 1 Dataset 2 Dataset 3Dataset 4 Dataset 5 Dataset 60.7 0.8 0.90.10.20.30.40.50.60.70.80.A B CD E F9x1x2  B1B2B3B4B5B60.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.10.20.30.40.50.60.70.80.91x1x2  B1B2B3B4B5B60.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.10.20.30.40.50.60.70.80.9x1x2  B1B2B3B4B5B60.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.10.20.30.40.50.60.70.80.91x1x2  B1B2B3B4B5B60.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.10.20.30.40.50.60.70.80.9x1x2 B1B2B3B4B5B60.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.900.10.20.30.40.50.60.70.80.91x1x2  B1B2B3B4B5B6Figure 1: (A)-(C): Simulated datasets representing different levels of user performance. (D)-(F): Parts of the real-world datasets from [14].Only the calibration steps are used which would have been recorded from the proposed calibration routine.Bereitgestellt von | KIT-Bibliothek | Karlsruher Institut für TechnologieAngemeldetHeruntergeladen am | 17.10.16 15:13710 | W. Doneit et al.: Control scheme selection in HMIs by analysis of activity signalsTable 2: Results for simulated (1–3) and real-world (4–6) benchmarkdatasets. Values below τ-thresholds are bold.Dataset Q1 Q2 Q3 Selection1 0.556 0.6672 0.2222 threshold2 0.956 0.2207 0.3065 classifier3 1 0.3005 0.6583 proportional4 1 0.064 0.7579 classifier5 1 0.0372 0.5945 classifier6 1 0.1742 0.6995 proportionala proportional control scheme. The calibration steps ofthe real-world datasets 4, 5 and 6 are discriminable, theirdispersions are low and so Q1 and Q3 are above theirthresholds. For datasets 4 and 5,Q2 is below its thresholds.That means the data of the calibration steps is different tothe intended signals and the proportional control schemeis not appropriate for the user. Instead the classifier-basedcontrol scheme is selected. Only for dataset 6 the propor-tional control scheme is selected.4 Discussion and outlookThe automatic control scheme selection helps to find acontrol scheme that fits to the user’s performance. Thus,frustration of the user by under- or overstraining can beavoided. The interpretability of the proposed criteria asuser performance allows the use in studies and medicalinvestigations. The user performance over several calibra-tions can be followed and changes in neurological andphysiological weaknesses can be quantified. Without ourcriteria, the information about user performance is lostby using the data just for training a model to estimateintended activations.The proposed criteria are able to represent differentphenomena of activation signal calibration data. The us-age of the criteria for selecting an appropriate controlscheme has been demonstrated with the help of three sim-ulated and three real-world datasets. The method is cur-rently used to prepare a study regarding user performanceevaluation.Author’s StatementResearch funding: The project is funded by the HelmholtzAssociation (Program bioInterfaces in technology andmedicine (RM, MR) and the German Federal Ministryof Education and Research BMBF [WD, DL, RR,MR,Grant No 13EZ1122(A–C)]. Conflict of interest: Authorsstate no conflict of interest. Informed consent has beenobtained from all individuals included in this study. Ethi-cal approval: The research related to human use complieswith all the relevant national regulations, institutionalpolicies and in accordance the tenets of the Helsinki Dec-laration, and has been approved by the authors’ institu-tional review board.References[1] Millán J, Renkens F, Mourino J, Gerstner W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE TransBiomed Eng. 2004;51:1026–33.[2] Vitiello N, Olcese U, Oddo CM, Carpaneto J, Micera S, CarrozzaMC, et al. A simple highly eflcient non invasive EMG-basedHMI. Conf Proc IEEE Eng Med Biol Soc. 2006;1:3403.[3] Knutson L, Soderberg G, Ballantyne BT, Clarke WR. A study ofvarious normalization procedures for within day electromyo-graphic data. J Electromyogr Kinesiol. 1994;4:47–59.[4] Ciaccio E, Dunn S, Akay M. Biosignal pattern recognitionand interpretation systems. IEEE Trans Biomed Eng.1993;12:89–95.[5] Choi K, Sato M, Koike Y. A new, human-centered wheelchairsystem controlled by the EMG signal. Proc IEEE Int Conferenceon Neural Networks. 2006;4664–71.[6] Reischl M, Gröll L, Mikut R. Evaluation of data miningapproaches for the control of multifunctional arm prostheses.Integr Comput-Aid E. 2011;18:235–49.[7] Loveless J, Seamone W. Chin controller system for poweredwheelchair; 1981.[8] Lund M, Christiensen H, Caltenco H, Lontis ER, BentsenB, Andreasen Struijk LN. Inductive Tongue Control ofPowered Wheelchairs. Conf Proc IEEE Eng Med Biol Soc.2010;2010:3361–4.[9] Min J, Lee K, Lim S, Kwon D. Human-friendly interfaces ofwheelchair robotic system for handicapped persons. ProcIEEE Conf on Intelligent Robots and Systems. 2002;2:1505–10.[10] Otto Bock Orthopädische Industrie. Patienteninformation fürSensorHand. 646D83; 1999.[11] Gopura R, Bandara D, Gunasekara J, Jayawardane T. Recenttrends in EMG-based control methods for assistive robots. In:H. Turker, editor, Electrodiagnosis in New Frontiers of ClinicalResearch InTech; 2013. p. 237–68.[12] Schmalfuß L, Rupp R, Tuga M, Kogut A1, Hewitt M, Meincke J,et al. Steer by ear: myoelectric auricular control of poweredwheelchairs for individuals with spinal cord injury. RestorNeurol Neurosci. 2015;34:79–95.[13] Doneit W, Tuga MR, Mikut R, et al. Kalibrierungs- undTrainingsstrategien zur individuellen Signalgenerierungfür die myoelektrische Steuerung technischer Hilfsmittel.Technisches Messen. 2015;82:411–21.[14] Stäb L. Entwicklung einer adaptiven Kalibrierungsroutine fürzweikanalige EMG-messungen. Bachelorarbeit, KarlsruherInstitut für Technologie; 2016.[15] Mikut R, Burmeister O, Braun S, Reischl M. The opensource Matlab toolbox Gait-CAD and its application tobioelectric signal processing. Proc DGBMT-WorkshopBiosignalverarbeitung. 2008;109–11.Bereitgestellt von | KIT-Bibliothek | Karlsruher Institut für TechnologieAngemeldetHeruntergeladen am | 17.10.16 15:13

Control scheme selection in human-machine-interfaces by analysis of activity signals

https://publikationen.bibliothek.kit.edu/1000060446/3917717

Control scheme selection in human-machine-interfaces by analysis of activity signals

Abstract

Similar works

Full text

Available Versions

KITopen