Many solutions have been proposed to help amputated subjects regain the lost functionality. In order to interact with the outer world and objects that populate it, it is crucial for these subjects to being able to perform essential grasps. In this paper we propose a preliminary solution for the online classification of 8 basics hand grasps by considering physiological signals, namely Surface Electromyography (sEMG), exploiting a quantitative taxonomy of the considered movement. The hierarchical organization of the taxonomy allows a decomposition of the classification phase between couples of movement groups. The idea is that the closest to the roots the more hard is the classification, but on the meantime the miss-classification error is less problematic, since the two movements will be close to each other. The proposed solution is subject-independent, which means that signals from many different subjects are considered by the probabilistic framework to modelize the input signals. The information has been modeled offline by using a Gaussian Mixture Model (GMM), and then testen online on a unseen subject, by using a Gaussian-based classification. In order to be able to process the signal online, an accurate preprocessing phase is needed, in particular, we apply the Wavelet Transform (Wavelet Transform) to the Electromyography (EMG) signal. Thanks to this approach we are able to develop a robust and general solution, which can adapt quickly to new subjects, with no need of long and draining training phase. In this preliminary study we were able to reach a mean accuracy of 76.5%, reaching
up to 97.29% in the higher levels

Moro, Michele

Pagello, Enrico

Stival, Francesca

English

Many solutions have been proposed to help amputated subjects regain the lost functionality. In order to interact with the outer world and objects that populate it, it is crucial for these subjects to being able to perform essential grasps. In this paper we propose a preliminary solution for the online classification of 8 basics hand grasps by considering physiological signals, namely Surface Electromyography (sEMG), exploiting a quantitative taxonomy of the considered movement. The hierarchical organization of the taxonomy allows a decomposition of the classification phase between couples of movement groups. The idea is that the closest to the roots the more hard is the classification, but on the meantime the miss-classification error is less problematic, since the two movements will be close to each other. The proposed solution is subject-independent, which means that signals from many different subjects are considered by the probabilistic framework to modelize the input signals. The information has been modeled offline by using a Gaussian Mixture Model (GMM), and then testen online on a unseen subject, by using a Gaussian-based classification. In order to be able to process the signal online, an accurate preprocessing phase is needed, in particular, we apply the Wavelet Transform (Wavelet Transform) to the Electromyography (EMG) signal. Thanks to this approach we are able to develop a robust and general solution, which can adapt quickly to new subjects, with no need of long and draining training phase. In this preliminary study we were able to reach a mean accuracy of 76.5%, reaching
up to 97.29% in the higher levels

Francesca Stival

Michele Moro

Enrico Pagello

Archivio istituzionale della ricerca - Università di Padova

Original Citation:A first approach to a taxonomy-based classification framework for hand graspsPublisher:Published version:DOI:Terms of use:Open Access(Article begins on next page)This article is made available under terms and conditions applicable to Open Access Guidelines, as described athttp://www.unipd.it/download/file/fid/55401 (Italian only)Availability:This version is available at: 11577/3270209 since: 2018-05-10T10:52:15ZUniversità degli Studi di PadovaPadua Research Archive - Institutional RepositoryA first approach to a taxonomy-basedclassification framework for hand graspsFrancesca Stival1 and Enrico Pagello21 Intelligent Autonomous Systems Lab (IAS-Lab)Department of Information Engineering (DEI)University of Padova{stivalfr,epv}@dei.unipd.ithttp://robotics.dei.unipd.it2 IT+Robotics Srl, Vicenza, ItalyAbstract. Many solutions have been proposed to help amputated sub-ject regain the lost functionality. In order to interact with the outer worldand objects that populate it, it is crucial for these subjects to beingable to perform essential grasps. In this paper we propose a preliminarysolution for the online classification of 8 basics hand grasps by consid-ering physiological signals, namely Surface Electromyography (sEMG),exploiting a quantitative taxonomy of the considered movement. The hi-erarchical organization of the taxonomy allows a decomposition of theclassification phase between couples of movement groups. The idea isthat the closest to the roots the more hard is the classification, but onthe meantime the miss-classification error is less problematic, since thetwo movements will be close to each other. The proposed solution issubject-independent, which means that signals from many different sub-jects are considered by the probabilistic framework to modelize the inputsignals. The information has been modeled oﬄine by using a GaussianMixture Model (GMM), and then testen online on a unseen subject,by using a Gaussian-based classification. In order to be able to processthe signal online, an accurate preprocessing phase is needed, in par-ticular, we apply the Wavelet Transform (Wavelet Transform) to theElectromyography (EMG) signal. Thanks to this approach we are ableto develop a robust and general solution, which can adapt quickly to newsubjects, with no need of long and draining training phase. In this pre-liminary study we were able to reach a mean accuracy of 76.5%, reachingup to 97.29% in the higher levels.Keywords: Electromyography, Taxonomy, Grasps, Prosthesis1 IntroductionNew and innovative prosthesis devices can improve the life quality of amputatedsubjects, helping them to interact with the surrounding world. Indeed, everydayeach of us use and grasp an high number of different objects with different shapes,dimensions and weight. The difference between the objects shape, weight, and2deformability comport disparate ways of interacting with them by using differ-ent grasping approaches. The scientific community have selected a number ofessential grasps, organizing them in taxonomies depending on several qualita-tive factors. Many different taxonomies have been proposed during the years,considering different aspects of hand motion. One of the most complete and de-tailed taxonomy has been proposed by Feix et al. in [6] by arranging togethermany different proposals. Their solution, called the GRASP taxonomy, organizesin a matrix 33 hand grasps, arranging the movements according to qualitativeforce parameters and fingers position. In this work we exploit a quantitativetaxonomy of 8 hand grasps starting from Feix’s one. The movements are ar-ranged in a binary tree, where movements belonging to the same subtree havea common underlying behaviour. The considered taxonomy is based on quan-titative parameters, i.e. it includes the information from the hand and fingerskinematics, and the muscular activation during the motion. In particular, themuscular activation and involvement is represented by the sEMG signals. Thesesignals are also a commonly used for the control of prosthesis, we consideredsEMG signals from the subjects with the goal of exploiting the physiologicalbehaviour in order to emulate what happens in the human body. sEMG signalshave the drawback of being non stationary, and very sensitive to the physicaland physiological state of the subject. In particular, studies shown a sensibilityfor muscular fatigue, also stress and physical weariness [5]. These characteristicsresult in a great variability of the sEMG signals, even if they are collected froma single subject in a limited period of time. As a consequence, the traditionalapproach when using sEMG signals is to focus on the signals of a single subject,without mixing signals from several subjects. Nevertheless, despite the preva-lence of subject-specific approaches, in literature there are some examples ofsubject-independent approaches. Castellini et al. [3] extend their previous worksin two ways: they apply their technique to 10 healthy subjects and they testtheir results in a Daily-Life Activity condition. Matsubara and Morimoto [8] im-plement a bilinear model able to reach an accuracy of 73% classifying differentmovements in a multi-user interface.Previous studies [10], [9] shown good results in the continuous online esti-mation of both upper and lower limbs movement, considering up to 40 differentsubjects. In our previous work we focused on subject-independent regression so-lutions for the continuous estimation of several joints for a fixed movement. Inthis paper we move to a subject-independent framework for the classification ofhand grasps. The fusion of signals from several different subjects produces gener-ally slightly worst performances, but has the vantage of obtaining a more robustand general model. Thanks to this generalization, a new subject can use theframework without the need of long training phases, or with no training phaseat all. In fact, the obtained model has a so large variability, to let it embodya wide number of possible subjects behaviours. Furthermore, we used a freelyavailable online dataset, the data was not registered ad-hoc for our intentions as-suring general results, and the availability of the dataset makes the experimentscomparable and reproducible.3In order to use the sEMG signals we need some additional tools capableof conforming the signals and to allow an online processing. The needing ofsuch tools is even more relevant for the subject-independent approach due tothe greater signals variability. A crucial part is committed to the preprocessingphase: it is fundamental to even differences between signals due to noise, muscu-lar and physical fatigue, and so on, but on the meantime to highlight the intrinsiccharacteristic of the movement. In order to be able to control a prosthetic devicewe need a twofold approach:– Obtaining good prediction of the movement, in order to be able to reproducecorrectly the requested movement;– Being able to process the information and the prediction in a short time, inorder to be able to work online.Besides the preprocessing phase, better results can be obtained by imple-menting new classification techniques. A large number of classification techniqueshave been proposed during the years to establish a robust interaction with pros-thesis devices. In [7] Ju et al. present the classification of 10 different grasps orin-hand manipulations using Fuzzy Gaussian Mixture Model, achieving an ac-curacy of 96.7%. Bu et al. [2] propose a framework based on Bayesian Networksfor motion classification of a cooking task. The manually designed Bayesian Net-works extract the statistical dependency between two continuous motions and itis combined with the output of a probabilistic Neural Network classifier, to im-prove stability and accuracy. In classification frameworks only a limited subsetof movements are considered. In real daily applications movements are complexand a single movement is composed by many simpler sub-movements. In [12]Yang et al. study how the composition of a movement affect the movementclassification, with the aim of using this technology in real Activities of DailyLiving (ADL). They proved that better results can be achieved by includingdynamic arm postures and varying muscular contractions in the training phase.In this paper we present a hierarchical cascade-based classification technique,to both reduce elaboration time and improve the prediction. The results can beachieved by discarding part of the movements while descending into the depthof the tree. Thus, the comparisons are made within a subset of the initial move-ments. This subset is continually refined and reduced until arriving at the clas-sification between a couple of movements.The basic idea behind this approach is that two close movements in thetaxonomy are more hard to distinguish than two distant movements. At the sametime, a misclassification between two close movements is less questionable than amisclassification between two distant ones. This approach is easily generalizableand scalable: new movements can be added in the framework, only by followingthe taxonomy structure.In this work we exploit the first results of the hierarchical and dependencyrelationship between the movements in order to develop a robust classificationframework, able to identify online the movement performed by a subject. Theproposed solution is subject independent, and it works also with new, unseen4subjects. The developed GMM based classification allows an incremental andprogressive classification of the samples, granting robust and online results.The remainder of the paper is structured as follows: Section ?? presents thepreprocessing procedure, together with the probabilistic model. In Section ??we shows and discuss some preliminary results, while Section ?? summarizes thework while proposing some future extensions.2 Methods2.1 Signal analysisIn order to be able to process the data online, an appropriate preprocessing ofthe signal is crucial. The data used in this study come from the NinaPro dataset,a freely online available dataset [1]. A complete description of the dataset can befound in [10], [9], [1]. We used data from N= 40 healthy subjects performing 8different hand grasps, each movement was repeated 6 times. The dataset containsboth information about the muscular activation and the physical movement, butwe only focused on the physiological signals from 8 sensors placed around theforearm.The best classification performances can be achieved if it is possible to detecta clear underlying behaviour between the considered signals. This goal is evenmore important to reach since we are considering signals from a large numberof different subjects. Since the good results obtained in previous studies [11],we started applying at the signal the Wavelet Transform [4] in order to allowan analysis in time and frequency, thus removing the dependence from time andgranting online processing. We used the db2 mother wavelet from the Daubechiesfamily and MAV as synthesis function, applying the Wavelet Transform to con-secutive windows of 200 samples. Finally data has been smoothed by applying amoving average lowpass filter and normalized, in order to regularize the output.2.2 Gaussian mixture model and classificationThe EMG signals from the selected channels plus the additional informationof the movement type, have been used to train oﬄine a probabilistic model,namely a GMM. The GMM approximates the input by using a weighted sumof K Gaussian components which better represent the input data, used to trainthe model. A complete overview of the GMM and how it is built can be foundin [10].Once built the model, we proceeded with the online classification phase. Aclassification technique makes us predict the kind of movement the subject isperforming starting from EMG signals from the subject’s muscles. In particular,after the oﬄine phase of GMM building, we processed online each single EMGsignal. Considering the data of a trial performed by a fixed subject, the samplesbelonging to the trial can be denoted as ξt0 = {ξ0, ξ1, ..., ξt}. We compute theProbability Density Function (PDF) of each new sample:ρm = PDF (ξs|γm) (1)5where 1 ≤ s ≤ S is the index of the considered sample within a trial. γm with1 ≤ m ≤M indicates the index of all the possible class of movements, and M isthe number of considered movements. The movement chosen for a fixed sampleis the one with the higher PDF Equation 2.φm = m : max {ρm, 1 ≤ m ≤M} (2)At each time instant t we computed the most probable movement up to thatmoment with 3:ϑs = s : max {φs, 1 ≤ s ≤ S} (3)2.3 TaxonomyFor simplify the work, we considered a subset of 8 movements extracted fromthe complete taxonomy. The chosen movements are classical every day grasps tointeract with common objects. In particular, we have considered the followingmovements: Index Finger Extension, Medium Wrap, Prismatic Four Fingers,Stick, Writing Tripod, Power Sphere, Extension Type, and Power Disk. In Fig-ure 3 is represented the considered taxonomy for the subset of chosen movements.Fig. 1. Subset of the General Taxonomy of Hand Grasps with the movements consid-ered for this experiment.63 ResultsWe tested the framework with a Leave-One-Out approach. The model has beenbuilt on N -1 subjects and tested on the remaining one. The whole process hasbeen repeated for all the N subjects. The testing phase has been organized inlevels of increasing complexity. Level 0 includes the classification between couplesof movement, i.e. writing tripod and prismatic four fingers, medium wrap andpower sphere. Level 1 rises of one level in the binary tree and compares a coupleof movement with a single one. Particularly, the movements of the previouslevel are considered as the same ”movement group”. Therefore, in level 1 weclassified between stick vs. writing tripod plus prismatic four fingers, and so on.A graphical example of this procedure is depicted in Figure 2.Fig. 2. Example of the levels organization in the taxonomy-based classification.Figure 2 summarize the achieved results, showing the accuracy obtained forthe executed tests. In particular, we computed the accuracy for each trial andmovement, in every classification level among all the 40 considered subjects. Thefinal accuracy has been computed by fixing the level and averaging among all theremaining factors. The obtained results are generally good, and they confirmedour expectations. Lower levels achieves the worst results, this probably happenssince at this level the two considered movements have the most similar sEMGsignals, thus it common to missclassificate them. This hypothesis is supported7by the fact that high level reaches higher accuracy. In particular, level 3 reachesan accuracy of 97.29%.Fig. 3. Accuracy obtained among all the considered levels for the subject-independenttaxonomy-based classification.4 ConclusionsIn this article we present some preliminar results of a novel approach for theclassification of hand grasps by considering physiological signals, namely sEMG.The proposed solution could help amputated subject to gain their lost function-ality by allowing them to interact with the outer world and with the objectsthat populate it. In the framework the critical part is the classification, espe-cially when we want to classify among a large number of groups misclassificationsare very common and this comports low accuracy. Our solution aims to limitthis problem by exploiting a hierarchical quantitative taxonomy of hand grasps.The binary tree structure of the taxonomy comports a classification betweentwo groups of movements, close to each other. The classification becomes moreprecise descending the tree and reaching the root, where the classification is re-stricted between a couple of movements. We built a general, subject-independentframework, able to adapt to new, unseen subject. In particular, we considereddata from 40 healthy subjects performing 8 common grasps. The information hasbeen modelized by using the GMM, and the classification process is determinedonline by using a gaussian-based framework. The obtained results are promising:we obtained a mean accuracy of 76.5%, reaching 97.29% in one of the higher lev-els. In some future works we want to try a new classification framework, and tocompare the accuracy obtained exploiting the taxonomy and the one obtainedwith the traditional approach.References1. Atzori, M., Gijsberts, A., Heynen, S., Hager, A.G.M., Deriaz, O., Van Der Smagt,P., Castellini, C., Caputo, B., Mu¨ller, H.: Building the ninapro database: A re-8source for the biorobotics community. In: Biomedical Robotics and Biomechatron-ics (BioRob), 2012 4th IEEE RAS & EMBS International Conference on. pp. 1258–1265. IEEE (2012)2. Bu, N., Okamoto, M., Tsuji, T.: A hybrid motion classification approach for emg-based human–robot interfaces using bayesian and neural networks. IEEE Transac-tions on Robotics 25(3), 502–511 (2009)3. Castellini, C., Fiorilla, A., Sandini, G.: Multi-subject/daily-life activity emg-basedcontrol of mechanical hands. Journal of NeuroEngineering and Rehabilitation 6(1),41 (2009), http://www.jneuroengrehab.com/content/6/1/414. Chun-Lin, L.: A tutorial of the wavelet transform. NTUEE, Taiwan (2010)5. Dimitrova, N., Dimitrov, G.: Interpretation of emg changes with fatigue: facts,pitfalls, and fallacies. Journal of Electromyography and Kinesiology 13(1), 13–36(2003)6. Feix, T., Romero, J., Schmiedmayer, H.B., Dollar, A.M., Kragic, D.: The grasptaxonomy of human grasp types. IEEE Transactions on Human-Machine Systems46(1), 66–77 (2016)7. Ju, Z., Ouyang, G., Wilamowska-Korsak, M., Liu, H.: Surface emg based hand ma-nipulation identification via nonlinear feature extraction and classification. IEEESensors Journal 13(9), 3302–3311 (2013)8. Matsubara, T., Morimoto, J.: Bilinear modeling of emg signals to extract user-independent features for multiuser myoelectric interface. Biomedical Engineering,IEEE Transactions on 60(8), 2205–2213 (2013)9. Stival, F., Michieletto, S., Pagello, E.: Subject-independent modeling of semg sig-nals for the motion of a single robot joint. In: Workshop of Robotics: Science andSystems (2015)10. Stival, F., Michieletto, S., Pagello, E.: Online subject-independent modeling ofsemg signals for the motion of a single robot joint. In: Biomedical Robotics andBiomechatronics (BioRob), 2016 6th IEEE International Conference on. pp. 1110–1116. IEEE (2016)11. Valentini, R., Michieletto, S., Spolaor, F., Sawacha, Z., Pagello, E.: Processing ofsemg signals for online motion of a single robot joint through gmm modelization.In: Rehabilitation Robotics (ICORR), 2015 IEEE International Conference on. pp.943–949. IEEE (2015)12. Yang, D., Yang, W., Huang, Q., Liu, H.: Classification of multiple finger motionsduring dynamic upper limb movements. IEEE journal of biomedical and healthinformatics 21(1), 134–141 (2017)

A first approach to a taxonomy-based classification framework for hand grasps

Padua Research Archive

https://www.research.unipd.it/bitstream/11577/3270209/1/Paper_2018_LAIAR_First_taxonomy_classification.pdf

A first approach to a taxonomy-based classification framework for hand grasps

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Università di Padova

Padua Research Archive