Kortier, H.G.

Schepers, H. Martin

Sluiter, Victor IJzebrand

Veltink, Petrus H.

English

Inertial and magnetic sensors, attached to various segments of the human hand, can be used to measure movements of the hand. This paper proposes a new method to assess hand kinematics by applying an Extended Kalman Filter in which prior information of the hand is fused with actual measurements obtained from various sensors

University of Twente Research Information

Ambulatory Assesment of Hand Kinematicsusing an instrumented gloveKortier H.G.1, Schepers H.M.2, Sluiter V.I.1, Veltink P.H.11 Biomedical Signals and Systems, MIRA Institute, University of Twente, Enschede, The Netherlands2 Xsens Technologies B.V., Enschede, the NetherlandsFirst author email: h.g.kortier@utwente.nlInertial and magnetic sensors, attached to various segments of the human hand, can be used to measure movements of the hand. This paper proposes a new method to assess hand kinematics by applying an Extended Kalman Filter in which prior information of the hand is fused with actual measurements obtained from various sensors. Keywords - hand kinematics, inertial movement sensing, instrumented glove, sensor fusion1. INTRODUCTION Analysis of hand kinematics is important in several application areas, such as rehabilitation, sports and ergonomics. In particular, ambulatory tracking of the whole hand configuration is valuable for kinematic assessment under daily life conditions. Current instrumented glove systems primary use resistive, magnetic or optical sensing methods [1]. A common drawback of these systems is that the sensing elements are mounted across the phalangeal joints, which requires an accurate alignment of body and sensor axes. Moreover, sensors that measure joints with multiple degrees of freedom (DoF) are difficult to calibrate because they often suffer from crosstalk due to misalignments. Finally, current instrumented gloves do not measure the translational and rotational movements of the complete hand, which is important when assessing the functionality of hand movements.Inertial sensors combined with magnetic sensors have proven to be accurate in estimating orientations without the need for external actuators or cameras [2]. The development of MEMS technology resulted in tiny and low-cost Inertial Measurement Units (IMU’s) that could be implemented in textile clothing easily without impairing the freedom of movement and tactile sensations. It is the objective within the PowerSensor project [3,4] to assess the kinematics of the hand using 3D inertial and magnetic sensors, which are attached to the various segments of the hand. To minimize hardware needs, our current version only allows for tracking of the thumb and index finger with 3D accelerometer and magnetometer pairs, and in addition to, tracking of the hand with a full IMU. However, the described algorithm can handle all fingers and thumb in parallel, so as to capture the movement of all degrees of freedom of the hand. 2. METHODSTheoryThe human hand is a highly articulated system but one that is also highly constrained. It’s skeleton can be modeled with 21 internal DoF (Fig 1). The distal interphalangeal (DIP) joints and proximal interphalangeal (PIP) joints of each finger have one DoF each, while the metacarpophalangeal (MCP) joints have two DoF. Unlike the fingers, the thumb has five DoF. There are two at the trapeziometacarpal (TM) joint, two DoF at the metacarpophalangeal (MCP) joint and the remaining DoF is located at the interphalangeal (IP) joint [1].Figure 1. (Left) Capital bones of the human hand. Joints are indicated with their abbreviations. (Right) Defined coordinate frames within the various bones of the index finger.DIPPIPMCPIPMCPTMDIPPIPMCPWristyzyzyzyz3DAHM201215Estimation of kinematics at any one time is calculated using an Extended Kalman Filter (EKF). The Kalman filter requires a model that describes the evolution of the process and the relation between the measurements taken and the current states, which are the relative segment orientations. The articulated finger and thumb were modeled as two kinematic chains originating from the hand coordinate frame (!! ). This frame is defined by the y-axis pointing to the MCP joint, the z-axis pointing outwards with respect to the dorsal side of the palm and the x-axis is defined according the right-handed coordinate frame (Fig. 1). The position of the fingertip (!!! ) in global frame (!! ) is obtained after transformation of the fingertip expressed in the distal segment (!!! ), which can be described by multiplication of subsequent transformation matrices (H) (Fig. 2):This net transformation !! !" ) contains a rotational part !!!!" !, which is the orientation matrix as function of the net quaternion, and a translational part !!! , which is the distal segment expressed in global frame. The former is estimated using the Kalman filter whereas the latter is measured in advance. In addition, theorientation of the sensors with respect to the particular segment is obtained during a calibration measurement. Relative orientation estimates between two linked segments are used for the Kalman’s filter input. To estimate the orientation we use a 3D accelerometer as an inclination estimate and a 3D magnetometer to estimate the heading. Now, solving the following cost function result in the optimal orientation estimate (!!! ) between two linked segments, for example the hand (H) and the proximal phalanx (M) [5]:The optimal orientation (!!" ) transforms the set of independent vectors measured at the proximal phalanxinto a second set of independent vectors, measured at the hand.Next, during a measurement update the Kalman filter applies dimensionality constraints of the particular jointusing a virtual measurement. The angle around the constrained axis of rotation, for example endo-exoration of the MCP joint, should be zero in the optimal case [2]. In case of the MCP joint this can be measured using the dot product of the hand’s x-axis (!!! ) and the z-axis of the proximal finger segment after transformation to the hand’s coordinate frame (!!!!! !!!! ).Figure 2. Modeled kinematic chain of four rigid bodies illustrating the hand with one finger. To all segments an accelerometer plus magnetometer pair was attached. Given are the vectors describing the position of both the joint and the IMU to its prior coordinate frame.with:!pGE1"= HGHHHMHMPHPD!pDE1"= HGD!pDE1"HGD =!R(qGD) pGD0T3 1"(1)(2)!G!HHandProx. PhalanxMed. PhalanxDist. Phalanx!SM!SP!SDpMSMpMPpPSPpPDpDSDpDEpHMxy!SHpHSH!M!P!DpGHpGEL(R(qHM )) :=12N#i$sH ! R(qHM )sM%23DAHM201216(a) (b)Figure 3. (a) Single sensor PCB containing a 3D accelerometer and 3D magnetometer (b) Instrumented glove. The PCB’s are mounted in a synthetic housing (white). These sensor housings can be expanded with markers for optical measurements. In addition a full 3D IMU (orange, back of the hand is visible. The subjectrepeatedly made contact with the tips of index finger and thumb (pinch task). Experimental methodsMultiple, custom made, printed circuit boards (PCB’s) equipped with a 3D accelerometer and magnetometer pair have been developed (Fig. 3a). They were mounted to the proximal and medial digits of both the index finger and the thumb. In addition a commercial IMU (Xsens technologies), containing a 3D rate gyroscope, 3D magnetometer and 3D accelerometer) was attached to the back of the hand (Fig. 3b). Sensor data is captured by a microcontroller and subsequently transmitted via USB to a computer where it has been processed offline in Matlab.3. RESULTSAs an example trial, one subject performed a pinching task, i.e. the thumb and index finger start completely extended and are subsequently brought in contact by flexing the particular joints (Fig. 3b and 4). Figure 4. Pose estimate of the hand, index finger and thumb during a pinch task (Fig. 3b). Both index and thumb chains originate from the wrist. The axes of joint coordinate frames and end-effectors of both thumb and finger chains are indicated (x is blue, y is green and z is red)3DAHM2012174. DISCUSSIONThe accuracy of the system is currently being evaluated using an optical reference system (PTI VisualEyez 4000). It should be noted that an inaccurate calibration of sensor to segment orientation causes incorrect orientation updates. We are currently working on a sophisticated calibration method that should result in a more accurate calibration.The current method determines the relative orientation using 3D accelerometer and magnetometer pair outputs. It is assumed that the difference in inertial acceleration between two subsequent segments is zero or at least small compared to the gravitational acceleration. Hence, during tasks with large finger accelerations (dynamic tasks like piano playing) the error of estimated orientation will be significant. Moreover, it is assumed that both magnetometers are exposed to the same magnetic fields. Local field disturbances to either of both sensors will also result in incorrect orientation estimates. Therefore a new glove system is currently under development where, in addition to the current sensory, 3D rate gyroscopes will be applied to estimate the orientation under conditions with large inertial acceleration and disturbed magnetic fields.  5. ACKNOWLEDGMENTThis research is supported by the Dutch Technology Foundation STW, which is part of the Netherlands Organization for Scientific Research (NWO) and partly funded by the Ministry of Economic Affairs, Agriculture and Innovation.6. REFERENCES[1] L. Dipietro, A. Sabatini, and P. Dario, “A survey of glove-based systems and their applications,” Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol. 38, no. 4, pp. 461–482, 2008.[2] H. Luinge, P. Veltink, and C. Baten, “Ambulatory measurement of arm orientation,” Journal of Biomechanics, vol. 40, no. 1, pp. 78–85, 2007.[3] H. G. Kortier and P. H. Veltink, “Load identification during object handling,” in Engineering in Medicine and Biology Society,EMBC, 2011 Annual International Conference of the IEEE, 2011, pp. 3500–3502.[4] P. Veltink, H. Kortier, and H. Schepers, “Sensing Power Transfer Between the Human Body and the Environment,” Biomedical Engineering, IEEE Transactions on, vol. 56, no. 6, pp. 1711–1718, 2009.[5] F. Markley and D. Mortari, “Quaternion attitude estimation using vector observations.,” Journal of the Astronautical Sciences, vol. 48, no. 2, pp. 359–380, 2000.3DAHM201218

Ambulatory Assesment of Hand Kinematics, using an instrumented glove

https://ris.utwente.nl/ws/files/5384911/Kortier12ambulatory.pdf

Ambulatory Assesment of Hand Kinematics, using an instrumented glove

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information