Capturing Human Hand Kinematics for Object Grasping and Manipulation

The aim of this thesis is to create a low-cost sensor equipped glove using commercially available components that can be used to obtain position, velocity and acceleration data for individual fingers of a hand within an optical motion capture environment. Tracking the full degrees of freedoms of the hand and finger motions without any hindrances is a challenging task in optical motion capture measurements. Attaching markers on every finger and hand joint makes motion capture systems troublesome due to practical problems such as blind spots and/or obtaining higher derivative motion constraints, such as velocities and accelerations. To alleviate this, we propose a method to capture the hand and finger kinematics with a reduced set of optical markers. Additionally inertial sensors are attached to the fingertips to obtain linear acceleration measurements. For optimal velocity estimation, a Kinematic Kalman Filter (KKF) is implemented and its result is compared to the time derivative of the Motion Capture System measurement. The higher derivative specifications are related to contact and curvature constraints between the fingers and the grasped object and are later used in formulating the synthesis task for the design of robotic fingers and hands. A preliminary prototype device has been developed to obtain position, velocity and acceleration information of each fingertip by incorporating multiple accelerometers into the basic design of reduced marker set

Hierarchical recognition of intentional human gestures for sports video annotation

We present a novel technique for the recognition of complex human gestures for video annotation using accelerometers and the hidden Markov model. Our extension to the standard hidden Markov model allows us to consider gestures at different levels of abstraction through a hierarchy of hidden states. Accelerometers in the form of wrist bands are attached to humans performing intentional gestures, such as umpires in sports. Video annotation is then performed by populating the video with time stamps indicating significant events, where a particular gesture occurs. The novelty of the technique lies in the development of a probabilistic hierarchical framework for complex gesture recognition and the use of accelerometers to extract gestures and significant events for video annotation

Measurement of the Flexible Bending Force of the Index and Middle Fingers for Virtual Interaction

AbstractIn this paper the development of a new low cost dataglove based on fingertip bending tracking techniques for measuring the fingers bending on various virtual interaction activities is presented as an alternative to the rehabilitation services enhancement in the betterment of the quality of life especially for the disabled person. The purpose of the research is to design a flexible control for measurement study of virtual interaction of index and middle fingers that are important in a variety of contexts as well as the deterministic approach. These analyses of fingers flexing of the system were using the flexible bend sensor functioning as a key intermediate of the process to track the fingertip positions and orientations. The main propose of the low cost dataglove is to provide natural input control of interaction in virtual, multimodal and tele-presence environments as an input devices provide as they can monitor the dexterity and flexibility characteristics of the human hand motion. Preliminary experimental results have shown that the dataglove capable to measure several human Degree of Freedom (DoF), “translating” them into commands for the interaction in the virtual world

Methods and hints to linearise the resistance values vs. bending angle relationship of bend sensors

The correct measure of static and dynamic postures of patients is a fundamental element for dispensing correct rehabilitation procedures. Nowadays there are different sensors and transducers useful to reach the aim of measuring human postures, even in a non uncomfortable way, during normal activities of everyday life. Among all these sensors we selected the flex ones stand their cheapness an good performances in terms of reliability and stability of electrical signal they provide. It is possible to measure flex-extension of human joints simply laying the flex sensors on wrist, knee, elbow, ankle, etc. But a drawback is paid for these sensors, because of a non linear function of their electrical resistance variation vs. bending angle. The non linearity involves a time consuming calibration, more complexity of the conditioning electronics, more troubles for drift problems and the issue to establish the best fit algorithm. So here we propose methodologies and hints to linearise the sensor’s electrical function

Photonic sensors based on flexible materials with FBGs for use on biomedical applications

This chapter is intended for presenting biomedical applications of FBGs embedded into flexible carriers for enhancing the sensitivity and to provide interference-free instrumentation.This work was fully supported by the Algoritmi’s Strategic Project UI 319-2011-2012, under the Portuguese Foundation for Science and Technology grant Pest C/EEI/UI0319/2011

Development of a Fingertip Glove Equipped with Magnetic Tracking Sensors

In this paper, we present the development of a data glove system based on fingertip tracking techniques. To track the fingertip position and orientation, a sensor module and two generator coils are attached on the fingertip and metacarpal of the corresponding finger. By tracking the fingertip, object manipulation tasks in a virtual environment or teleoperation system can be carried out more precisely, because fingertips are the foremost areas that reach the surface of an object in most of grasping processes. To calculate the bending angles of a finger, we also propose a method of constructing the shape of the finger. Since the coils are installed on the fingertips and metacarpals, there is no contact point between the sensors and finger joints. Hence, the shape of the sensors does not change as the fingers are bending, and both the quality of measurement and the lifetime of the sensors will not decrease in time. For the convenience of using this glove, a simple and efficient calibration process consisting of only one calibration gesture is also provided, so that all required parameters can be determined automatically. So far, the experimental results of the sensors performing linear movement and bending angle measurements are very satisfactory. It reveals that our data glove is available for a man-machine interface

Segmentation of intentional human gestures for sports video annotation

We present results on the recognition of intentional human gestures for video annotation and retrieval. We define a gesture as a particular, repeatable, human movement having a predefined meaning. An obvious application of the work is in sports video annotation where umpire gestures indicate specific events. Our approach is to augment video with data obtained from accelerometers worn as wrist bands by one or more officials. We present the recognition performance using a Hidden Markov Model approach for gesture modeling with both isolated gestures and gestures segmented from a stream

An Inertial Measurement System for Hand and Finger Tracking

The primary Human Computer Interfaces (HCI) today are the keyboard and mouse. These interfaces do not facilitate a fluid flow of thought and intent from the operator to the computer. A computer mouse provides only 2 Degrees of Freedom (2DOF). Touch interfaces also provide 2DOF, but with multiple points, making the touch interface far more expressive. The hand has 6 Degrees of Freedom (6DOF)by itself. Combined with the motion of the fingers, the hand has the potential to represent a vast array of differing gestures. Hand gestures must be captured before they can be used as a HCI. Fortunately, advances in device manufacturing now make it possible to build a complete Inertial Measurement Unit (IMU) the size of a fingernail. This thesis documents the design and development of a glove out tted with six IMUs. The IMUs are used to track the finger and hand positions. The glove employs a controller board for capturing IMU data and interfacing with the host computer. Python™ software on the host computer captures data from the glove. MATLAB™ is used to perform IMU calculations of the incoming data. The calculated data drives a 3D visualization of the glove rendered in Panda3D™. Future work using the glove would include improved IMU algorithms and development of gesture pattern recognition

Wrist-worn gesture sensing with wearable intelligence

This paper presents an innovative wrist-worn device with machine learning capabilities and a wearable pressure sensor array. The device is used for monitoring different hand gestures by tracking tendon movements around the wrist. Thus, an array of PDMS-encapsulated capacitive pressure sensors is attached to the user to capture wrist movement. The sensors are embedded on a flexible substrate and their readout requires a reliable approach for measuring small changes in capacitance. This challenge was addressed by measuring the capacitance via the switched capacitor method. The values were processed using a programme on LabVIEW to visually reconstruct the gestures on a computer. Additionally, to overcome limitations of tendon’s uncertainty when the wristband is re-worn, or the user is changed, a calibration step based on the Support Vector Machine (SVM) learning technique is implemented. Sequential Minimal Optimization (SMO) algorithm is also applied in the system to generate SVM classifiers efficiently in real-time. The working principle and the performance of the SVM algorithms demonstrate through experiments. Three discriminated gestures have been clearly separated by SVM hyperplane and correctly classified with high accuracy (>90%) during real-time gesture recognition

MOCA: A Low-Power, Low-Cost Motion Capture System Based on Integrated Accelerometers

Human-computer interaction (HCI) and virtual reality applications pose the challenge of enabling real-time interfaces for natural interaction. Gesture recognition based on body-mounted accelerometers has been proposed as a viable solution to translate patterns of movements that are associated with user commands, thus substituting point-and-click methods or other cumbersome input devices. On the other hand, cost and power constraints make the implementation of a natural and efficient interface suitable for consumer applications a critical task. Even though several gesture recognition solutions exist, their use in HCI context has been poorly characterized. For this reason, in this paper, we consider a low-cost/low-power wearable motion tracking system based on integrated accelerometers called motion capture with accelerometers (MOCA) that we evaluated for navigation in virtual spaces. Recognition is based on a geometric algorithm that enables efficient and robust detection of rotational movements. Our objective is to demonstrate that such a low-cost and a low-power implementation is suitable for HCI applications. To this purpose, we characterized the system from both a quantitative point of view and a qualitative point of view. First, we performed static and dynamic assessment of movement recognition accuracy. Second, we evaluated the effectiveness of user experience using a 3D game application as a test bed