Doctor of Philosophy

dissertationWhen interacting with objects, humans utilize their sense of touch to provide information about the object and surroundings. However, in video games, virtual reality, and training exercises, humans do not always have information available through the sense of touch. Several types of haptic feedback devices have been created to provide touch information in these scenarios. This dissertation describes the use of tactile skin stretch feedback to provide cues that convey direction information to a user. The direction cues can be used to guide a user or provide information about the environment. The tactile skin stretch feedback devices described herein provide feedback directly to the hands, just as in many real life interactions involving the sense of touch. The devices utilize a moving tactor (actuated skin contact surface, also called a contactor) and surrounding material to give the user a sense of the relative motion. Several game controller prototypes with skin stretch feedback embedded into the device to interface with the fingers were constructed. Experiments were conducted to evaluate user performance in moving the joysticks to match the direction of the stimulus. These experiments investigated stimulus masking effects with both skin stretch feedback and vibrotactile feedback. A controller with feedback on the thumb joysticks was found to have higher user accuracy. Next, precision grip and power grip skin stretch feedback devices were created to investigate cues to convey motion in a three-dimensional space. Experiments were conducted to compare the two devices and to explore user accuracy in identifying different direction cue types. The precision grip device was found to be superior in communicating direction cues to users in four degrees of freedom. Finally, closed-loop control was implemented to guide users to a specific location and orientation within a three-dimensional space. Experiments were conducted to improve controller feedback which in turn improved user performance. Experiments were also conducted to investigate the feasibility of providing multiple cues in succession, in order to guide a user with multiple motions of the hand. It was found that users can successfully reach multiple target locations and orientations in succession

Master of Science

thesisHaptic interactions with smartphones are generally restricted to vibrotactile feedback that offers limited distinction between delivered tactile cues. The lateral movement of a small, high-friction contactor at the fingerpad can be used to induce skin stretch tangent to the skin's surface. This method has been demonstrated to reliably communicate four cardinal directions with 1 mm translations of the device's contactor, when finger motion is properly restrained. While earlier research has used a thimble to restrain the finger, this interface has been made portable by incorporating a simple conical hole as a finger restraint. An initial portable device design used RC hobby servos and the conical hole finger restraint, but the shape and size of this portable device wasn't compatible with smartphone form factors. This design also had significant compliance and backlash that must be compensated for with additional control schemes. In contrast, this thesis presents the design, fabrication, and testing of a low-profile skin-stretch display (LPSSD) with a novel actuation design for delivering complex tactile cues with minimal backlash or hysteresis of the skin contactor or "tactor." This flatter mechanism features embedded sensors for fingertip cursor control and selection. This device's nonlinear tactor motions are compensated for using table look-up and high-frequency open-loop control to create direction cues with 1.8 mm radial tactor displacements in 16 directions (distributed evenly every 22.5°) before returning to center. Two LPSSDs are incorporated into a smartphone peripheral and used in single-handed and bimanual tests to identify 16 directions. Users also participated in "relative" identification tests where they were first provided a reference direction cue in the forward/north direction followed by the cue direction that they were to identify. Tests were performed with the user's thumbs oriented in the forward direction and with thumbs angled inward slightly, similar to the angledthumb orientation console game controllers. Users are found to have increased performance with an angled-thumb orientation. They performed similarly when stimuli were delivered to their right or left thumbs, and had significantly better performance judging direction cues with both thumbs simultaneously. Participants also performed slightly better in identifying the relative direction cues than the absolute

Tactile Weight Rendering: A Review for Researchers and Developers

Haptic rendering of weight plays an essential role in naturalistic object interaction in virtual environments. While kinesthetic devices have traditionally been used for this aim by applying forces on the limbs, tactile interfaces acting on the skin have recently offered potential solutions to enhance or substitute kinesthetic ones. Here, we aim to provide an in-depth overview and comparison of existing tactile weight rendering approaches. We categorized these approaches based on their type of stimulation into asymmetric vibration and skin stretch, further divided according to the working mechanism of the devices. Then, we compared these approaches using various criteria, including physical, mechanical, and perceptual characteristics of the reported devices and their potential applications. We found that asymmetric vibration devices have the smallest form factor, while skin stretch devices relying on the motion of flat surfaces, belts, or tactors present numerous mechanical and perceptual advantages for scenarios requiring more accurate weight rendering. Finally, we discussed the selection of the proposed categorization of devices and their application scopes, together with the limitations and opportunities for future research. We hope this study guides the development and use of tactile interfaces to achieve a more naturalistic object interaction and manipulation in virtual environments.Comment: 15 pages, 2 tables, 3 figures, surve

Haptic-Motor Transformations for the Control of Finger Position

abstract: Dexterous manipulation relies on modulation of digit forces as a function of digit placement. However, little is known about the sense of position of the vertical distance between finger pads relative to each other. We quantified subjects' ability to match perceived vertical distance between the thumb and index finger pads (d[subscript y]) of the right hand (“reference” hand) using the same or opposite hand (“test” hand) after a 10-second delay without vision of the hands. The reference hand digits were passively placed non-collinearly so that the thumb was higher or lower than the index finger (d[subscript y] = 30 or –30 mm, respectively) or collinearly (d[subscript y] = 0 mm). Subjects reproduced reference hand d[subscript y] by using a congruent or inverse test hand posture while exerting negligible digit forces onto a handle. We hypothesized that matching error (reference hand d[subscript y] minus test hand d[subscript y]) would be greater (a) for collinear than non-collinear d[subscript y]s, (b) when reference and test hand postures were not congruent, and (c) when subjects reproduced dy using the opposite hand. Our results confirmed our hypotheses. Under-estimation errors were produced when the postures of reference and test hand were not congruent, and when test hand was the opposite hand. These findings indicate that perceived finger pad distance is reproduced less accurately (1) with the opposite than the same hand and (2) when higher-level processing of the somatosensory feedback is required for non-congruent hand postures. We propose that erroneous sensing of finger pad distance, if not compensated for during contact and onset of manipulation, might lead to manipulation performance errors as digit forces have to be modulated to perceived digit placement.The article is published at http://journals.plos.org/plosone/article?id=10.1371/journal.pone.006614

Separating haptic guidance from task dynamics: A practical solution via cutaneous devices

There is much interest in using haptic feedback for training new skills or guiding human movement. However, the results of studies that have incorporated haptic guidance to train new skills are mixed, depending on task complexity and the method by which the haptic guidance is implemented. Subjects show dependency on the guidance forces and difficulty in discerning which aspects of the haptic feedback are related to the task dynamics and which are meant to convey task completion strategies. For these reasons, new methods to separate haptic cues for guidance from haptic feedback of task dynamics are needed. In this experiment, 30 subjects completed a trajectory following task using a wrist exoskeleton which also rendered task forces. To assist subjects, guidance cues were provided in one of three forms: (1) cutaneous forces from a wearable skin-stretch device on the ipsilateral forearm and (2) contralateral forearm, and (3) kinesthetic forces from a kinematically similar wrist exoskeleton operated by the contralateral arm. The efficacies of each guidance condition are compared by examining subject performance and learning rates. The results indicate that cutaneous guidance is nearly as effective as kinesthetic guidance, making it a practical and cost-effective alternative for spatially separated assistance

Tactile-STAR: A Novel Tactile STimulator And Recorder System for Evaluating and Improving Tactile Perception

Many neurological diseases impair the motor and somatosensory systems. While several different technologies are used in clinical practice to assess and improve motor functions, somatosensation is evaluated subjectively with qualitative clinical scales. Treatment of somatosensory deficits has received limited attention. To bridge the gap between the assessment and training of motor vs. somatosensory abilities, we designed, developed, and tested a novel, low-cost, two-component (bimanual) mechatronic system targeting tactile somatosensation: the Tactile-STAR—a tactile stimulator and recorder. The stimulator is an actuated pantograph structure driven by two servomotors, with an end-effector covered by a rubber material that can apply two different types of skin stimulation: brush and stretch. The stimulator has a modular design, and can be used to test the tactile perception in different parts of the body such as the hand, arm, leg, big toe, etc. The recorder is a passive pantograph that can measure hand motion using two potentiometers. The recorder can serve multiple purposes: participants can move its handle to match the direction and amplitude of the tactile stimulator, or they can use it as a master manipulator to control the tactile stimulator as a slave. Our ultimate goal is to assess and affect tactile acuity and somatosensory deficits. To demonstrate the feasibility of our novel system, we tested the Tactile-STAR with 16 healthy individuals and with three stroke survivors using the skin-brush stimulation. We verified that the system enables the mapping of tactile perception on the hand in both populations. We also tested the extent to which 30 min of training in healthy individuals led to an improvement of tactile perception. The results provide a first demonstration of the ability of this new system to characterize tactile perception in healthy individuals, as well as a quantification of the magnitude and pattern of tactile impairment in a small cohort of stroke survivors. The finding that short-term training with Tactile-STARcan improve the acuity of tactile perception in healthy individuals suggests that Tactile-STAR may have utility as a therapeutic intervention for somatosensory deficits

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Doctor of Philosophy

dissertationVirtual environments provide a consistent and relatively inexpensive method of training individuals. They often include haptic feedback in the form of forces applied to a manipulandum or thimble to provide a more immersive and educational experience. However, the limited haptic feedback provided in these systems tends to be restrictive and frustrating to use. Providing tactile feedback in addition to this kinesthetic feedback can enhance the user's ability to manipulate and interact with virtual objects while providing a greater level of immersion. This dissertation advances the state-of-the-art by providing a better understanding of tactile feedback and advancing combined tactilekinesthetic systems. The tactile feedback described within this dissertation is provided by a finger-mounted device called the contact location display (CLD). Rather than displaying the entire contact surface, the device displays (feeds back) information only about the center of contact between the user's finger and a virtual surface. In prior work, the CLD used specialized two-dimensional environments to provide smooth tactile feedback. Using polygonal environments would greatly enhance the device's usefulness. However, the surface discontinuities created by the facets on these models are rendered through the CLD, regardless of traditional force shading algorithms. To address this issue, a haptic shading algorithm was developed to provide smooth tactile and kinesthetic interaction with general polygonal models. Two experiments were used to evaluate the shading algorithm. iv To better understand the design requirements of tactile devices, three separate experiments were run to evaluate the perception thresholds for cue localization, backlash, and system delay. These experiments establish quantitative design criteria for tactile devices. These results can serve as the maximum (i.e., most demanding) device specifications for tactile-kinesthetic haptic systems where the user experiences tactile feedback as a function of his/her limb motions. Lastly, a revision of the CLD was constructed and evaluated. By taking the newly evaluated design criteria into account, the CLD device became smaller and lighter weight, while providing a full two degree-of-freedom workspace that covers the bottom hemisphere of the finger. Two simple manipulation experiments were used to evaluate the new CLD device

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

A Haptic Feedback System for Lower Limb Amputees Based on Gait Event Detection

Lower limb amputation has significant effects on a person’s quality of life and ability to perform activities of daily living. Prescription of prosthetic device post amputation aims to help restore some degrees of mobility function, however studies have shown evidence of low balance confidence and higher risk of falling among amputee community, especially those suffering from above knee amputation. While advanced prostheses offer better control, they often lack a form of feedback that delivers the awareness of the limb position to the prosthetic user while walking. This research presents the development and evaluation of a wearable skinstretch haptic feedback system intended to deliver cues of two crucial gait events, namely the Initial Contact (IC) and Toe-off (TO) to its wearer. The system comprises a haptic module that applies lateral skin-stretch on the upper leg or the trunk, corresponding to the gait event detection module based on Inertial Measurement Unit (IMU) attached at the shank. The design and development iterations of the haptic module is presented, and characterization of the feedback parameters is discussed. The validation of the gait event detection module is carried out and finally the integration of the haptic feedback system is described. Experimental work with healthy subjects and an amputee indicated good perceptibility of the feedback during static and dynamic (walking) condition, although higher magnitude of stretch was required to perceive the feedback during dynamic condition. User response time during dynamic activity showed that the haptic feedback system is suitable for delivering cues of IC and TO within the duration of the stance phase. In addition, feedback delivered in discernible patterns can be learned and adapted by the subjects. Finally, a case study was carried out with an above-knee amputee to assess the effects of the haptic feedback on spatio-temporal gait parameters and on the vertical ground reaction force during treadmill and overground walking. The research presented in this report introduces a novel design of a haptic feedback device. As such, the outcome includes a well-controlled skin-stretch effect which contributes to the research by investigating skin-stretch feedback for conveying discrete event information rather than conveying direction information as presented in other studies. In addition, it is found that stretch magnitude as small as 3 mm could be perceived in short duration of 150 ms during dynamic condition, making it a suitable alternative to other widely investigated haptic modality such as vibration for ambulatory feedback application. With continuous training, the haptic feedback system could possibly benefit lower limb amputees by creating awareness of the limb placement during ambulation, potentially reducing visual dependency and increasing walking confidence