We propose a haptic system that applies forces or skin deformation to the
user's arm, rather than at the fingertips, for believable interaction with
virtual objects as an alternative to complex thimble devices. Such a haptic
system would be able to convey information to the arm instead of the
fingertips, even though the user manipulates virtual objects using their hands.
We developed a set of haptic sketches to determine which directions of skin
deformation are deemed more believable during a grasp and lift task. Subjective
reports indicate that normal forces were the most believable feedback to
represent this interaction.Comment: 2 pages, work in progres

Di Luca, Massimiliano

Okamura, Allison M.

Sarac, Mine

English

arXiv

We propose a haptic system that applies forces or skin deformation to the user's arm, rather than at the fingertips, for believable interaction with virtual objects as an alternative to complex thimble devices. Such a haptic system would be able to convey information to the arm instead of the fingertips, even though the user manipulates virtual objects using their hands. We developed a set of haptic sketches to determine which directions of skin deformation are deemed more believable during a grasp and lift task. Subjective reports indicate that normal forces were the most believable feedback to represent this interaction

University of Birmingham Research Portal

  University of BirminghamHaptic Sketches on the Arm for Manipulation inVirtual RealitySarac, Mine; Okamura, Allison M.; Di Luca, MassimilianoCitation for published version (Harvard):Sarac, M, Okamura, AM & Di Luca, M 2019, Haptic Sketches on the Arm for Manipulation in Virtual Reality. inworld haptics conference. IEEE World Haptics Conference.Link to publication on Research at Birmingham portalGeneral rightsUnless a licence is specified above, all rights (including copyright and moral rights) in this document are retained by the authors and/or thecopyright holders. The express permission of the copyright holder must be obtained for any use of this material other than for purposespermitted by law.•	Users may freely distribute the URL that is used to identify this publication.•	Users may download and/or print one copy of the publication from the University of Birmingham research portal for the purpose of privatestudy or non-commercial research.•	User may use extracts from the document in line with the concept of ‘fair dealing’ under the Copyright, Designs and Patents Act 1988 (?)•	Users may not further distribute the material nor use it for the purposes of commercial gain.Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.When citing, please reference the published version.Take down policyWhile the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has beenuploaded in error or has been deemed to be commercially or otherwise sensitive.If you believe that this is the case for this document, please contact UBIRA@lists.bham.ac.uk providing details and we will remove access tothe work immediately and investigate.Download date: 14. Jun. 2020Haptic Sketches on the Arm for Manipulation in Virtual RealityMine Sarac1, Allison M. Okamura1, and Massimiliano Di Luca2Abstract— We propose a haptic system that applies forcesor skin deformation to the user’s arm, rather than at thefingertips, for believable interaction with virtual objects as analternative to complex thimble devices. Such a haptic systemwould be able to convey information to the arm instead ofthe fingertips, even though the user manipulates virtual objectsusing their hands. We developed a set of haptic sketches todetermine which directions of skin deformation are deemedmore believable during a grasp and lift task. Subjective reportsindicate that normal forces were the most believable feedbackto represent this interaction.I. INTRODUCTIONIn the real world, mechanical properties of objects, such asmass, stiffness and temperature, are mostly sensed throughtouch. Haptic devices aim to recreate the same feelingfor virtual interactions, typically at the fingertip. Multi-degree-of-freedom fingertip devices (Fig. 1 (a)) provide skindeformation for compelling and realistic feedback [1], [2].Even though these multi-degree-of-freedom devices improveperformance and perceived realism of manipulation tasks,they are complex, bulky, and expensive.We propose a different approach to artificial haptic feed-back, by relocating the haptic feedback that conveys infor-mation about mechanical properties of manipulated virtualobjects from the fingertip to the arm (Fig. 1 (b)). Rather thanrecreating realistic feedback to match the stimulation of realobjects, we posit that sensory signals, which merely hint atthe real properties of an object, might be sufficient to createinterpretable or “believable” interactions. Such a concept hasbeen applied, for example, in pseudo-haptic rendering, wherethe visual analogies are used to inform the user of whathaptic effects they would have experienced.The typical requirement of realism turns into the newrequirement of believability, if haptic feedback accompaniesvirtual manipulation tasks with visual cues. In this context,a believable haptic feedback can convey information aboutmaterial properties of objects to be useful for manipulationtasks, so that it qualitatively adds to (rather than detractsfrom) the user experience.Prior work has examined methods for haptic feedback atthe arm. For example, “CUFF” applies normal and tangentialforces to render interaction forces between objects in theenvironment, and a hand prosthesis, which is worn by thesame user who wears the “CUFF” [3]. “hBracelet” alsoapplies normal and tangential forces to render interactionforces that occur during virtual manipulation or teleoperation1 Department of Mechanical Engineering, Stanford University, Stanford,CA. msarac@stanford.edu, aokamura@stanford.edu2 Facebook Reality Labs, and University of Birmingham, UKmax.diluca@oculus.comFig. 1. Haptic feedback modalities: (a) current haptic devices where afingertip device conveys grasping (Fg) forces, and (b) the proposed hapticsystem where grasping (Fg) forces are rendered at the wrist or arm.tasks [4]. Moriyama et al. [5] also developed a haptic devicewith two five-bar linkage mechanisms, applying normal andtangential forces, to represent interaction forces acting onthe index finger and the thumb independently during virtualgrasping tasks. Even though these devices were developedwith different motivations, they share the mutual goal (andassociated design complexity) of creating realistic feedbackfor the user.Simpler haptic devices could instead be designed to renderbelievable information to the user about the physical proper-ties of virtual objects during manipulation tasks. To investi-gate the use of skin stretch applied at the wrist as a way tocreate believable sensations, we investigated which directionsof skin deformation on a user’s arm would result in the mostbelievable feedback. We created a set of haptic sketches inwhich feedback forces (applied via skin deformation) werestimulated manually. Here we present these haptic sketchesand our findings, and discuss future work toward designinga wearable haptic device.II. HAPTIC SKETCHES FOR VIRTUAL MANIPULATIONWe investigated how users perceived interactions duringvirtual manipulation tasks presented via skin deformation onthe arm, while the users viewed the manipulation task in thevirtual environment. We implemented simple haptic sketches,which were introduced by Moussette [6]. Haptic sketchesaim to stimulate feedback forces manually or through simpletools (like magnets) instead of complex mechanical devices.Even though applying forces manually might not correlatewith the virtual interactions accurately, the repetition ofthese tasks might result in sufficient believability to compareamong different feedback modalities. Furthermore, removingthe need for a mechatronic device gave us the opportunityto quickly test several types of stimulation.Fig. 2 shows the setup we employed for haptic sketches.During trials, a user was asked to sit in front of a monitor andarXiv:1911.08528v1  [cs.RO]  14 Nov 2019Fig. 2. Haptic sketches: (a) the setup with a virtual environment, twomotion trackers to be worn at the fingertips on the index and middle fingers,and a folded wooden stick to stimulate haptic feedback manually by theexperimenter, and (b) different haptic feedback modalities applied to theuser, such as normal, sliding and stretch forces. Skin stretch forces areimplemented using a double-sided tape between the stirrer and user’s skin.wear electromagnetic finger trackers on the thumb and indexfinger. On the monitor, the user sees a representation of avirtual box and their virtual fingertips, whose movements aremapped to the actual fingers. The user is asked to grasp theobject and move it around. In the meantime, the experimenterapplies feedback in the form of skin stretch to the user’s armat a single point using a folded wooden stick (made from acoffee stir stick). The stick is folded into a U-shape so thatthe middle area is used as the contact area, and the openextremities are used as a handle by the experimenter to applyforces. For normal and sliding forces, the stick contacts theuser’s skin directly, while for stretch forces, double-sidedtape is used. The experimenter aims to provide feedbackforces to reflect the stiffness of the object being grasped.The haptic sketches were tested on five subjects, allstudents who are familiar with haptics. Each user experiencedall feedback modalities repeatedly until they felt confidentwith the perception. After each feedback modality, subjectswere asked whether they could feel the feedback throughoutthe whole manipulation task, how they felt about its cor-respondence with the manipulation task, and whether theycould learn to interpret the mechanical properties of objectswith it. Finally, they were asked to compare all modalities.All five users reported the same experience. Such anagreement convinced us that we have a reliable approachfor future research. The results suggest that normal forcesand stretch forces led to a believable experience of materialproperties of objects, while sliding sensations were notperceived to be related to manipulation actions or mechanicalproperties in the virtual environment. Even though usersreported that they could perceive mechanical properties of theobjects using both normal and stretch forces, normal forcesfelt more realistic. It is possible that users relate normalforces to natural muscle contraction that occurs as they movetheir fingers.Even though normal and stretch forces might have similarimpact on user perception, they are different in terms ofmechanical implementation. Skin stretch highly depends onhow the end-effector of the haptic device is attached to theuser’s skin; we need to prevent it from slipping. Even thoughtwo-sided tape or glue can be used to attach mechanical partsto the skin, its practicality as well as implementation on hairyskin might be problematic. Furthermore, the mechanical partsneed to be grounded on the arm, which results in reaction orgrounding forces that also affect user perception. Yet, normalforces are expected to be much simpler to implement.As a result of these haptic sketches and implementationconsiderations, we conclude that a believable haptic deviceshould apply normal forces as users manipulate virtualobjects and view their actions in a virtual environment.III. FUTURE WORKHaving decided to apply normal forces during virtualmanipulation tasks, we will implement a mechanical devicefor further investigations. Even though our haptic sketchesapplied forces to a single area on the surface of the user’sarm, we expect that perception can be improved if normalforces are applied at multiple contact points or differentlocations. The performed haptic sketches focused on stiffnessproperties of the object, since the forces were applied basedon the finger movement with which the user grasped theobject. In future implementations, the haptic device couldalso render inertial forces based on how heavy the object isand the manner in which users lift it.In summary, we propose that normal force on the armcould provide information that is sufficient to make a manip-ulation task occurring in the virtual environment believable.However, even the best implementation of this idea might beinsufficient, due to the lack of feedback at the fingertip. Inthis scenario, we could introduce a simple fingertip device,which can render event-based feedback to represent contactevents only and compare the performance of the overallsystem to existing multi-degree-of-freedom fingertip devices.REFERENCES[1] J. M. Suchoski, S. Martinez and A. M. Okamura, “Scaling InertialForces to Alter Weight Perception in Virtual Reality,” IEEE Interna-tional Conference on Robotics and Automation, 2018, pp.484-489.[2] D. Leonardis, M. Solazzi, I. Bortone and A. Frisoli, “A 3-RSR HapticWearable Device for Rendering Fingertip Contact Forces,” in IEEETransactions on Haptics, vol. 10, no. 3, pp. 305-316, 2017.[3] S. Casini, M. Morvidoni, M. Bianchi, M. Catalano, G. Grioli and A.Bicchi, “Design and realization of the CUFF - clenching upper-limbforce feedback wearable device for distributed mechano-tactile stim-ulation of normal and tangential skin forces,” IEEE/RSJ InternationalConference on Intelligent Robots and Systems, Hamburg, 2015, pp.1186-1193.[4] L. Meli, I. Hussain, M. Aurilio, M. Malvezzi, M. K. OMalley andD. Prattichizzo, “The hBracelet: A Wearable Haptic Device for theDistributed Mechanotactile Stimulation of the Upper Limb,” in IEEERobotics and Automation Letters, vol.3, no.3, pp. 2198-2205, 2018.[5] T. K. Moriyama, A. Nishi, R. Sakuragi, T. Nakamura and H. Kajimoto,”Development of a wearable haptic device that presents haptics sensa-tion of the finger pad to the forearm,” 2018 IEEE Haptics Symposium(HAPTICS), San Francisco, CA, 2018, pp. 180-185.[6] C. Moussette. “Simple haptics: Sketching perspectives for the designof haptic interactions.” PhD dissertation, Umea University, 2012.

Haptic Sketches on the Arm for Manipulation in Virtual Reality

Abstract

Similar works

Full text

Available Versions

University of Birmingham Research Portal

University of Birmingham Research Portal