We present Dexmo: an inexpensive and lightweight mechanical
exoskeleton system for motion capturing and force feedback
in virtual reality applications. Dexmo combines multiple
types of sensors, actuation units and link rod structures to
provide users with a pleasant virtual reality experience. The
device tracks the user’s motion and uniquely provides passive
force feedback. In combination with a 3D graphics rendered
environment, Dexmo provides the user with a realistic sensation
of interaction when a user is for example grasping an
object. An initial evaluation with 20 participants demonstrate
that the device is working reliably and that the addition of
force feedback resulted in a significant reduction in error rate.
Informal comments by the participants were overwhelmingly
positive.This is the accepted manuscript. The final version is available at http://dl.acm.org/citation.cfm?doid=2858036.2858487

Bian, Y

Gu, X

Kristensson, PO

Sun, W

Zhang, Y

Zhou, D

We present Dexmo: an inexpensive and lightweight mechanical exoskeleton system for motion capturing and force feedback in virtual reality applications. Dexmo combines multiple types of sensors, actuation units and link rod structures to provide users with a pleasant virtual reality experience. The device tracks the user's motion and uniquely provides passive force feedback. In combination with a 3D graphics rendered environment, Dexmo provides the user with a realistic sensation of interaction when a user is for example grasping an object. An initial evaluation with 20 participants demonstrate that the device is working reliably and that the addition of force feedback resulted in a significant reduction in error rate. Informal comments by the participants were overwhelmingly positive

CUED - Cambridge University Engineering Department

Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in VR

Apollo (Cambridge)

Dexmo: An Inexpensive and Lightweight MechanicalExoskeleton for Motion Capture and Force Feedback in VRXiaochi Gu1,21Department of EngineeringUniversity of CambridgeCambridge, United KingdomYifei Zhang, Weize Sun,Yuanzhe Bian, Dao Zhou2Dexta RoboticsShenzhen, ChinaPer Ola KristenssonDepartment of EngineeringUniversity of CambridgeCambridge, United KingdomFigure 1. A person using Dexmo to control a virtual exoskeleton model in virtual reality.ABSTRACTWe present Dexmo: an inexpensive and lightweight mechani-cal exoskeleton system for motion capturing and force feed-back in virtual reality applications. Dexmo combines multipletypes of sensors, actuation units and link rod structures toprovide users with a pleasant virtual reality experience. Thedevice tracks the user’s motion and uniquely provides passiveforce feedback. In combination with a 3D graphics renderedenvironment, Dexmo provides the user with a realistic sen-sation of interaction when a user is for example grasping anobject. An initial evaluation with 20 participants demonstratethat the device is working reliably and that the addition offorce feedback resulted in a significant reduction in error rate.Informal comments by the participants were overwhelminglypositive.ACM Classification KeywordsH.5.2. Information Interfaces and Presentation (e.g. HCI):Input devices and strategiesAuthor KeywordsVirtual reality; force feedback; exoskeleton; motion captureINTRODUCTIONThere are many ways for people to bring their motion intothe the virtual world, however, there is little feedback backto the real world. Current force feedback devices are bulky,non-portable, expensive and difficult to manufacture. There isstill a lack of a light, easy-to-use and affordable force feedbackapproach for people to touch or sense in the digital world.In this paper we present Dexmo, a mechanical exoskeletonthat is a lightweight, inexpensive, compact, reliable and safesolution for providing force feedback and motion capture inaugmented and virtual reality environments. Figure 1 illus-trates a user wearing Dexmo and using it to interact with avirtual world. Rather than applying torque control at each in-dividual joint of the exoskeleton directly, Dexmo uses a microservo unit to shift stopping blocks linearly to stop the rotationof all joints, and forms a rigid body.While achieving a similar sensation to the user as prior meth-ods, our approach has three advantages. First, it can be mademuch smaller and more affordable. Since the joint does notneed to actively provide a torque, there is no need for ex-pensive motors, complex transmissions or sensor modules.Second, only a small shifting servo is needed, and all the bend-ing moment provided by the users is directly exerted to therigid body in the mechanism instead of the motor, making itmore durable, and less torque is required from the servo thuscut the cost. Third, it is safer because the mechanism doesnot actively apply forces to the user. If the controlling unitmalfunctions, no harm will be inflicted.Figure 2. The mechanical design of the exoskeleton.BACKGROUNDEarly attempts to implement haptic interfaces for human handsinclude the PHANTOM [7], which measures users’ hand posi-tion with a grounded robotic arm and exerts controlled pointforce vector on users’ hand. PHANTOM achieved precisestiffness control by adjusting the torque of three DC brushedmotors with encoders. This technology is essentially a trans-mission between the motors and the human hand. Thereforethe workspace for the user and the mobility is highly limited.Moreover, this system fails to produce feedback for individualfingers, reducing the credibility of the haptic experience.The Rutgers Master II ND [2] utilizes pneumatic actuatorsarranged in center of palm and achieves force feedback bydirectly driving the fingers. This device uses the non-contactHall effect and IR sensors for motion capturing for durabilityreasons, yet this approach raises manufacture costs. Specifica-tions of the RMII-ND haptic glove are comparable to those ofthe CyberGrasp [9], another well-known haptic glove system.CyberGrasp uses resistive bend sensors for motion capturing.This system uses a DC motor and cable-pulley transmissionson an exoskeleton to pull users’ finger backward in order tosimulate the exerted force (“active haptics”). Primarily in-tended for corporations, the army and medical rehabilitation,CyberGrasp system is not a consumer-grade product. Whilethese two systems are capable of offering precise force control,they are large in size and expansive due to complexity of thedesign.Other systems include ‘Haptic Telexistence” [8], HIRO III[3], a Japanese system [5] and RML Glove [6]. While thislatter system introduced a passive elastic element (springs) intheir mechanical exoskeleton, it uses motors directly to applytorque on the joints. This setup makes it impossible to providerigid force feedback.Our system Dexmo, in contrast, uses active passive forcefeedback, which largely reduces the size by integrating all thedrivers on one single device. This novel approach allow us tocreate a light-weight device with a low manufacturing cost.Figure 3. The mechanical design of the force feedback unit.SYSTEM DESIGNThe Dexmo exoskeleton mainly consists of the following com-ponents: main controller, force feedback units, upper andlower connectors and finger caps (Figure 2). The finger capsare attached to users’ individual fingers, and the main con-troller is attached to the back of the user’s hand. Each forcefeedback unit has two degrees of freedom of rotation on thesame plane, and they are mechanically jointed to the maincontroller and the finger caps via connector bars, forming alink-bar mechanism. A special extra piece of linkage is intro-duced for the thumb skeleton. The rotational axis of sensorsat two ends are orthogonal so the exoskeleton does not blockthe users’ thumb movement.The design of the force feedback unit is shown in Figure 3.The force feedback unit is the core of the design of Dexmo,it consists both the sensory module as well as the actuationmodule. Two rotational sensors are installed to the case andthey detects the relative rotation of both ends to the case. Amicro servo unit drives two stopping sliders and locks theratchet wheel firmly in place when the force feedback unit isactivated. The transmission mechanism can be arbitrary. Inthis case we used a crown gear connected to the end of theservo, and two teeth bars lying in parallel with each other thatpoints in opposite directions.The accuracy for each joint is limited by the sampling rate ofthe analog-to-digital (ADC) converters. For our device, weused a 12 bit ADC, which means we can collect angular dataat a 0.09 degree accuracy in theory. However since this is awearable device there is not rigid connection and the actualaccuracy is around 0.5 degrees. The response time for theforce feedback unit greatly depends on the type of actuatorimplemented. The response time of the coreless motors in ourdesign has a response time of around 50 ms.The force feedback function of Dexmo works as follows: whenthe user moves their hands and fingers around, the data is sentback to the upper computer which is converted into 3D graph-ics in a virtual space. The virtual hand moves with the userin real time. When a collision is detected between the virtualhand and an virtual object, a command is sent back to thedevice which activates the force feedback units. Inside eachforce feedback unit, the actuators drive the stopping mecha-nism and locks the joint in place, which turns the mechanisminto a rigid body. This reassembles our physical reality. Justas a person’s physical hand touches a real object, the objectprevents the finger from moving inwards. The rigid exoskele-ton exerts an opposing force to the user’s finger tips, and thusprovides the user with force feedback.Motion CaptureWhen the users move their fingers while wearing Dexmo, itslink rod mechanism moves accordingly. The shape of theexoskeleton was designed in a way that fits the user’s handshape so it does not block users’ hand movement regardlessof whether they are clenching their fist or fully extend theirhands. The rotational sensors reads three to four values forthe thumb and one to two values for the rest of each finger.The data is collected by the controlling unit and sent to anupper computer where the kinematics regeneration algorithmis performed. The device use the data to draw out a forwardkinematics chain for each finger, and calculate out the relevantposition of each end point with respect to the origin.The sensors on Dexmo do not have as many degrees of free-dom (DOF) as a human hand. The device only tracks theMetacarpophalangeal joint on each finger directly using rota-tional sensors, while leaving the other two joints for interpo-lation. Therefore, to enable skeletal tracking, we assume thatthere exists a certain relationship between rotation angles ateach finger joint when clenching a fist. More specifically, weassume the angles at the Distal Interphalangeal (DIP) joint andthe Proximal Interphalangeal (PIP) joint will be f (x) and g(x)respectively, where x is the angle at the Metacarpophalangeal(MCP) joint.To investigate the assumed linear relationship between f (x)and g(x), we developed a naïve computer vision-based track-ing system using a web camera and color straps. These colorstraps were put on the MCP, PIP, DIP joints and fingertip ofuser’s index finger and the camera is targeted at the side offinger. The system detected these color straps as blobs andthen connected the centres of them. These connections wereour estimation of the phalanges. Angles between these esti-mated phalanges were recorded as the user was asked to gentlyrepeat the action of bending a finger inwards.We used linear regression to fit the dataset 1000 samples intotal from five participants with different hand sizes (200 sam-ples per participant). We further assumed PIP and DIP jointson all other fingers have similar relationships as those of theindex finger. As the visualization of this model showed areasonable resemblance to the original hand motion, we pro-ceeded with the simplified tracking model.Dexmo started with 16 DOF, however, after further investi-gation we found that 11 DOF is also sufficient for providingenough data to perform the skeletal reconstruction. The po-sitioning of this device can be easily achieved using existingtechnology: inertial measurement units on both arms, opticaltracking, grounded robot arm, etc.Figure 4. The game scenario used in the evaluation.DESIGN TRADE-OFFS AND PARAMETERSSince Dexmo does not require a tethered power supply it ismobile. Due to Dexmo’s compact, light-weight characteristicsand low power consumption, Dexmo can run for 4 hours withonly a 800mAh battery. Other commercial haptics devices(Cybergrasp) that provide continuous force feedback requiremuch more power than Dexmo.A similar type of magnetorheological (MR) brake-based ex-oskeleton [1] uses 5 MR brakes for force feedback,in total itweights over 600 g, while Dexmo weights less than 270 g.Dexmo is a lot much more inexpensive to manufacture due toits modular design and the avoidance of expensive motors, incontrast to prior solutions. A single motor from a high-endhaptics device can cost more than the entire Bill of Materialsprice for Dexmo.Finally, Dexmo is also less fragile and more safe compared tothe string-pulley-motor type of exoskeletons, since its mech-anism is enclosed in a case instead of exposed strings andpulleys, when dropped to the ground or being hit by otherimpacts, Dexmo stands a higher chance of not being damaged.However, a disadvantage with Dexmo is that it only providesbinary haptics feedback which can only provide feedbackabout whether something is present, the softness of the digitalobject will not be perceived.This might lower users’ immersiveexperience. However, Dexmo’s Force Feedback Unit (FFU)can be individually controlled, which still allows a user to feelthe shape of an irregular shaped object.Another disadvantage is that Dexmo uses a shifting mechanismwhich introduces delays to the system. During our testing wefound that the delay for each FFU is between 20–40ms, whichcould affect the users’ immersive experience. The continuoustype haptics devices generally has a reaction delay lower than30 ms.EVALUATIONThe efficacy of the system has been verified in informal labtrials. See also our supplemental video which demonstratesthe tracking system in action.The purpose of this evaluation was to validate that Dexmo canbe worn and used by typical end-users in a typical playful VRtask. As part of future work we will carry out more extensivecontrolled experiments.MethodWe convenience sampled 20 participants from diverse back-grounds (four art students, three athletes, three engineers, threecleaning ladies, three game developers and four factory work-ers). There were ten women and ten men recruited in total.We developed a playful game scenario as games are a commonapplication for VR environments. We built an archery gamescenario using Unity1 and combined it to our device with ahead mounted display (see Figure 4 for an illustration).The main purpose was to validate that the system would workwell in a typical VR scenario and to gather users’ intial per-ceptions of the system. In addition, we wanted to investigate ifthe introduction of the force feedback resulted in an improvedperformance (lower error rate). We therefore decided to testthe null hypothesis H0 that force feedback (FORCEFEEDBACKcondition) does not result in a significant difference in er-ror rate compared to a baseline condition that does not useforce feedback (NOFORCEFEEDBACK condition). Our ex-periment was a within-subjects design with one independentvariable with two levels: FORCEFEEDBACK and NOFORCE-FEEDBACK).Participants were asked to pick up an arrow from a basket, putit on a bow, pull the bowstring and then release the arrow toshoot at a target. The score was then recorded. Each partici-pant was first asked to wear the Dexmo exoskeleton with forcefeedback enabled and then perform the archery task. Afteran hour of resting to avoid muscle memory, each participantwas asked to wear the exoskeleton without the force feedbackabilities, and then shoot ten arrows one more time. Theirperformances were recorded and analyzed afterwards. Theconditions were not counterbalanced but this of little conse-quence in this instance for three reasons. First, participantshad a one hour break between the conditions. Second, as thegame was very simple, any learning effect is negligible. Third,the condition that could hypothetically benefit from a learn-ing effect was the baseline condition NOFORCEFEEDBACK,which only means we are less likely to be able to reject thenull hypothesis.ResultsThe mean error rate was 61% (lower 95% confidence interval:54.9%; upper 95% confidence interval: 67.1%) in the NO-FORCEFEEDBACK and 44% (lower 95% confidence interval:1http://www.unity3d.com0	  0.2	  0.4	  0.6	  0.8	  1	  ForceFeedback	   NoForceFeedback	  Error	  Rate	  Figure 5. Mean error rate and 95% confidence interval as a function ofcondition in the formative evaluation.36.7%; upper 95% confidence interval: 51.3%) in the FORCE-FEEDBACK condition (see also Figure 5). Repeated measuresanalysis of variance at significance level α = 0.05 showed thatthis difference was statistically significant (F1,19 = 45.380,η2p = 0.705, p < 0.0001). The low p-value in conjunctionwith the high effect size indicate that the detected differenceis robust and substantial.As expected, force feedback had a positive effect on perfor-mance. In addition, participants’ responses were very positivewith representative quotes such as “this is really cool”, “itfeels very real”, “I never had this kind of experience before”,“this is very light, it’s not as uncomfortable as I expected”and “this looks very futuristic”. However, we caution againstover-interpretation as there is a likely a novelty effect.DISCUSSION AND CONCLUSIONSIn this paper we have presented Dexmo: a mechanical ex-oskeleton system for motion capturing and force feedback.Compared to most the other force feedback devices, Dexmoonly provides on/off force feedback, but the shifting mecha-nism that this design implements helps Dexmo avoid the usageof bulky motors and expensive sensory modules. Thus boththe weight and the size of the design can be reduced whichresults in a light-weight, pleasant and low-cost mass-consumerdevice.When the joint is locked, the clutch and lock slider forms arigid body which makes this design infinitely more rigid thanany motor simulated force feedback. This also better modelsthe way users experience the physical world when interactingwith rigid bodies. In addition, this design also results in alower power consumption.The functionality of Dexmo has been verified in lab measure-ments (see also the supplemental video). In addition, we havecarried out an evaluation with 20 participants. Our resultsdemonstrate that the device is working reliably and are similarto a previously study on the experience of “passive haptics”[4]. In addition, we found that force feedback did result in asignificant reduction in error rate. Informal comments by theparticipants were overwhelmingly positive.REFERENCES1. J. Blake and H.B. Gurocak. 2009. Haptic Glove With MRBrakes for Virtual Reality. Mechatronics, IEEE/ASMETransactions on 14, 5 (Oct 2009), 606–615. DOI:http://dx.doi.org/10.1109/TMECH.2008.20109342. M. Bouzit, G. Burdea, G. Popescu, and R. Boian. 2002.The Rutgers Master II-new design force-feedback glove.Mechatronics, IEEE/ASME Transactions on 7, 2 (Jun2002), 256–263. DOI:http://dx.doi.org/10.1109/TMECH.2002.10112623. T. Endo, H. Kawasaki, T. Mouri, Y. Doi, T. Yoshida, Y.Ishigure, H. Shimomura, M. Matsumura, and K. Koketsu.2009. Five-fingered haptic interface robot: HIRO III. InEuroHaptics conference, 2009 and Symposium on HapticInterfaces for Virtual Environment and TeleoperatorSystems. World Haptics 2009. Third Joint. 458–463. DOI:http://dx.doi.org/10.1109/WHC.2009.48108124. Brent Edward Insko. 2001. Passive Haptics SignificantlyEnhances Virtual Environments. Ph.D. Dissertation.University of North Carolina at Chapel Hill.http://www.cs.unc.edu/techreports/01-017.pdf5. T. Koyama, I. Yamano, K. Takemura, and T. Maeno.2002. Multi-fingered exoskeleton haptic device usingpassive force feedback for dexterous teleoperation. InIntelligent Robots and Systems, 2002. IEEE/RSJInternational Conference on, Vol. 3. 2905–2910 vol.3.DOI:http://dx.doi.org/10.1109/IRDS.2002.10417136. Zhou Ma and P. Ben-Tzvi. 2015. RML Glove;AnExoskeleton Glove Mechanism With Haptics Feedback.Mechatronics, IEEE/ASME Transactions on 20, 2 (April2015), 641–652. DOI:http://dx.doi.org/10.1109/TMECH.2014.23058427. T. H. MASSIE. 1994. The PHANToM Haptic Interface :A Device for Probing Virtual Objects. Proceedings of theASME Winter Annual Meeting, Symposium on HapticInterfaces for Virtual Environment and TeleoperatorSystems, Chicago, IL, Nov., 1994 (1994).http://ci.nii.ac.jp/naid/10018008840/en/8. Katsunari Sato, Kouta Minamizawa, Naoki Kawakami,and Susumu Tachi. 2007. Haptic Telexistence. In ACMSIGGRAPH 2007 Emerging Technologies (SIGGRAPH’07). ACM, New York, NY, USA, Article 10. DOI:http://dx.doi.org/10.1145/1278280.12782919. Cyberglove Systems. 2015. Hand Motion Capturing andForce Feedback with Cybergrasp. (2015).http://www.cyberglovesystems.com/ [Online; accessed25-September-2015].

Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in VR

Abstract

Similar works

Full text

Available Versions

CUED - Cambridge University Engineering Department

Apollo (Cambridge)