B Pflesser

D Morris

GJ Wiet

JB Hochman

M Agus

MS Sorensen

Journal of Otolaryngology : Head and Neck Surgery

English

Jordan Brent Hochman

Nariman Sepehri

Vivek Rampersad

Jay Kraut

Milad Khazraee

Justyn Pisa

Bertram Unger

Crossref

Hochman et al. Journal of Otolaryngology - Head and Neck Surgery 2014, 43:23http://www.journalotohns.com/content/43/1/23HOW I DO IT ARTICLE Open AccessMixed reality temporal bone surgical dissector:mechanical designJordan Brent Hochman1,6*, Nariman Sepehri3, Vivek Rampersad3, Jay Kraut4, Milad Khazraee3, Justyn Pisa5and Bertram Unger2AbstractObjective: The Development of a Novel Mixed Reality (MR) Simulation.An evolving training environment emphasizes the importance of simulation. Current haptic temporal bone simulatorshave difficulty representing realistic contact forces and while 3D printed models convincingly represent vibrationalproperties of bone, they cannot reproduce soft tissue. This paper introduces a mixed reality model, where the effectiveelements of both simulations are combined; haptic rendering of soft tissue directly interacts with a printed bone model.This paper addresses one aspect in a series of challenges, specifically the mechanical merger of a haptic device with anotic drill. This further necessitates gravity cancelation of the work assembly gripper mechanism. In this system, the hapticend-effector is replaced by a high-speed drill and the virtual contact forces need to be repositioned to the drill tip fromthe mid wand.Previous publications detail generation of both the requisite printed and haptic simulations.Method: Custom software was developed to reposition the haptic interaction point to the drill tip. A custom fitting, tohold the otic drill, was developed and its weight was offset using the haptic device. The robustness of the system todisturbances and its stable performance during drilling were tested. The experiments were performed on a mixed realitymodel consisting of two drillable rapid-prototyped layers separated by a free-space. Within the free-space, a linear virtualforce model is applied to simulate drill contact with soft tissue.Results: Testing illustrated the effectiveness of gravity cancellation. Additionally, the system exhibited excellentperformance given random inputs and during the drill’s passage between real and virtual components of the model.No issues with registration at model boundaries were encountered.Conclusion: These tests provide a proof of concept for the initial stages in the development of a novel mixed-realitytemporal bone simulator.Keywords: Haptic, 3D, Virtual, Reality, Temporal, Bone, Surgery, Training, Education* Correspondence: jordanhochman@hotmail.com1Neurotologic Surgery, Department of Otolaryngology Head and NeckSurgery, Faculty of Medicine, University of Manitoba, GB421, 820 SherbrookStreet, Winnipeg, Manitoba, Canada6Laboratory for Surgical Modeling Simulation and Robotics, University ofManitoba, Winnipeg, Manitoba, CanadaFull list of author information is available at the end of the article© 2014 Hochman et al.; licensee BioMed CentCommons Attribution License (http://creativecreproduction in any medium, provided the orDedication waiver (http://creativecommons.orunless otherwise stated.ral Ltd. This is an Open Access article distributed under the terms of the Creativeommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andiginal work is properly credited. The Creative Commons Public Domaing/publicdomain/zero/1.0/) applies to the data made available in this article,Figure 1 Current state of temporal bone surgical simulation. a) 3D printed temporal bone model and b) Stereoscopic Bimodal HapticGraphic Virtual Temporal Bone Simulation. Both simulations were developed at the University of Manitoba, Laboratory for Surgical ModelingSimulation and Robotics.Figure 2 Work assembly of haptic gripper. Solid-Works model of theassembled gripper; permitting union of the haptic device and otic drill.Hochman et al. Journal of Otolaryngology - Head and Neck Surgery 2014, 43:23 Page 2 of 5http://www.journalotohns.com/content/43/1/23IntroductionConcern for patient safety and outcomes underscore theimportance of the ever-increasing pace of developmentin medical simulation.Simulators are now ubiquitous in training [Figure 1].Interactive computer-driven simulations provide a safetraining environment for learning anatomy and procedures.Several haptic temporal bone simulators are currentlyavailable [1-5]. A haptic device is a robotic system designedto apply forces through an end-effector. As a virtual toolcomes into contact with virtual tissues, reaction forces aresimulated. Unfortunately, owing to fundamental limitationsin mechanical design, existing haptic simulations are unableto realistically reproduce the vibration and contact forcesexperienced during surgery.Printed models provide another approach to temporalbone simulation and can be created from virtual modelsusing a variety of techniques. A plaster based printer(Z-Corporation, Rock Hill, SC) uses digitized data andplaces successive layers of material, building a physicalmodel (additive manufacturing). The printed modelsprovide a realistic sense of hardness, but are ill-equippedto present soft tissues [6].Each of the former simulations have been generated inour Lab [7], however, there is an opportunity to addressthe limitations in both with an entirely new mixed-realityparadigm. Mixed reality (MR) techniques fuse digitaldata with the human perception of the real environment.The MR simulation will combine a physical printed bone-model with a virtual soft-tissue model. The virtual modelwill employ a haptic device (HD2, Quansar Corporation,Markham, Ontario) that provides real-time contact forcerepresentation. This device is a 6 Degree of Freedom(DOF), 10 newton per axis manipulandum. The hapticdevice will generate forces encountered during interactionwith virtual soft tissues, such as decompression of a sinusor dural plate, as well as permit metric assessments. Theresulting MR model should provide a platform that iscapable of both the realistic force-feedback and visual-ization surgical trainees require.Developmental steps in generation of a MR Simulator:1. Segment DICOM data2. Mechanical merger of a six DOF haptic device withotic drill3. Generate a virtual haptic model4. Generate a 3D printed temporal bone model5. Develop registration techniques to co-locate printedand virtual models6. Develop control algorithms for virtual force-feedback interaction7. Validate systemThe challenges in creation of novel MR Temporal BoneSimulation are considerable. This paper addresses justone aspect, specifically the mechanical merger of a sixDOF haptic device with an otic drill system (Medtronic,Minneapolis MN). This is accomplished with a customgripper mechanism. In this structure, the drill becomesthe haptic end-effector.In the MR simulation, a user must be able to employthe drill as they would in surgery. The merged hapticdrill device needs to successfully navigate three differentcontact conditions during device operation: movementFigure 3 Converting haptic interaction point to drill tip from mid-wand. Haptic contact forces should be felt at drill tip rather than at themid-point of the haptic wand. This necessitates the following changes to the forces created by the haptic device, where x is the cross product,t the distance between drill tip and haptic wand mid-point and l is handle length.Hochman et al. Journal of Otolaryngology - Head and Neck Surgery 2014, 43:23 Page 3 of 5http://www.journalotohns.com/content/43/1/23in free space, contact with rapid-prototyped bone, andcontact with virtual soft tissue. Throughout these condi-tions and in transitions between them, the user shouldfeel only the weight of the drill. Further, real and virtualforces need be displaced to the drill tip from the normativemid-shaft of the haptic end-effector.MethodsWe have developed a custom fitting for the haptic devicewhich holds an otic drill. The fitting consists of severalpieces [Figure 2] and secures the drill, which then actsas the haptic end-effector.(a)Figure 4 Stability assessment while dissecting a simple mixed realitygripper assembly. Note the simple 3-layer rectangular printed model. b) shinterfaces between real and virtual modes of operation. At position A, the drilhaptic device. This results in a steady 2.5 N z-axis force. The user moves the dto C, the user encounters real forces from drilling in addition to the constantmodel layers and a virtual soft tissue force is generated by the haptic device,with the second real bone model layer and the virtual force is off. At E, the drcompensation forces are present. c) shows a recording of the drill’s z-axis posby the haptic device. The system remains stable throughout, without high-freContact forces with virtual objects are normally appliedat mid-point of the haptic end-effector. A user would feelthat contacts are coming from this location, at the pointwhere the end-effector is grasped. In order for the userto feel that the forces are the result of drill interactionswith the virtual environment, the contact forces mustbe re-located to the tip of the drill. This is accomplishedin software, adjusting the haptic device force and torqueoutputs to take drill length, position and orientation intoaccount [Figure 3]. The result is natural-feeling force-feedback during bone drilling.Software was also developed to offset gravitationalforces permitting the user to feel only the weight of the(b)                                                       (c) model. a) represents a mixed reality model with HD2, otic drill andows the vertical path of the drill through the model with labels atl is in free space, with only gravity compensation forces generated by therill down to interact with the printed model surface at location B. From Bgravity compensation forces. From C to D the drill is between real boneincreasing with depth of penetration. From D to E, the drill is engagedill enters the free space below the second bony layer where only gravityition in metres (upper axis) and force in Newtons (lower axis) generatedquency or underdamped oscillations at boundaries between models.Hochman et al. Journal of Otolaryngology - Head and Neck Surgery 2014, 43:23 Page 4 of 5http://www.journalotohns.com/content/43/1/23drill and not the custom fitting. The software guides thehaptic device motors to dynamically generate an upwardsforce, equal to that of the drill fittings, compensating fortheir additional weight.A series of tests were performed to ensure the operabilityand stability of the re-designed system under normativeconditions. The first set of tests determined the ability ofFigure 5 Operation with gravity compensation throughout the devicereality system within its nominal workspace (x, y, z position in meters - topforces (Fz in Newton- bottom axis) and is present between times A and B.or undamped oscillations in the device position recordings.the system to generate sufficient power to offset the weightof the custom end effector.A second set of tests determined the effectiveness ofthe system in position control mode during disturbancesfrom the user.The third set of tests examined system stability duringdrilling with a simple MR model [Figure 4]. The modelwork space. These graphs show user manipulation of the mixed-three axes). Gravity compensation for the gripper is provided by z-axisNo instability is seen, as manifested by the absence of high frequencyHochman et al. Journal of Otolaryngology - Head and Neck Surgery 2014, 43:23 Page 5 of 5http://www.journalotohns.com/content/43/1/23consists of two drillable rapid-prototyped layers, separatedby a free-space. Within the free-space, a virtual springforce model is applied by the haptic system. Force feed-back increases steadily with depth of penetration throughthe free space. The real and virtual models are registered/co-located to each other so that, as the drill penetrates thefirst real layer of the model, only gravity compensation isengaged. When the empty space below the first layer isencountered, the virtual spring model is additionallyemployed. Once the drill has passed through the emptygap, it again encounters the real model and only gravitycancelation is engaged.ResultsGravity cancellation of the gripper assembly was effectivewithout visible drift, throughout the nominal deviceworkspace [Figure 5]. A force of 2.5 N in the z-axis, and0.1 Nm pitch-torques and 0.02 Nm roll-torques wererequired; less than the maximum continuous outputcapabilities of the device (7.67 N z-axis and 0.948 Nm).The system remained stable during random input fromexperimenters during operation of the otic drill up toits maximum of 42,000 rpm.Stability of the system during the drill’s passagethrough a MR model was monitored [Figure 4]. Noissues with registration or instability at model bound-aries were encountered.DiscussionThe integration of printed and virtual simulation providesa platform which can represent otic drill forces as wellas contact of the drill (or potentially other instruments)with embedded soft tissue structures and facilitate metricassessments. A MR system can complement existing sim-ulations in creating a realistic and reproducible surgicaltraining platform, which can be used to teach/assesstrainees. The system permits mistakes and explorationsof technique and represents an immense opportunity toimprove patient safety.Several significant challenges remain to be addressed.Highly accurate co-location of the virtual and printedsimulations is requisite and will be complicated by theneed to change the position of the simulation duringdissection, as in real surgery. The virtual force algo-rithms need to be modified for the HD2 haptic as theywere initially developed for a 3 DOF device. Further,investigations of construct and concurrent validity arenecessary.These tests provide evidence for the effective mechan-ical and software design of a novel system, integrating anotic drill with an existing haptic device. The new systemprovides gravity compensation and operational stabilityduring interactions with a simple mixed-reality model.ConclusionThese tests provide a proof of concept for the initial stagesin the development of a novel mixed-reality simulator.Research supportMedtronic: Donation of Otic Drill SystemNSERC-Engage Grant (Industry Collaborator: Quanser)Health Sciences Centre FoundationUniversity of Manitoba Faculty of MedicineAuthor details1Neurotologic Surgery, Department of Otolaryngology Head and NeckSurgery, Faculty of Medicine, University of Manitoba, GB421, 820 SherbrookStreet, Winnipeg, Manitoba, Canada. 2Clinical Learning and SimulationFacility, Department of Medical Education, Faculty of Medicine, University ofManitoba, Winnipeg, Manitoba, Canada. 3Faculty of Engineering, University ofManitoba, Winnipeg, Manitoba, Canada. 4Department of Medical Education,Faculty of Medicine, University of Manitoba, Winnipeg, Manitoba, Canada.5Surgical Hearing Implant Program, Department of Otolaryngology - Head andNeck Surgery, Health Sciences Centre, GB421, 820 Sherbrook Street, Winnipeg,Manitoba, Canada. 6Laboratory for Surgical Modeling Simulation and Robotics,University of Manitoba, Winnipeg, Manitoba, Canada.Received: 28 January 2014 Accepted: 27 June 2014Published: 8 August 2014References1. Morris D, Sewell C, Barbagli F, Salisbury K: Visuohaptic simulation of bonesurgery for training and evaluation. IEEE Comput Graph App 2006, 26(6):48–57.2. Wiet GJ, Stredney D, Sessanna D, Stredney D, Schmalbrock P, Welling B:Virtual temporal bone dissection simulation. Stud Health Technol Inform2000, 70:378–384.3. Pflesser B, Tiede U, Heinz KH, Leuwer R: Volume cutting for virtual petrousbone surgery. Comput Aided Surg 2002, 7(2):74–83.4. Agus M, Giachetti A, Gobbetti E, Zanetti G, Zorcolo A, John NW, Stone RJ:Mastoidectomy simulation with combined visual and haptic feedback.Medicine meets virtual reality. Stud Health Technol Inform 2002, 85:17–23.5. Sorensen MS, Mosegaard J, Trier P: The visible ear simulator: A public PCApplication for GPU-Accelerated Haptic 3D simulation of ear surgerybased on the visible ear data. Otology & Neurotology 2009, 30(4):484–487.6. Hochman JB, Kraut J, Kazmerik K, Unger B: Generation of a 3D printedtemporal bone model with internal fidelity and validation of themechanical construct. Otolaryngol Head Neck Surg 2014, 150(3):448–454.7. Kraut J, Unger B, Hochman JB: Temporal Bone Surgical Simulation Employinga Multicore Architecture. Electrical and Computer Engineering, 2013 26thAnnual IEEE Conference. 1–6. 10.1109/CCECE.2013.6567771.doi:10.1186/s40463-014-0023-9Cite this article as: Hochman et al.: Mixed reality temporal bone surgicaldissector: mechanical design. Journal of Otolaryngology - Head and NeckSurgery 2014 43:23.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color ﬁgure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submit

Journal of Otolaryngology - Head and Neck Surgery

Mixed reality temporal bone surgical dissector: mechanical design

https://journalotohns.biomedcentral.com/track/pdf/10.1186/s40463-014-0023-9?site=journalotohns.biomedcentral.com

Mixed reality temporal bone surgical dissector: mechanical design

Abstract

Similar works

Full text

Available Versions

Crossref