Through innovative haptic and holography application, Haptic Holography focused on developing a more realistic and accurate three-dimensional (3D) ‘synthetic reality’ for purposes of pre-commercialization to enhance current medical training

Borins, Myles

Harrison, Paul

Laverick, Alex

leslie, Ryan

Logan, Natalie

Page, Michael

Paterson, Finlay

Vasilliev, Andrei

OCAD University Open Research Repository

OCAD University Open Research RepositoryFaculty of Design2012 FedDev Ontario’s ARC Initiatives OCAD University Project# 1 – Haptic holographyPage, Michael and Logan, Natalie and Harrison, Paul and Vasilliev, Andrei and Borins, Myles and Paterson, Finlay and Laverick, Alex and leslie, Ryan Suggested citation: Page, Michael and Logan, Natalie and Harrison, Paul and Vasilliev, Andrei and Borins, Myles and Paterson, Finlay and Laverick, Alex and leslie, Ryan (2012) FedDev Ontario’s ARC Initiatives OCAD University Project# 1 – Haptic holography. Project Report. OCAD University. Available at http://openresearch.ocadu.ca/id/eprint/1057/Open Research is a publicly accessible, curated repository for the preservation and dissemination of scholarly and creative output of the OCAD University community. Material in Open Research is open access and made available via the consent of the author and/or rights holder on a non-exclusive basis. Haptic Holography - June 2012       Through innovative haptic and holography application, Haptic Holography focused on developing a more realistic and accurate three-dimensional (3D) ‘synthetic reality’ for purposes of pre-commercialization to enhance current medical training.Entact Robotics Inc. is a lead-ing innovator of haptic (sense of touch) devices in Canada and was the colaborating part-ner for the Haptic Holography project contributing to OCAD University’s extended history of over 30 years experience of innovation in holography. Entact Robotics Inc. focuses on the development of tele-op-eration (machine operation at a distance) robotic systems with haptic feedback, engineer-ing design and the ability to work with clients in real-time to prototype, test and integrate their haptic/robotic systems.In addition to the development of proprietary devices, Entact Robotics Inc. consults with research institutes and industry regarding custom applications. Technological developments in digital holography and haptics made further investigations for this collaboration possible.The Haptic Holography team collaborated with Entact Robotics Inc. to investigate merging holographic im-ages and sensory perception (haptics) with various medi-ums for application in medi-cal and surgical training. FedDev Ontario’s ARC Initiatives OCAD UniversityProject # 1 – Haptic HolographyThe research methodology for the Haptic Holography project resulted in the following work plan:1.   Survey of existing publications, technology, co-axial viewing 2.   Full literature review on sensory feedback to holographic projection3.   Identify real-time interactive Graphics Processing Unit (GPU) 4.   Development of Graphic User Interface (GUI) to control the device5.   Identify a process for mapping haptic data (‘touch’ in 3D space)6.   Production of digital hologram prototypes7.   System design for controlling super bright LEDs8.   Completion of a prototype workstation9.   Creation of multiple ‘synthetic’ experiences 10. Analysis of ield tests in medical education and design industries11. Final reportImage from holographic simulation showing the point of entry of the lumbar puncture and 3D graticule guiding the angle of entry.Haptic Holography - June 2012      2 The ields of science and medicine were early adopters of ‘haptics’, sense of touch, and ‘holography’ - Greek for ‘holos’ meaning ‘whole’ and ‘grafe’ meaning ‘writing or drawing’. brates and bones of patients, in physical form. However, current models do not allow practitioners to see beneath the surface and are not paired with instant feedback systems.Haptic Holography’s research found that existing training tools and equipment could vastly be improved upon, by integrating visual awareness through ho-lographic overlay and sensory touch with haptics.Based on the research con-ducted, a holographic scene that was useful for training medical professionals in critical medical procedures was cre-ated. A literature review and survey of current technology and ex-isting medical training devices in the ield were done by re-searching a centralized base of those engaged in the devel-opment of haptics, holography and H3D software (an open source program for haptics), used in haptic physics environ-ments at universities and in research labs around the world. Although current 3D imaging systems in the medical ield exist, they primarily employ the use of latex models to simulate medical training procedures.The built internal structures of these dummies replicate the actual tissue, ligaments, verte-Haptic Virtual ViewThe initial prototype for the Haptic Holography worksta-tion began with the creationof a holographic ‘species’ and a haptic ‘scalpel’ to un-derstand the application inmedical training simulations.Haptic Holography - June 2012       3 Digital and analogue holo-grams were used to create multiple prototype simulations that proved convincing to ob-servers. Observers repeatedly failed to notice the partition between 3D animation, video and hologra-phy media, making the differen-tiation between simulation and actual, virtually unrecognizable.In medical procedures such as lumbar puncture or spinal tap, extreme precision is required to safely and accurately carry out the procedure. The Haptic Ho-lography workstation supports training by facilitating learning and building expertise The model produced by the Haptic Holography project, incorporating holographic scene and haptic workstation, greatly improves the visual simulation training cur-rently available in the medical ield. The Haptic Holography workstation is auto-stereoscopic, pro-viding co-axial viewing so that the holographic illusion can be viewed from all angles in a 3D capacity. It also eliminates the need for cumbersome head-gear, goggles or glasses, creating a more ‘realistic’ experience.Haptic Holography - June 2012       4 The workstation produces a holographic ‘reality’ designed to train medical workers with a sophisticated integration of haptics and holography to simulate lumbar puncture or spinal tap operations. The workstation’s haptic con-troller is in the same position as the virtual object, which may be a scalpel or a needle.The simulation provides haptic feedback, giving the user the sense of shape and feel through the controller. As well, the simulation provides audible cues and text speaking to the degree of penetration. Full immersion with the 3D display screen is simulated, by provid-ing the user with the freedom to use the controller to move, angle, adjust and “feel”, the structures necessary to the medical proce-dure, thus, which results in a superior-responsive training tool. The Haptic Holography project resulted in a rapid-responsive, customized, interac-tive workstation fully suitable for the task of inputting haptic data, head tracking data, processing, outputting to video, audio and creating LED control in real-time.Haptic Holography - June 2012       5 The Haptic Holography Workstation has a number of modules responsible for displaying imagery and creating interactivity:The Haptic Holography work-station provides lexibility in resizing so the user can re-hearse the procedure at an image of 1:3 of human scale, with less critical precision, to 1:2 with more precision and then to an ‘actual’ 1:1 size to train for accuracy and extreme precision. Hologram and holder - This device holds the hologram, in trans-mission or relection mode in a precise postion, aligned with the OLED display.OLED display - This display produces very bright and chroma satu-rated images; so that the video image will be as saturated as the ho-lographic scene. The computer controls all the input data, real-time processing and output data. The display is viewable at any angle without degradation of the image. The laser module - This low-power, inexpensive laser illuminates the hologram in transmission mode.The Kinect module - This device tracks the viewer’s head position and instantly modiies the video parallax to conform with that of the hologram.The Super-bright LED module - In relection mode, this device changes the reconstruction angle of the hologram, which animates the scene. The module is controlled by the computer and changes the holographic scene to accommodate the “zoom” of the image.Haptic Holography - June 2012       6 The Haptic Holography work-station was irst demonstrated at Canada 3.0 in 2012, at the OCAD U booth. The technol-ogy of the workstation received positive feedback from medical professionals in 3D imaging ield.  In late 2012, the project work was presented at The Subtle Technologies Conference and tentative plans include an ex-hibit to show the technology at an international conference on haptics in Finland.Entact Robotics Inc., has plans to develop the Haptic Holography workstation for market commercialization. Users of the Haptic Holography technology can ind application in medical and sur-gical training procedures as well as providing a new form of ‘synthetic reality’ for art-ists and designers. Haptic Holography Principle Investigator and OCAD University Assistant Professor Michael Page, presented this research at The 9th International Symposium on Display Holography (ISDH 2012) at MIT Media Lab, in Cambridge, Massachusetts where the next generation haptic holography workstation was installed.Haptic Holography - June 2012       7 Phase Labhttp://research.ocadu.ca/ar-tresearchHaptic Holography projecthttp://thehapticholographypro-ject.ca/The Photon Leaguehttp://thephotonleague.org/FedDev Ontario’s ARC Initiatives OCAD UniversityProject # 1 – Haptic HolographyProject TeamMichael Page, Principal Investigator, Assistant Professor OCAD UniversityNatalie Logan, BFA, Research Assistant OCAD UniversityPaul Harrison, BSc, Research Assistant OCAD UniversityAndrei Vasilliev, BSc, Research Assistant OCAD UniversityMyles Borins, BA, Research Assistant OCAD UniversityFinlay Paterson, BFA, Research Assistant OCAD UniversityAlex Laverick, The Photon LeagueRyan Leslie, Founder, Director of Engineering, Entact Robotics Inc.Entact Robotics Inc. http://www.entactrobotics.com/OCAD University http://www.ocadu.ca/OCAD University Researchhttp://www.ocadu.ca/research.htmOCAD University Research Projectshttp://research.ocadu.caHaptic Holography - June 2012       8Credits:Sang Hoon Hwang, renderings pgs 1, 3, 5-7Finlay Paterson, image pg 2Michael Page, photo pg 4Source: H3D http://www.h3dapi.org/Software donation in support of Phase Lab provided by Autodesk.

FedDev Ontario’s ARC Initiatives OCAD University

Project# 1 – Haptic holography

http://openresearch.ocadu.ca/id/eprint/1057/1/Haptic_Holography.pdf

FedDev Ontario’s ARC Initiatives OCAD University Project# 1 – Haptic holography

Abstract

Similar works

Full text

Available Versions

OCAD University Open Research Repository