Development of 3-D Medical Image Visualization System

This paper reports the development of a holographic video (holovideo) rendering system that uses standard 2-D medical imaging inputs and generates medical images of human body parts as holographic video with three-dimensional (3-D) realism. The system generates 3-D medical images by transforming a numerical description of a scene (such as in data from URI, CAT, PET, and X-ray databases) into a holographic fringe pattern and then displays the images on the image volume of a holovideo display system. The system uses specialized digital signal processors to scale up the computation and rendering speed of the holovideo computing system beyond what exists today. Holograms developed under this research have horizontal (holographic) resolution high enough for smooth binocular parallax and a (video) resolution in the vertical direction comparable to NTSC television. Thus, the holovideo rendering and display system provides medical personnel with the information essential for viewing internal organs of humans with accurate 3-D realism. It is envisioned that the commercializable system that will ultimately be developed in the course of this research program will be compact enough to be used as desktop equipment in a medical imaging platform and economical enough to be made available in adequate numbers to hospitals

Delivering real-time holographic video content with off-the-shelf PC hardware

Thesis (M. Eng. and S.B.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2004.Includes bibliographical references (leaves 75-76).We present a PC based system to simultaneously compute real-time holographic video content and to serve as a framebuffer to drive a holographic video display. Our system uses only 3 PCs each equipped with an nVidia Quadro FX 3000G video card. It replaces a SGI Onyx and the custom built Cheops Image Processing System that previously served as the platform driving the MIT second-generation Holovideo display. With a prototype content generation implementation, we compute holographic stereograms and update the display at a rate of roughly 2 frames per second.by Tyeler Quentmeyer.M.Eng.and S.B

Generated holographic stereograms in photorefractive polymer

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 77-80).This thesis aims to assess the feasibility of an updatable three-dimensional display based on the direct fringe writing of computer-generated holographic gratings into a novel photorefractive polymer. The photorefractive polymer in question has been developed by Nitto Denko Technical Corporation and has many attractive properties for the 3-D display application, including long image persistence, rapid erasure, high diffraction efficiency, and large area; however, current holographic display systems based around its use involve interference methods that complicate their optical and computational architectures. The direct fringe writing architecture under question is poised as a simplifying and enhancing alternative to previous demonstrations of updatable holographic displays in photorefractive polymeric materials based around such conventional interference-based holographic stereogram techniques. In addition to simplifying optical architectures, direct fringe writing can allow for complete control of recorded hologram characteristics; interference fringes can be computed to simulate any arbitrary reference beam geometry and wavefront curvature. The system concept - comprised of fringe pattern generation on computer, fringe pattern transfer from SLM to photorefractive polymer, and spatial multiplexing for large-image generation - reintroduces accommodation cues to the resulting holographic images and represents a reduction of system footprint, complexity, and cost relative to the current interference-based systems. The adaptation of the Diffraction Specific Coherent Panoramagram fringe computation method - originally developed to drive AOM-based holographic displays at video rates while preserving all depth cues, including accommodation - to the current display architecture is presented and methods for direct fringe transfer from SLM to photorefractive polymer are depicted. Such methods for direct fringe writing are explored in simulation and experiment. Theoretical arguments for system performance are formulated in the context of a wave optics-based system analysis. Preliminary results of horizontal parallax-only images on this display are presented and directions for performance improvements and system extensions are explored.by Sundeep Jolly.S.M

Haptic holography : an early computational plastic

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2001.Includes bibliographical references (p. 135-148).This dissertation introduces haptic holography, a combination of computational modeling and multimodal spatial display, as an early computationalplastic In this work, we combine various holographic displays with a force feedback device to image free-standing material surfaces with programmatically prescribed behavior. We present three implementations, Touch, Lathe, and Poke, each named for the primitive functional affordance it offers. In Touch, we present static holographic images of simple geometry, reconstructed in front of the hologram plane (in the viewer's space), and precisely co-located with a force model of the same geometry. These images can be visually inspected and haptically explored using a hand-held interface. In Lathe, we again display holo-haptic images of simple geometry, this time allowing those images to be reshaped by haptic interaction in a dynamic but constrained manner. Finally in Poke, we present a holo-haptic image that permits arbitrary reshaping of its reconstructed surface. As supporting technology, we offer a new technique for incrementally computing and locally updating interference-modeled holographic fringe patterns. This technique permits electronic holograms to be updated arbitrarily and interactively, marking a long-held goal in display holography. As a broader contribution, we offer a new behavior-based spatial framework, based on both perception and action, for informing the design of spatial interactive systems.Wendy J. Plesniak.Ph.D

Multiple viewpoint rendering for three-dimensional displays

Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1997.Includes bibliographical references (leaves 159-164).Michael W. Halle.Ph.D

Hologram synthesis for photorealistic reconstruction

Computation of diffraction patterns, and thus holograms, of scenes with photorealistic properties is a highly complicated and demanding process. An algorithm, based primarily on computer graphics methods, for computing full-parallax diffraction patterns of complicated surfaces with realistic texture and reflectivity properties is proposed and tested. The algorithm is implemented on single-CPU, multiple-CPU and GPU platforms. An alternative algorithm, which implements reduced occlusion diffraction patterns for much faster but somewhat lower quality results, is also developed and tested. The algorithms allow GPU-aided calculations and easy parallelization. Both numerical and optical reconstructions are conducted. The results indicate that the presented algorithms compute diffraction patterns that provide successful photorealistic reconstructions; the computation times are acceptable especially on the GPU implementations. © 2008 Optical Society of America

Trends in development of dynamic holographic displays

Creation of a dynamic 3-D display based on holography, in which a 3-D scene is encoded in terms of optical diffraction, transformed into the fringe patterns of the hologram that is further converted into a signal for a spatial light modulator (SLM) and displayed in real time, is an extremely challenging enterprise. There are various approaches targeted to solve associated problems

Valokentistä aaltokentiksi: hologrammien generointi perspektiivisistä kuvista

In this thesis, the link between the ray-optics and wave-optics formalisms of light propagation modeling is studied through light field (LF) and holography. Multi-perspective images, such as captured by multicamera arrays, are utilized to obtain the discrete LF information. Three di erent computer generated hologram (CGH) representations are discussed in the thesis: holographic stereogram (an example for incoherent CGH), phase-added stereogram and diffraction specific coherent panoramagram (examples for coherent CGH). Comparative analysis of these three different holographic representation techniques is carried out through experiments simulating the viewing process of the holograms by the human eye. In particular, reconstructed image quality is compared for different scenes at different viewpoints. The accommodation responses of each technique is also evaluated via changing the focal length of the lens in the human eye model to focus the eye at different distances. The prominent issue of speckle noise apparent in hologram reconstruction process is particularly addressed in detail, since it heavily affects the quality of the reconstructed images. In addition to existing solutions analyzed in the thesis, random averaging and pixel separation, a speckle suppression method based on pixel separation for coherent holograms is proposed. The proposed method is shown to further enhance the reconstructed image quality with respect to existing speckle reduction techniques. Besides the perceived image quality, another topic that is seen to be critical in the context of the thesis is simplifying the capture process of LF. In this aspect, the strict camera sampling requirements in LF capture for holographic stereograms are shown to be relieved considerably through the use of shearlet-based LF reconstruction algorithm. This enables utilization of more appropriate capture devices, e.g. multi-camera arrays, instead of conventionally used camera rigs.Tämän työn tavoitteena on tarkastella valon säde- ja aalto-optiikkaa valokenttien ja holografian kautta. Moniperspektiivisiä kuvia käytetään tallentamaan diskreetin valokentän informaatio. Kolme eri digitaalista hologrammiesitystä valittiin tähän työhön vertailtavaksi: holographic stereogram (esimerkkinä inkoherenteista hologrammeista), phase-added stereogram ja diffraction specific coherent panoramagram (esimerkkeinä koherenteista hologrammeista). Näiden hologrammiesitysten välisiä eroja analysoidaan ihmisnäköä numeerisesti simuloivien kokeiden avulla. Erityisesti eri hologrammitallenteista saatujen rekonstruktiokuvien visuaalista laatua vertaillaan simuloimalla katsojaa eri näkökulmista. Holografiseen rekonstruktioprosessiin liittyvää pilkkuhäiriötä käsitellään yksityiskohtaisesti, sillä se heikentää havaittujen kuvien laatua huomattavasti. Nykyisten ratkaisujen, kuten satunnaiskeskiarvottamisen ja pikseliseparaation lisäksi johdetaan pikseliseparaatioon pohjautuva pilkkuhäiriötä vähentävä menetelmä koherenteille hologrammeille. Kokeiden perusteella tämän menetelmän osoitetaan parantavan rekonstruktiokuvien laatua. Havaitun kuvanlaadun lisäksi kriittinen aihe tämän työn kontekstissa on valokentän tallentamisen helpottaminen. Tiukkoja näytteistämisvaatimuksia tähän liittyen voidaan keventää huomattavasti shearlet-muunnokseen pohjautuvan valokentän rekonstruktioalgoritmin avulla, mahdollistaen perinteisesti käytettyjen järjestelmien sijaan käytännöllisempien kameraryhmien käytön

Volumetric rendering for holographic display of medical data

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1988.Includes bibliographical references.Work funded by a joint IBM/MIT agreement.by Wendy J. Plesniak.M.S

Coherent and Holographic Imaging Methods for Immersive Near-Eye Displays

Lähinäytöt on suunniteltu tarjoamaan realistisia kolmiulotteisia katselukokemuksia, joille on merkittävää tarvetta esimerkiksi työkoneiden etäkäytössä ja 3D-suunnittelussa. Nykyaikaiset lähinäytöt tuottavat kuitenkin edelleen ristiriitaisia visuaalisia vihjeitä, jotka heikentävät immersiivistä kokemusta ja haittaavat niiden miellyttävää käyttöä. Merkittävänä ratkaisuvaihtoehtona pidetään koherentin valon, kuten laservalon, käyttöä näytön valaistukseen, millä voidaan korjata nykyisten lähinäyttöjen puutteita. Erityisesti koherentti valaistus mahdollistaa holografisen kuvantamisen, jota käyttävät holografiset näytöt voivat tarkasti jäljitellä kolmiulotteisten mallien todellisia valoaaltoja. Koherentin valon käyttäminen näyttöjen valaisemiseen aiheuttaa kuitenkin huomiota vaativaa korkean kontrastin häiriötä pilkkukuvioiden muodossa. Lisäksi holografisten näyttöjen laskentamenetelmät ovat laskennallisesti vaativia ja asettavat uusia haasteita analyysin, pilkkuhäiriön ja valon mallintamisen suhteen. Tässä väitöskirjassa tutkitaan laskennallisia menetelmiä lähinäytöille koherentissa kuvantamisjärjestelmässä käyttäen signaalinkäsittelyä, koneoppimista sekä geometrista (säde) ja fysikaalista (aalto) optiikan mallintamista. Työn ensimmäisessä osassa keskitytään holografisten kuvantamismuotojen analysointiin sekä kehitetään hologrammien laskennallisia menetelmiä. Holografian korkeiden laskentavaatimusten ratkaisemiseksi otamme käyttöön holografiset stereogrammit holografisen datan likimääräisenä esitysmuotona. Tarkastelemme kyseisen esitysmuodon visuaalista oikeellisuutta kehittämällä analyysikehyksen holografisen stereogrammin tarjoamien visuaalisten vihjeiden tarkkuudelle akkommodaatiota varten suhteessa sen suunnitteluparametreihin. Lisäksi ehdotamme signaalinkäsittelyratkaisua pilkkuhäiriön vähentämiseksi, ratkaistaksemme nykyisten menetelmien valon mallintamiseen liittyvät visuaalisia artefakteja aiheuttavat ongelmat. Kehitämme myös uudenlaisen holografisen kuvantamismenetelmän, jolla voidaan mallintaa tarkasti valon käyttäytymistä haastavissa olosuhteissa, kuten peiliheijastuksissa. Väitöskirjan toisessa osassa lähestytään koherentin näyttökuvantamisen laskennallista taakkaa koneoppimisen avulla. Kehitämme koherentin akkommodaatioinvariantin lähinäytön suunnittelukehyksen, jossa optimoidaan yhtäaikaisesti näytön staattista optiikka ja näytön kuvan esikäsittelyverkkoa. Lopuksi nopeutamme ehdottamaamme uutta holografista kuvantamismenetelmää koneoppimisen avulla reaaliaikaisia sovelluksia varten. Kyseiseen ratkaisuun sisältyy myös tehokkaan menettelyn kehittäminen funktionaalisten satunnais-3D-ympäristöjen tuottamiseksi. Kehittämämme menetelmä mahdollistaa suurten synteettisten moninäkökulmaisten kuvien datasettien tuottamisen, joilla voidaan kouluttaa sopivia neuroverkkoja mallintamaan holografista kuvantamismenetelmäämme reaaliajassa. Kaiken kaikkiaan tässä työssä kehitettyjen menetelmien osoitetaan olevan erittäin kilpailukykyisiä uusimpien koherentin valon lähinäyttöjen laskentamenetelmien kanssa. Työn tuloksena nähdään kaksi vaihtoehtoista lähestymistapaa ristiriitaisten visuaalisten vihjeiden aiheuttamien nykyisten lähinäyttöongelmien ratkaisemiseksi joko staattisella tai dynaamisella optiikalla ja reaaliaikaiseen käyttöön soveltuvilla laskentamenetelmillä. Esitetyt tulokset ovat näin ollen tärkeitä seuraavan sukupolven immersiivisille lähinäytöille.Near-eye displays have been designed to provide realistic 3D viewing experience, strongly demanded in applications, such as remote machine operation, entertainment, and 3D design. However, contemporary near-eye displays still generate conflicting visual cues which degrade the immersive experience and hinders their comfortable use. Approaches using coherent, e.g., laser light for display illumination have been considered prominent for tackling the current near-eye display deficiencies. Coherent illumination enables holographic imaging whereas holographic displays are expected to accurately recreate the true light waves of a desired 3D scene. However, the use of coherent light for driving displays introduces additional high contrast noise in the form of speckle patterns, which has to be taken care of. Furthermore, imaging methods for holographic displays are computationally demanding and impose new challenges in analysis, speckle noise and light modelling. This thesis examines computational methods for near-eye displays in the coherent imaging regime using signal processing, machine learning, and geometrical (ray) and physical (wave) optics modeling. In the first part of the thesis, we concentrate on analysis of holographic imaging modalities and develop corresponding computational methods. To tackle the high computational demands of holography, we adopt holographic stereograms as an approximative holographic data representation. We address the visual correctness of such representation by developing a framework for analyzing the accuracy of accommodation visual cues provided by a holographic stereogram in relation to its design parameters. Additionally, we propose a signal processing solution for speckle noise reduction to overcome existing issues in light modelling causing visual artefacts. We also develop a novel holographic imaging method to accurately model lighting effects in challenging conditions, such as mirror reflections. In the second part of the thesis, we approach the computational complexity aspects of coherent display imaging through deep learning. We develop a coherent accommodation-invariant near-eye display framework to jointly optimize static display optics and a display image pre-processing network. Finally, we accelerate the corresponding novel holographic imaging method via deep learning aimed at real-time applications. This includes developing an efficient procedure for generating functional random 3D scenes for forming a large synthetic data set of multiperspective images, and training a neural network to approximate the holographic imaging method under the real-time processing constraints. Altogether, the methods developed in this thesis are shown to be highly competitive with the state-of-the-art computational methods for coherent-light near-eye displays. The results of the work demonstrate two alternative approaches for resolving the existing near-eye display problems of conflicting visual cues using either static or dynamic optics and computational methods suitable for real-time use. The presented results are therefore instrumental for the next-generation immersive near-eye displays