Ando, Takahisa

Matsumoto, Toshiaki

Shimizu, Eiji

Takahashi, Hideya

Osaka City University Repository

Mem. Fac. Eng., Osaka City Univ., Vol. 40, pp. 1 - 6 (1999)
Head Mounted Display for Mixed Reality
using Holographic Optical Elements
Takahisa ANDO*, Toshiaki MATSUMOTO**,
Hideya Takahashi*** and Eiji SHIMIZU****
(Received September 30,1999)
Synopsis
We propose the stereoscopic Head Mounted Display(HMD) using Holographic Optical
Elements(HOE). This display uses HOE as the holographic combiner instead of half mirror that is used by
conventional HMD. A single HOE can record the multiple optical images by the characteristic in itself,
that is angular selectivity a~d wavelength selectivity: Therefore, the HOE as the combiner has the feature
which can separate the stereoscopic images onto the left and right eyes and it has high efficiency both
transparency for outside world and reflectance of virtual images shown by electrical display like as small
LCD or CRT and so on.
In this paper we proved that the HMD using Lippmann hologram has the potentiality of
miniaturization, lightening, wide field of vision for outside world. In addition, we have developed the
Mixed Reality system by using oUr new HMD. This HMD can be suitable to the Virtual Reality field,
especially Mixed Reality we have to observe both the real outside world and the virtual world at the same
time.
KEYWORDS: HMD, HOE, HUD, Virtual Reality, Mixed Reality, Augmented Reality
Introduction
Recently, the Mixed-Reality (MR) is remarkable in many fields, such as medical and game fields,
multimedia, and attractions for the various events. MR technique is the concept that includes Augmented
Reality (AR) and Augmented Virtuality(AV)I11121. AR is to augment the real world by electrical virtual
information, and AV is to augment the virtual world by real information. But the boundary between AR
and AV is not defined accurately. In other words, the Mixed Reality is to combine the real world and
virtual world by using the electrical visual display system. Therefore the display we can see the real and
virtual world is necessary for MR technique. Recently the Mixed Reality technique is researched by many
other Virtual Reality researchers who want to apply this technology to practical usage like as the
consumer electronics because only Virtual Reality technology which has been investigated over a decade
can't realize the satisfied application. Although we can use the various type of display for MR, see
through Head Mounted Display (HMD) is considered as the main device for MR now.
But conventional HMD, that is made by optical lenses magnifying the small LCD, has varies defects.
For example it has a narrow field of vision for outside world and a tradeoff between transparency for
outside world and reflectance of LCD images. And it has a limitation of miniaturization, lightening for
practical industrial application because two pieces of optical components(lens and combiner) are
needed.I·~JTherefore we have to overcome these defects and develop the new see through HMD for Mixed
Reality in order to observe the clear and wide outside world as well as bright virtual stereoscopic images.
We propose that Holographic Optical Elements(HOE) can be used instead of half mirror of
conventional HMD. By using the HOE as the combiner on our new HMD, the function of lens and
* R&D Headquarters, Sanyo Co., Ltd.
** Student, Master Course of Dept. of Electrical Engineering
*** Lecturer, Master Course of Dept. of Electrical Engineering
**** Professor, Master Course of Dept. of Electrical Engineering
1
2combiner and binocular stereoscopy can be integrated into one piece of HOE, therefore it can provide us
the wide field of vision of real outside world as well as both high transparency for real world and
intensity for virtual images.
Principle of HOE for see-through HMD
Angular Selectivity of HOE
The HOE is diffraction grating and has the characteristic of angular selectivity and wavelength
selectivity. The angular selectivity is that the diffractive intensity of HOE in reconstruction depends on
the incident angle of illumin~ting beam which is equivalent to reference beam used in recording. In a
word, the diffraction light can not be seen if the view point is slipped off slightly. HOE can be exposed
by multiple beams and diffract many 'illuminating lights to the specific directions. Fig.l shows the basic
principle of the angular selectivity of HOE concerning about transmission type and Lippmann type.'
HOE r----~_----..&....---_
HOE
wavelength of ray 1 = wavelength of ray2
(a) Transmission type of HOE (b) Lippmann type of HOE
Fig.l Basic principle of angular selectivity of HOE
Wavelength Selectivity of HOE
The wavelength selectivity of HOE is that the diffractive intensity of HOE in reconstruction depends
on the wavelength of illuminating beam in case the incident angle of illuminating beam is equivalent to
reference beam used in recording. In other words, HOE can diffract the slight width of wavelength in the
illuminating beam. The characteristic of Lippmann type of HOE is very remarkable. It is 'possible to
transmit the visible rays and to reflect the narrow band of specific wavelength of rays (for example red).
holographic combiner(HOE)
display(ray2)
Fig.2 Principle of holographic combine
Therefore, we can make the holographic combiner we can see both the outside world and the specific
wavelength image shown by display. Fig.2 shows the principle ofholographic combiner. Its principal is the
same as the Head Up Display(HUD). In case of HUD, we can not see the stereoscopic images on the
display like as the LCD and so on. Nonnally, the viewing distance of HUD is long(focal distance is
60cm'-2m) and the screen size is not so large(about IOcm square), therefore we have to observe small
images ofHUD on both eyes. Furthennore if we move our head slightly, we can not see the images because
the exit angle of HUD is very narrow.
Binocular Stereoscopy of HOE
According to the angular selectivity and wavelength selectivity, single HOE exposed by multiple
recording beams can display the binocular images diffracted from two illuminating beams on each left and
right eye separately. It is very easy for us to understand that single HOE has the function providing the
binocular parallax images like as parallax barrier display or lenticular display without special glasses. And
we propose that we can see the binocular stereoscopic images through the HOE like as HUD ifit is located
close to our eyes and two rays including the parallax information are illuminated from different direction.
Fig.3 shows the principle of binocular parallax display of HOE.
3
HOE '---_.......- - __---
view of real world
(a) Transmission type (b) Lippmann type
Fig.3 Principle of binocular parallax display of HOE
Feature ofHMD using HOE
We propose the HMD using HOE as the combiner instead of the conventional HMD by using the-
feature of HOE shown in fig.! and 2. The HOE is most suitable to realiz,e the see-through HMD for MR
because it has the function both the combiner and the optical lens. Fig.4(a) shows the optical configuration
of conventional HMD and Fig.4(b) shows the optical configuration of HMD using HOE.
view of real world
halfmirror~
/----
.,#:
reflector I
I
I
I
o
display display
(a) Conventional HMD (b) New HMD using HOE
Fig.4 Concept of optical configuration of HMD
4The HMD using HOE is more useful than the HMD using a half mirror as the combiner, especially at
the Mixed Reality environment. The conventional HMD has the optical system close to the eyes and it is
limited the field of view for the real world. Fig.4(a) shows the field of view for the real world about the
conventional HMD. Meanwhile the HMD using HOE has only one grating in front of the face and its
field of view for the real world is very wide. Fig.4(b) shows the field of view for the real world about the
HMD using HOE. Furthermore all wavelength rays of.the real world can be transparent and only specific
wavelength rays illuminated by a display can be diffracted to a certain direction. Therefore, this HOE can
provide the binocular stereoscopic images.
Optical characteristics of HOE
A prototype of HOE is 4 inches long by 5 inches wide. The distance between eye and HOE is 4 inches.
The pupil distance is 65 mm. Fig.5 shows the optical characteristic of HOE. In recording, an object beam
and a reference beam are diverging rays, the angle between them is 120 degree. This angle can put the
small LCD display on both side of head and can prevent the reflection light on HOE from being incident
upon the human eyes. We can see the binocular images modulated by illuminating light for reconstruction
Image
Virtual Image for Left Eye
•
Virtual Image for Right Eye
o
5 inches wide
Image
Fig.5 Optical Specification of HOE
virtual images
L R
illuminating ray
\ IIIIl 0 illuminating ray
diffuser,\ " H Y/diffuser
h~~ :\~~
c:r ~LCD// ~
'- 'COllimator lens /~ semiconductor laser
(a) Optical system for reconstruction (b) Appearance of HMD using HOE
Fig.6 Prototype of optical system
at the viewing point. The rays from left is diffracted by HOE and can be seen on the left eye, the rays
from right is diffracted and can be seen on the right eye, therefore only one HOE can separate the
stereoscopic images to left and right eye. In addition, we can see the rays both the real world and the
diffracted images at the same time because HOE can diffract only specific wavelength rays without
reducing the intensity of the visible rays. We made the prototype of HOE shown in fig.5 by way of the
two-beam interference method and we made the experimental optical system for reconstruction by using
the semi-conductor laser as the illuminated light. Fig.6(a) shows the experimental optical system for
reconstruction. It 'uses the He-Ne laser (wavelength:632.5nm) in recording and semiconductor laser
(wavelength:635nm) in reconstructing. The reconstructed image was red color only.
This optical system uses 1.3 inches LCD display as the spatial modulation devices. Fig.6(b) shows the
appearance of HOE mounted on the helmet. Fig.? shows the diffracted images displayed by this optical
system. Fig.?(a) shows the diffractive image for left eye and Fig.?(b) shows the diffractive image for right
eye. Both the diffractive images and the outside world through HOE can be seen in the ordinary office
room which illuminating intensity was about 900lux. According to Fig.?, we can distinguish the left and
right parallax about a triangle and a ellipse on reconstructed diffracted Ught.
(a) Image for left eye (b)Image for right eye
Fig.? Diffracted images for binocular parallax recognition
Interactive MR System
Structure of MR System
We have developed the interactive MR system by using this HMD. In this MR system, we can
observe the virtual stereoscopic images among the real world and we can feel the virtual objects floating
in the real environment through HMO using HOE because the view of real world behind the virtual
hologram images is very wide. Fig.8 shows the structure of MR system. It uses graphic workstation for
generating the stereoscopic images and the magnetic sensor for head tracking. Graphic workstation
creates a pair of VGA resolution images for each left and right parallax and those images are transmitted
to the LCD attached on the helmet. The images on LCD can be diffracted by HOE and holographic
images are reconstructed and we can see virtual images through HOE. If the observer looks around in the
real world by moving his head, he can see the virtual objects according to his heading by magnetic head
tracking sensor. In order to create the high definition images for interaction of walk though and to grasp
the objects, SGI Indigo2 Maximum IMPACT as the hardware and Division dVISE which is a virtual
reality application package software are used.
Concept of Floating Image Media
We propose the Floating Image Media as the new expression. Floating Image Media means to express
5
6the virtual images among the real environment, namely, new media expression by Mixed Reality
technology. Our ultimate end for Mixed Reality is to display the electro-holography among the real world
because we can see the real world naturally as well as the natural electronic stereoscopic image with full
parallax . But it is very difficult to complete the electro-holography because there is no device and
calculator which can display the plenty of interference fringes. We can managed to use the optical see-
through HMO now for this purpose, but the HMO we can use now is not so good for combine both the
real world and the virtual images. Therefore we developed new HMO using HOE which is very effective
for viewing the natural real environment. By using our new HMO, we can complete the Floating Image
Media system without the electro-holography. The software contents of the Floating Image Media is
molecular simulation at this first prototype although it is possible to express the any shape and motion.
Fig.9 shows the example of Floating Image as the molecular simulation. We can look around the various
molecular model according to the observer's heading of his head.
We demonstrated the HMO using HOE on the interactive MR system at SIGGRAPH99(4). Total
number of 656 subjects tried it and all except 3 subjects who are originally stereo blind could observe the
binocular stereoscopic images among the real world through HOE.
frame
helmet
LJ magnetic source
magnetic sensor ~
R image
Graphic Workstation
Fig.8 Interactive MR System Fig.9 Example of Floating Images
Conclusion
We have completed the prototype' of HMO using HOE for the purpose of usage of MR environment.
And we proved that only one piece of HOE can separate the binocular parallax images onto left and
right eye. Furthermore we showed that the HOE as the combiner can improve the defects of half mirror
which is the tradeoff between transparency and reflectance. This time prototype is a little heavy and is not
miniaturized because it was made by our own hands. In addition, the field of view of virtual image is a
little narrow because angle of view is about 20 degrees. And the virtual image is only one color because
we used just red semiconductor laser. In future, we have to improve these factors.
References
[1] Y.Ohta, H.Tamura, "Mixed Reality" ,Ohmsha,1999
[2] R.T.Azuma: "A survey of augmented reality", Presence: Teleoperators and Virtual Environments,
Vo1.6, No4, pp.355-385 (1997)
[3] J.E.Melzer, K.Moffitt, "Head Mounted Display,Designing for the user", McGraw-Hill, 1997
[4] T.Ando, E.Shimizu, H.Takahashi, K.Yamasaki, T.,Matsumoto, "HologramlHead Mounted Display",
SIGGRAPH Conference Abstracts and Applications, pp.177, (1999)


Head Mounted Display for Mixed Reality using Holographic Optical Elements

http://dlisv03.media.osaka-cu.ac.jp/infolib/user_contents/kiyo/DB00000881.pdf

Head Mounted Display for Mixed Reality using Holographic Optical Elements

Abstract

Similar works

Full text

Available Versions

Osaka City University Repository