Virtual reality (VR) is an exciting use of advanced hardware and software technologies to achieve an immersive simulation. Until recently, the majority of virtual environments were merely 'fly-throughs' in which a user could freely explore a 3-dimensional world or a visualized dataset. Now that the underlying technologies are reaching a level of maturity, programmers are seeking ways to increase the complexity and interactivity of immersive simulations. In most cases, interactivity in a virtual environment can be specified in the form 'whenever such-and-such happens to object X, it reacts in the following manner.' CLIPS and COOL provide a simple and elegant framework for representing this knowledge-base in an efficient manner that can be extended incrementally. The complexity of a detailed simulation becomes more manageable when the control flow is governed by CLIPS' rule-based inference engine as opposed to by traditional procedural mechanisms. Examples in this paper will illustrate an effective way to represent VR information in CLIPS, and to tie this knowledge base to the input and output C routines of a typical virtual environment

Engelberg, Mark L.

English

NASA Technical Reports Server

N95-19765
C:i V/J/
Using CLIPS to Represent Knowledge
in a VR Simulation
Mark Engelberg
LinCom Corporation
role @gothamcity.jsc.nasa.gov
September 13, 1994
Abstract
Virtual reality (VR) is an exalting use of advanced hardware and
software techonologies to achieve an immersive simulation. Until re-
cently, the majority of virtual environments were merely "fly-throughs"
in which a user could freely explore a 3-dimensional world or a visual-
ized dataset. Now that the underlying technologies are reaching a level
of maturity, programmers are seeking ways to increase the complexity
and intera_tivity of immersive simulations. In most cases, intera_tiv-
ity in a virtual environment can be specified in the form "whenever
such-and-such happens to object X, it reacts in the following man-
ner." CLIPS and COOL provide a simple and elegant framework for
representing this knowledge-base in an efficient manner that ca_ be
extended incrementally. The complexity of a detailed simulation be-
comes more manageable when the control flow is governed by CLIPS'
rule-based inference engine as opposed to by traditional procedural
mechanisms. Examples in this paper will illustrate an effective way to
represent VR information in CLIPS, and to tie this knowledge base
to the input and output C routines of a typical virtual environment.
' 363
https://ntrs.nasa.gov/search.jsp?R=19950013349 2020-06-16T09:20:55+00:00Z
1 Background Information
1.1 Virtual Reality
A virtual experience, or more precisely, a sense of immersion in a computer
simulation, can be achieved with the use of specialized input/output devices.
The head-mounted display (HMD) is perhaps the interface that most char-
acterizes virtual reality. Two small screens, mounted close to the user's eyes,
block out the real world, and provide the user with a three-dimensional view
of the computer model. Many HMDs are mounted in helmets which also
contain stereo headphones, so as to create the illusion of aural, as well as vi-
sual immersion in the virtual environment. Tracking technologies permit the
computer to rewt the position and angle of the user's head, and the scene is
recalculated accordingly (ideally at a rate of thirty times a second or faster).
[1]
There are many types of hardware devices which allow a user to interact
with a virtual environment. At a minimum, the user must be able to navigate
through the envrionment. The ability to perform actions or select objects
in the environment is also critical to making a virtual environment truly
interactive. One popular input device is the DataGlove which enables the
user to specify actions and objects through gestures. Joysticks and several
variants are also popular navigational devices.
1.1.1 Training
Virtual reality promises to have a tremendous impact on the way that train-
ing is done, particularly in areas where hands-on training is costly or dan-
gerons. Training for surgery, space missions, and combat all fall into this
category; these fields have already benefitted from existing simulation tech-
nologies [2]. As Joseph Psotka explains, _Virtual reality offers training as
experience _ [3, p. 96].
1.1.2 Current Obstacles
VIi hardware is progressing at an astonishing rate. The price of HMDs and
graphics workstations continues to fall as the capabilities of the equipment
increase. Recent surveys of the literature have concluded that the biggest
364
obstacle right now in creating complex virtual environments is the software
tools. One study said that creating virtual environments was "much too
hard, and it took too much handcrafting and special-casing due to low-level
tools" [5, p. 6].
VR developers have suffered from a lack of focus on providing interactiv-
ity and intelligence in virtual environments. Researchers have been "most
concerned with hardware, device drivers and low-level support libraries, and
human factors and perception" [5, p. 6]. As a result,
Additional research is needed to blend multimodal display, mul-
tisensory output, multimodal data input, the ability to abstract
and expound (intelligent agent), and the ability to incorporate hu-
man intelligence to improve simulations of artifical environments.
Also lacking are the theoretical and engineering methodologies
generally related to software engineering and software engineer-
ing environments for computer-aided virtual world design. [4,
p. 10]
1.2 CLIPS
VR programmers need a high-level interaction language that is object-oriented
because "virtual environments have potentially many independent but inter-
acting objects with complex behavior that must be simulated" [5, p. 6]. But
unlike typical object-oriented systems, there must be "objects that have time-
varying behavior, such as being able to execute Newtonian physics or systems
of rules" [5, p. 7].
CLIPS 6.0 can fill this need for a high-level tool to program the interac-
tions of a virtual environment. COOL provides the object-oriented approach
to managing the large number of independent objects in complex virtual en-
vironments, and CLIPS 6.0 provides the ability to construct rules which rely
on pattern-matching on these objects.
CLIPS is a particularly attractive option for VR training applications
because many knowledge-bases for training are already implemented using
CLIPS. If the knowledge-base about how different objects act and interact in
a virtual environment is implemented in the same language as the knowledge-
base containing expert knowledge about how to solve tasks within the envi-
ronment, then needless programming effort can be saved.
365
2 Linking CLIPS with a Low-level VR Library
2.1 Data Structures
Every VR library must use some sort of data structure in order to store all
relevant positional and rotational information of each object in a virtual envi-
ronment. These structures are often called nodes and nodes are often linked
hierarchically in a tree structure known as a scene. In order to write CLIPS
rules about virtual objects, it is necessary to load all the scene information
into COOL.
The following NODE class has slots for a parent node, children nodes, z-,
y-, and z-coordinates, and rotational angles about the z-, y-, and z-axis.
(defclass NODE
(is-a USER)
(role concrete)
(slot node-index (visibility public)
(create-accessor read-write) (type INTEGER))
(slot parent (visibility public)
(create-accessor read-write) (type INSTANCE-NAME))
(multislot children (visibility public)
(create-accessor read-write) (type INSTANCE-NAME))
(slot x (visibility public) (create-accessor read) (type FLOAT))
(slot y (visibility public) (create-accsssor read) (type FLOAT))
(slot z (visibility public) (create-accessor read) (type FLOAT))
(slot xrot (visibility public) (create-accessor read) (type FLOAT))
(slot yrot (visibility public) (create-accessor read) (type FLOAT))
(slot zrot (visibility public) (create-accessor read) (type FLOAT)))
Itisa trivialmatter to write a converter which generates NODE instances
from a scene file.For example, a typicalscene might convert to:
(definstances tree
(BODY of NODE)
(HEAD of NODE)
366
; and so on
)
(modify-instance [BODY]
(x 800.000000)
(y -so.oooo0o)
(z 280.000000)
(xrot 180.000000)
(yrot 0.000000)
(zrot 180.000000)
(parent [REF])
(children [HEAD] [Rpalm]
; and so on
[Chair] ))
2.2 Linking the Databases
Note that the NODE class has no default write accessors associated with slots
x, y, z, xrot, yrot, or zrot. Throughout the VR simulation, the NODE
instances must always reflect the values of the node structures stored in
the VR library, and vice versa. To do this, the default write accessors are
replaced by accessors which call a user-defined function to write the value to
the corresponding VR data structure immediately after writing the value to
the slot. Similarly, all of the VR functions must be slightly modified so that
any change to a scene node is automatically transmitted to the slots in its
corresponding NODEinstance.
The node-index slotof the NODE classis used to give each instance a
unique identifyingintegerwhich serves as an index into the C array of VR
node structures.This helps establishthe one-to-one correspondence between
the two databases.
2.3 Motion Interaction
A one-to-one correspondence between the database used internally by the VR
library and COOL objects permits many types of interactions to be easily
programmed from within CLIPS, paxticulaxly those dealing with motion.
The following example illustrates how CLIPS can pattern-match and change
slots, thus affecting the VR simulation.
367
(defrule rabbit-scared-of-blue-carrot
(object (name [RABBIT]) (x ?r))
(object (name [CARROT]) (x ?c_:(< ?c (+ ?r 5))&:(> ?c
i>
(send [RABBIT] put-x (- ?r 30)))
(- ?r 5))))
Assuming the rabbit and blue carrot can only move alon 8 the z-axis, this
rule can be paraphrased as "whenever the blue carrot gets within 5 inches of
the rabbit, the rabbit runs 30 inches away."
2.4 The Simulation Loop
Most interactivity in virtual environments can be almost entirely specified
by rules analogous the above example. A simulation then consists of the
following cycle repeated over and over:
1. The low-level VR function which reads the input devices is invoked.
2. The new data is automatically passed to the corresponding nodes in
COOL, as described in Section 2.1.
3. CLIPS is run, and any rules which were actiwted by the new data are
executed.
4. The execution of these rules in turn triggers other rules in a domino-like
effect until all relevant rules for this cycle have been fired.
. The V1% library renders the new scene from its internal database (which
reflects all the new changes caused by CLIPS rules), and outputs this
image to the user's HMD.
While the low-level routines still provide the means for reading the input
and generating the output of VR, the heart of the simulation loop is the
execution of the CLIPS rule-base. This provides an elegant means for in-
crementally increasing the complexity of the simulation; best of all, CLIPS'
use of the Rete algorithm means that only relevant rules will be considered
on each cycle of execution. This can be a tremendous advantage in complex
simulations.
368
2.5 More VR functions
There are some VR interactions that cannot be accomplished solely through
motion. Fortunately, CLIPS provides the ability to create user-defined func-
tions. This allows the programmer to invoke external functions from within
CLIPS. User-defined CLIPS functions can be provided for most of the useful
functions in a VR function library so that the programmer can use these func-
tions on the fight-hand side of interaction rules. Some useful VR functions
include:
make-invisible Takes the node-index of an instance as an argument.
chauge-color Requires the node-index of an instance and three floats spec-
ifying the red, green, and blue characteristics of the desired color.
test-collision Takes two node-index integers and determines whether
their corresponding objects are in contact in the simulation.
play-sotmd Takes an integer which corresponds to a soundfile (this corre-
spondence is established in another index file of soundfiles).
The play-sound function takes an integer as an argument, instead of
a string or symbol, because there is slightly more overhead in processing
and looking up the structures associated with strings, and speed is crucial
in a virtual reality application. Similarly, all user-defined functions should
receive the integer value stored in an instance's node-index slot, instead of
the instance-name, for speed purposes.
It is also possible to create a library of useful deffunctions which do many
common calculations completely within CLIPS. For example, a distance func-
tion, which takes two instance-names and returns the distance between their
corresponding objects, can be written as follows:
(deffunction distance (?nl ?n2)
(sqrt (+ (** (- (send ?n2 get-x)
(** (- (send ?n2 get-y)
(** (- (send ?n2 get-z)
(send ?nl get-x)) 2)
(send ?nl get-y)) 2)
(send ?nl get-z)) 2))))
369
3 Layers upon Layers
Once basic VR functionalityisadded to CLIPS, virtualenvironments can
be organized in CLIPS according to familiarobject-orientedand rule-based
design principles. For example, a RABBIT class could be defined, and the
example rules could be modified to pattern match on the is-afieldinstead of
the name field.Ifthiswere done, any RABBIT instance would automatically
derive the appropriate behavior. Once a libraryof classesisdeveloped and
an appropriate knowledge-base to go with it,creating a sophisticatedvirtual
environment ismerely a matter of instantiatingthese classesaccordingly.
Consider thisfinalexample, illustratingpoints from Sections 2.5 and 3.
(defrule shy-rabbit-behavior
?rabbit <- (object (is-a RABBIT) (personality shy)
(x7) (y 7) (z?))
?band <- (object (is-a RAND) (x 7) (y ?) (z ?))
m>
(if (< (distance 7rabbit ?hand) 100) then
(bind ?rabbit-index (send 7rabbit get-node-index))
(if (evenp (random)) then
(change-color ?rabbit-index I 0 O) ; rabbit blushes
else
(make-invisible 7rabbit-index)) ))
This rule becomes relevant only ifthere isa shy rabbit and a hand in the
simulation. Ifso, shy-rabbit-behavior isactivated whenever the hand or
the rabbit moves. Ifthe hand gets closeto the rabbit,there isa 50% chance
that the rabbit willblush,and a 50% chance that the rabbit willcompletely
vanish.
4 Conclusion
CLIPS has been successfullyenabled with virtualrealityprogramming capa-
bilities,using the methodologies described in thispaper. The two testusers
of this approach have found CLIPS to be a simpler, more natural paradigm
for programming virtualrealityinteractionsthan the standard approach of
managing and invoking VR functionsdirectlyin the C language. Hopefully,
370
by using high-level languageslike CLIPS in the future, more VR program-
mers will be freed from their current constraints of worrying about low-level
details, and get on with what really matters -- creating complex, intelligent,
interactive environments.
References
[1] Ben Delaney. State of the art VR technology circa 1994. New Media,
4(8):44-45, August 1994.
[2] Ben Delaney. Virtual reality goes to work. New Media, 4(8):40-48, August
1994.
[3] Joseph Psotka. Synthetic environments for training. In Second Annual
Synthetic Environments Conference, pages 79-107. Technical Marketing
Society of America, March 1994.
[4] U.S. Army Training and Doctrine Command and U.S. Army Research
Office. Executive Summary -- Virtual Reality/Synthetic Environments
in Army Training, October 1992.
[5] Andries van Dam. VR as a forcing function: Software implications of
a new paradigm. In IEEE 1993 Symposium on Research Frontiers in
Virtual Reality, pages 5-8, Los Alamitos, California, October 1993. IEEE
Computer Society Technical Committee on Computer Graphics, IEEE
Computer Society Press.
371


Using CLIPS to represent knowledge in a VR simulation

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server