One approach used to better understand the impact of visual flow on control tasks has been to use synthetic perspective flow patterns. Such patterns are the result of apparent motion across a grid or random dot display. Unfortunately, the optical flow so generated is based on a subset of the flow information that exists in the real world. The danger is that the resulting optical motions may not generate the visual flow patterns useful for actual flight control. Researchers conducted a series of studies directed at understanding the characteristics of synthetic perspective flow that support various pilot tasks. In the first of these, they examined the control of altitude over various perspective grid textures (Johnson et al., 1987). Another set of studies was directed at studying the head tracking of targets moving in a 3-D coordinate system. These studies, parametric in nature, utilized both impoverished and complex virtual worlds represented by simple perspective grids at one extreme, and computer-generated terrain at the other. These studies are part of an applied visual research program directed at understanding the design principles required for the development of instruments displaying spatial orientation information. The experiments also highlight the need for modeling the impact of spatial displays on pilot control tasks

Bennett, C. Thomas

Johnson, Walter W.

Perrone, John A.

Phatak, Anil V.

English

NASA Technical Reports Server

N90-22956
SYNTHETIC PERSPECTIVE OPTICAL FLOW:
INFLUENCE ON PILOT CONTROL TASKS
C. Thomas Bennett, Walter W. Johnson, John A. Perrone
NASA Ames Research Center
Moffett Field, California
Anil V. Phatak
Analytical Mechanics Associates
Palo Alto, California
INTRODUCTION
Computational and empirical analyses of optical flow have led to a more complete under-
standing of pilot control tasks. Such analyses are based on the premise that a primary stimulus for
the perception of self-motion is the flow of optical texture in the visual field (Gibson, 1950;
Koenderink and van Doom, 1976). It has been further recognized that there are both local and
global optical variables that might influence control behavior (Owen and Warren, 1982; Uttal,
1985). With this realization came the understanding that to study how optical flow influences con-
trol tasks, it is essential that the complex visual scene be decomposed into observable flow patterns
(Regan and Beverly, 1985).
One approach used to better understand the impact of visual flow on control tasks has been to
use synthetic perspective flow patterns. Such patterns are the result of apparent motion across a
grid or random dot display. Unfortunately, the optical flow so generated is based on a subset of
the flow information that exists in the real world. The danger is that the resulting optical motions
may not generate the visual flow patterns useful for actual flight control.
We have conducted a series of studies directed at understanding the characteristics of syn-
thetic perspective flow that support various pilot tasks. In the first of these, we examined the con-
trol of altitude over various perspective grid textures (Johnson et al., 1987). Another set of studies
has been directed at studying the head tracking of targets moving in a three-dimensional coordinate
system. These studies, parametric in nature, have utilized both impoverished and complex virtual
worlds represented by simple perspective grids at one extreme, and computer-generated terrain at
the other.
These studies are part of an applied visual research program directed at understanding the
design principles required for the development of instruments displaying spatial orientation infor-
mation. The experiments also highlight the need for modeling the impact of spatial displays on
pilot control tasks.
40-1
https://ntrs.nasa.gov/search.jsp?R=19900013640 2020-03-19T22:06:42+00:00Z
ALTITUDE CONTROL
Introduction
The purpose of this experiment was to examine the characteristics of "wire frame" perspec-
tive grids as support for altitude control. Wolpert, Owen, and Warren (1983) reported that splay
angle information was one of the most important indicants of altitude change. In their study, they
used ground surface textures consisting of equally spaced lines either parallel to the direction of
travel (meridian texture), orthogonal to the direction of travel (latitudinal texture), or both (square
texture).
There are two limitations of Wolpert's work that have relevance to the current study. The
first is that discrete-trial, passive-response methodology was used. This is in contrast with a set-
ting where a person is required to continuously monitor a perspective scene, and where his or her
responses result in feedback control of perspective dimensions of the stimulus.
The second limitation derives from the fact that subjects could have monitored the location at
which any meridian texture line intersected the bottom edge of the screen. As a result, a subject
could tell if altitude had changed by merely observing the movement and intersection without mon-
itoring the splay angle at all.
Methods
Subjects were flown at a constant velocity, at three different altitudes, over each of the three
grid types mentioned above. The display was generated by an Evans and Sutherland PS-2 graph-
ics system. The "aircraft" was buffeted by both lateral and vertical winds. Each of the distur-
bances was defined by its own sum of 13 sine waves. The five subjects were required to maintain
a constant height above the grid by means of a joy stick. The primary performance metric was
adjusted root mean square error (ARMSE) from the assigned altitude.
The important point here is that because of the lateral noise imposed on the craft position, the
meridian lines moved left and right irrespective of the actual change in altitude. As a result, sub-
jects could not determine altitude change by only the movement of the meridian lines. Changes in
altitude would have to be determined by changes in density (lower density corresponds to a lower
altitude) and splay angles (the greater the angle the lower the altitude) of the grid structure.
Results and Discussion
Based on the work previously cited, it was expected that ARMSE sould be lowest for the
meridian surface and highest for the latitude surface. This was not the case (fig. 1).
Because of the unexpected larger ARMSE values obtained when flying over the meridian
surface texture, it was decided to look more critically at a single subject's performance. A detailed
power frequency analysis was performed and showed that the meridian grid resulted in (1) less
stick power associated with the vertical disturbance than any of the other grid textures; and (2) the
most power in the stick movement associated with the lateral input signal (fig. 2).
40-2
These analyses indicate that the subject (1) was less reactive to the information specifying
true changes in altitude when flying over the meridian texture; and (2) tended to confuse lateral
with vertical motion in displays where only splay information was available.
PERSPECTIVE FLOW FIELDS AND HEAD TRACKING IN A 3-D VIRTUAL
WORLD
Introduction
In the previous study, we discussed the impact of perspective flow displays on a manual
control task that regulated the altitude of a simulated aircraft. In certain military rotorcraft, systems
exist in which movement of a sensor system is slewed to the crewmember's head motion. Cur-
rently there is only standard flight symbology in this helmet-mounted display to indicate altitude,
attitude, and heading. A small portion of the display provides information concerning the field of
view and field of regard of the sensor.
Despite the fact that these systems are currently fielded, little systematic data exist concerning
how a pilot uses flight/target information presented on a helmet-mounted display. Even less data
are available on alternative display configurations that might make a pilot more sensitive to changes
in aircraft state.
As part of a program to better understand helmet-mounted flight displays, we conducted a
study to validate a laboratory simulation of the currently fielded system. A perspective flow field
was used to create the virtual world that was the basis for this simulation. A detailed report of this
study is in preparation.
Methods
A wire-frame perspective grid was displayed to six subjects by means of a head-mounted
1 in. Sony electronic viewfinder. Head position was monitored by means of a Polhemus head
tracker. As the subjects moved their heads, they were able to "look" around the virtual world.
Six subjects were "flown" over the grid at two different altitudes and three different veloci-
ties. Positioned on the surface was a wire frame cube. The target was offset to the left or right of
the direction of travel. The subject could "track" the target by means of a cross hair that was gen-
erated in the middle of the monocular display. Tracking ARMSE was determined by subtracting
line of sight (LOS) to target from the visual LOS.
Results and Discussion
Figure 3 shows the mean screen errors for the different offsets, as a function of slant angle to
the target. The term slant angle incorporates elevation and azimuth components. It is important to
remember here that as range to the target decreases, optical (apparent) velocity increases. So,
during the course of the "flight," the target was in fact accelerating, even though "aircraft" speed
40-3
wasconstanthroughouttheflight. A 3x4x2 (speedx offset x altitude) repeated-measures
analysis of variance was conducted on the mean ARMSE values for each subject. This analysis
indicated that as optical velocity increased, there was a significant increase in screen error
(p < 0.001). This was true irrespective of whether the increase in optical velocity was produced
by changes in slant range or "vehicle" speed.
In figure 4 is shown the change in both ground error and screen error as a function of slant
angle. To calculate ground error, the target and visual LOSs were first projected to the ground
plane. Ground error was then given as the distance between those two intersections. As slant
angle increased, ground error did not significantly change (p > 0.46). One interpretation of these
data is that the subjects were treating the task as a true three-dimensional LOS problem. If the
subjects had maintained screen error constant (as in an arcade game), ground error would have
directly varied with slant range. A second interpretation is that subjects tried to maintain a constant
screen error, but were unable to do so because of the accelerating optical velocity of the target.
HEAD TRACKING DURING SIMULATED AUTOMATED AND MANUAL
HELICOPTER FLIGHT
Introduction
A model of head tracking in a 3-D world (represented by a perspective flow field) was devel-
oped and tested in the previous study. The purpose of the present experiment was to (1) validate
the laboratory simulation, and (2) model the trade-offs that pilots make when they are required to
control their craft and simultaneously head-track targets. A detailed report of this study is in
preparation.
Methods
Six AH-64 Apache helicopter pilots took part in a simulation of the pilot night-vision system
(PNVS). The study took place in a fixed-base mock-up of the helicopter. The visual scene was a
complex, computer-generated world in which a stationary helicopter served as the target. Each
pilot was initially flown "automatically" in either a rectilinear or curvilinear path past the target.
This served to simulate a copilot/gunner or a pilot in an automated flight mode. The pilot was then
required to duplicate the ground track in manual flight mode while simultaneously tracking the tar-
get. The spread of target ranges extended from approximately 6,000 to 400 ft. In the trials
reported here, own-ship velocities never exceeded 80 mph.
Head-tracking ARMSE was calculated as in the previous study. Ground-track error was also
measured. This was the difference in feet between the flightpaths in the automated versus manual
flight modes. During the manual flights, pilots were informed that target tracking was the primary
task, but that ground track error was being measured.
40-4
Results and Discussion
Figure 5a shows the averaged screen errors in the manual and automatic flight modes, as a
function of slant angle. A repeated-measures analysis of variance revealed a significant effect of
slant angle (p < 0.005) as well as significant slant angle by flight mode interaction (p < 0.001).
The inference is that screen error is greater near the end of a manual flight than it is at the end of an
automatic flight.
At first glance this makes a great deal of intuitive sense. During manual flight, the pilot is not
only head-tracking a target, but also manually flying the helicopter. However, inspection of fig-
ure 5b reveals another explanation of the increased screen error. As can be seen, optical velocities
during the manual flight mode are significantly greater than during automated flight. Additionally,
a multivariate regression revealed a significant positive correlation (p < 0.0001) between optical
velocity and screen error, when the effect of slant angle is statistically removed. This analysis is
consistent with the interpretation that optical velocity is a major source of head-tracking error.
An interesting question that arises from these data is why optical velocities are greater during
manual flight. Presumably, given that the pilot is under control of the craft, he or she could have
biased the flightpath to decrease optical velocity, and, hence, screen error.
Figure 5c provides some understanding of the complex trade-offs that the pilots were mak-
ing. This figure shows that as slant angle increased (and slant range decreased), the magnitude of
the ground error decreased significantly (p < 0.005), then gradually increased. As with the second
experiment, the data reported here are consistent with the interpretation that the pilots were treating
the task as a true 3-D problem. Otherwise, there would have been no reason why they would not
have simply held screen error constant and allowed ground error to vary. Also, although they flew
a flightpath that increased the problem of head tracking (by increasing optical velocity), their man-
ual flightpath resulted in, if not a constant, at least a minimal ground error. This, of course, is the
name of the game for a combat pilot.
GENERAL DISCUSSION
Pilot control tasks include both manual flight control and the control of head-slaved sensor
systems. Three studies were presented to highlight the nature of the design considerations that are
important in the development of displays that convey spatial orientation information. Factors
emphasized included the need to characterize both optical/visual flow fields and the control
dynamics of manual and head-slaved systems.
40-5
REFERENCES
Gibson, J. J.: The perception of the visual world. Houghton Mifflin, 1950.
Johnson, W. W.; Bennett, C. T.; Tsang, P. S.; and Phatak, A. V.: The visual control of simu-
lated altitude. Proc. Fourth Symposium on Aviat. Psychol., 1987, in press.
Koenderink, J. J.; and van Doom, A. J.: Local structure of movement parallax of the plane. J.
Opt. Soc. Amer., vol. 66, 1976, pp. 717-723.
Owen, D. H.; and Warren, R.: Optical variables as measures of performance during simulated
flight. Proc. of Human Factors Soc. 27th Ann. Meeting, 1982, pp. 312-315.
Regan, D.; and Beverley, K. I.: Visual flow and direction of locomotion. Science, vol. 227,
1985, pp. 1063-1064.
Uttal, W. R.: The detection of nonplanar surfaces in visual space. Lawrence Erlbaum Associates,
Hillsdale, NJ, 1985, pp. 1-22.
Wolpert, L.; Weon, D. H.; and Warren, R.: The isolation of optical information and its metrics for
the detection of descent. Optical flow and texture variables useful in simulating self motion
(II), Final Technical Report for Grant No. AFOSR-81-0078. Columbus, OH: The Ohio State
Univ., Depart. Psychol., Aviation Psychol. Lab., 1983.
40-6
t.kl
¢/3
et"
110
100
9O
80
70
GRID TYPE
MERIDIAN GRID
C) LATITUDE GRID
O SQUARE GRID
t I
ol I
=o °
lee
60 = l I
1000 1500 2000
ALTITUDE, ft
Figure 1.- Mean median ARMSE as a function of grrd type and altitude (ARMSE measures based on
10-sec intervals.
v--
(a 30
z
<¢
gfl
re
_ 25
,..I
_ 20
_ 15
k-
0
_ 5
re
20
GRID TYPE
MERIDIAN GRID
O
rn
........ rn ............. -_
' ' 0'01000 1500 2 0
ALTITUDE, ft
Figure 2.- Control-stick activity associated with lateral disturbance as a function of grid type and
altitude (subject 5).
40-7
3g,
"o
m
o
rr
==2
z
tll
g.I
rr
t_
O_
--
0 I I I I I I
40 50 60 70 80 90
SLANT ANGLE, deg
Figure 3.- Head tracking/virtual world, screen error versus offset.
3 7
-_2
m
O
tr
re
lit
Z
ILl
ILl
er
ol
oO
GROUND ERROR
0 I I I I I I 0
40 50 60 70 80 90
SLANT ANGLE, deg
5
¢"
4_
LU
(3
3Z
Z)
O
CIE
2 _
Figure 4.- Head tracking/virtual world (all conditions).
40-8
6"0 5
er
o
=4
t_
uJ
z3
_:2
c_
1
.,f
 ..j' oToMAT,c
ia) i l i i I _ i i i
5O
40
_ 30
0
..I
_, 2o
.J
_ 10
i-
e_
o
MANUA/
. IC
))l I l i I i i I
5O
_40
30
nr
LU
_ 20
o
_ 10
V AUTOMATIC
(C) i l
'5'''8'0'20 30 40 0 60 70 90
SLANT ANGLE, deg
Figure 5.- Manual versus automatic flight, all conditions. (a) Screen error.
(c) Ground error.
(b) Optical velocity.
40-9



Synthetic perspective optical flow: Influence on pilot control tasks

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server