A real time model of the visible output from a 94 GHz sensor, based on a radiometric simulation of the sensor, was developed. A sequence of images as seen from an aircraft as it approaches for landing was simulated using this model. Thirty frames from this sequence of 200 x 200 pixel images were analyzed to identify and track objects in the image using the Cantata image processing package within the visual programming environment provided by the Khoros software system. The image analysis operations are described

Kasturi, Rangachar

English

NASA Technical Reports Server

13346
Analysis of Simulated Image Sequences from Se,tsors for Restricted-Visibility Operations
Ra.ngachar Kasturi
t.l_eai tltll_llt of r.lectnt:al"" " ' and i-ompmer Engineering
Penn State University
University Park, Pennsylvania 16802
tt , ,Abstract
As part of the Advanced SenSor and Imaging_System Technology (ASSIST) program, imaging systems and interface
requirements are being evaluated in order to enhance a pilot's view of the outside environment under restricted
visibility conditions. A sequence of simulated images as seen from an aircraft as it approaches a runway was
obtained from a model of a passive millimeter wave sensor operating at 94 GHz. Atmospheric attenuation is known
to be a local minimum at this frequency. These images were analyzed to delineate objects of interest such as
runways, taxiways, buildings, etc. Information about the position and motion of various objects on the ground may
be computed by tracking the regions in such .segmented image sequence. This information can be combined with
data from other sensors to alert the pilots of any objects which are likely to be a hazard for continuing the flight.
1. Introduction
Enhancing the safety of operation of aircraft under restricted visibility conditions has been an important research
topic for many years [ 1]. There is a great deal of interest in imaging sensors with the capability to see through fog
and produce a real-world display which, when combined with symbolic or pictorial guidance information, could
provide the basis for a landing system with lower visual minimum capability than those presently being used
operationally [21. An example of the interest in sensor and imaging systems technology is the FAA/DOD/Industry
program entitled, Synthetic Vision [31.
Since the attenuation of radiation in the visible spectrum due to fog is very large [4] (Figure 1), sensors are being
designed to operate at lower frequencies at which the attenuation is smaller (providing the ability to see through
fog). One such frequency of choice is 94 GHz since the clear-air attenuation at this frequency is a local minimum
and fog attenuation is low (Figure 1). In a joint research project, NASA Langley Research Center and TRW are
developing an imaging system to capture images using a passive sensor operating at this frequency. The goal of
this project is to establish a foundation for imaging sensor and pictorial graphics display technology and their
integration into future cockpit systems to enable transport operations under restricted-visibility conditions [2].
Extensive enhancement and analysis of images acquired using such sensors is essential in order to identify and track
various objects such as buildings, runways, vehicles, etc. as the aircraft approaches a runway for landing.
Information extracted from such an analysis is useful to generate warning signals to the pilot of any potential
hazards.
A real-time model of the visible output from a 94 GHz sensor, based on a radiometric simulation of the sensor, has
been developed. A sequence of images as seen from an aircraft as it approaches for landing has been simulated
using this model. Thirty frames from this sequence of 200 x 200 pixei images (For: H x V) were analyzed to
identify and track objects in the image using the Cantata image processing package within the visual programming
environment provided by the Khoros software system 151. The image analysis operations are described in this
paper.
2. Analysis of Simulated Images
The image analysis operations include noise filtering, edge detection, edge closing, region labeling, and matching
of corresponding objects in the image sequence. An example of the iconic Cantata workspaee used in this analysis
is shown in Figure 2. Three frames from the simulated image sequence on which this analysis was performed are
shown in Figure 3. The image analysis operations are described in this ,section.
125
https://ntrs.nasa.gov/search.jsp?R=19920004627 2020-03-17T13:45:52+00:00Z
Noise Filtering: Since a model for the noise phenomena is not yet available, random noise was added to the images
during simulation. A simple weighted spatial averaging filter (module marked vconvolve in Fig. 2) [6] was then
used to decrease the effect of this additive noise. Although this is adequate to filter noise in these simulated images,
a filter designed to match the noise in actual, real-world images will have to be developed later, when such data
is available.
Edge Detection: Edge pixels in the images were detected using the Difference Recursive Filter (DR.F; vdrfin Fig.
2) [7]. This filter, which is based on principles similar to those of the well-known Laplacian of Ganssian filter [8],
has been shown to be an optimal filter for detection of step edges in white noise [7]. The smoothing function used
by this filter is a symmetrical exponential function.
Edge Closing: Edge pixels detected by the DRF filter are not guaranteed to form closed object boundaries. Such
gaps in boundaries are bridged using an edge-closing operator (vclose in Fig. 2). The edge-closing operator uses
as its input two images: the output of the DRF filter and a gradient image (eg., output of a Sobel edge detector -
vd/ffin Fig. 2). It follows a path of maximum gradient between the two end points being bridged. The resulting
closed edges detected in corresponding images of Figure 3 are shown in Figure 4.
Region Labeling: Connected pixels belonging to the same object in the image are given a unique label. 4-
connectivity [6] is used in this labeling scheme (pixels which are connected at the top, bottom, left, and right - but
not diagonally - are said to be four-connected to the pixel at the center). The region labeling operator (vlabel in
Fig. 2) us_.,_ the output of the edge-closing module; alternatively, it is also possible to label objects by operating
directly on the noise filtered image using region growing techniques.
Region Matching: Objects in consecutive frames of the image sequence are compared to identify corresponding
objects. The matching (vrmatch in Fig. 2) is done using a probabalistic rule which is based on the size and shape
of regions. It computes the similarity of objects in successive frames. To illustrate objects which are matched they
are assigned the same gray level in both frames in the displayed image. Two such frames after matching are shown
in Figure 5.
3. Conclusions
A software system for analyzing simulated image sequences from a passive millimeter wave imaging system was
described in this paper. The analysis/tracking system consists of the following stages: smoothing using a spatial
averaging filter for noise reduction; detection of edge pixels using a recursive filter; bridging discontinuities in
detected edges using an edge-linking operator; labeling objects in each image in the sequence; and comparing objects
in consecutive frames to locate corresponding objects.
The passive millimeter wave imaging system will facilitate operation in restricted visibility conditions. Automated
analysis of sensor-captured images to locate and track moving objects will provide critical information necessary
for detecting potential runway incursions. Quantifying the amount of apparent motion of static objects in the scene
from consecutive image frames is also useful as a cross-check of the position and velocity of the approaching
aircraft. The preliminary results presented here using simulated images clearly demonstrate the potential of image
analysis methods for detecting and tracking objects in a dynamic scene. Analysis of real images is well known to
be a much more difficult task, however, particularly in real-time. To realize a practical system, new vision
algorithms for analyzing sensor-captured images, and new display concepts for fusing information from various
sensors, are necessary.
R_ferences
1. Glines, C.V., "Synthetic vision will let pilots .see through precipitation," Professional Pilot, Vol. 24, pp. 62-67,
Aug. 1990.
2. Hatfield, J.J and R. V. Parrish, "Advanced cockpit technology for future civil transport aircraft," Proc. llth
Annual IEEE/AESS Dayton Chapter Symposium, pp. , Dayton, Ohio, Nov. 1990.
126
3. Burgess, M.A., "Safety in future aircraft systems," Proc. Flight Safety Foundation's 34th Corporate Aviation
Safety Seminar, pp. 76-99, Dcmtbom, Michigan, April 1989.
4. Young, S.K., R.A. Davidhei_r, B. Harem, P.S.C. Lee, M. Musw, tto, M.M. Shoucri, L. Yujiri, "Passive
millin_t_r-wave imaging," TRW Space & Defense - Quest, Vol. 13, No. 2, pp. 3-20, Winter 1990191.
5. Rasum, J., D. Argiro, T. Sauer, and C. Williams, "A visual language and software development environment
for image processing," Intl. J. Imaging Systems and Technology, Vol. 2, pp. 183-199, 1990.
6. Gonzalez, R.C. and P. Wintz, Digital Image Processing, second edition, Addison-W_ley, Reading,
Massachusetts, 1987.
7. Shen, J. and S. Castan, "An optimal linear operator for edge detection," Proc. IEEE Conf. Computer Vision
and Pattern Recognition, pp. 109-114, Miami Beach, Florida, 1986.
8. Mart, D. and E. Hildreth, "Theory of edge detection," Proc. Royal Society, London, Set. B, Vol. 207, pp.
187-217, 1980.
lo(x)
, lOO
lO
o.1
0.01
10 GHz 100 GHz I THz 10 THz 100 THz 1000 THz
Figure 1. Atmospheric effects on electromagnetic radiation [4].
[mm..mmil+ m+Imm_.+I+ +++I
p++
I kernels
lmmmHmmml
++'+-'l-i_+I
I+,-Ii +,li
'?+°+I+,J
+++m.mll+mml+
+I-I+ t----ml..,,Im
Figure 2. Block diagram of the image analysis and tracking operations presented as a Cantata workspace.
127
ORIGINAL PAGE IS
OF POOR QUALITY
Figure 3. Frames 1, 16, and 30 in the sequence of simulated images. 
Figure 4. Runway, taxiway, horizon, and buildings delineated in the images of Figure 3. 
Figure 5. Two consecutive frames in the image sequence after detection and tracking of objects; 
corresponding oh.jects in the two images are given the same gray level. 
128 


Analysis of simulated image sequences from sensors for restricted-visibility operations

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server