Image analysis often starts with some preliminary segmentation which provides a representation of the scene needed for further interpretation. Segmentation can be performed in several ways, which are categorized as pixel based, edge-based, and region-based. Each of these approaches are affected differently by various factors, and the final result may be improved by integrating several or all of these methods, thus taking advantage of their complementary nature. In this paper, we propose an approach that integrates pixel-based and edge-based results by utilizing an iterative relaxation technique. This approach has been implemented on a massively parallel computer and tested on some remotely sensed imagery from the Landsat-Thematic Mapper (TM) sensor

Lemoigne, Jacqueline

English

NASA Technical Reports Server

IMAGE ANALYSIS
OF DISPARATE
BY INTEGRATION
INFORMATION
N93-25973
Jacqueline Le Moigne
Information Science and Technology Office and
Center of Excellence in Space Data and Information Sciences
Mail Codes 930.4 and 930.5
NASA Goddard Space Flight Center
Greenbelt, MD 20771
Tel: (301) 286-8723
Fax: (301) 286-5152
lemoigne@nibbles.gsfc.nasa.gov
ABSTRACT
Image analysis often starts with some
preliminary segmentation which provides a
representation of the scene needed for further
interpretation. Segmentation can be performed in
several ways, which are categorized as pixel-
based, edge-based, and region-based. Each of
these approaches are affected differently by
various factors, and the final result may be
improved by integrating several or all of these
methods, thus taking advantage of their
complementary nature.
In this paper, we propose an approach that
integrates pixel-based and edge-based results by
utilizing an iterative relaxation technique. This
approach has been implemented on a massively
parallel computer and tested on some remotely
sensed imagery from the Landsat-Thematic
Mapper (TM) sensor.
1. INTRODUCTION
After pre-processing of some original data,
image segmentation is the process which
generates a spatial description of an image as a
set of specific parts, regions or objects. The
"segmented" output is then utilized by a higher-
level image interpretation process. There is no
single standard approach to segmentation which
would be "successful" for any type of data, but
some general methods have been developed
based on the two main characteristics of regions
or objects in an image:
(1) each region or object exhibits an internal
uniformity with respect to some image
property (e.g., gray level, color, texture),
(2) each region or object presents some
contrast with its surroundings.
These two properties lead to three different
types of segmentations, pixel-based, region-
based and edge-based segmentations.
Each of these approaches is affected
differently by various factors. Pixel-based
methods form their decision only based on the
information given at each pixel, while the two
other types of segmentation take into account the
information contained in the surrounding pixels.
Usually in a pixel-based approach, all of the
original information is utilized, thus avoiding
selection process. Such methods are also easier
to integrate in a learning process, but their main
133
https://ntrs.nasa.gov/search.jsp?R=19930016784 2020-03-17T05:35:40+00:00Z
drawback is that they do not take into account
spatial information. Conversely, edge- and
region-based approaches base their decision on
spatial information. Edge-based methods
measure the variation of intensity between pixels
belonging to different objects; they may produce
excellent results for unevenly illuminated images,
but they also can be very sensitive to noise.
Region-based approaches, which measure the
internal uniformity of some intensity or texture
function, often produce spurious segmentation
under non-uniform lighting conditions, but are
usually less sensitive to noise.
In general, these three types of
segmentation may be improved by integrating
them and by taking advantage of their
complementary nature. We previously proposed
an approach that integrates region segmentation
and edge detection results by interpreting a binary
tree representation (Le Moigne, 1992), thus
producing a refined region segmentation. This
algorithm has been tested on Landsat-TM data.
The integration of edge and region data may also
be performed by a relaxation method and has
been proposed in (Le Moigne, 1989); in the work
described in this paper, we refine this relaxation
method for the purpose of integrating edge and
classification information and we implement it on
a massively parallel computer, the MasPar MP-1.
Then, we test this approach on remotely sensed
imagery, such as Landsat-TM data.
2. RELAXATION TECHNIQUES
a. Overview
A large number of iterative relaxation
schemes have been proposed to improve the
results given by such basic processes as edge
detection, region segmentation or pixel
classification (Davis, 1981; Hummel, 1987;
Faugeras, 1981; Peleg, 1980; Zucker, 1977).
The principle of these algorithms is to utilize
contextual information for iteratively changing
the initial labeling of the objects in a scene toward
optimal labeling.We will concentrate on
relaxation methods for which the decisions at
each point are taken in a probabilistic fashion.
This general class of relaxation techniques is
described in (Davis, 1981). Let us assume that
we have a set of N objects {O1, O 2 ..... ON}
(e.g., the pixels) to be labeled into one of L
n
classes {CI, C2 ..... CL} and that Pi [Ck] is the
probability that the object Oi is assigned to the
class Ck at the iteration n. The principle of the
relaxation algorithms, then, is to build a series of
probability sets {pn[ck] ;1 < i < N; I < k _< L},
where each new iteration step, n, adjusts the
probabilities according to the contextual
information, and these probabilities satisfy the
conditions (1) and (2):
for every i, for every k,
n
0 < Pi [Ck] < 1, (1)
and
t.
every i, __._pn[ck] = 1. (2)for
k=a
Thus, the problem is to define the updating
n
formula for Pi [Ck]. The simplest updating (which
we use in this work) is described below.
The relaxation scheme assumes that the
class assignments of each object depend on the
class assignment of the "other" objects; the
"other" objects can be defined, for example, by
neighboring objects. Therefore, we define
134
c[i,k;j,l] as the compatibility coefficient between
the object Oi with the label Ck and the object Oj
with the label Cl. We assume that the c[i,k;j,ll's
belong to the range interval [0,+1] and are
positive if there is compatibility and equal to 0 if
there is high incompatibility. The coefficient
c[i,k;j,l] can be defined as a conditional
probability, p(i e Ck/j • CI), which provides a
n Cprobabilistic framework. Let qi I k] be the global
compatibility for the object Oi with the label Ck; it
is defined by
n 1 j=V[i] I=L
c[i,k;j,1] (3)
- p;[cl]qi [Ck] V[i] j=_i 1=_1
where vii] is the number of objects in the
neighborhood of object Oi. Then nqi [Ck] is the
"increment" which is applied to update pn[c k]
and compute the new probability set {pln+l)[ck]}
(see (Davis, 198 I) for details):
n n
Pi [Ck] x qi[Ck]
Pn + 1[Ck] - I=L
"_ n nPi [C1] x qi [C1]
(4)
1=1
This multiplication still ensures that the
conditions (1) and (2) are satisfied. Besides, if
the global compatibility of the object Oi with the
label Ck is higher than all the compatibilities of
Oi with the other labels CI, then the pn[ck]'S
a
increase relative to the other Pi [Cil S. That means
that this scheme provides an overall improvement
of the labeling but it does not guarantee the
convergence toward stable labeling.
Other relaxation schemes (Faugeras, 1981;
Peleg, 1980) utilize different formulas or
different frameworks. For example in (Faugeras,
1981), another way of updating the probabilities
pn[c k] is given; the principle of this other method
is to minimize a criterion by the "projected
gradient" optimization method. The criterion
measures the ambiguity and the consistency of
the current labeling at each step. This algorithm
provides us with a converging sequence of
probabilities pn. Starting from an initial point pO,
the method converges toward a local minimum in
the vicinity of _.
b. Utilizing Relaxation to Integrate
Disparate Information
We now describe how a relaxation
technique, such as the one described above, can
be utilized to integrate knowledge from edge
detection and pixel classification.
Previously, Zucker and Hummel (Zucker,
1977) simultaneously used edge and region data
in a relaxation process for labeling dots. Their
goal was to provide a low-level description of the
roles of dots in cluster analysis. Unlike our
approach, they used these two types of data
"equally", i.e., both edges and regions defined
the labels and the initial probabilities. There were
ten different labels, with eight edge labels at
various orientations, one "region" label called
"interior point" label, and one "noise" label.
In our approach, the labels are all region
labels, and edge data are utilized to update the
labeling by way of the compatibility coefficients.
This approach tends to be more general and this
algorithm can be very easily applied to the fusion
of various types of data. In our work, the
135
definition of the labels and the initial probabilities
are provided by a neural network pixel
classification (Chettri, 1992), then the relaxation
process updates the initial labeling by using some
edge detection results, e.g., from a Canny
operator (Canny, 1986). These standard
techniques, pixel classification and edge
detection, can easily be changed without altering
the definition of the overall relaxation algorithm.
Once the initial probabilities are given, the
fusion is realized by computing the
compatibilities between neighboring pixels.
Coefficients c[i,k;j,l] in formula (3) represent the
compatibilities between object Oi with the label
Ck and object Oj with the label C1. If a relaxation
is performed only on regions, these coefficients
represent the compatibilities of neighboring
pixels belonging to given regions. If both regions
and edges are considered, both information can
be integrated into the compatibility coefficients by
utilizing the following equation:
c[i,k;j,1]= pr(Ck/C1) x Fk&l(i,j) (5)
where pr(Ck/Cl) is the "region-probability"
(independent of edges) of i belonging to the class
Ck if the neighbor j belongs to the class CI; this
probability could be estimated from some
previous ground truth data. The last term in Eq.
(5), Fk&l(i,j), is a function that varies in [0,11 and
is:
-closer to 1 if Ck = Cl (or, more generally, if
regions C"kand CI are similar in tone) and i is not
an edge point, or if regions Ck and CI are
different and i is an edge point,
- closer to 0 if Ck = CI and i is an edge point, or
if regions Ck and C1 are different and i is not an
edge point.
For example, F can be defined by:
Fk&l(i,j) = Kk&l x Mag(i)
+ (1- Kk&l ) x (1- Mag(i) )
where Kk&l is equal to 0 if k=l and equal to 1 if
k;el, and Mag(i) is the magnitude of the gradient
computed at the point i and normalized between 0
and 1.
The magnitude of the gradient at neighboring
point j would also be taken into account and we
could utilize the following formula:
Fk&l(i,j)= Kk&l x Mag(i) x Mag(j)
+ (1- Kk&l ) x (1- Mag(i) x Mag(j) ).
This updating has not been implemented yet, but
will be considered in future work.
Therefore, both region and edge
information participate simultaneously in the
updating of the labeling. Results are presented in
Section 4.
3.PARALLEL IMPLEMENTATION
Computation time is the main concern of
relaxation techniques. However these techniques
are characterized by parallel local processing
which can be easily implemented on an
architecture that favors computations between
adjacent pixels (Fishier, 1987). The method
discussed in this paper has been implemented on
a MasPar MP-1. The MasPar Parallel Processor
is a fine-grained, massively parallel SIMD
architecture, with 16,384 parallel processing
elements arranged in a 128x128 matrix and
connected by an eight nearest neighbors
interconnection network.
136
The parallel implementation of the
relaxationalgorithmis straightforward,with the
quantities n nPi[Ck]andqi [Ck]computedin parallel
at eachpixel. Timings aregiven in Table 1 for
various size images and various numbers of
labels.
4. RESULTS
Figure 1 shows the results of this algorithm
on a test image. In this example, we assume that
the "ideal" image (or "ground truth") is
composed of two distinct regions and we choose
some compatibility coefficients which reflect this
assumption; the initial probabilities present three
classes, two classes corresponding to the two
"ideal" regions and one "artefact" class. The edge
image presents strong edges at the border
between the two "ideal" regions and the edge
magnitudes decrease with the distance to this
border. For this example, we notice that if the
relaxation algorithm is utilized without any edge
information, one of the labels "takes over" the
whole image after only three iterations. If the
edge information is integrated in the relaxation
process, the two regions are still separated after
10 iterations and the labeling seems to be stable.
Figures 2 to 5 present results of the
algorithm applied on a Landsat-TM scene
("Washington D.C. region") shown at the top left
corner of Figure 2. Initial probabilities for this
scene have been obtained from a classification
into seven labels, utilizing a probabilistic neural
network (see (Chettri, 1993) for details). The 7
labels correspond to the classes "urban",
"agriculture", "rangeland", "forest",
"waterbodies", "wetland", and "bareland".
The algorithm described in section 2 was
applied to this initial classification, using two
different numbers of labels; first, we grouped
these 7 labels into the 3 classes "urban",
"agriculture", and "other". Figures 2 and 3 show
the results without and with the edge
information. When no edge information is
utilized (Figure 2), one of the labels (label 0) has
"disappeared" after 20 iterations. When the edge
information is taken into account (Figure 3), the
three labels are still present after 30 iterations,
and the labeling seems to stay stable after 20
iterations.
Then, the 7 labels defined previously are
considered, and we obtain similar results: see
Figures 4 and 5. When no edge information is
utilized, only three labels are Ieft after 30
iterations and one of the three is covering most of
the image; but when the edge information is
integrated in the updating formula, all the initial
labels are still represented after 30 iterations and
the labeling seems to stabilize after 20 iterations.
Also, the results obtained after 10 or 30 iterations
can be compared to the ground truth data shown
at the top left corner of Figure 5: qualitatively, we
can observe an overall improvement of the
segmentation (e.g., suppression of small isolated
pixels or groups of pixels), but some features
(e.g., a road) have been regrouped with a
neighboring region. A quantitative evaluation of
these results will be performed later.
The previous results show the importance
of the edge information and how the integration
of edge- and the pixel-based segmentations can
improve the final result.
Other similar results have also been
obtained on AVHRR data and will be presented
at the conference.
137
5. CONCLUSION
The results presented in this paper show
how the integration of complementary
information, such as pixel- and edge-based
techniques, can improve the final segmentation.
More work needs to be done, especially in
the definition of the initial probabilities, in the
choice of the compatibility coefficients, and in the
quantitative evaluation of the results. Also, the
results presented in section 4 seem to show that,
when the edge information is utilized, the
relaxation process becomes "stable" after a
certain number of iterations: this issue of
"convergence" will be studied, and different
schemes, such as the one presented in (Faugeras,
1981) will be investigated.
Acknowledgments
The author wishes to thank James C. Tilton and
Samir R. Chettri for the original data that were
used in this work, as well as Samir R. Chettri for
the neural network classification results that were
used to build the initial probabilities of Figures 2
to 5.
REFERENCES
Canny, J., (1986), "Computational Approach to Edge
Detection," IEEE Trans. Pattern Analysis and Machine
Intelligence, Vol. PAMI-8,679-698.
Chettri, S.R., (1993), "The Probabilistic Neural Network
Architecture for High Speed Classification of Remotely
Sensed Imagery," submitted to 1993 Goddard Conference
on Space Applications of Artificial Intelligence.
Davis, L.S., & Rosenfeld, A., (1981), "Cooperating
Processes for Low-Level Vision: A Survey," Artificial
Intelligence 17, 245-264.
Faugeras, O.D., & Berthod, M., (1981), "Improving
Consistency and Reducing Ambiguity in Stocchastic
Labeling: An Optimization Approach," IEEE Trans. on
Pattern Analysis and Machine Intelligence, Vol. PAMI-3,
No. 4, 412-424.
Fishier, M.A., & Firschein, O., (1987), "Parallel
Computer Architectures for Computer Vision," in
Readings in Computer Vision: Issues, Problems,
Principles and Paradigms, edited by M.A. Fishier and O.
Firschein, Morgan Kaufmann Publishers, Los Altos,
California, 768-772.
Hummel, R.A., & Zucker, S.W., (1987), "On the
Foundations of Relaxation Labeling Processes," in
Readings in Computer Vision: Issues, Problems,
Principles and Paradigms, edited by M.A. Fishier and O.
Firschein, Morgan Kaufmann Publishers, Los Altos,
California, 585-605.
Le Moigne, J., (1989), "Data Fusion by Relaxation," in
Proc. of Intelligent Autonomous Systems Intern.
Conference, Amsterdam, December 11-14, 368-377.
Le Moigne, J., & Tilton, J.C., (1992), "Refining Image
Segmentation by Integration of Edge and Region Data,"
IGARSS'92, Houston, Texas, May 26-29.
Peleg, S., (1980), "A New Probabilistic Relaxation
Scheme," IEEE Trans. on Pattern Analysis and Machine
Intelligence, Vol. PAMI-2, No. 4,362-369.
Zucker, S.W., & Hummel, R.A., (197"/), "Computing
the Shape of Dot Clusters, I: Labelling Edge, Interior, and
Noise Points," University of Maryland Computer Science
Center Technical Report No 543.
138
Image Size # Labels Edges ?
Time per
Iteration
256*256 7 no 0.36
256*256 7 yes 0.41
256*256 3 no 0.10
256*256 3 yes 0.11
512"512 2 no 0.17
512"512 2 yes 0.17
Table 1
Timings Obtained for One Relaxation Iteration on a MasPar MP-1
139
"Ideal" Edges I n _d---_Probabilities
I_ 1/no edges Ite_2/no edges I_ 3/no edges Du 1_ no edges
l_r I/edges l _er 31 edges I_ I_ edges
Results of our Method on a Test Example
140
Relaxation Results Without Edge Information
on a Landsat.TM scene (3 Labels)
141
Results of the "Relaxation With Edges" Method
on a Landsat.TM scene (3 Labels)
142
Relaxation Results Without Edge Information
on a Landsat-TM scene (7 Labels)
143
Results of the "Relaxation With Edges" Method
on a Landsat.TM scene (7 Labels)
144


Image analysis by integration of disparate information

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server