Detecting edges of objects in their images is a basic problem in computational vision. The scale-space technique introduced by Witkin [11] provides means of using local and global reasoning in locating edges. This approach has a major drawback: it is difficult to obtain accurately

the locations of the 'semantically meaningful' edges. We have refined the definition of scale-space, and introduced a class of algorithms for implementing it based on using anisotropic diffusion [9]. The algorithms involves simple, local operations replicated over the image making parallel

hardware implementation feasible. In this paper we present the

major ideas behind the use of scale space, and anisotropic diffusion for edge detection, we show that anisotropic diffusion can enhance edges, we suggest a network implementation of anisotropic diffusion, and provide

design criteria for obtaining networks performing scale space, and edge detection. The results of a software implementation are shown

Malik, Jitendra

Perona, Pietro

English

Caltech Authors - Main

A NETWORK FOR MULTISCALE IMAGE SEGMENTATION 
Pietro Perona Jitendra Malik 
Department. of Ekd.rical Engineering am! Computer Science 
University of California at I3crkdcy- CA 94720- U.S.A. 
e-mail to: perona @united. I3crkdey.ed n.arpa 
ABSTRACT 
Deterling edges of ol>jecfs in their images is a ba.<ic ]>roblc»> in com-
puta.lional vision. The scale-space technique inii'Oduced by Witkin [I 1} 
provides means of using local and global r·m.<cmi»g i» lora.ling edges. 
This approach has a major drawback: il is diflicu/1 to obtain accumtely 
the /ocati011S of /he 'semantically meaningful' edges. We have refined 
the deflnilion of scale-space,'""' inlroduced a class of algorithms for im-
plementing it based ou using anisotropic dij]u.<iou {.9}. The algorithms 
iul'Oh•es .<implc, local operations replicated o1•er /he image makiug par-
allel ha.rdwar·e implementation feasible. In /his J>aJ>er· we present /he 
major ideas behind the use of scale space, and anisolmpic diffusion for· 
edge deleclion, we show thai anisotropic dilfusion can enhance edges, 
we suggest a networ·k implementation of anisofr·o]>ic dilfusion, and pro-
vide design criteria for obtaining networks performing scale space, and 
edge detection. The resulls of a software implcmeutatiou are shown. 
1 INTRODUCTION 
Detecting edges of objects in their i1w1ges is a hm;ir problem in compu-
tational vision and much effort. has been devoted to it. so far. Surfaces 
in nature usually [7,G] have a hierarchical organization composed of a 
small discrete number of levels. At t.he finest level, a tree is composed 
of leaves with an intricate structure of veins. At the next level, each leaf 
is replaced by a single region, and at the highest. level there is a single 
blob corresponding to the tret'(.op. There is a n:1tnral range of resolu-
tions (intervals of the scale-space parameter) corresponding t.o each of 
these levels of description. Furthermore at earl1 level of description, the 
regions (leaves, treetops or forests) have well-defined boundaries. 
Noise, finer detail, and the structure of the edges the1nselves, makes 
local criteria (i.e. decision rules, or operators, based 011 small fixed-
size 11eighbourhoods ) insufficient for det.enniuing the exist.e11ce a11d 
location of the edges at different scales of resolnt.ion. On the other hand 
global criteria prove to be hard to formulate, difficult to implement, and 
cmnputat.ionolly expensive. The scale-space technique provid('s nwa.us 
for taki11g advantage of bot.h local and global reasoning on images. 
The impot·tallce of multi-scale descriptions of inwges has bee11 rec-
ognized from the early days of comput.er vision e.g. Rosenfeld and 
Thurston [10]. A clean formalism for tl1is problem is the idea of scale-
space filtering i11troduced by Witkin [11] and furt.her developed in 1\oen-
derillk [6], Uabaud, Duda a11d Witkin [1], Yuille and Poggio [12], a11d 
Hummel [4]. 
The essential idea of t.his approach is quite simple: embed tl1e origi11al 
image i11 a family of derived images J(.r, y, I) obt.aiued by coBvolviBg the 
origi11al image with lowpass kernels of 11perture t· 
Larger values oft, the scale-space parameter, correspo11d to images 
at coarser resolutions. See fig. 1. 
1\oeBderiBk motivates the choice for a particular lowpass kernel by 
stating two criteria : 
1. Causality : A11y feature at. a coarse level of resolntio11 is required 
t.o possess a (not necessarily unique) ~'cause~~ at. a fillf>r level of res-
olution although the reverse 11eed 11ot be true. In other words, 110 
ISCAS'88 
2565 
Figure 1: A family of I 0 images l(x,t.) obtai11ed by convolvi11g the 
original one (top) with gaussian kemds whose variance increases from 
top to bottom. (Adapted from Witkin). 
spurious detail should be generated wht•n the resolution is dimin-
ished. 
2. Homogeneity and lsolro]>y : The blurring is required to be space 
irtvarinttt. 
It turns out that only I he gaussian kernel generates a scale space 
satisfying these requirements. As pointed out by l\oenderi11k [6], and 
llummel [4] t.he one parameter family of derived in1ages may equiv-
alent-ly be viewed a.s the solution of the heat conduction or diffusion 
equation 
I,= c(/xx + lyy) 
with the initial condition l(.>',y,O) = J0 (J·,y), the original image. 
Unfortunately, in the scale-space thus derived the true location of 
the boundaries at coarse scales is not directly available in the coarse 
srale images. The edges at the coarse level 11 are shifted from their 
true locations due to the intrraction of the different brightness "blobs" 
during diffusion. The only way to obtain the true position of the edges 
that have been detected at a coarse srale is by tracking across the scale 
spare, to their locations in the original image. 
The renson for this spill-ial distortion is quite obvious-Gaussian blur-
riug docs not 'respect' the natural boundaries of oliject.s. Suppose we 
have the picture of a treetop with the sky as background. The gaussian 
blurring process would result in the green of the leaves getting 'mixed' 
with the blue of I he sky, long before the treetop emerges as a feature 
(aft.er the leaves have been blurred toget.lwl"). 
Figure 1 shows a sequence of coa.rser images obtained by gaussian 
blurring which illustrates this phenomenon. It may also be noted that 
the regim1 boundaries are generally quite diffuse instead of being sharp. 
v\'ith this as mot-ivation, we have enunciated the criteria which we 
believe any candidate paradigm for generating multi-scale 'semantically 
meaningful' descriptions of images must satisfy. 
CH2458-8/88/0000-2565$1.00 © 1988 IEEE 
1. Causality : As pointed out. by Witkin and Koenderink, a scale-
space representat.ion should have the property t.hat. no spurious 
detail should be generated. 
2. Immediate Localization: At. ellch resolution, the region bound-
llries should be sharp and coincide with the semantically meaning-
ful boundaries at that. resolut.ion. 
3. Piecewise Smoothing: At. all scales, int.ra~region smoothing 
should occur prefrrent.ially over int.er~region smoot.hing. In the tree 
example mentioned earlier, t.he lellf regions should be collapsed t.o 
a treet.op before being merged with the sky background. 
In (.he next section we will outline a framework for achieving these 
objectives. 
2 ANISOTROPIC DIFFUSION 
Consider the anisotropic diffusion equa.t.ion 
I,= rliv(c(J:,y,t)'\71) = c(a·,y,t)6I + VC· vi 
where we indicate with div the divergence opera.t.or, and with '\7, and 
6 the gra!lient., and laplacian operators with respect to the space vari-
ables. It reduces t.o the isotropic heat. diffusion equat.ion 11 = c6I if 
c(.r:,y,t) is a constant. Suppose at t.he time (scale) I, we knew t.lte lo-
cations of the region boundaries appropriate for that. scale. We would 
want. t.o encourage smoothing within a region in preference to smoothing 
acmss the boundaries. This could be achieved by setting the conduct. ion 
coefficient. to be 1 in the interior of each region and 0 at (.he bound-
aries. The blurring would then take place separately in each region with 
no interaction between regions. The region boundaries would remain 
sharp. 
or course, we do not know in advance the region boundaries a.t each 
scale (If we did the problem would already have been solved!). What can 
be computed is a. current best estimate of the loca.t.ion of the boundaries 
(edges) appropriate t.o t.ha.t scale. Let. E(x, y, t) be a. vect.or valued 
funct.ion defined on the image with the following ideal properties: 
1. E( a:, y, t) = 0 in the interior of each region. 
2. E(.l!,y,l) = 1\e(x,y,t) a.t each edge point, where e is a. unit. vec-
tor tangent. to the edge a.t the point, and 1\ is the local contrast 
(difference in the image intensities on the left. and right ) of the 
edge. 
Note that (.he word edge as used above has not been formally defined~ 
we mean here the perceptual subject.ive notion of a.n edge as a region 
boundary. A completely satisfactory formal definition is likely to be 
part of the solution, rather than the problem definition! 
Once a.n estimate of E(x,y,t) is available, c(x,y,t) is chosen to be a 
funct.ion c = g(\IE\1) of the magnitude of E. According to the previously 
stated st.rat.egy g(·) has to be a. nonnegative monotonically decreasing 
funct.ion wit.h g(O) = 1. This way the diffusion process will mainly 
take place in t.he interior of regions, and it will not affect. the region 
boundaries where the magnitude of E is large. There are many possible 
choices for g(·) the simplest being a binary valued funct.ion. We will 
also briefly discuss the fact that the funct.ion g( ·) need not be time 
invariant, and actually sometimes it should vary with time. 
The way the conduction coefficient is chosen guarantees that. the diffu-
sion process sa.t.isfies the third requirement. for generating a. scale-space, 
namely piecewise smoothing. The analysis peformed in sed.ion 3, and 
the experiment.a.tion ( see fig. 4 and fig. 5), will show that t.he second 
criterion-immediate localization~is also satisfied. It is important to 
know which possible choices of t.he conduction coefficient c(·) , if any, 
respect the first criterion, causality. 
For studying causality we make use of a. classical result of the theory 
of partial differential equations : the maximum principle. This the-
orem guarantees t.ha.t the maxima and minima of the solution of the 
anisotropic diffusion equa.t.ion belong to t.he initial condition ( the origi-
nal image in our case ), and to the boundaries of the domain of interest 
(the boundaries of the pi ct. nrc), provided that. the conduction coefficient 
c(x, y, t) is differentiable and non-nega.t.ive. 
This theorem is relevant. to our discussion because if we identify "fea-
t.ltres~ in the ima.gt'S with "blobs~ of the brightness function I(x,y, I) 
for different values of the scale parameter, t.ltt'n the maximum pt·inciple 
guarantees that. no new features are creat.ed for incrt'asing t. In fact 
the birth of a new "blob., would imply the creation of a. ma.xin~um, or 
a. minimum, (.hat would have to belong to the interior of the domain of 
interest. in the scale space. 
A more precise st.at.rment. of this theorem, and a proof t.ha.t. 
anisotropic diffusion satisfies it. may be found in [9], or ma.y be derived 
using techniques in [5]. 
3 EDGE ENHANCEMENT 
A big problt'lll with ronvent.iona.llow-pass filt.t>ring and diffusion, is that 
the price pa.yed for eliminating t.he noise, and for performing scale space, 
is the blurring of edges. This causes their detection, and localization to 
be difficult .. 
Hummel [3] has suggested enhancing edges by running backwards the 
diffusion equa.t.ion. This is a.n ill-posed problem, and gives rise to nu-
merically unstable computational metods, unless proper regularization 
is employed. 
We will show here briefly that a. proper choice of the conduct.ion 
coefficient makes the anisotropic diffusion enhance edges while running 
forward in time, and still being numerically stable. 
Let our model of an edge be a. mollified version of a step , for ex-
ample a. step function convolved with a. gaussian kemel. A change of 
coordinates will align t.he step wit.lt the y axis, thus reducing t.lte 2-D 
prohlem t.o 1-D. Thus / 1 = div(c(x,y,t)'\71) = f,:(c· I:c)· 
Take c to be locally determined, just a. funct.ion of the gradient of I : 
c(x,y,t) = g(I .... (:t:,y,t)). Call <p(·) the product c·lx: cp(Ix) = g(I .... )·lx. 
We are int.erest.ed in looking at. t.he slope of the edge while this is 
being diffused. Consider t.hen : f,(I .... ). Since the functions that we are 
dealing wit.h are smooth the order of differencia.tion may be inverted 
obtaining: 
a a a a 
Dt(Ix)= Dx(I,)= Dx(Dx(c·Ix))= 
a (a ( , 2 , 
=Ox ox'Plx))='P ·lxx +'P ·lux 
Suppose the edge is oriented in such a. wa.y t.ha.t. I, > 0 ( if not, reverse 
the orientation of the x axis ). At the point of inflect.ion lxx = 0, and 
I,xx < 0 since the point. of inflection conesponds, or is very close to, 
t.he point with maximum slope. Then in a. neigh hour hood of the point 
of iuflect.ion f,U:c) has sign opposit.e t.o <p'(I.r ). If <p'(I.r) > 0 the slope 
of the edge will decrea.~e; if, on the contrary cp'(Ix) < 0 t.lte slope will 
increase. 
Notice that. this increase in slope cannot. be caused by a scaling of 
the edge, beca.nse t.his would violate the maximum principle. The edge 
becomes sharper. 
Observe t.hat. such a funct.ion <p( ·)is easy to obtain: take, for example, 
c(x,y,t) = g(Jx) = l+(~)'h with a> 0. Then there exists a certain 
threshold value related to l\ and o, below which cp(·) is monot.onically 
increasing, and beyond which <p(·) is monotonically decreasing, giving 
the desirable result of blurring small discontinuities, and sharpening 
edges. 
Therefore with passive diffusion, and local reMoning we may obtain 
edge enhancement if we choose t.lte conduct.ivity funct.ion g(·) appro-
priatt'ly. Increasing progressively the value of the constant. J( we may 
obtain scale- space, the more a.nd more edges disappearing from the 
picture. 
2566 
4 NETWORK IMPLEMENTATION 
In section 2 we have introduced a class of diffusion e<Juat.ions t.lta.t gener-
at.e families of images satisfying the 3 scale-space criteria. that we have 
enunciated in section 1. Section 2 also hints on ways to choose g(·), 
and E and bounds are put on the possible choices of c. In this section 
Figure 2: The st.mct.ure of a network implementing anisotropic diffu-
sion. 
we will indicate t.he st.ruct.ure of a simple network that. implements !.he 
anisotropic diffusion equation, and provide design crii.Nia for obtaining 
scale space, and edge detection. 
ConsidN the following simple discret.izat.ion of the second term of t.he 
anisotropic diffusion equation : 
1 tl.x tl.x 
lx(x, y);:;; tl.x (I(x + 2 , y)- l(x- 2' y)) 
1 tl.y tl.y 
Iy(>:,y);:;; -(I(x,y+ -'-)- I(x,y- -)) 
tl.y 2 2 
div(c(x,y,t)"VI);:;; 
1 tl.x 
;:;; tl.x 2 (c(x + 2 , y) · (I(x + tl.x, y)- I(x, y))-
tl.x 
c(x- 2' y) · (I(x, y))- l(x- tl.x, y)))+ 
1 tl.y " 
+ tl.y2 (c(J:, y + 2 ) · (l(x, y + tl.y)- l(J:, y))-
tl.y 
c(x, y- 2 ) · (I(x, y))- I(x, y- tl.y))) 
Introducing t.he notation: 
cN(x,y) = c(x, y + ~) · b 
cE(J:, y) = c(x + ¥· y) · .o,~> 
c5 (:c,y) = c(J:,y- ~) · b 
cw(x,y) = c(x- ¥,y) · .o,~> 
'\lNl(x,y):: l(x,y+ tl.y)- l(x,y) 
'\7 sl(x, y) = I(x, y- tl.y)- I(x, y) 
etc. we obtain : 
This corresponds to diffusion of a quantity I associated to the nodes 
of a lattice with arcs of conductivity c . An elect.rical network with the 
topology of a square lattice, charges wit.h initial value equal to the image 
bright.ness I associated to the nodes, and resistors with conductance c 
associated to the arcs does the job. 
'Ne may perform the same analysis as in the previous sect.ion using 
the discretized equations obtaining the same qualitative result: a lattice 
I =+(AV) 
AV 
Figure 3: The qualit.a.t.ive behavior of a nonlint>ar resistor needed to 
rt>alize scale space ( solid line ) compared with the linear resistor ( 
dotted line ). 
network whost> nodes are only neart>s!.-neighbour connect.ed, and whose 
arc elements have only th(' couple of assoriat.ed nodes as imputs can 
perform scale-spare and edge det.cct.ion, including th(' edge ('nha.ncement 
effect., if the arc dements are rhosen to have a nonlinear conduct.ance 
as indicated in section 4. See also fig. 3. 
5 EXPERlMENTAL RESULTS 
We have simulated in soft. ware I. he anisotropic difl'usion scale space and 
edge detection network that we propose. 
The computational setup had 11 lat.t.ice st.ruct.ure, with brightness val-
ues associated t.o the vert.ices, aud conduction coelficit>nts to t.he arrs. 
A constant value of 1 for the conduct.ivit.y led t.o gaussian blurring. 
In a first set of experiments we used t.he absolute value of t.he gradit>nt 
as the est.imate for II Ell . Different. functions were used for g( ·) giving 
perceptually similar result.s. The image in fig. 4 [2) was obtained using 
(vi)- c g -1+(-jf-)2 
Notkt> that. the smoot.hed image has a l\1ondrian-like appearanee. 
This is the effect of the piecewise smoothing, and mak('s edge detection 
extremely simple: it is sulficient. to t.hr('shold t.lte magnitude of t.he 
gradient of the brightness. Not.ic~ the bt'haviour of t.lte edge detect.or 
on t.he water, t.he pavement, and the archit.ect.ural details. 
There is an addit.ional remark to be made: if the function g( ·) ap-
proaches Z('I"O very quickly for incrt>asing contrast. values the edges may 
be too impermeable i.e. t.he image is transformed to a piecewise-
constant Mondrian, and the diffusion is very slow in getting rid of the 
fine detail. This may he correct.ed by let.t.ing the value of g( ·) increase in 
time so that t.he weaker edges become unst.ablt' and are diffused. If this 
is done slowly compared wit.h the spet'd of t.he diffusion process, t.he cri-
t.eria that we have enunciated, namely 'localization of boundaries', and 
'piecewise smoothing' are respected, while the comput.ation becomes 
more efficient. There are other reasons for allowing t.he function g(·) 
vary with time. One of these is the presence of high levels of noise in 
the image. In this case the cotHluct.ion coelficient has t.o start off wit.h a 
high value in order to let t.he diffusion average .the noise, and decrt'ase 
in the course of the diffusion process. We discuss this in detail in [8). 
References 
2567 
[1) J. llabaud, A. Witkin, Daudin, and R.Duda. Uniqueness of the 
gaussian kernel for scale-space filtering. IEEE Tmns. on PAM!, 8, 
January 1986. 
[2) Canaletto. View in Venice. Circa 1740. 
Figure 4: The effect of anisotropic diffusion on an image. From top 
to bottom: a) The original image . b) The same image at a coarser 
resolution. 
Figure 5: The brightness patt.ern along a horizontal line crossing the 
lower part. of the image in fig.4 . The top plot. represents the bright.ness 
of the original image, while the bottom one the brightness of the coarse 
resolution image 4.b. The middle one corresponds to an image at an 
int.enHediate resolution. Observe that the position of the edges does 
not vary across the scale space. 
[3] H.A. Hummel. Deblurring Gau.ssian Blur. Technkal Ileport., New 
York University, Courant Institute of Mathematical Sciences. 
[4] R.A. II ummel. Representations based on Zer·o-crossings in Scale-
Space. Technical Report 225, New York University, Courant Insti-
tute of Mathematical Sciences, 1986. 
[5] Fritz John. Partial Differential Equations. Springer Verlag, New 
York, 1982. 
[6] .J .J. Koenderink. The structure of images. Biological Cybemetics, 
50:363-370, 1984. 
[7] David 1\Iarr. Vision. W.II.Fre~man & Co., 1982. 
[8] Pietro Perona and Jit.endm Malik. 1t>chnical Ileport, Deptt .. of 
EECS, U.C.llerkeley. In preparat.ion. 
[9] Pietro Perona and .Jitendt·a Malik. Scale space,and edge detection 
using anisotropic diffusion. In IEEE Computer Society Workshop 
on Computer Vision - Jlliami, 1987. 
[10] A. Rosenfeld and M. Thurston. Edge and curve rletect.ion for visual 
scene analysis. IEEE Tr·ans. 011 Comtruters, C-20:562-569, May 
1971. 
[11] A.P. Witkin. Scale-spare filtering. In lnternatio11al Joint Co11fer-
ence on Artificial Intelligence, pages 1019-1021, 1983. Karlsruhe. 
[12] A. Yuille and T. Poggio. Scaling t.heorems for zero crossings. IEEE 
Transactions on Patter-.. Analysis, and Machine Intelligence, 8, 
January 1986. 
Acknowledgements 
This work has been supported by grant 82-11-008 of the Semiconductor 
Research Corporation , and an IBM faculty developement award. 
2568 


A network for multiscale image segmentation

Daudin, and R.Duda. Uniqueness of the gaussian kernel for scale-space filtering.

View in Venice. Circa 1740.

View in Venice. Circa 1740. Figure 4: The effect of anisotropic diffusion on an image. From top to bottom: a) The original image . b) The same image at a coarser resolution.

http://authors.library.caltech.edu/32000/1/PERiscas88.pdf

A network for multiscale image segmentation

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main