For the image processing of an object in its natural image, it is necessary to extract in advance the object to be processed from its image. To accomplish this the outer shape of an object is extracted through human instructions, which requires a great deal of time and patience. A method involving the setting of a model of color distribution on the surface of an object is described. This method automatically provides color recognition, a piece of knowledge that represents the properties of an object, from its natural image. A method for recognizing and extracting the object in the image according to the color recognized is also described

Mifune, Mitsuru

Mugitani, Takako

Nagata, Shigeki

English

NASA Technical Reports Server

Recognition of Object Domain by Color Distribution 
43053809f Tokyo DAINANAKAI CHTSHIKI KOGAKU SYMPOSIUM in Japanese 
22-23 Mar 88 pp 125-128 
[Article by Takako Mugitani, Information Processing and Manufacturing 
System Division, NEC Corp.; Mitsuru Mifune, Second System Engineering 
Division, Chugoku NEC Software Co. , Ltd. ; Shigeki Nagata, Second 
Application System Division, NEC Software Co. , Ltd. : "Setting of Surface 
Model of Reflected Objects; Verification of Surface Model of Reflected 
Objects; Recognition of Object Colors and Extraction of Object Domain"] 
[Text] 1. Introduction 
For the image processing of an object in its natural image, it is necessary 
to extract in advance the object to be processed from its image. To 
accomplish this the outer shape of an object is extracted through human 
instructions, which requires a great deal of time and patience. 
This paper describes a method involving the setting of a model of color 
distribution on the surface of an object, thereby automatically providing 
"color" recognition, a piece of knowledge to represent the properties of an 
object, from its natural image. In addition, it describes a method for 
recognizing and extracting the object in the image according to the color 
recognized. 
2.  Setting of Surface Hodel of Reflected Objects 
In computer graphics, various methods of expressing the surface of an 
object have been proposed in order to obtain realistic images. The authors 
studied a color distribution model with the light constantly contained in 
natural images taken into consideration by applying a model phong, which 
is often used for expressing bright surfaces, etc. 
A phong model is often used for plastic-like material expression. The 
reflected light S, at point P on the surface can be expressed by the 
expression (1): 
S p ( X )  = C,(X) (cos(i) (1 - d) + d) 
+ W(X,i) (cos(s))" 
49 
https://ntrs.nasa.gov/search.jsp?R=19890010557 2020-03-20T03:48:50+00:00Z
where 
C,: 
i: 
w: mirror reflectivity 
s:  
n: 
d: ambient light reflection coefficient 
diffuse reflectance at point P on an object 
angle of incidence of illumination light 
angle between the directions of mirror reflection and the line of 
sight 
power value for mirror reflection model by each material 
Since S, is a function of light wavelength A ,  the three stimulus values of 
R, G, and B can be obtained by integrating them with sensor sensitivity; 
Since the terms dependent on wavelengths are C ( A )  and W(A,i) alone, 
substituting expression (1) for expression ( 2 ) ,  we kave 
(p)=(g!) (cos(i) (1 - d) + d) 
where 
R,, = S C,(X)r(X)d X R,W(i) = (W(X,i)r(X)d X 
GO S C,(X)g(X)d G,W(i) fW(A,i)g(X)d 
Bo = J Cp(X)b(X)d X B,W(i) = fW(X,i)b(X)d X 
It is considered that W(i) is dependent on color term i of the light source 
and is separable. Setting a - cos(i) (1 - d) + d, B = W(i) (cos(s))”, we 
find that expression ( 3 )  results in 
where (R, , G o ,  Bo) t and (R, , G, , B, ) t can be regarded as colors appearing 
in diffuse reflection of an object (hereafter called object color) and 
those of illumination light (hereafter called light source color), 
respectively. It is considered that the color of reflected light at each 
point changes according to the surface direction and illumination at each 
point and the positional relation between itself and the observer, and that 
these factors are mixed at rates of a and 0. 
If the expression ( 4 )  model is correct, the color of the object surface 
must exist in a plane passing the black, the light source color and the 
50 
object color. The report by Tajima and his colleague' indicates that the 
color of an object is distributed on a plane and that the slippage between 
this plane and the one passing the black, the light source color and the 
object color is about 5 degrees, with these two planes almost according 
with each other. 
Figure 1 shows a projection drawing in which the body color of an 
automobile parked on a street is projected on this distribution plane with 
the black-white axis from the black ( O , O , O )  in RGB space to the white 
(255,255,255) displayed at the same time. From the picture, the light 
source color seems to be white, while the color of the object surface is 
distributed not only in the plane decided by the black, light source color 
and object coior, but also in a trtangle with the Slack, light source color 
(white) and object color as its vertexes. From this, expression (5) is 
considered to hold: 
We will call this color distribution model a reflected object surface 
model. 
Color dittribution 
Figure 1. Color Distribution in Automobile Body 
3 .  Verification of Reflected Object Surface Model 
We decided to verify the efficiency of the reflected object surface model 
of the expression ( 5 )  proposed in section 2 with several natural images. 
The processing for the distribution drawing in Figure 1, however, was 
conducted by deciding a distribution plane after the dispersion was 
obtained, so that it required about 40 minutes with a minicomputer with 
processing performance of 3 mips. To facilitate the verification, we 
considered a method for expressing color distribution without seeking 
51 
Figure 2. Color Distribution in Automobile Body 
dispersion and succeeded in the processing above with a 16-bit personal 
computer. Figure 2(a) and (b) show the result of the above processing for 
the same picture of the automobile. 
The axis of abscissas in Figure 2(a) is the black-white axis where factors 
of R, G, and B become equivalent, while its axis of ordinates represents 
the distance from the black-white axis. Figure 2(b) shows individual 
points projected on plane vertical to the black-white axis. If examination 
of a natural image of an object other than an automobile shows its color to 
be distributed as in this distribution drawing, it will demonstrate that 
the reflected object surface model is correct. 
Figure 3 .  Color Distribution in Cleaner 
Figure 3(a) and (b) show a color distribution of a natural image of a 
cleaner with a plastic-like surface for a phong model. Here, too, the 
light source color can be assumed to be white, and it can be found that as 
is the case with an automobile, the color is distributed in a triangle with 
the black, light source color and object color as its vertexes. In 
addition, the authors examined such objects of materials with great mirror 
reflection as a TV set, a telephone and a desk, and obtained a similar 
result, thus confirming the reflected object surface model as correct. 
G 
Figure 4. Color Distribution in Suit 
52 
With respect to the surface of a suit material lacking mirror reflection, a 
similar result to that shown in Figure 4(a) and (b) was obtained, 
indicating that its color is distributed in the same way. With respect to 
interior light equipment, which has great mirror reflection and is made of 
material permitting light to permeate, the result shown in Figure 5(a) and 
(b) was obtained, and it has been confirmed that its color is distributed 
as well. 
Figure 5. Color Distribution in Interior Lighting Equipment 
From the above results, it has been confirmed that the color of the surface 
of an object, irrespective of the materials used in the object surface, is 
distributed in a triangle, with the black, light source color and object 
color as its vertexes. Therefore, the reflected object surface model the 
authors have proposed is applicable, irrespective of the materials used in 
the object surface. 
4. Recognition of Object Color and Extraction of Object Domain 
Since the light source color can very often be assumed as white, as is the 
case with sunlight the light source color is regarded as white in the 
following description. In a reflected object surface model, the color of 
the object surface exists on a plane with a black-white axis as its bottom 
end and the object color itself is the one farthest from the black-white 
axis. Therefore, if any color on an object is specified and its 
distribution plane is decided, the color farthest from the black-white axis 
on this plane can be recognized as the object color. The authors 
considered that by doing so, what entered the scope of a triangle with the 
white, black, and object color as its vertexes could be automatically 
extracted as a portion of the object. 
However, as verified in section 3 ,  the object color in a natural image is 
distributed not in a perfect plane, but in a certain width. The lightness 
of color changes due to the effect caused by light sources such as shadows, 
but the hue does not, so it can be considered that the width of color 
distribution extends in the direction vertical to the hue surface. When 
viewed in the color distribution in Figure 2, since the plane with the 
black-white axis as its end can be approximately regarded as the hue 
surface in RGB space, the water droplet type shown in Figure 2(b) can be 
regarded as the width of color distribution. For practical performance, 
the authors conducted an experiment having a water droplet type approximate 
to a fan type, confirming that use of a fan-type model with a given 
interior angle permits the object color to be recognized without posing any 
problem for its practical use. 
53 
Table 1. Width of Color Distribution 
Object Interior angle 
Automobile 
Cleaner 
suit 
Interior lighting equipment 
58 
28 
42 
37 
The object color could be recognized by using a fan-type model with a given 
interior angle; however, with respect to the automatic extraction of a 
portion of an object, since the width of the interior angle of one image is 
largely different from that of another, the unrecognizable portion of the 
object increases when the interior angle is too narrow, while something 
other than the object is also recognized by mistake when the width of the 
interior angle is too wide. For the automatic extraction of object domain, 
it is necessary to strictly set the interior angle. The authors succeeded 
in extracting object domain by providing the dispersion of hue in terms of 
parameter through visual observation. 
5. Conclusion 
The authors have so far conducted an experiment on the automatic 
recognition of object domain from its natural image, but they could not 
automatically recognize the width of color distribution. From the analysis 
result of natural images obtained so far, the width of color distribution 
is considered to be caused by materials of the object surface; therefore, 
if knowledge of materials with the width of color distribution taken into 
consideration can be obtained, more perfect automatic extraction of object 
domain together with color will be possible. 
References 
1. Yoji Tajima and Takako Mugitani, "Color Changing Algorithm for Color 
Image Portion," Information Processing Society's 35th National 
Convention. 
2. B.T. Phong, "Illumination for Computer Generated Pictures," COMM. ACM, 
Vol 18, No 6, 1975. 
20117/9365 
END 
54 


Recognition of object domain by color distribution

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server