Contains fulltext :
                  241512.pdf (Publisher’s version ) (Open Access)Symposium on The Art and Science of Pattern Recognitio

Kisters, Peter M.F.

Schouten, Theo E.

van den Berg, Anja

van den Broek, Egon

van Rikxoort, Eva M.

Vuurpijl, Louis G.

English

Broek, E.L. Van Den

Rikxoort, E.M. van

Kisters, P. M. F.

Schouten, Th.E.

Radboud Repository

Human-Centered Computer Vision
Egon L. van den Broek
Department of AI, Vrije Universiteit Amsterdam
De Boelelaan 1081a, 1081 HV Amsterdam, NL
egon@few.vu.nl http://www.few.vu.nl/˜egon/
Eva M. van Rikxoort, Peter M. F. Kisters, Theo E. Schouten, and Louis G. Vuurpijl
Digital media are rapidly replacing their analog counter-
parts. This development is accompanied by (i) the in-
creasing amount of images present on the Internet, (ii)
the availability of the Internet for an increasing number
of people, (iii) a decline in digital storage costs, and (iv)
the developments in personal digital video/photo cam-
era’s [1, 2, 3]. In anticipation of these developments,
the fields of computer vision (CV) and content-based im-
age retrieval (CBIR) evolved rapidly. Driven by a tech-
nology push, a range of CV/CBIR techniques were de-
veloped [3, 4]. However, seldomly the user and his
characteristics were taken into account and subsequently,
limitations of mere technical solutions became appar-
ent [1, 5, 6].
The NWO ToKeN Eidetic project: Intelligent CBIR,
aims to bridge the semantic gap present in the field of
CV/CBIR, with the successful launch of the CBIR system
Vind(X) as its foundation. However, the Vind(X) systems
suffers from two drawbacks [7], it depends on: (i) the
cooperative annotation of its users to fill its database of
outlines [8] and on (ii) outline-outline (or shape) match-
ing [7].
To enable a full analysis of image content (e.g., through
object recognition), color and texture analysis has to be
done as well as segmentation and shape extraction, to fa-
cilitate shape matching. Since each of these topics is es-
sential for CV/CBIR, each of them was addressed in the
Eidetic research line and will be discussed, before com-
bining them.
Most images present on the Internet and in databases
are color images. Moreover, the analysis of color in the
image is not only used for the analysis of color distri-
butions but is also used in texture analysis, image seg-
mentation, and shape extraction. Hence, color analysis
is of the utmost importance, for bridging the semantic
gap [5], since color captures essential information about
our environment [1, 2, 9]. Therefore, we started with fun-
damental research toward human color processing [10].
This resulted in a unique color space segmentation, driven
by experimental data concerning the 11 color categories,
known to be used by humans since half a century [11, 12].
This color space segmentation can function as a highly ef-
ficient, human-based color quantization scheme [1, 11].
Texture is the second feature, widely used for image
analysis, CV, and CBIR purposes. Most texture analysis
techniques are intensity-based [13, 14]. However, multi-
ple problems can arise with texture analysis of color im-
ages, when their color is ignored (e.g., two distinct col-
ors can have the same intensity). Therefore, intensity-
based and colorful texture analysis were compared, using
several color spaces and quantization schemes [13, 14].
This resulted in the new, parallel-sequential texture anal-
ysis approach with a 96% correct classification perfor-
mance [14]. Moreover, in the research line “mimick-
ing human texture classification”, various texture analysis
techniques were compared with human texture classifica-
tion [9].
Using the 11 color categories and the texture analysis
scheme developed, coarse image segmentation was con-
ducted, the first phase of shape extraction [15]. Next,
the exact shapes were extracted from such coarse seg-
ments by pixelwise classification, followed by smoothing
operators [7]. The shapes extracted were analyzed us-
ing the Vind(X) engine. With that, for all three features,
human-based techniques have been developed to extract
them from not annotated image material. Moreover, the
segmentation and shape extraction enabled us to conduct
object-based image retrieval [7].
In order to test the feature extraction techniques, an on-
line CV/CBIR benchmark was developed [1, 2]. Retrieval
was conducted utilizing: (i) color, (ii) texture, (iii) shape,
(iv) color and texture combined, and (v) the three features
combined. Object-based image retrieval, exploiting color,
texture, and shape features, resulted in a retrieval preci-
sion up to 80% [7]. Hence, the first human-based, com-
putationally very efficient, OBIR engine was launched.
In parallel to the research discussed so far, a re-
search line in computational geometry emerged. Within
this line of research the Fast Exact Euclidean Distance
(FEED) transform was developed [16]. FEED was used
in [11] and exploited for the Weighted Distance Map-
ping paradigm [12]. Recently, a parallel implementation
of FEED: timed FEED was launched [17] and 3D-FEED
was developed [18].
References
[1] E. L. van den Broek, P. M. F. Kisters, and L. G. Vu-
urpijl, “The utilization of human color categoriza-
tion for content-based image retrieval,” in Proceed-
ings of Human Vision and Electronic Imaging IX,
B. E. Rogowitz and T. N. Pappas, Eds., San Jose,
CA, USA, 2004, vol. 5292, pp. 351–362.
[2] E. L. van den Broek, P. M. F. Kisters, L. G. Vuurpijl,
and J. C. M. von Schmid, “Content-based image
retrieval benchmarking: Utilizing color categories
and color distributions,” The Journal of Imaging
Science and Technology, [accepted].
[3] M. Israël, E. L. van den Broek, P. van der Put-
ten, and M. J. den Uyl, “Real time automatic
scene classification,” in Proceedings of the Six-
teenth Belgium-Netherlands Artificial Intelligence
Conference, R. Verbrugge, N. Taatgen, and L. R. B.
Schomaker, Eds., 2004, pp. 401–402.
[4] M. Israël, E. L. van den Broek, P. van der Putten,
and M. J. den Uyl, “Automating the construction of
scene classifiers for content-based video retrieval,”
in Proceedings of the Fifth International Work-
shop on Multimedia Data Mining (MDM/KDD’04),
L. Khan and V. A. Petrushin, Eds., Seattle, WA,
USA, 2004, pp. 38–47.
[5] E. L. van den Broek, L. G. Vuurpijl, P. M. F. Kisters,
and J. C. M. von Schmid, “Content-based image re-
trieval: Color-selection exploited,” in Proceedings
of the 3rd Dutch-Belgium Information Retrieval
Workshop, M.-F. Moens, R. de Busser, D. Hiemstra,
and W. Kraaij, Eds. December 2002, vol. 3, pp. 37–
46, University of Leuven, Belgium.
[6] E. L. van den Broek, P. M. F. Kisters, and L. G. Vu-
urpijl, “Design guidelines for a content-based im-
age retrieval color-selection interface,” in Proceed-
ings of the conference on Dutch directions in HCI,
B. Eggen, G. van der Veer, and R. Willems, Eds.,
Amsterdam, The Netherlands, 2004, ACM Interna-
tional Conference Proceeding Series, ACM Press -
New York, NY, USA.
[7] E. L. van den Broek, E. M. van Rikxoort, and Th. E.
Schouten, “Human-centered object-based image re-
trieval,” [submitted].
[8] L. Vuurpijl, L. Schomaker, and E. L. van den Broek,
“Vind(x): Using the user through cooperative anno-
tation,” in Proceedings of the Eighth International
Workshop on Frontiers in Handwriting Recognition,
S. N. Srihari and M. Cheriet, Eds., Ontario, Canada,
2002, pp. 221–226.
[9] E. M. van Rikxoort, E. L. van den Broek, and Th. E.
Schouten, “Mimicking human texture classifica-
tion,” in Proceedings of Human Vision and Elec-
tronic Imaging X, B. E. Rogowitz, T. N. Pappas,
and S. J. Daly, Eds., San Jose, CA, USA, 2005, vol.
5666, pp. 215–226.
[10] E. L. van den Broek, M. A. Hendriks, M. J. H. Puts,
and L. G. Vuurpijl, “Modeling human color catego-
rization: Color discrimination and color memory,”
in Proceedings of the 15th Belgium-Netherlands Ar-
tificial Intelligence Conference, T. Heskes, P. Lucas,
L. Vuurpijl, and W. Wiegerinck, Eds. 2003, pp. 59–
68, SNN, Radboud University Nijmegen.
[11] E. L. van den Broek, Th. E. Schouten, and P. M. F.
Kisters, “Efficient color space segmentation based
on human perception,” [submitted].
[12] E. L. van den Broek, Th. E. Schouten, P. M. F.
Kisters, and H. Kuppens, “Weighted Distance Map-
ping (WDM),” in Proceedings of the IEE Inter-
national Conference on Visual Information Engi-
neering: Convergence in Graphics and Vision, [in
press], Ed., 2005.
[13] E. L. van den Broek and E. M. van Rikxoort, “Eval-
uation of color representation for texture analysis,”
in Proceedings of the 16th Belgium-Netherlands
Artificial Intelligence Conference, R. Verbrugge,
N. Taatgen, and L. R. B. Schomaker, Eds., Gronin-
gen - The Netherlands, 2004, pp. 35–42.
[14] E. L. van den Broek and E. M. van Rikxoort, “Col-
orful texture analysis,” [in preparation].
[15] E. M. van Rikxoort, E. L. van den Broek, and Th. E.
Schouten, “Object-based image retrieval: Utilizing
color and texture,” [submitted].
[16] Th. E. Schouten and E. L. van den Broek, “Fast
Exact Euclidean Distance (FEED) Transformation,”
in Proceedings of the 17th International Confer-
ence on Pattern Recognition (ICPR 2004), J. Kittler,
M. Petrou, and M. Nixon, Eds., Cambridge, United
Kingdom, 2004, vol. 3, pp. 594–597, IEEE Com-
puter Society.
[17] Th. E. Schouten, H. C. Kuppens, and E. L. van
den Broek, “Timed Fast Exact Euclidean Distance
(tFEED) maps,” in Proceedings of Real Time Elec-
tronic Imaging X, N. Kehtarnavaz and P. A. La-
plante, Eds., San Jose, CA, USA, 2005, vol. 5671,
pp. 52–63.
[18] Th. E. Schouten, H. C. Kuppens, and E. L. van den
Broek, “Three dimensional fast exact euclidean dis-
tance (3D-FEED) maps,” in Proceedings of the 3rd
International Conference on Advances in Pattern
Recognition [in preparation], 2005.


Human-Centered Computer Vision

University of Twente Research Information

Human-Centered Computer VisionEgon L. van den BroekDepartment of AI, Vrije Universiteit AmsterdamDe Boelelaan 1081a, 1081 HV Amsterdam, NLegon@few.vu.nl http://www.few.vu.nl/˜egon/Eva M. van Rikxoort, Peter M. F. Kisters, Theo E. Schouten, and Louis G. VuurpijlDigital media are rapidly replacing their analog counter-parts. This development is accompanied by (i) the in-creasing amount of images present on the Internet, (ii)the availability of the Internet for an increasing numberof people, (iii) a decline in digital storage costs, and (iv)the developments in personal digital video/photo cam-era’s [1, 2, 3]. In anticipation of these developments,the fields of computer vision (CV) and content-based im-age retrieval (CBIR) evolved rapidly. Driven by a tech-nology push, a range of CV/CBIR techniques were de-veloped [3, 4]. However, seldomly the user and hischaracteristics were taken into account and subsequently,limitations of mere technical solutions became appar-ent [1, 5, 6].The NWO ToKeN Eidetic project: Intelligent CBIR,aims to bridge the semantic gap present in the field ofCV/CBIR, with the successful launch of the CBIR systemVind(X) as its foundation. However, the Vind(X) systemssuffers from two drawbacks [7], it depends on: (i) thecooperative annotation of its users to fill its database ofoutlines [8] and on (ii) outline-outline (or shape) match-ing [7].To enable a full analysis of image content (e.g., throughobject recognition), color and texture analysis has to bedone as well as segmentation and shape extraction, to fa-cilitate shape matching. Since each of these topics is es-sential for CV/CBIR, each of them was addressed in theEidetic research line and will be discussed, before com-bining them.Most images present on the Internet and in databasesare color images. Moreover, the analysis of color in theimage is not only used for the analysis of color distri-butions but is also used in texture analysis, image seg-mentation, and shape extraction. Hence, color analysisis of the utmost importance, for bridging the semanticgap [5], since color captures essential information aboutour environment [1, 2, 9]. Therefore, we started with fun-damental research toward human color processing [10].This resulted in a unique color space segmentation, drivenby experimental data concerning the 11 color categories,known to be used by humans since half a century [11, 12].This color space segmentation can function as a highly ef-ficient, human-based color quantization scheme [1, 11].Texture is the second feature, widely used for imageanalysis, CV, and CBIR purposes. Most texture analysistechniques are intensity-based [13, 14]. However, multi-ple problems can arise with texture analysis of color im-ages, when their color is ignored (e.g., two distinct col-ors can have the same intensity). Therefore, intensity-based and colorful texture analysis were compared, usingseveral color spaces and quantization schemes [13, 14].This resulted in the new, parallel-sequential texture anal-ysis approach with a 96% correct classification perfor-mance [14]. Moreover, in the research line “mimick-ing human texture classification”, various texture analysistechniques were compared with human texture classifica-tion [9].Using the 11 color categories and the texture analysisscheme developed, coarse image segmentation was con-ducted, the first phase of shape extraction [15]. Next,the exact shapes were extracted from such coarse seg-ments by pixelwise classification, followed by smoothingoperators [7]. The shapes extracted were analyzed us-ing the Vind(X) engine. With that, for all three features,human-based techniques have been developed to extractthem from not annotated image material. Moreover, thesegmentation and shape extraction enabled us to conductobject-based image retrieval [7].In order to test the feature extraction techniques, an on-line CV/CBIR benchmark was developed [1, 2]. Retrievalwas conducted utilizing: (i) color, (ii) texture, (iii) shape,(iv) color and texture combined, and (v) the three featurescombined. Object-based image retrieval, exploiting color,texture, and shape features, resulted in a retrieval preci-sion up to 80% [7]. Hence, the first human-based, com-putationally very efficient, OBIR engine was launched.In parallel to the research discussed so far, a re-search line in computational geometry emerged. Withinthis line of research the Fast Exact Euclidean Distance(FEED) transform was developed [16]. FEED was usedin [11] and exploited for the Weighted Distance Map-ping paradigm [12]. Recently, a parallel implementationof FEED: timed FEED was launched [17] and 3D-FEEDwas developed [18].References[1] E. L. van den Broek, P. M. F. Kisters, and L. G. Vu-urpijl, “The utilization of human color categoriza-tion for content-based image retrieval,” in Proceed-ings of Human Vision and Electronic Imaging IX,B. E. Rogowitz and T. N. Pappas, Eds., San Jose,CA, USA, 2004, vol. 5292, pp. 351–362.[2] E. L. van den Broek, P. M. F. Kisters, L. G. Vuurpijl,and J. C. M. von Schmid, “Content-based imageretrieval benchmarking: Utilizing color categoriesand color distributions,” The Journal of ImagingScience and Technology, [accepted].[3] M. Israël, E. L. van den Broek, P. van der Put-ten, and M. J. den Uyl, “Real time automaticscene classification,” in Proceedings of the Six-teenth Belgium-Netherlands Artificial IntelligenceConference, R. Verbrugge, N. Taatgen, and L. R. B.Schomaker, Eds., 2004, pp. 401–402.[4] M. Israël, E. L. van den Broek, P. van der Putten,and M. J. den Uyl, “Automating the construction ofscene classifiers for content-based video retrieval,”in Proceedings of the Fifth International Work-shop on Multimedia Data Mining (MDM/KDD’04),L. Khan and V. A. Petrushin, Eds., Seattle, WA,USA, 2004, pp. 38–47.[5] E. L. van den Broek, L. G. Vuurpijl, P. M. F. Kisters,and J. C. M. von Schmid, “Content-based image re-trieval: Color-selection exploited,” in Proceedingsof the 3rd Dutch-Belgium Information RetrievalWorkshop, M.-F. Moens, R. de Busser, D. Hiemstra,and W. Kraaij, Eds. December 2002, vol. 3, pp. 37–46, University of Leuven, Belgium.[6] E. L. van den Broek, P. M. F. Kisters, and L. G. Vu-urpijl, “Design guidelines for a content-based im-age retrieval color-selection interface,” in Proceed-ings of the conference on Dutch directions in HCI,B. Eggen, G. van der Veer, and R. Willems, Eds.,Amsterdam, The Netherlands, 2004, ACM Interna-tional Conference Proceeding Series, ACM Press -New York, NY, USA.[7] E. L. van den Broek, E. M. van Rikxoort, and Th. E.Schouten, “Human-centered object-based image re-trieval,” [submitted].[8] L. Vuurpijl, L. Schomaker, and E. L. van den Broek,“Vind(x): Using the user through cooperative anno-tation,” in Proceedings of the Eighth InternationalWorkshop on Frontiers in Handwriting Recognition,S. N. Srihari and M. Cheriet, Eds., Ontario, Canada,2002, pp. 221–226.[9] E. M. van Rikxoort, E. L. van den Broek, and Th. E.Schouten, “Mimicking human texture classifica-tion,” in Proceedings of Human Vision and Elec-tronic Imaging X, B. E. Rogowitz, T. N. Pappas,and S. J. Daly, Eds., San Jose, CA, USA, 2005, vol.5666, pp. 215–226.[10] E. L. van den Broek, M. A. Hendriks, M. J. H. Puts,and L. G. Vuurpijl, “Modeling human color catego-rization: Color discrimination and color memory,”in Proceedings of the 15th Belgium-Netherlands Ar-tificial Intelligence Conference, T. Heskes, P. Lucas,L. Vuurpijl, and W. Wiegerinck, Eds. 2003, pp. 59–68, SNN, Radboud University Nijmegen.[11] E. L. van den Broek, Th. E. Schouten, and P. M. F.Kisters, “Efficient color space segmentation basedon human perception,” [submitted].[12] E. L. van den Broek, Th. E. Schouten, P. M. F.Kisters, and H. Kuppens, “Weighted Distance Map-ping (WDM),” in Proceedings of the IEE Inter-national Conference on Visual Information Engi-neering: Convergence in Graphics and Vision, [inpress], Ed., 2005.[13] E. L. van den Broek and E. M. van Rikxoort, “Eval-uation of color representation for texture analysis,”in Proceedings of the 16th Belgium-NetherlandsArtificial Intelligence Conference, R. Verbrugge,N. Taatgen, and L. R. B. Schomaker, Eds., Gronin-gen - The Netherlands, 2004, pp. 35–42.[14] E. L. van den Broek and E. M. van Rikxoort, “Col-orful texture analysis,” [in preparation].[15] E. M. van Rikxoort, E. L. van den Broek, and Th. E.Schouten, “Object-based image retrieval: Utilizingcolor and texture,” [submitted].[16] Th. E. Schouten and E. L. van den Broek, “FastExact Euclidean Distance (FEED) Transformation,”in Proceedings of the 17th International Confer-ence on Pattern Recognition (ICPR 2004), J. Kittler,M. Petrou, and M. Nixon, Eds., Cambridge, UnitedKingdom, 2004, vol. 3, pp. 594–597, IEEE Com-puter Society.[17] Th. E. Schouten, H. C. Kuppens, and E. L. vanden Broek, “Timed Fast Exact Euclidean Distance(tFEED) maps,” in Proceedings of Real Time Elec-tronic Imaging X, N. Kehtarnavaz and P. A. La-plante, Eds., San Jose, CA, USA, 2005, vol. 5671,pp. 52–63.[18] Th. E. Schouten, H. C. Kuppens, and E. L. van denBroek, “Three dimensional fast exact euclidean dis-tance (3D-FEED) maps,” in Proceedings of the 3rdInternational Conference on Advances in PatternRecognition [in preparation], 2005.

NARCIS 

Automating the construction of scene classiﬁers for content-based video retrieval,”

Content-based image retrieval benchmarking: Utilizing color categories and color distributions,”

Content-based image retrieval: Color-selection exploited,”

Design guidelines for a content-based image retrieval color-selection interface,”

Efﬁcient color space segmentation based on human perception,” [submitted].

Evaluation of color representation for texture analysis,”

Fast Exact Euclidean Distance (FEED) Transformation,”

Isra¨ e l ,E .L .v a nd e nB r o e k ,P .v a nd e rP u t

Mimicking human texture classiﬁcation,”

Modeling human color categorization: Color discrimination and color memory,”

Object-based image retrieval: Utilizing color and texture,” [submitted].

The utilization of human color categorization for content-based image retrieval,”

Three dimensional fast exact euclidean distance (3D-FEED) maps,”

Timed Fast Exact Euclidean Distance (tFEED) maps,”

Vind(x): Using the user through cooperative annotation,”

Weighted Distance Mapping (WDM),”

Human-Centered Computer Vision

Abstract

Similar works

Full text

Available Versions

Radboud Repository

University of Twente Research Information

NARCIS

NARCIS