Spectral image sensors provide images with a large number of contiguous spectral channels per pixel. Visualization of these huge data sets is not a straightforward issue. There are three principal ways in which spectral data can be presented; as spectra, as image and in feature space. This paper describes several visualization methods and their suitability in the different steps in the research cycle. Combinations of the three presentation methods and dynamic interaction between them, adds significant to the usability. Examples of some software implementations are given. Also the application of volume visualization methods to display spectral images is shown to be valuabl

Heijden, G.W.A.M., van der

Polder, G.

English

Name not available

Visualization of spectral images

van der Heijden, G.W.A.M.

NARCIS 

Wageningen Yield 

Visualization of Spectral Images
Gerrit Poldera,b and Gerie W.A.M. van der Heijdena
aPlant Research International, PO-Box 16, 6700 AA, Wageningen, The Netherlands
bDelft University of Technology, Faculty of Applied Physics, Pattern Recognition Group,
Lorenzweg 1, 2628 CJ, Delft, The Netherlands
ABSTRACT
Spectral image sensors provide images with a large number of contiguous spectral channels per pixel. Visualization
of these huge data sets is not a straightforward issue. There are three principal ways in which spectral data can be
presented; as spectra, as image and in feature space. This paper describes several visualization methods and their
suitability in the different steps in the research cycle. Combinations of the three presentation methods and dynamic
interaction between them, adds significant to the usability. Examples of some software implementations are given.
Also the application of volume visualization methods to display spectral images is shown to be valuable.
Keywords: Spectral images, feature space, volume visualization
1. INTRODUCTION
Recent developments in optics and sensors allow imaging spectrometry: the creation of spectral images, i.e. images
with wavelength-specific measurements over a large part of the spectrum.1,2 In relation to gray-value images, these
images are an order of magnitude larger. Besides a 2D spatial dimension of e.g. 256x256 pixels, a spectral image
also has a third spectral dimension of e.g. 256 wavelengths. In literature the terms multispectral and hyperspectral
imaging are often used. Multispectral is mostly used for images with 10-100 wavelength bands, where hyperspectral
is used for images with more than 100 bands. In this paper we do not make this distinction, we use the term spectral
image independent of the number of wavelength bands.
Spectral imaging is commonly used in the area of remote sensing, where images of the earth surface are taken
from satellite, or from an airplane.3,4 Recently cheap and easy to use sensors came available, and now this technique
is more and more used on macroscopic and microscopic scale. This paper focuses on the latter two. Spectral
images can be described as a three-dimensional data cube. Visualization of these data is not a straightforward issue.
More complex data require more complex and sophisticated data analysis procedures if their full potential is to be
achieved. The matter of how the variations are represented mathematically and conceptually is an important first
step in defining how the analysis process should proceed. In fact each step in the research cycle, comprising data
gathering, data exploration, data analysis and presentation of results, has specific requirements for the visualization
of spectral image data. In table 1 the research cycle steps related to their visualization requirements are tabulated.
Landgrebe5 distinguished three principal ways in which spectral images can be represented quantitatively:
• As spectra, which shows variations of a pixel or region as a function of wavelength
• In image form, where pixels are displayed in geometric relationship to one another
• In feature space, where pixels are displayed as points in an N-dimensional space
In this paper we will present several visualization methods for these data representations and give examples of their
use, related to data analysis. Also visualization methods as a combination of different data representation modes are
discussed, including volume rendering and ray tracing techniques. The paper is organized as follows. First spectral,
image and feature based representation of spectral images are described. Then some examples of visualization
methods and software are given. We end the paper with discussion and some concluding remarks.
Further author information: (Send correspondence to G. Polder)
G. Polder: E-mail: G.Polder@plant.wag-ur.nl
G.W.A.M. van der Heijden: E-mail: G.W.A.M.vanderHeijden@plant.wag-ur.nl
132
Visualization and Optimization Techniques, Yair Censor, Mingyue Ding, Editors,
Proceedings of SPIE Vol. 4553 (2001) c©2001 SPIE . 0277-786X/01/$15.00
Table 1. Visualization requirements of spectral image data related to the steps in the research cycle.
Step Purpose Visualization requirements
1 Data gathering First data screening Speed and easy of use
2 Data exploration Generation of hypotheses Versatile and flexible, interaction needed
3 Data analysis Pattern recognition techniques Versatile and flexible, no interaction
4 Presentation Presentation and publication of the results Eye-catching, informative picture
2. SPECTRAL BASED REPRESENTATION
In the research cycle spectral based representation is most often used at the data gathering and the data exploration
step. From this finding it is obvious that many algorithms for the analysis of spectral images, which appear in
literature, begin with a representation of a response function as a function of wavelength. Identifiable features within
a spectral response function, such as absorption bands due to resonances at the molecular level, can be used to identify
compounds associated with a given spectrum. This approach, arising from the concepts of chemical spectroscopy,
which has long been used in the laboratory for molecular identification, is perhaps one of the most fundamentally
cause/effect based approaches to the analysis of spectral images. Traditionally spectroscopy analysis techniques
are used to analyze each pixel in the image, without using the spatial relationship. From groups of pixels, which
are related in spatial or spectral sense, the mean spectra with its confidence limits can be calculated. The spectra
are visualized using a two-dimensional graph of the wavelength verses the absorption or reflection value, sometimes
extended with an indication of dispersion using error bars. Pure spectral representation is not used in the last step
in the research cycle, but combinations of spectral representation with image or feature based representation can be
useful for presentation.
3. IMAGE BASED REPRESENTATION
Image based representation is important in all steps in the research cycle. Therefore the image form is perhaps
the first form one thinks of when considering spectral imaging as a source of information. Indeed, the spatial
relationship between the spectra is crucial for most applications. However, for classification purposes, it is not so
straightforward to maintain spatial relationship when performing spectral analysis, e.g. before decomposing spectra
in principal components, the image is unfolded resulting in a one-dimensional array of spectra, disregarding the
spatial relationship between the data.6 Nevertheless data in image form serve as the human/data interface in that
image space helps the user to make the connection between individual pixel areas and the surface features they
represent. To display a spectral image, a first step would be to reduce it into a two-dimensional gray value image,
e.g. by taking the mean of the wavelength values, or by taking a specific wavelength band from the data. The data
can also be reduced to a RGB color image (which in the strict sense is a spectral image with 3 bands) by taking
three specific bands from the spectral image, or by applying a weigh function to the wavelength data. In these cases
a false color image is produced. When the wavelengths of the spectral image cover the visible range, sensible weights
are the spectral tristimulus values as defined for a standard colorimetric observer by CIE (1964). In this case a true
color image is produced.
4. FEATURE BASED REPRESENTATION
Often spectral image analysis is used to automatically find and measure objects in the image. Pattern recognition
techniques are used to extract the features from the data. In the research cycle this is done in the data exploration and
the data analysis steps. For the presentation step, feature based presentation is crucial. Feature based representation
begins with a spectral focus, but it is less related to pictures or graphs. The sampled discrete spectrum of a pixel can
be regarded as a feature vector in an N-dimensional space, where N is the number of spectral bands. This conversion
of the information from a continuous function of wavelength to a discrete point in a vector space is not only inherent
in the operation of a spectral sensor, it is a crucial representation for pattern recognition, i.e. analysis by a machine-
implemented algorithm. High dimensional spaces can not be visualized directly. Visualization is restricted to only a
few (typically 2 or 3) dimensions. Therefore, from a visualization point of view, the N-dimensional vector needs to be
reduced by the selection or combination of features in order to visualize it in two or three dimensional scatterplots.
Proc. SPIE Vol. 4553 133
Figure 1. Trellis plot of PCA analysis of four tomatoes which differ in maturity. The eigenimages, eigenvectors and
eigenvalues are shown.
Several strategies and criterion functions are possible for searching good subsets. One of the most well known and
often used methods is Principal Component Analysis. In PCA the original feature vector is decomposed into a new
set of features, being linear combinations of the original features, with the addition that the first few features contain
most of the variation present in the original set. Feature reduction can be obtained by selecting only the first M
features where M ¿ N . The most important scores or loadings can be visualized in a two- or three-dimensional
graph.6 In addition to the visualization possibilities and an improved computational speed in lower dimensional
feature spaces there might also be an increase in the accuracy of the classification algorithms.7,8 This is caused
by the fact that for smaller feature sets less parameters have to be estimated. Besides PCA, other ways exist to
reduce the dimensionality by mapping the data on a linear or nonlinear subspace. Examples of these techniques
are Fisher linear or quadratic discriminant analysis, independent component analysis, neural networks and support
vector machines.
5. COMBINING SPECTRAL, IMAGE AND FEATURE BASED REPRESENTATION
Combinations of the three presentation methods and dynamic interaction between them, adds significant to the
usability. This section describes some examples.
5.1. Combination of image and spectral display
In the data gathering and data exploration steps, both spectral and image based representation have a valuable
contribution. The combination of these two representation modes is even more valuable, when data visualized in
one form can be linked interactively with data shown in the other form. Isaac (Imaging Spectroscopy Acquisition
and Control) (see http://www.ph.tn.tudelft.nl/ polder/isaac.html) is a system for recording and screening spectral
images. It consists of specialized hardware and software for the acquisition of spectral images on a macroscopic scale.
The main purpose of the software is not visualization of the acquired images, but some simple tools are provided, in
order to quickly evaluate the scanned images. The spatial information of different spectral bands can be displayed
concurrently. Furthermore of each pixel, the spectral information can be shown. There is a close interaction between
the two visualization methods: while clicking or dragging in the image window, the spectrum of the selected pixel is
shown. The wavelength plane shown can be selected with a slider. Also tools for contrast stretching the image are
provided.
5.2. Combination of image and feature display
Since the spatial relationship of the measured features is important, the combination of image and feature display is
important in the data exploration, data analysis and the presentation step. By considering a spectral image as an
134 Proc. SPIE Vol. 4553
Figure 2. Spectral image of four roses. On the left the first PC component, on the right the PCA score plot of
scores 1 and 2. Interactive selections are made in the image, which results in the corresponding scores in the score
plot.
Figure 3. The same spectral image as in figure 2. Interactive selections are made of clusters in the score plot,
pixels within this cluster are marked in the image window.
image sequence, a powerful analysis and visualization tool is obtained. Visualization of spatial bands is obtained by
showing the image in a Trellis plot, i.e. showing the image at different spectral bands in a grid. Since there are a
large number of bands, it is not possible to display each plane in its original size. Figure 1 shows a trellis plot of
PCA analysis on a spectral image of tomatoes of different maturity. Besides the score-images, also the eigenvalues
and eigenvectors are shown for each principal component. This figure is made with Pixies (Physiological Information
eXtraction from ImagE Sequences), by Apteryx, which is a user-friendly application dedicated to the analysis of
biomedical image series. A detailed description can be found at http://www.apteryx.fr/en/pixies/. These examples
display the data in a static way.
Dynamic interaction between the two visualization methods adds significant to the value. In figure 2 and 3 a
spectral image of four different roses is shown. On the left is the sum image of all the wavelength bands, on the
right a score plot from score 1 and 2 of the PCA analysis is plotted. In figure 2 regions in the image are selected
interactively. The corresponding scores are marked in the score plot. Figure 3 shows the other way around, the
corresponding pixels of selected clusters in the score plot are marked in the image. Additionally zooming and panning
tools are provided. This interactive visualization method was accomplished with in-house software written in C and
implemented in Scil-Image (TNO-TPD, Delft, NL).
5.3. Volume visualization techniques applied to spectral images
Since a spectral image is organized as a 3D sampled scalar field, standard volume visualization techniques can be
applied to visualize the data. In contrast with real 3D images (e.g obtained with Magnetic Resonance Imaging of
Confocal Scanning Laser Microscopy) a voxel does not represent a 3D spatial point, instead it represents the reflection
value of a 2D spatial point at a certain wavelength. Figure 4 shows a combination of slices of a spectral image of two
flowers of Saint Paulia, violet and pink. The violet flower has much lower reflection values in the lower wavelength
Proc. SPIE Vol. 4553 135
Figure 4. Direct volume rendering of a spectral image of a violet (upper left) and a pink (lower right) flower of
Saint Paulia. To visualize several aspects of this image, a cross-section at 75 in the spectral direction is cut out. At
the bottom plane number 50 is displayed.
bands. Spatial information is preserved, but only at the selected cross-section. Since a voxel in a spectral image does
not represent a spatial 3D data point it was not to be expected that iso-surfaces would be a useful representation
method. But due to the fact that in most images there is some shading from the border to the center of the objects
this technique gave unexpectedly good results (figure 5). The volume visualization in this section is all done in
MATLAB (The Mathworks inc.).
6. DISCUSSION
A number of methods can be used to visualize spectral images. All these methods make use of one or more combi-
nations of the spectral, spatial and feature nature of the data. It is difficult to visualize all the aspects of a spectral
image in one picture, because of the large size of the data. Displaying one or more image bands gives information
on the spatial information in the selected band, but the relation between spectral and spatial information is more
difficult to investigate. Adding interactive control, such as selecting a certain band with a slider, or showing the
spectrum at selected pixels contributes to the explorative value of this visualization method. Displaying all image
bands at once gives more information about the spectral relationship of the image bands. Since the images need to be
reduced significantly to fit on the screen, it is difficult to see details in the spatial information. Mapping the features
to lower dimensional space (figure 1) can also contribute to the understanding of the data. From spectral band 5
onwards the data contains mainly noise, so displaying image 1-4 at their original sizes gives most of the information.
Adding user-interaction contributes significant, as can be seen from figure 2 and 3. Display of pixels in either
image or feature space as a result of interactive selection of points in the other space is useful while exploring the
data. Volume visualization techniques can be applied to visualize combinations of spectral and spatial information
(figure 4). Both spectral and spatial characteristics are shown in detail, but only at the selected cross-section. To
explore the whole data set, interactive selection of cross-sections is needed and real-time update of the display is
136 Proc. SPIE Vol. 4553
Figure 5. Iso-surface of a spectral image of four tomatoes of different maturity. The lower right one is the greenest
one, then upper right, lower left and the upper left the ripest. Using a cross section the reflection values at wavelength
band 28 are also visualized.
desirable. This also applies to rotating the data set, the viewpoint or the illumination direction. The same accounts
for iso-surfaces where the threshold level (iso-contour) needs to be selected in an interactive way, with fast update
of the display.
For presentation of data, an important aspect is to gather as much information in a single picture as possible
without losing insight. Figure 4 shows a nice example of combining the spectral information at interesting cross-
sections, with a spatial representation in the form of an isocap. Although at the moment hardly possible, user
interaction could also be useful for presentation purposes. For presentation on the internet compression algorithms
for spectral images and VRML-browsers with volume rendering capability are required.
REFERENCES
1. E. Herrala and J. Okkonen, “Imaging spectrograph and camera solutions for industrial applications,” International
Journal of Pattern Recognition and Artificial Intelligence 10(1), pp. 43–54, 1996.
2. T. Hyva¨rinen, E. Herrala, and A. Dall’Ava, “Direct sight imaging spectrograph: a unique add-on component
brings spectral imaging to industrial applications,” in SPIE symposium on Electronic Imaging, vol. 3302, 1998.
3. A. F. H. Goetz, G. Vane, J. E. Solomon, and B. N. Rock, “Imaging spectrometry for earth remote sensing,”
Sience 228, pp. 1147–1153, 1985.
4. G. Vane and A. F. G. Goetz, “Terrestrial imaging spectrometry: Current status, future trends,” remote Sensing
of Environment 44, pp. 117–126, 1993.
5. D. Landgrebe, “On information extraction principles for hyperspectral data, a white paper,” tech. rep., School
of Electrical& Computer Engineering, Purdue University, 1997.
6. P. Geladi and H. Grahn, Multivariate image analysis, John Wiley & Sons, New York, 1996.
7. B. D. Ripley, Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996.
8. R. P. W. Duin, PRTools, Version 3.0, A Matlab Toolbox for Pattern Recognition. Delft, 2000.
Proc. SPIE Vol. 4553 137


Direct sight imaging spectrograph: a unique add-on component brings spectral imaging to industrial applications,”

Imaging spectrograph and camera solutions for industrial applications,”

Imaging spectrometry for earth remote sensing,”

Multivariate image analysis,

On information extraction principles for hyperspectral data, a white paper,” tech. rep.,

Pattern Recognition and Neural Networks,

PRTools, Version 3.0, A Matlab Toolbox for Pattern Recognition.

Terrestrial imaging spectrometry: Current status, future trends,” remote

Visualization of spectral images

Abstract

Similar works

Full text

Available Versions

Name not available

NARCIS

Wageningen Yield