Se proponen novedosas fórmulas para evaluar la certeza de la cartografí

Albrecht, F.

García Abril, Antonio

Hernando, A.

Lang, S.

Tiede, D.

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

* Corresponding author 
NOVEL PARAMETERS FOR EVALUATING THE SPATIAL AND THEMATIC 
ACCURACY OF LAND COVER MAPS  
 
 
Hernando A. a*, D. Tiede b, F. Albrecht b, S. Lang b, García-Abril A.a 
 
a Technical University of Madrid (U.P.M), E.T.S.I de Montes, Silvanet Research Group, Ciudad Universitaria s/n, 
28040 Madrid, Spain – (ana.hernando, antonio.garcia.abril)@upm.es 
 
b Salzburg University, Centre for Geoinformatics Z_GIS, Schillerstrasse 30 * 5020 Salzburg, Austria- (dirk.tiede, 
Florian.Albrecht, Stefan.Lang )@sbg.ac.at 
 
 
 
 
KEY WORDS: Accuracy, spatial, thematic, assessment, OFA-matrix. 
 
ABSTRACT: 
 
Correct delineation and classification of land cover maps is an important aspect for its management and monitoring. Based on 
orthophotos, object-based image analysis (OBIA) can be used to produce detailed land cover maps. However, these new products 
require measures to evaluate and quantify both thematic (difference in assigned labels) and spatial (difference in delineated object 
boundaries) accuracy of the outputs. We propose to extend the Object Fate Analysis (OFA) comparison approach with an OFA-
matrix to validate thematic and spatial accuracy together. Its diagonal shows the relative area of spatial and thematic coincidence 
between a reference map and a classification map. Novel parameters STL (Spatial Thematic Loyalty), STLOVERALL (Spatial Thematic 
Loyalty Overall) and MIO (Maximal Interfering Object) summarize the new OFA-matrix accuracy assessment. Land cover map 
generated by OBIA (classification) was compared with the map performed using photo interpretation and field data (reference). The 
results show for the OFA-matrix good spatial and thematic accuracies (>65 %) for all the classes except for one of them, with a good 
STLOVERALL=69.8 %. Overall, OFA-matrix could be used for OBIA accuracy assessment. 
 
 
 
 
 
1. INTRODUCTION 
Object Based Image Analysis (OBIA) has been recently 
applied for processing land cover maps using automated 
methods for the analysis of very high resolution images (Hay et 
al., 2005; Hernando et al., 2012). OBIA, alternative to the 
traditional pixel approach, describes the imaged reality using 
spectral, textural, spatial, topological and hierarchical objects 
characteristics (Lang, 2008; Blaschke, 2010). This method 
raises concerns about the subsequent validation strategies. It 
was suggested that OBIA products would preferably be assessed 
by replacing the standard sampling units by objects (Congalton 
and Green, 2009). Accuracy assessment of object-based maps 
should include both thematic and spatial components. 
Therefore, for the OBIA accuracy assessment new methods 
explicitly built on the concept of objects are needed (Albrecht, 
2010; Lang et al., 2010).  
Considering the thematic accuracy (difference in 
assigned labels), in a review of 20 recent papers on OBIA the 
point–based sampling, error matrix (Congalton and Green, 
1999) was still used in the majority of the cases (Radoux et al., 
2010). However, even if it is recommended to set up the 
accuracy assessment on an object-based sampling, there is no 
state-of-the-art approach available at the present. Nevertheless, 
some strategies using the bias and the variance of the overall 
accuracy have been proposed (Radoux et al., 2010). Related to 
the spatial accuracy (difference in delineated object boundaries), 
boundaries are more relevant for OBIA than had been for 
traditional pixel-based image analysis approach. For this 
purpose new methods for validation are being developing such 
as positional error (Radoux and Defourny, 2007) or object fate 
analysis (Schöpfer et al., 2008). The methodology presented by 
Radoux and Defourny (2007) showed that the use of simple 
statistics, bias and standard deviation, provide robust 
quantitative spatial quality assessment. The Object Fate 
Analysis (OFA) is a method presented by Schöpfer et al. (2008) 
for investigating the spatial relationships of corresponding 
objects in two different representations. OFA was successfully 
applied for object-based change detection and for the 
assessment of object boundary accuracy (Schöpfer et al., 2008).  
Both methods summarize spatial quality and thereby 
complement thematic accuracy assessments. In any case, none 
of the above studies undertook both thematic and spatial 
accuracy at the same time. In this paper we propose to extend 
the OFA comparison approach with an OFA-matrix and derived 
OFA-parameters to validate thematic and spatial accuracy 
simultaneously.     
 
2. METHODS  
2.1 Thematic and spatial assessment: OFA-matrix. 
Within OFA the topological relationships between overlapping 
objects are categorized by an error band and by evaluating 
Proceedings of the 4th GEOBIA, May 7-9, 2012 - Rio de Janeiro - Brazil. p.613
613
  
whether the centroid of the classification object fell inside the 
reference object (C2: Good I, Good II, Expanding) or not (C1: 
Invading, Not interfering I, Not interfering II), (Tiede et al., 
2010). The two data sets are named classification and reference. 
For the spatial comparison, all classified objects were compared 
to all reference objects in the manner illustrated in Fig. 1.A. 
Then, the reference object boundary and a buffer applied to this 
boundary serve for subdividing the relationships for each cluster 
into its three sub-types. The OFA concept is implemented in a 
tool called LIST (Landscape Interpretation Support Tool) as an 
extension for ArcGIS (Weinke et al., 2007). To illustrate this 
concept we explain a possible case. We compare a one 
reference object A (area: s1) with different classified objects (a, 
b, c...) as illustrated in Fig. 1.B.  
 
 
 
Figure.1.A: Interaction between classified objects (rose solid 
color), reference objects (grey outline) and buffer (dashed red 
outline). 1.B: Comparison between one reference object A 
(area: S1) with different classified objects (a, b, c...) and (p) the 
overlapping part (e.g. pAIa: correct thematic classified object 
(a) compared to the reference (A) and OFA type spatial 
delineation: I (Good I). 
             Considering the thematic accuracy some of them are 
correct thematic classified (a) while others (b and c) are not. 
Each classified object is compared to the reference only with the 
overlapping part between the reference object and the classified 
object (p). For a reference (A) those classified (a) are correct 
thematically. Besides, we can distinguish the OFA spatial 
accuracy (I) Good I, (II) Good II. Therefore, we obtain pAIa, 
pAIIa. Other examples would be pAIb thematic wrongly 
classified and good I spatial classified and pAIIIc both thematic 
and spatial incorrectly classified. 
We propose now to combine the OFA concept with 
the traditional error matrix structure to create the OFA-matrix, 
for validating both thematic and spatial assessments. The OFA-
matrix is a square array of numbers set out in rows and columns 
that considering a reference category (T) expresses the relative 
area ( rTCt ) of classified objects (t) regarding the OFA types (C) 
–Good I; Good II; Expanding; Invading; Not Interfering II- and 
the thematic correspondence (Fig. 2). 
If finally we compare all these relationships following 
these criteria in an OFA- matrix, we formulate that rTCt  is the 
sum of the relative areas of the t classified objects considering a 
OFA type (C) defined by Eq. (1):        
                                                           
      
          (1) 
where pTCti  are the areas within the reference compared to the 
reference category area ST. The diagonal shows the relative area 
of spatial and thematic coincidence between the reference and 
the classification.   
Furthermore, to characterize overall spatial and thematic 
accuracy we group the total OFA-matrix information into 
clusters C2 and C1 (Fig. 3.A).   
We proposed three specific measures. Firstly, STL (Spatial 
thematic Loyalty) indicates the percentage of spatial and 
thematic loyalty between one reference (T) and the classified 
objects (t) for a C2 OFA type given by Eq. (2):  
 
   2
2
2
TC
tTC
tTC R
r
STL =
                                                            (2)   
 
 
where RTC is the total relative area in a reference T in a type C1 
or C2 given by Eq. (3):  
     
                       (3) 
Secondly, STLOVERALL (Spatial Thematic Loyalty Overall) is the 
mean of the STL defined in Eq. (4):  
w
SLT
STL
tTC
OVERALL
∑
=
2
    
    
                                  (4)
 
∑= TCtTC rR
∑
=
=
t
i T
iTCt
TCt S
p
r
1
614
  
 where w is the number of the references categories (A, B, 
C…w...T).  
Thirdly, MIO (Maximal Interfering Object) is the percentage of 
the class (different from the reference) invading more the 
reference for a C1 OFA type given by Eq. (5).   
1
1
TC
máxTC
R
r
MIO =
    
                                                 (5) 
 
 
Figure 2. Structure of the OFA-matrix where each of the cells 
contain  rTCt which is the sum of the relative areas of the 
classified objects (t) considering a OFA type (C). 
  
 
 
3. RESULTS 
In our case of study we considered two land cover 
maps: the first one is one performed using photo interpretation 
and field data (considered as the reference) and the stands map 
obtained with the OBIA (considered as the classification). The 
reference map was obtained delineating visually homogeneous 
vegetation patches and confirming the class assignment with a 
field survey (a regular grid of 156 circular plots, r=10 meters). 
The study site is a forest public land, called “Dehesa Boyal” 
(815 ha), located in central Spain, Ávila, Region of Castilla y 
León (Fig.1). It is on flat or gently sloping area (altitudes 1220-
1395 m, slope 5-12%) with oligotrophic acid soils. The climate 
is Temperate Mediterranean, with dry hot summers, cool and 
rainy winters and a mean annual rainfall between 600-2000 mm. 
This site is included in the Mediterranean biogeographic region 
of the Natura 2000 network. Based on field work, we can 
distinguish within this area the following land cover classes: 
shrub (<1 m tall); low pole (> 1 m tall and a diameter breast 
height (DBH) < 10 cm); high_ pole (10 cm < DBH < 20 cm); 
High_ forest (20 cm < DBH < 35 cm), the largest trees in the 
forest tall_ shrubs and pasture  (Hernando et al., 2010). 
 
 
The summarized OFA- matrix shows the results 
(Fig.3.B). We appreciate that more than ¾ of the classified 
objects belong to the cluster C2 for all the categories (77.9%, 
83.4%, 79.9%, 78.9%, 74.4%, 84.2%) which means a good 
relative spatial coincidence. For example, the shrub has a 77.9% 
spatial accuracy but there is only a thematic coincidence with 
shrub for the 24.7%, that is a STL (shrub) = 31.8%. Even if the 
class shrub was adequately segmented there is thematic 
confusion with the high polewood (33.6%). The highest STLs 
belong to tall_ shrubs (83.6 %) and high_pole (87.2%) which 
represent the best spatial and thematic assignment. 
 
The STLOVERALL for all the classification is 69.8%, it 
indicates the percentage which was correctly segmented of the 
type C2 and well classified thematically. Regarding the C1 
group we obtain a different view for the incorrectly classified 
objects (type C1). For example, regarding the class s-shrub 
(22.1%) is mainly invaded by the low_ pole (9.9%) and 
therefore the MIO_shrub= 44.6%. The highest MIO (75.8%) 
belongs to the reference High_ forest which is invaded by the 
16% of High_ forest.    
 
 
4. DISCUSSION 
The presented approach to assess the accuracy of the 
object-based image classification of “Dehesa Boyal” showed 
that with the OFA-matrix it is possible to compare the 
classification to the reference on the basis of objects. Spatial 
boundary inconsistencies are accounted for by assigning each 
spatial relationship between a classification object and a 
reference object to an OFA category. Thematic comparisons 
then are handled separately within each category.  
Each object comparison is taken into account by its 
overlapping area that is covered by both involved objects. By 
this approach, each area segment of the reference is only 
counted once in the matrix and all matrix cells add up to form 
the total area of the reference. On this basis the OFA-matrix is 
comparable to the traditional thematic error matrix. As an 
advantage, the OFA-matrix enhances the thematic object 
comparison by distinguishing between objects that are spatially 
sound (and well delineated C2) and objects that do not meet this 
spatial accuracy C1. Obviously the categorization process has to 
be lead with care in order to get optimal information about the 
implications of spatial accuracy on thematic accuracy. 
Furthermore, the proposed new parameters STL (Spatial 
Thematic Loyalty), STLOVERALL (Spatial Thematic Loyalty 
Overall) and MIO (Maximal Interfering Object) summarize the 
accuracy assessment. These parameters could be use to compare 
different Object-based Image studies as we used to do with the 
Kappa index (Congalton and Green, 1999; Congalton, 2001) in 
the traditional pixel approach.  
615
  
 
Figure 3. A: Example of the structure of the summarized OFA-matrix and structure 3.B: application for the “Dehesa Boyal” example.  
.  
 
616
  
5. CONCLUSION  
The OFA-matrix is explicitly built on the concept of 
objects and fulfills this initial requirement for an object-based 
accuracy assessment. Thematic accuracy and spatial accuracy 
can be assessed together in a single approach. The proposed 
OFA-matrix, with its parameters STLOVERALL and MIO, assesses 
about the thematic and spatial accuracy jointly for the OBIA. 
This matrix has shown to be an useful and alternative tool to the 
confusion matrix for validation or comparison in classifications 
based on objects. This fact confirms the transferability of OFA-
matrix to other OBIA application contexts. 
 
Acknowledgments 
 
The research leading to these results has received funding from 
Spanish Ministry of Science and Innovation through 
MODELIDAR (AGL2009-07140) and DECOFOR (AGL2009-
08562) projects and the European Community's Seventh 
Framework Programme (FP7/2007- 2013) under grant 
agreement n° 263479 (MS.MONINA). We are also grateful to 
Technical University of Madrid (U.P.M) for fellowships 
awarded to Ana Hernando for her Ph.D. We would like to thank 
all of the members of the ‘‘Silvanet’’ research group. 
REFERENCES 
Albrecht, F., 2010. Uncertainty in image interpretation as 
reference for accuracy assessment in object-based image 
analysis. In, Accuracy 2010 Symposium, Leicester, UK. 
Blaschke, T., 2010. Object based image analysis for remote 
sensing. Isprs Journal of Photogrammetry and Remote Sensing 
65, 2-16. 
Congalton, R.G., 2001. Accuracy assessment and validation of 
remotely sensed and other spatial information. International 
jorunal Wildland Fire 10, 321-328. 
Congalton, R.G., Green, K., 1999. Assessing the accuracy of 
Remote Sensed Data: Principles and Practices. Lewis 
publishers, Boco Raton. 
Congalton, R.G., Green, K., 2009. Assesing the accuracy of 
Remote Sensed Data: Principles and Practicises. Boca 
Raton:CRC Press. 
Hay, G.J., Castilla, G., Wulder, M.A., Ruiz, J.R., 2005. An 
automated object-based approach for the multiscale image 
segmentation of forest scenes. Int. J. Appl. Earth Obs. Geoinf. 
7, 339-359. 
Hernando, A., Arroyo, L., Velazquez, J., Tejera, R., 2012. 
Object based image analysis for mapping Natura 2000 habitats 
to improve forest management. Photogrammetric Engineering 
& Remote Sensing (in press). 
Hernando, A., Tejera, R., Velazquez, J., Nunez, M.V., 2010. 
Quantitatively defining the conservation status of Natura 2000 
forest habitats and improving management options for 
enhancing biodiversity. Biodivers. Conserv. 19, 2221-2233. 
Lang, S., 2008. Object-based image analysis for remote sensing 
applications:modeling reality-dealing with complexity. In: 
Blaschke, T., Lang, S., Hay, G.J. (Eds.), Object-Based Image 
Analysis. Springer Berlin Heidelberg, pp. 3-27. 
Lang, S., Albrecht, F., Kienberger, S., Tiede, D., 2010. Object 
validity for operational tasks in a policy context. J. Spat. Sci. 
55, 9-22. 
Radoux, J., Bogaert, P., Defourny, P., 2010. Overall accuracy 
estimation for geographic object-based image classification. In, 
Accuracy 2010 Symposium, Leicester, UK. 
Radoux, J., Defourny, P., 2007. A quantitative assessment of 
boundaries in automated forest stand delineation using very 
high resolution imagery. Remote Sens. Environ. 110, 468-475. 
Schöpfer, E., Lang, S., Albrecht, F., 2008. Object-fate analysis: 
spatial relationships for the assessment of object transition and 
correspondence. In: Blaschke, T., Lang, S., Hay, G.J. (Eds.), 
Object-Based Image Analysis. Springer Berlin Heidelberg, pp. 
786-801. 
Tiede, D., Lang, S., Albrecht, F., Holbling, D., 2010. Object-
based Class Modeling for Cadastre-constrained Delineation of 
Geo-objects. Photogrammetric Engineering and Remote 
Sensing 76, 193-202. 
Weinke, E., Lang, S., Tiede, D., 2007. Landscape Interpretation 
Support Tool (LIST). Shaker Verlag, Entwicklerforum 
Geoinformationstechnik 2007. ISBN: 978-3-8322-6403-1. 
 
 
617


Novel parameters for evaluating the Spatial and Thematic accuracy of land cover maps.

Archivo Digital UPM

http://oa.upm.es/21040/2/INVE_MEM_2012_129477.pdf

Novel parameters for evaluating the Spatial and Thematic accuracy of land cover maps.

Abstract

Similar works

Full text

Available Versions

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Archivo Digital UPM