The author has identified the following significant results. Labeling of wheat rather than total grains, particularly with only one acquisition, led to significant overestimates in some segments. The Image-100 software and procedures were written to facilitate classification of the LACIE segments but were not designed to record data for later accuracy assessment. A much better evaluation would have been possible if accuracy assessment data had been collected following each satisfactory classification

Hocutt, W. T.

English

NASA Technical Reports Server

General Disclaimer 
One or more of the Following Statements may affect this Document 
 
 This document has been reproduced from the best copy furnished by the 
organizational source. It is being released in the interest of making available as 
much information as possible. 
 
 This document may contain data, which exceeds the sheet parameters. It was 
furnished in this condition by the organizational source and is the best copy 
available. 
 
 This document may contain tone-on-tone or color graphs, charts and/or pictures, 
which have been reproduced in black and white. 
 
 This document is paginated as submitted by the original source. 
 
 Portions of this document are not fully legible due to the historical nature of some 
of the material. However, it is the best reproduction available from the original 
submission. 
 
 
 
 
 
 
 
Produced by the NASA Center for Aerospace Information (CASI) 
https://ntrs.nasa.gov/search.jsp?R=19790010247 2020-03-21T23:20:45+00:00Z
JSC-14606
	
Lockheed OCKHEEDA SUf1WARY OF	 7.9 ^	 14 Q
 CORPORATION 	 A Oft 
^
..sElectronics Tel. 7 A 3 5oad 1, Houston, Texas 77058 ^^`^^^ ^^^3
Company, Inc.
Ref: 642-6990
Contract NAS 9-15200
Job Order 73-783-11
TECHNICAL MEMORANDUM
ANALYSIS OF THE CLASSIFICATION OF U.S. AND CANADIAN
INTENSIVE TEST SITES USING THE IMAGE 100
HYBRID CLASSIFICATION SYSTEM
1.43de available unQir NASA sponsorship
in ttce	 ^i car,y and' ;aids dis-
By 	 e+, Earih Rescssces Surrey
F ► ^„ •arn .t; Ct^sr < !in!1 :a.a e^iih(wI liability
	
W. T. Hocutt	 ±o ► ana o« a1a w- '",, , E of. '
Approved By: &hf _
E. M. Hsu, Supervisor
Accuracy Assessment Section
(E79-10156) ANALYSIS OF THE CLASSIFICATION
	 N79-18418
OF US AND CANADIAN INTENSIVE TEST SITES
USING THE IMAGE 100 HYBRID CLASSIFICATION
SYSTEM (lockheed Electronics Co.) 8 g HC	 Unclas
A02/MF A01	 CSCL 02C G3/43 00156
k'
ti
November 1978	 LEC-12393
1. INTRODUCTION
During Phase III of the Large Area Crop Inventory Experiment (LACIE), 24 U.S.
and 10 Canadian intensive test sites were scheduled for processing on the Inter-
active Multispectral Image Analysis System, Model 100 (Image 100) using the
Procedure 1 hybrid classification system. This report describes an evaluation
of the results obtained.
In this study the only segments analyzed were those with (1) satisfactory
classifications and (2) complete classification and ground-truth data. This
eliminated 22 of the 34 segments: 4 segments were not classified, 6 segments
were eliminated because of unsatisfactory classifications, and 12 segments
were eliminated because of inadequate classification or ground-truth data.
This left 12 segments with 13 estimates. (Segment 1968 is a mixed-wheat seg-
ment, so estimates for this segment are available for both spring and winter
wheat.)
Originally, it was intended to investigate both labeling accuracy and propor-
tion estimation accuracy. However, there were not enough dot-labeling data
available, so this part of the investigation was abandoned.
2. METHOD
In the procedures used on the Image 100, the analyst could give proportion
estimates for winter wheat, winter grains, spring wheat, or spring grains.
However, these could not be compared directly with a corresponding ground-
truth proportion, since the ground truth did not cover the whole segment.
Therefore, an analyst estimate of the proportion in the ground-truth area was
obtained and compared with the ground-truth value. This was done in the
following manner.
The image, classification map, and a map of the ground-truth area were read
into the Image 100. Using existing hardware and software, the number of
pixels within the ground-truth area and the number of these pixels which were
classified as wheat or grains were counted. These pixel counts, after delet-
ing those which had been designated other (00) pixels, were used to calculate
the "uncorrected" proportion estimate for the ground-truth area. The bias
correction alpha table, as stored in the Image 100, was then used to obtain a
bias-corrected estimate for the ground-truth area.
3. RESULTS
The results of the evaluation are shown in table 1. Each segment is identi-
fied in the table together with the acquisition used for the estimate, the
type of estimate (i.e., spring wheat, winter wheat, spring grains, and winter
grains), and the Robertson biostage as determined from the adjustable crop
calendar for the latest imagery used. In each case, the uncorrected propor-
tion estinate, R, and the bias-corrected proportion estimate, RC , which are
expressed as percentages, are given for the entire segment. The corresponding
quantities Y and Y  for the ground-truth area within the segment are given in
the next two columns. Since some of the estimates passed to the Crop Assess-
ment Subsystem (CAS) were not corrected for bias, each estimate passed is
identified by a superscript b. The ground-truth proportion, Y, which was
determined from field reports, is also given, along with the difference,
D = Y  - Y, which is the proportion error for the ground-truth area.
LACIE segment 1973 (Whitman County, Washington) had the largest difference
value, D = -33.8. Labeling data were available for this classification, and
they showed that all labeling errors were for wheat called nonwheat. Out of
98 labeled type 1 and type 2 dots, approximately 25 percent were wheat dots
which were labeled nonwheat. Note that 90 percent of the cultivated crops
within the around-truth area (excluding pasture and summer fallow) were
winter wheat.*
*Ground truth was availTab"le for approximately one-t-iird of tfis -segment is
area contained 53.5 percent winter wheat, 3.9 percent spring barley, 2.4 per-
cent dry peas, and 40.1 percent fallow and pasture.
2
F .
6
3N
W
H
W
zoo
6-0N
W
H
Z
M
4
M
0d
G
Z
Q
zW
NrN
O_
m
02
O
e—
W
C9
H
W
2
F-
C7
ZMN
NW
NM
N
IL
O
Z
O_
H
Q
t.aM
LL.
NNd
JV
H
M
W
N
Q.
WM
vd
J
r
W
J
00
Q
F--
V
L
, .+ c c
tOlOw
m cm
a
C C G Lu T
^^ p rN ^ N Y B
I	 1 
{{I
	
t
qE
N
LaQ
E ^ 6 1► CD O <b ^ ^ to r A CO t+l Pf N
1	 ' N O	 M, t0 N	 V m1 ^1 ^ F% fu	 3.I	 ? 1	 1	 I
:
._+ c
Liu M^..4!12!CRC9 C 0! 'iliiug
p1 M V•uq	 10 N	 PY tr	 NC
^a
mac'V
N V
 6UIt?V O CO	 n00 1D N0 1!t m i0 O N
9ILO m 4 Y PN	 C 1 wf N16 V 1^L a fn MN ^	 ^ .-	 M NIS G u
v
am
t s+
C E
^t
t0	 N
OIY
CO V h N r, VV N to to r
M N N '~ n ^"r' V10tl0'1	 N
ooO
LOW V N
N +^+ 1G a wf 01 O V1 N V CC CD Y9 N
V
_q U V
L lv
_
	
to ^ 
N
O^ CC
q
tb L V M N	 Ca	 a	 a	 c ^a a aE++
^+ e
v
y
C L pi w
yC G ty! N	 CC
	 11:	 IA	 to	 W 1!f	 4 1Q CC q m	 V
COp
NCOI+ O O a c
v+ u a^
u
c
om
LagOi
V 1n n m N O CC 11 O1 O O CT v
m N 1"t 10 N N N V en N V n V 10 m
o
s a
1, d
o
y 
+^
ssL^ N7 t271 q s	 S	 i N H[1
^' a
0
m N {! N N++.^ ^y 7 h h n n n n h h	 nn n h n n n h h	 n n h nn n n
m A
N
O •^
L•^v
y N mG .- M .- - .- .. N M	 P'1 N Pf .-
E c 7
2 V
q
o mac	 o	 +°^ dr
.+ tT	 O
19
	 q r
oft
C
qq
	
Lpp N Gq ^qq	 ~	
Cq
pCq
'Aqq
C
^t at Z	 '^  V C C^	 1L N i i
L
qm ^1Ppp1	 to	 F,	 P,	 ^O	 t^Mp	 1pp0pp
01 ^	 a T ^ d1 T
N
cis
01	 OI	 d1
r
The largest overestimate (17.2 percent) occurred for,LACIE segment 1992.
This segment had only 2.3 percent spring wheat within the ground-truth area,
but the large amount of barley raised the total spring grains to 40 percent.
The high wheat estimate was caused primarily by the inclusion of a large
portion of barley in the wheat estimate. It is particularly difficult to
separate spring wheat and spring barley without an acquisition showing a
difference in crop development such as occurs near the ripe stage of barley.
The estimate for LACIE segment 1975 probably is better than is indicated by
the table 1 data. Examination of the imagery and the classification map
indicates that the wheat proportion probably was much higher in the area out-
side rather than inside the ground-truth area. The bias correction factors,
which were calculated for the entire segment, probably are valid for the
segment; however, because the wheat proportions are significantly different
in the ground-truth area, these bias correction factors are not appropriate.
Even so, the bias correction was applied to the ground-truth area for con-
sistency. (See table l.)
Figure 1 shows a plot of the proportion errors as a function of the ground-
truth proportions for all 13 estimates. It will be seen that the errors are
overestimates for segments with low wheat proportions and that the errors are
i underestimates for segments with high wheat proportions. This behavior is
similar to that observed for blind sites (ref. 1) during LACIE Phase I and
Phase II.
With the exception of LACIE segment 1973, which was discussed earlier, fig-
ure 1 shows that the largest proportion errors occurred in spring wheat and
spring grains. Of the five spring wheat and spring grains estimates, two
(segments 1958 and 1992) had very large overestimates of spring wheat in
segments with very large proportions of barley and oats (46.0 and 38.6 per-
cent, respectively).
r
4
20
cd
u4d
CL
10
s
L
0
V
aL
m
E	 -10
d
AE
i+
°a -20
u
L
a
v
-30co
•
n
W.
I
30
i
10	 20	 30	 40
Ground truth (Y), percent
n
• U.S. WINTER WHEAT
OR WINTER GRAINS
n U.S. SPRING WHEAT
OR SPRING GRAINS
♦ CANADA SPRING WHEAT
OR SPRING GRAINS
•
-40
50 60
♦
Figure l.— Plot of proportion errors as a function of
ground-truth proportions.
5
r. .
4. CONCLUSIGNS
Insufficient data are available to provide a clear picture of the accuracy of
the Image 100 classifications, but some specific observations are valid.
a. Labeling of wheat rather than total grains, particularly with only one
acquisition, led to significant overestimates in some segments.
b. The Image 100 software and procedures were written to facilitate classifi-
cation of the LACIE segments but were not designed to record data for
later accuracy assessment. A much better evaluation would have been pos-
sible if accuracy assessment data had been collected following each
satisfactory classification.
c. A more thorough quality assurance check on the Image 100 analysis probably
would have prevented reporting to GAS some of the segments (such as 1973
and 1992) which had large estimation errors.
5. REFERENCE
1. LACIE Phase 1 and Phase II Accuracy Assessment Final Report. LACIE-00450
(JSC-13136), April 1978.
6


Analysis of the classification of US and Canadian intensive test sites using the Image 100 hybrid classification system

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server