The Moderate Resolution Imaging Spectrometer (MODIS) observing facility will operate on the Earth Observing System (EOS) in the late 1990's. It is estimated that this observing facility will produce over 200 gigabytes of data per day requiring a storage capability of just over 300 gigabytes per day. Archiving, browsing, and distributing the data associated with MODIS represents a rich opportunity for testing and applying both lossless and lossy data compression methods

Salomonson, Vincent V.

English

NASA Technical Reports Server

THE MODERATE RESOLUTION IMAGING SPECTROMETER: AN EOS FACILITY
INSTRUMENT CANDIDATE FOR APPLICATION OF DATA COMPRESSION METHODS
Vincent V. Salomonson /Q_ ] /
Earth Sciences Directorate
NASA/Goddard Space Flight Center
Greenbelt, MD 20771
Abstract. The Moderate Resolution Imaging Spectrometer (MODIS) observing facility will operate on
the Earth Observing System (EOS) in the late 1990's. It estimated that this observing facility will
produce over 200 gigabytes of data per day requiring a storage capability of just over 300 gigabytes per
day. Archiving, browsing, and distributing the data associated with MODIS represents a rich
opportunity for testing and applying both lossless and lossy data compression methods.
1. Introduction
MODIS is a multispectral imaging system to be flown on the EOS in the late 1990's. The capability of
the MODIS instrument derives from and expands upon some instruments that have been successfully
flown on spacecraft or aircraft and used for many years to observe properties of the earth-atmosphere
system and to develop data bases for studies of global change. These instruments are the Advanced
Very High Resolution Radiometer (AVHRR) and the High Resolution Infrared Sounder (HIRS) being
currently flown on the NOAA operational meteorological satellites, the Coastal Zone Color Scanner
(CZCS) flown on the Nimbus-7, the Landsat Thematic Mapper, and various aircraft scanners.
The MODIS system will be composed of two cross-track scanning instruments. One instrument is
called MODIS-N (nadir), indicating a multispectral scanner that will not be tilted and provide a
continuous cross-track scan. The other instrument is called MODIS-T (tilt), indicating a scanner that
will allow the cross-track scan to be tilted 50 ° fore and aft. The MODIS-N will have 36 spectral bands
covering spectral bands in the visible, near-infrared (0.7-1 microns), the short-wave infrared (1-3
microns), and the thermal infrared (3-15 microns). Tables 1 and 2 summarize MODIS-N and Table 3
summarizes MODIS-T. The purposes of the bands in Table 2 are only indicative and not complete.
Further details concerning MODIS-T and MODIS-N instrumentation are given in Salomonson [2] and
Magner and Salomonson [ 1].
Data volume coming from the MODIS observing facility will be on the order of a terabit of data per day
depending upon the total time the instruments are on and the amount of ancillary information acquired
concerning spacecraft attitude, etc. It is quite appropriate, therefore, to consider the MODIS as a case
study for data compression methods that would reduce the volume of data involved for archiving,
distribution, and analysis. This paper will describe in more quantitative detail the volumes and rates of
data associated with the MODIS. The objective of this discussion is to provide those interested in
applying data compression methods to MODIS, as a relevant and challenging example, with particulars
that should help in assessing the magnitude and complexities of the task.
2. MODIS Data Volumes and Rates
In the broadest sense, it is envisioned that data compression methods might be most appropriately
applied to MODIS data for the purposes of data storage, data distribution, providing a browsing
capability, or facilitating the direct broadcast of MODIS data to terminals on the ground where only a
subset or specific parameter of the data is needed or where reduced data volume or rate is needed in
order to be compatible with limited receiving or processing capabilities. It is assumed in this paper that
all information for MODIS must be retained in processing, at least through level 1, if not to level 2. The
13
https://ntrs.nasa.gov/search.jsp?R=19920003209 2020-03-17T13:52:32+00:00Z
reasonfor beingresolutewith regardto the level 1dataprocessingis thatfrom thesedataarederivedall
level 2 products. The level 2 products,of which therewill be asmany as 100,must utilize all the
radiometricand calibrationfidelity in level I. Even in the storageof MODIS data,particularlyin the
caseof level 1, all information must be retained. This, therefore,indicatesthat only losslessdata
compressionmethodsshouldbe applied to datastoragefor level 1, and perhapslevel 2 and above.
Lossydatacompressionmethodsaredeemed,at thispoint in theauthor'sunderstanding,appropriatefor
producingbrowsedataor for very specificapplicationswhereinthelossof informationcanbetolerated.
Figure 1 depictsthe overall dataflow for the MODIS. This figure showsthat MODIS datawill flow
from theEOSplatform throughtheTrackingandDataRelaySatellite(TDRS)to theCustomerandData
OperationsSystems(CDOS) and into the GoddardSpaceFlight Center(GSFC) Distributed Active
Archive Center(DAAC). In theDAAC, datawill beprocessedafter algorithmshavebeendeveloped
and checkedfor accuracyand quality by the MODIS ScienceTeam Membersusing the severaland
distributedTeamMember ComputingFacilities (TMCF's)and the MODIS TeamLeaderComputing
Facility (TLCF). Whenproductsareproducedin the appropriateDAAC, they will be archivedand
distributedto the scientific communityand anypart of thepublic at largethat desiresto useMODIS
data. The distributionof thedatawill occur throughtheEOSDataand InformationSystem(EOSDIS)
electronicnetworkthatexistswhenMODIS becomesoperational.
As furtherdetail,Table4 showsspecificsconcerningcalculateddataratesandvolumesassociatedwith
MODIS-N and-T. In this table,it is worthnotingthatthe13thbit for MODIS-T datais only includedto
flag thegainusedin sendingMODIS-Tdatato theground(see[1]). In going from level 1A (calibration
andnavigationinformationprovidedin theheader,butnot applied)to level 1B(calibrationinformation
dataappliedandpixel locationavailable),the increasedvolumeis dueprincipally to convertingthe 12-
bit informationto 16bitsandaddingnavigation,calibration,andbrowseinformation.
Table 5 shovesthearchivingrequirementsin gigabytesperday tbr MODIS-N and-T. Thesearerough
estimatesbasedonpreliminaryestimatesusingexistingorplannedalgorithmsfor theprincipal products
to be derived from the MODIS. In manycasestheseestimatesare baseduponexperiencefrom the
heritageinstrumentsindicatedin the Introduction. From a similar perspective,the expectedlines of
codeestimatesshownin Table 6 havebeenderived. In general,the bulk of the effort for producing
linesof codeand storingthe resultsfalls in producingat-satelliteradiances(levels 1A and 1B)and in
producingwaterleavingandlandleavingradiances.
Table 7 showsthe estimatedload on the datadistribution systemin providing MODIS data to other
DAAC's in theEOSDIS. TheothermainDAAC, besidestheGSFCDAAC, isat theEROSDataCenter
(EDC)in SiouxFalls, SouthDakota. At theEDC,all level 2 productsproducedover Iandareaswill be
archivedand all level 3 land productswill be producedand distributed from the EDC. Other key
DAAC's areat theLangleyResearchCenterin Virginia andtheNational SnowandIce DataCenterin
Boulder,Colorado. The assumptionsasto thefractionof MODIS datathatwill go to theseDAAC's is
providedin thethirdcolumnof Table7.
With regardto directbroadcastof MODIS dataandthevolumeof MODIS browsedata,the following
statementsare provided. For direct broadcastit hasbeenassumedthat 100 percentof the raw data
would be involved,but, in this instance,on-boarddatacompressiontechniquescould beexamined. If
suchanapproachis to beused,however,thatmustbedecidedsoon(i.e.,in a yearor 2) in orderfor it to
be implementedwith the instrumentor on the EOS spacecraft. The situations surroundingdirect
broadcastfrom theEOSarerelativelyundefined,butonemayassumethata 15megabitdirectbroadcast
link will be availableto be sharedamongthe instruments. MODIS, of course,has the potentialfor
occupyinga largeshareof thiscapabilityunlessdatacompressionis appliedappropriately. In thecase
of browsedata,by assumingthat browsedatawill be comprisedof 5 percentof levels 1B, 2, and 3,
14
O3
IT
ILl
Or)
O n
l.-
Z
o N'
!
--LL
I:1 o
O_
- _I-
E
,a E
c
G=- i-_>_1
1 _ __
_ - __ /
co _ _
n
I
I
g_
15
resulting in an estimate of about 12 gigabytes of data per day. Browse data is a prime candidate for
applying lossy compression methods.
In the case of archiving MODIS data, it has already been indicated that no information should be lost in
archiving the data. However, lossless compression methods could, and perhaps should, be applied that
allow the progressive extraction of archived data at various levels of accuracy depending upon the
amount of information actually needed. This means that if the data are compressed appropriately, one
could access the archive and extract first-order information. If this initial extraction indicates further
information is needed, another pass through the compressed archive could result in higher-order
information. Ultimately it appears that data compression methods are available for archived data
wherein the complete information available in the original data stream ultimately could be retrieved.
3. Summary and Conclusions
The MODIS provides a rich opportunity for applying data compression methods for archiving,
browsing, and distribution. Lossless methods should be developed for archiving that allow eventual
extraction of all the information contained in the MODIS. Lossy methods can very appropriately be
applied in order to browse MODIS data and distribute it for quick-look analyses. The challenges
include costs of developing and applying data compression methods including associated hardware
costs, the availability of off-the-shelf versus special purpose hardware and software, demonstrating
reliability and low risk of losing information for lossless methods plus, ira many cases, making the
application of data compression transparent to the average user.
Acknowledgments
The author wishes to acknowledge the assistance of Dr. A. Fleig, Dr. D. Hart, and members of the
Research and Data Systems Corporation (RDC) staff for developing the tables and figure in this paper.
Kelly Pecnick prepared the final drafts of the paper. Her help and expertise are very much appreciated.
Ill
[2]
References
Magner, T. J., and V. V. Salomonson, 1991: Moderate Resolution Imaging Spectrometer-Tilt
(MODIS-T). International Journal of Imaging Systems and Technology, VoI. 3.
Salomonson, V. V., 1991: Execution Phase (C/D) Spectral Band Characteristics of the EOS
Moderate Resolution Imaging Spectrometer-Nadir (MODIS-N) Facility Instrument. Advances in
Space Research, Vol. 3, pp. 231-236.
16
TABLE 1
MODIS-Nadir (N) Summary
PARAMETERS DESIGN SPECIFICATIONS OR
EXPECTED PERFORMANCE
PLATFORM ALTITUDE
IFOV (no. of bands @ IFOV)
SWATH
SPECTRAL BANDS
RADIOMETRIC ACCURACY
QUANTIZATION
POLARIZATION SENSITIVITY
MODULATION TRANSFER FUNCTION
S/N PERFORMANCE
(70 DEGREE SOLAR ZENITIq/OCEANS)
NEDT PERFORMANCE (THERMAL BANDS)
@ 300 DEG K/WINDOW BANDS
SCAN EFFICIENCY
INTEGRATION TIME
SIZE (APPROX)
WEIGHT
POWER
PEAK DATA RATE
DUTY CYCLE
705 KM
29@ 1000M
5@ 500M
2 @ 250 M
110 DEG/2330 KM
36 BANDS TOTAL
(19/0.4-3.0 I_m; 17/3-15 _tm)
5% ABSOLUTE, < 3 I_m
1% ABSOLUTE, > 3 _tm
(possibly < 0.3%)
2% REFLECTANCE
12 BIT
2% MAX, < 2.2 _m
0.3 AT NYQUIST
830:1 (443 nm)
745:1 (520 nm)
503:1 (865 nm)
LESS THAN 0.05
(TO BE DETERMINED)
(TO BE DETERMINED)
1X1.6X1M
APPROX 200 kg
250 w
11 MBS (daytime)
100%
17
TABLE 2
MODIS-NBands
BAND
1
2
CENTER * IFOV (m) WIDTH PURPOSE
LAND AND CLOUD BOUNDARIES BANDS
659 250 50 VEG CHLOROPHYLL ABS
LAND COVER TRANS.
865 250 40
LAND AND CLOUD PROPERTIES BANDS
3 470 500 20
4 555 500 20
5 1240 500 20
6 1640 500 20
7 2130 500 50
OCEAN COLOR BANDS
8 415 1000 15
9 443 1000 10
10 490 1000 10
11 531 1000 10
12 565 1000 10
13 653 1000 15
14 681 1000 10
15 750 1000 10
16 865 1000 15
ATMOSPHERE/CLOUDBANDS
17 905 1000 30
i8 936 1000 10
19 940 1000 50
THERMAL BANDS
20 3.75 1000 0.18
21 3.75 1000 0.05
22 3.96 1000 0.05
23 4.05 1000 0.05
24 4.47 1000 0.05
25 4.52 1000 0.05
26 4.57 1000 0.05
27 6.72 1000 0.36
28 7.33 1000 0.30
29 8.55 1000 0.30
30 9.73 1000 0.30
31 11.03 1000 0.50
32 12.02 1000 0.50
33 13.34 1000 0.30
34 13.64 1000 0.30
35 13.94 1000 0.30
36 14.24 1000 0.30
* BAND CENTER AND BANDWIDTH ARE
MICROMETERS FOR BANDS 20-36
CLOUD AND VEGETATION
LAND CO_R TRANSF.
SOIL, VEG D_RNCS
GREEN VEGETATION
LEAF/CANOPY PROPRT_S
SNOW/CLOUD DIFFRNCES
LAND & CLOUD PROPRTIES
CHLOROPHYLL
CHLOROPHYLL
CHLOROPHYL
CHLOROPHYLL
SEDIMENTS
SEDIMENTS, ATMOSPHERE
C_OR. _UORESCENCE
AEROSOL PROPERTIES
AEROSOL/ATM PRPRTS
CLOUD/ATM PROPERTIES
CLOUD/ATM PROPERTIES
CLOUD/ATM PROPERTIES
SEA SURFACE TEMP
FOREST FIRES/VOLCANOES
CLOUD/SFC TEMPERATURE
CLOUD/SFC TEMPERATURE
TROP TEMP/CLD FRACTION
TROP TEMP/CLD FRACTION
TROP TEMP/CLD FRACTION
MID-TROP HUMIDITY
UPPER-TROP HUMIDITY
SFC TEMPERATURE
TOTAL OZONE
CLOUD/SFC TEMPERATURE
CLOUD/SFC TEMPERATURE
CLD HEIGHT & FRACTION
CLD HEIGHT & FRACTION
CLD HEIGHT & FRACTION
CLD HEIGHT & FRACTION
IN NANOMETERS FOR BANDS 1-19 AND
18
pARAMETERS
TABLE 3
MODIS-Tilt (T) Summary
PLATFORM ALTITUDE
IFOV
SWATH
SPECTRAL BANDS (10-15 nm WIDTH)
DYNAMIC RANGE
RADIOMETRIC ACCURACY
QUANTIZATION
POLARIZATION SENSITIVITY
MODULATION TRANSFER FUNCTION
SIN PERFORMANCE (SPEC)
(70 DEGREE SOLAR ZENITH)
NEDT PERFORMANCE (THERMAL BANDS)
SCAN EFFICIENCY
INTEGRATION TIME
MODE)
COLLECTING APERTURE (DIA)
SIZE (APPROX)
WEIGHT
POWER
PEAK DATA RATE
DUTY CYCLE
DESIGN
SPECIFICATIONS
OR EXPECTED
PERFQRMANGE
705 KM
1.4 MRAD (1.1 KM)
90 DEG/1500 KM
32 (400-880 nm.)
(AREA ARRAY)
Lmax 95% @ 22.5 deg
solar zenith angle
5% absolute
2% relative to the sun
12 BIT
<2.3 %
(< 20 deg tilt)
0.3 AT NYQUIST
835:1(440 nm)
685:1(625 nm)
400:1 (845 nm)
N/A
25 %
1.127 MSEC (COMPOSITE
34 MM
75 X 140 X 100 cm
-170 kg
-130 w
~3 mbps (day)
DAYTIMEI100%
19
TABLE 4
MODIS-N and MODIS-T Data Rate and Volume Estimates
Earth Radius (kin)
Satellite Altitude (kin)
Orbital Period (rain)
6371
705
98.9
Modis-N # 1003 m REF channels
Modis-N # 500 m REF channels
Modis-N # 250 m REF channels
Modis-N # 1030 m TIR channels
Modis-N # 500 m NIR channels (1.6, 2.1 nm)
Modis-T # 1.1 km REF channels
12
3
2
17
2
32
MODIS-N # bits/REF channel
MODIS-N # bits/TIR channel
MODIS-T # bits/REF channel
12
12
13
MODIS-N REF Duty Cycle
MODIS-N TIR Duty Cycle
MODIS-T REF Duty Cycle
50%
100%
45%
MODIS-N # Along-track IFOVs
MODIS-T # Along-track IFOVs
MODIS-N # Detectors
MODIS-T # Along-track detectors
8
30
648
30
MODIS-N # Maximum scan angle (deg)
MODIS-T # Maximum scan angle (deg)
MODIS-N # IFOV FWHM (deg)
MODIS-T # WOV FWHM (deg)
MODIS-N # pixels along-scan/on-Earth
MODIS-T # pixels along-scan/on-Earth
55
45
8.13E-02
8.94E-02
1354
1007
MODIS-N Scan Period (sec)
MODIS-T Scan Period (sec)
MODIS-N VIS Data (megabits/scan)
MODIS-N TIR Data (megabits/scan)
MODIS-N Daytime Data (megabits/scan)
MODIS-T Daytime Data (megabits/sean)
MODIS-N # Scans/Orbit
MODIS-T # Scans/Orbit
1.2
4.6
7.3
3.2
10.5
12.6
5000
579
MODIS-N Daytime Data Rate (mbps)
MODIS-N Nighttime Data Rate (mbps)
MODIS-T Daytime Data Rate (mbps)
8.9
2.7
2.7
MODIS-N Orbital Ave Data Rate (mbps)
MODIS-T Orbital Ave Data Rate (mbps)
5.8
1.2
MODIS-N Daily Data Volume (gigabytes)
MODIS-T Dally Data Volume (gigabytes)
Total Daily Data Volume (gigabytes)
62.6
13.1
75.8
MODIS-N Volume (gigabytes) Level-1A
MODIS-T Volume (gigabytes) Level-lA
Total Daily Volume (gigabytes) @ IA
65.8
13.8
79.6
MODIS-N Volume (gigabytes) Level-IB
MODIS-T Volume (gigabytes)Level-lB
Total Daily Volume (gigabytes) @ IB
113.6
23.1
136.7
Total Daily Volume (gigabytes) @ 1A&IB 216.3
2O
TABLE 5
MODIS Long-Term Archive Storage Requirements
(Gigabytes Per Day)
PRODUCT LEVEL
DATA PRODUCT 1A 1B 2/T 2/N 3 TOTAL
Navigation 18.7 18.7
Cafibrafion 6, 8 6.8
Spacecraft Ancillary 4.3 4.3
At-Satellite Radiances 75.3 104.7 180.0
Water-Leaving Radiances 10.1 4.0 6.6 20.7
Single Scattering Aerosol Radiances 8.2 2.6 10.9
Angstrom Exponents 0.3 0.4 0.8
Chlorophyll-A Concentrations (Case 1) 0.3 0.4 0.2 1.0
Chlorophyll-A Concentrations (Case 2) 0.0 0.0 0.2 0.3
ChlorophylI-A Fluorescence 0.3 0.4 0.2 1.0
CZCS Pigment Concentrations 0.3 0.4 0.2 1.0
Sea-Surface Temperature 1.1 0.2 1.3
Sea-Ice Cover 0.1 0.1 0.2
Attenuation at 490 nm 0.3 0.4 0.2 1.0
Detached Coccolith Concentration 0.1 0.1 0.2 0.5
Phycoerythrin Concentrations 0.3 0.2 0.5
Dissolved Organic Matter 0.3 0.4 0.2 1.0
Suspended Solids 0.3 0.4 0.2 1.0
Glint Field 0.3 0.4 0.2 1.0
IPAR 0.1 0.1 0.1 0.2
Ocean Cal Data Sets 0
Primary Production (Oceans) 0.3 0.4 0.2 1.0
Land-Leaving Radiances 2.7 14.4 2.8 20.0
Topographically Corrected Radiance 2.7 14.4 2.8 20.0
Vegetation Index 3.8 1.7 5.5
Polarized Vegetation Index 3.8 1.7 5.5
Land Surface Temperature 0.5 0.2 0.6
Thermal Anomalies 0.5 0.5
Evapowanspiration 0.1 0.1
Primary Production (Land) 0.1 0.1
Snow Cover 0.2 0.0 0.2
Spatial Heterogeneity (not sized here) 0
Land Cover Type 0.0 0.0
Bidirectional Reflectance, BRDF 0.0 0.0
Cloud Mask 0.3 1.6 1.9
Cloud Fraction 0.0 0.0
Cloud Effective Emissivity 0.1 0.0 0.1
Cloud-Top Temperature and Pressure 0.3 0.0 0.3
Cloud Optical Thickness (0.66 fro) 0.1 0.0 0.1
Cloud Particle Effective Radius 0. I 0.0 0.1
Cloud Particle Thermodynamic Phase 0.0 0.0 0.0
Aerosol Optical Depth (0.41 to 2.13fm) 0.0 0.0
Aerosol Size Distribution " 0.0 0.0
Aerosol Mass Loading 0.0 0.0
Atmospheric Stability 0.1 0.0 0. I
Total Precipitable Water 1.7 0.0 1.7
Total Ozone 0.1 0.0 0.1
Browse 6.5 1.4 2.7 1.0 5.0
Metadata (Not sized here) 0
Ocean Discipline Subtotal (I.,-2/3) 21.8 12.1 9.5 43.4
Land Discipline Subtotal (I.,-2/3) 5.4 37.6 9.5 52.5
Atmosphere Discipline Subtotal (L-2/3) 0.3 4.0 0.0 4.3
Total 79.6 136.7 28.9 56.4 20.0 315.0
21
TABLE 6
EstimatedMODISDataProcessingRequirements
(Linesof Code)
PROCESSINGLEVEL
Level-IA
LAUNCH LOG
25,000
GROWTH LOC
25,000
Level- IB 25,000 30,000
Calibration/Monitor 72,000 144,000
Level-2 Ocean 12,000 24,000
Level-2 Land 40,000 80,000
Level-2 Atmosphere 20,000 40,000
Level-2 Shell 30,000 30,000
Level-2 Utility 40,000 80,000
Level-2 IDS Products 36,000 72,000
Level-3 30,000 60,000
Near-Real-Time 17,800 81,500
Subtotal 347,800 666,500
Supporting Software
(validation) 552,000
TQ_I 1.218.500
22
TABLE7
MODISDataDistribution
(GigabytesPerDay)
FROM
CDOS
GSFC
TO
GSFC
MODIS Investigators
DATA DESCRWTION
All Level-0 Products
10% of Level- 1A Products
50% of Level- 1B Products
100% of Level-2 Products
100% of Level-3 Products
DATA VOLUME
76
182
GSFC Other Investigators 5% of Level-lB Products
10% of Level-2 Products 17
10% of Level-3 Products
GSFC EDC Level-lB for Land Products 41
GSFC Langley Research Center 100% of Level-lB Products 137
GSFC National Snow and Level-lB for Snow and
Ice Data Center (NSIDC) Ice Products 4
23



The moderate resolution imaging spectrometer: An EOS facility instrument candidate for application of data compression methods

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server