The topics are presented in viewgraph form and include the following: system functionality; characteristics; data sources; hardware/software systems; and performance assessments

Behnke, J.

Halem, M.

Palm, N.

Pease, P.

English

NASA Technical Reports Server

Architectural Assessment of
Mass Storage Systems at GSFC
M. Halem, Code 930.0
J. Behnke, Code 633.1
P. Pease, Code 935.0
N. Palm, Code 931.0
NASA/Goddard Space Flight Center
Greenbelt, MD 20771
599
https://ntrs.nasa.gov/search.jsp?R=19930022791 2020-03-17T05:18:44+00:00Z
Architectural Assessment of
Mass Storage Systems at GSFC
NDADS: National Space Science Data Center
Data Archive and Distribution Service
GDAAC V.0: Earth Observing System Data Information System
Goddard Distributed Active Archive Center
M(DS)2: NASA's Center for Computational Science
Mass Data Storage and Delivery System
by
Dr. M. Haleru, J. Behnke, P. Pease and N. Palm
Space Data and Computing Division
NASA/Goddard Space Flight Center
Presentation for
Goddard Conference on
Mass Storage Systems &
Technologies
NASAJGSFC, Bldg. 8
September 24,1992
OVERVIEW
Background
System Functionality
Characteristics
Data Sources
Hardware/Software Systems
Performance Assessments
Conclusions
BACKGROUND OF MASS DATA STORAGE SYSTEMS
NDADS:
Prototype of the Hubble Space Telescope Data Archive and Distribution
Service (HST-DADS) contracted to Loral AeroSys in 1989. Evolved as the
Astrophysics and Space Physics archiving system for the National Space
Science Data Center to maintain a mix of near and on-line data and manage a
deeper data storage archive
GDAAC/V.0:
EOS prototype archive and distribution systems initiated in FY91 and planned
for operational availability in FY94. One of nine geographically distributed
discipline-oriented interoperable DAAC's
M(DS)2:
A mass storage and delivery system serving more than 1400 users within the
NASA Computational Science Center at Goddard that has to manage both the
high-speed computer-generated simulation data, as well as space-borne
observational data
SYSTEM REQUIREMENTS
NDADS GDAAC V.0 M(DS)2
NEAR-ONLINE 2.6 TB/6 TB 10TB/$TB 7TB/$b'TB
STORAGE/DEEP lSGB DASD 16GB/DASD 240 GB/DASD
SCALABLE UP TO I0 TB/S01"B 18 TB 22S TB/SOO TB
100 GB DASD 100 GB/DASD 3 TB/DASD
INGEST 13 GB/DAY 30 GB/DAY 90 GB/DAY
0_ATE)
DISTRIBUTE loso MB/DAY - NET 1S0 GB/DAY 100 GB/DAY
(RATE) 700 MB/DAY - TAPE
100 PHOTOffDAY
146
CONCURRENT 240 CATALOG QUERIES/HR
USERS
602
100 MIN. 128
MIN. 32 SIMULTANEOUS
FFP TRANSFERS
SYSTEM FUNCTIONAL CHARACTERISTICS
Data and Metadata Functions
Network Access (Ethernet, FDDi, DecNet, UltraNet)
Security
Integrity and Quality Control
Automated Data Migration and Compaction
User Ingest and Retrieval
Remote Orderin8 and Delivery Service
Catalogues and Inventories
Browse - On-Line
NDADS
l
Barrier
GDAAC
g
Barrier
MDSDS
It
RACF/C2
a •
Partial •
Partial Partial •
• X
• z User
lnteroperability z Partial
Database Queries and Subsettin 8 • a
• Partial Partial
l
Partial
Portable Software Operating Systems
Incorruptible Archive
Remote Back-Up/Safe Store
Data Compression
Redundancy (NSPOF) Partial
Scalable Upjtrade
Accounting & Monitoring
• l •
• Partial •
NDADS Data Sources:
Astrophysics:
Wave Length Project DataTypes iGranules An:hived Total Size
High Ener_?
Ultra Violet
Copernicus
Infrared
Radio
Miscellaneous
TOTAL
FXOSAT 10 MK 150 GB
2 10K 100% 4 GB
200 100K 20GB
HEAO-3
HEASARC
EINSTEIN 15K 305 120 GB20
ASTRO-D launch 1993 -- m :1650GB
XTE launch 1996 --
GRO 100 _ , none 50GBl_ear
ROSAT 60 • 100K 30% 100 GB
VELA SB 1 IK 100%
IUE 618 OOK 100%
EUVE launch 1992 -- _ SOGB
HST ' 4 1GB/4GB
COBE 6 looK 0 50 GB
6 I50K 1005/105 5GB+78GBIRAS
3GB
200 GB/TMGB
VLA 1 _ 60% 1OO0 GB
ADC Catalogs Various IK 100% 1GB
2.41 + (1.75)
Space Ph
Mission
SKYLAB
ISTP-GOES
ISTP-IMP8
ISTP-GEOTAIL
Atomic Physi_
DE-!
VO41AGER
_rAL
,sics:
Data Types
1
iGranules
3500
Archived
lOO%
Total Size
10 GB
.4 GB
.4 GB8
S 1.0 GB
2 100 105 1GB
100GB
2GB
114 GB
603
GSFC V0 DAAC Data Volume Requirements (by Project)
q_
e
g
.o
O
20000
18000
16000
14000
12000
10000
8000
6000
4000
2000
0
FY9t
I !
FY92 FY93 FY94 FY95 FY96 FY97
(51_92)
! ATMOS
.g AV_qR
S_WFS
III TOMS2
l_l TOYS
, I LVU_
_1 CO_S
ICZ_
• Pt._
Projec t
Science Project Data Products
Product Description
UARS !.
SeaWiFS
Protlles of IS trace species, temperature, and wind
Solar UV lrrndlance measurements (115 - 400 nm)
Oce_ pigment, chlorophyll a concentrations
5 wilier leaving radiances, 3 aerosol radiances
Diffuse attenuation coefficient
Atlas / l'rofl_ ot 30+ upper atmosphere trace species
ATMOS Upper atmmpherlc temperature profiles
TOMS2
AVHRR
Pathfinder
TOVS
Pathfinder
Total ozone, effective tropospheric refiectivily
6 baclkxatteced UV radiances (313 - 340 nm)
• Binned 5 chmnel clear sky radiances
• Daily ¢lond fraction, height, and refiectivity at 9 km and I degree spatial
resolutions
• Daily, weekly and seasonal surface reflectance NDV! at 9 km resolution
• Dally Surface albedo at 9 km resolution
• Aerocml optical thickness I Iongwave surface flux
• Proflkm of atmospheric temperature, humidity, and geopotential height
• Precipitable water in 6 tropospheric layers, total ozone, and tropopause
preuure
• Surface air lad skin temperatures, 3.7 micron bidirectional surface
reflectance, and S0 GHz sui'face microwave emiulvily
• Cloud fraction, cloud top pressure, precipitation estimate, visible reflectance,
oulgoing lemgwave radiation I and Ionlgwave cloud [orcinlg
Goddard Version 0 DAAC
604
M(DS)2 DATA SOURCES
PROJECTS: NIMBUS/TOMS
ISTP
IUE
GRO
MODELING: DATA ASSIMILATION
COUPLED OCEAN/ATMOSPHERE/STRATOSPHERE
GEODYNAMICS
SPACE PHYSICS PLASMA MODELING
ANALYSIS: TOVS PATHFINDER
ALGORITHM DEVELOPMENT (TRMM, MODIS...)
HST IMAGE DEBLURRING
HPCC: EARTH AND SPACE SCIENCE TESTBEDS
Mass Storage Hardware Systems
NDADS
3VAX641ffs
(14GB/DASD)
• 2 CYGNET WORM Jukeboxes
with 4 SONY drives
GDAAC
• 2 SGI 4D/440
(16 GB DASD)
• 2 CYGNET WORM Jukebox
with 4 ATG 9001 drives (24 TB)
• I Metrum ACS (8.TES) w/
4 drives
• 2 9-track Dec tape drives
• 2 8ram Exabyte tape drives
• 2 4n'_a Winchester tape drives
• 1 I/O power channel 80MB/s
• 5 6250 9 track tape
•4 Exabyte 8 nun tape drives
• 2 3480 tape cartridge
• CD-Rom pre-mastering
• 2 4ram DAT tape drives
MDSDS
• IBM ES 9021/500
(56 ch, 128 MB
• IBM 3980 - (240 GB/DASD)
• Convex 3240 - (512 MB)
• 3 STK 440 (4 grB)
• 1 Dataware WORM Jukebox 3
34.___ (t 2 TB)
• B-Test Helical E-Systems
Tower (8.2 TB)
• 8 ESCON Channel
• Ultrane|
• 40 Memorex 3480
compatible tape drives
605
Z>
0
Z
0
r,_
¢,)
0
k
=
.=
Iold
,<
,<
I,,,,
o
607
National Space Science Data Center
NDADS Computer Cluster
Archlve
VAX
Local Area (thernet
Cygnet 1802 Jukebox
2 SONY WORM drives
51 Idots
334 Olyule
VAX
Cygnet 1803 Jukebox
2 SONY WORM drives
131 slot8
U8 GBytes
CluIIIr inleroonnecl
Host
VAX
StandsIone SONY optical drives Controller devices for
periphtral equipment
accessible cSuster-wida
2 LAOO Tape Drives
'F.XABYTE"
$ Olyle capacity
2 Winchester Tape drives
OAT
s olym capacity
8ram tape
4ram takoe
2 DEC 9*tmck tape drives
11S00/62S0 BPI
13 magnetic disk drives
20 OByles momge
608
Mass Storage Software Systems
Software S_aem
ClienUServer
Operating System
Networks Supported
Dalsbsse/lJbcery Management
System
Physical Storage Device Driver
Seflwsre
User Interface
AUTOMATED RETRIEVALMAIL
NDADS
VAX VMS
DECnet
TCP/IP
SPRINTnet
GDAAC
TCP/IP
FSTAGE/t bl URE
SYBASE
INGRES
JIMS
SOAR
NCDS
ARMS •
NSSDC ARCHIVE
Unitr_
ORACLE
Unitrse
Drivers
NCDS/PLDS
EOSDIS/IMS
MDSDS
MVS, AIX
TCP/IP
Ethernet
BrrNET
NSFNET
InterNet
InterLink
UltraNet
HSM
Unitree
Oracle (Opt)
Dstawsre
STK
Unitree Drivers
USER CONTROL
YSTEM
System Assessments
NDADS GDAAC MDSDS
Strengths: • Project customization
• FTP Accessible
• Distribution of archive
media
• Intelligent data access
and optimization
• Data compression
• Metadata search/browse
• Remote back-up
• Project customization • FTP Accessible
• FIT Accessible worldwide
• Intemperable • Archival/retrieval by
• Metadata search/browse user request
NCDS/PCDS Experience • Intelligent hierarchical
Intelligent Data Mgmt. storage migration
SpatiaiTemp Data Fusion
• Metadata search/browse
• IEEE Mass Storage
Compliant
• Open
• Remote back-up option
• IEEE Mass Storage
Compliant
Weaknesses: • Non-portable systems
• In-house customization
of software and
hardware
• In-house customization
of software
• Embryonic HW/SW
systems
• Embryonic HW/SW
systems
• Costs
6O9
PERFORMANCE CHARACTERISTICS
THESE ARE ALL PRELIMINARY ESTIMATES FOR THE NDADS M(DS)2 AT THIS POINT!!
(DOES NOT INCLUDE PROJECT SPECIFICS)
NDADS
SONY Optical Disk Drives: Read: 600 KB/sec
Write: 300 KBIsec
Actual Rates: Read: 250 KBIsec
Write: 107.S2 KBIsec (averase)
250 KB/sec (max)
Matter load speed: 11 seconds
Data Storage: Current: 120GBytes
Growth: 240 GBytes/year
Storage Input: S GB/day (average), 17 GB/day (max)
Storage Output: 330 files/day, electronically
Inquiries on the archive: 24/day
M(DS)2
DASD: 80 MB/S Throughput
Mass Storage: 18 MB/S Throughput
CONCLUSION
Mass storage systems allow scientists to perform research previously
impossible because of logistic burdens and maintain pace with rapid
data growth arising from increasing computational power and
observational resolution
Mass storage hardware systems technology evolving faster than
software available to integrate into system. IEEE mass storage
standards model changing faster than vendors can keep up with; still
need standards.
Community needs to acquire much more performance test data
reliability, stability and data access speeds across small to large mass
storage systems
Not yet clear whether many small distributed client-servers are more
effective than fewer large-scale client servers
Mass storage management systems need to become more robust and
more stable
610


Architectural assessment of mass storage systems at GSFC

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server