A summary and viewgraphs of a discussion presented at the National Space Science Data Center (NSSDC) Mass Storage Workshop is included. Some of the experiences of the Scientific Computing Division at the National Center for Atmospheric Research (NCAR) dealing the the 'data problem' are discussed. A brief history and a development of some basic mass storage system (MSS) principles are given. An attempt is made to show how these principles apply to the integration of various components into NCAR's MSS. Future MSS needs for future computing environments is discussed

Olear, Bernard T.

English

NASA Technical Reports Server

_I9 _- 15 04_
MASS STORAGE SYSTEM
EXPERIENCES AND FUTURE NEEDS
AT
THE NATIONAL CENTER FOR ATMOSPHERIC
RESEARCH
Summary of the Presentation to the Conference on
Mass Storage Systems and Technologies for
Space and Earth Science Applications
July 23-25, 1991
by
Bernard T. O'Lear
Manager, Systems Programming
Scientific Computing Division
National Center for Atmospheric Research
257
https://ntrs.nasa.gov/search.jsp?R=19930005857 2020-03-17T08:32:51+00:00Z
3/5/91
This is a summary of the presentation given at the Conference on Mass Storage
Systems and Technologies for Space and Earth Science Applications. The presen-
tation was compiled at the National Center for Atmospheric Research (NCAR),
Boulder, Colorado. NCAR is operated by the University Corporation for
Atmospheric Research and is sponsored by the National Science Foundation. Any
opinions, findings, conclusions, or recommendations expressed in this paper are
those of the author and do not necessarily reflect the views of the National Science
Foundation.
258
This presentation is designed to
relate some of the experiences of the
Scientific Computing Division at NCAR
dealing with the "data problem." A brief
history and a development of some basic
Mass Storage System (MSS) principles are
given. An attempt is made to show how
these principles apply to the integration
of various components into NCAR's MSS.
There is discussion of future MSS needs
for future computing environments.
NCAR provides supercomputing and
data processing for atmospheric, oceanic
and related sciences. This service is
provided for university scientists and for
scientists located at NCAR. There is a total
of about 1200 users.
The data problem for this
community can briefly be summarized as
follows; Historical atmospheric data is
archived, programs are saved and the data
which model the atmosphere, oceans and
sun are saved. The NCAR storage
experience is based upon current
supercomputing megaflop rates which
produce a number of terabytes archived
on a yearly basis. There is a history of
data growth and file growth. The NCAR
data storage experience has been as
follows; There are about 500 bytes of
information archived for each megaflop
of computing. When NCAR had an X-
MP/48, the archive rate for the utilized
megaflop compute rate was 3 terabytes
per year. The installation of a Y-MPS/864
increased the archival rate to 6 terabytes
per year. Forecasting future computing
configurations and atmospheric models
being planned we are now approximating
a 30-50 terabyte archive per year rate by
the year 1993 or 1994.
Data has been saved in many forms
over NCAR's existence and then migrated
to machine-readable media. Some of the
data has come from handwritten logs,
from punch cards, half-inch tape. All of
this has been collected and is now
archived on IBM 3480 cartridge tape. One
of the basic principles for archiving this
data is to identify certain classes of data.
Archive data is kept forever. Long-term
data is kept for 10 to 15 years. Near-term
data is kept for 1 month to 1 year and a
category called scratch data is killed after
1 month and cannot be recovered
automatically by the system.
One of the other basic principles
that has been identified is that dataset
sizes continue to grow as a function of
supercomputing sizing. The amount of
data that can be saved is bound in storage
by media capacities. That is, these criteria
are established for determining which
data will be saved and for how long
because there is not an infinite media
capacity at this time. Our experience has
shown that every 10 to 15 years the data
in the MSS will need to be migrated to a
new media base because of changing
systems and obsolescence of existing
media. Usually the media or the drives
cannot be purchased anymore. This
migration takes place not because the data
is bad on the media, but because the drives
will not be available.
Another problem is that a number
of companies have provided the capability
for this massive storage, but the small
companies tend to disappear within five
years. The drive components that have
been furnished for mass data storage
disappear in five to eight years no matter
what company they come from.
The next basic principle is that the
migration of the mass storage system data
to a new media base, which is now several
ten's of terabytes, is not a trivial
operation. The migration does not take
place in a short amount of time. For
instance, one-time migrations can run for
long periods of time, necessarily years to
move terabytes data. It is very difficult to
guarantee that the data is migrated
absolutely without reading it back, which
is time consuming. These migrations are
very costly and in my opinion shouldn't
be done. We have developed the concept
of "DATA OOZE," and we prefer this
technique over migration right now. The
way DATA OOZE works is that it is a
continuous movement of data within the
system. The data is moving across the
2storage hierarchy and across the
changing media types under the control
of the MSS. The migration path for this
data in the hierarchy can be from
memory to solid state disk to high speed
disk to disk arrays or farms, and from
there out to some kind of tape. Later on as
new data storage media become available,
the data is migrated onto these media in
real time, since every day some amount of
the data is migrated as it is being used.
Our conclusions from these
experiences have been that new
components and media types are_
integrated according to the following
rules; Use standard components. The
standards may be real or de facto and
apply in the areas of channels, interfaces,
operating systems, media, etc. We look for
media that is easy to obtain and is cost
effective. We look for the long-term
viability of the vendor and mu|[lpIe
sources for the many system components.
In the area of mass storage system
integration we look at access speeds, ease
of expandability, heterogeneous host
access, maintenance costs, media costs and
systems costs.
There are a number of future
growth issues for the NCAR MSS. The
Scientific Computing Division (SCD)
continues to develop future configuration
scenarios. These scenarios try to
anticipate the functional requirements
we anticipate providing for our scientific
community. There are three key
components we need to address: network
services and access, the large scale
computing (Big Iron), and the data
archives. Of course, these all play within
the context of distributed computing.
The near-term issues for the NCAR
MSS focus on some immediate upgrades
which will deal with the MSS growth for a
couple of years. The entire archive will
be migrated onto double density 3490 and
3490-compatible media. The mid-90s to
late 90s became more interesting because
of the expanding interest in archiving
vast data collections.
The issues of future growth will be
centered in three areas of ongoing
development: the various MSS software
packages, the data storage components
and the networks.
The questions then become how all
of these components get assembled and
which ones do we plan to use. Will SCD be
able to construct on effective peta-byte
MSS by the end of the decade? Which of
our basic principles can we apply to
insure that such a system can be built?
###
26O
f
Mass Storage System
Experiences and Future Needs
at
The National Center for Atmospheric Research
Conference on Mass Storage Systems
and Technologies for
Space and Earth Science AppHcatloos
July 23-25, 1991
Bernard T. O'Lear
Manager, Systems Programming
Scientific Computing Division
National Center for Atmospheric Research
| NCAR Sd_lflto COml_lr_l OIV_
J
The following presentation was compiled at the
National Center for Atmospheric Research
Boulder, Colorado
• The National Center for Atmospherk Research Is operated by the
University Corporation for Atmospheric Research and is sponsored by the
NationaiScience Foundation.
• Any opinions, findings, conclusions, or recommendations expressed
in this talk are those of the author(s) and do not necessarily reflect the views
of the National Science Foundation.
ak4xreoml_li_ll • _. t:_m
261
Introduction
The experiences of a scientific center dealing with
"The Data Problem"
• Brief history
• The current computing environment
• Development of some basic principles
• How the principles apply
• Future needs for future computing environments
_. ! NC:kqSet,ratineC_ D:v_'_n
J
History
NCAR provides supercomputtng and data processing for atmospherk,
oceanic, and related sciences:
• At universities
• At NCAR
. Totals about 1200 users
The data problem for this community
• Save and archive historical atmospheric data
• Save programs and data which model the atmosphere, oceans, and sun
262
The NCAR Storage Experience
• 500 Bytes per million flop
• Arddv_l rmgefor model output
- 4 TBytes/year with X-MP/48
- 8 TBytes/year wtth Y-MPS/864
.40 TBytes for climate simulation
J
NCAiI IkNII-DMa GNmk
IMblJ, H
Im , 147._IRID
m
|
I!"11 s_p_o_po_, c_.._., o,,,- J
263
T14mum
12_Mm
12000m
!
C
t
0
/ NCAR._tentJfk OemDt,'t_ Olv_tm
'_,,,_t..n"'a _,_,__.o.,.. J
NCAR Mass Storage Systems
(MSS)
Usage Data ]
101,700 tape cartridges in use
Over 18.5 Tbytes of data stored
Over 710,000 files
Average filelength26.2MB
s 2 minute delivery
/...p,,c_....,_t_l..._ C___g _DM_i_9..- J
E
fHistory
(Continued)
Data saved in many forms - then migrated to machine readable media:
• Handwritten logs ---> Punched mrds
• Punched cards ..... > One-half inch tape
• One-half inch tape ---> AMPEX TBM tape
• AMPEX TBM tape .... > IBM 3480 tape
• IBM 3480 tape ---> IBM 3490-E tape
• IBM 3490-E tape .... • ? ? ?
/ _ ,s_,_ c_ oh,_o_ J
Basic Principles
Identification of data classes:
• Archive data = keep forever
• Long.term data -- keep 10-15 years
• Near-term data = keep I month to I year
• Scratch data = kill after I month
/._¢ ___,_.__j_N.___c__ 0_..
265
BasicPrinciples(Continued)
• Dataset sizes continue to grow as a function of supercomputer sizing
• Dataset sizes are constrained in Storage by media capacities
• Every ten to fifteen years, the data in the MSS will need to be migrated
to a new media base
Basic Principles
(Continued)
Migration:
• Not because the data is bad on the media..
• But because the drives will not be there
- Haft life of a start-up company = 5 years
• Half life of dflve electroalcs = 5-8 _srs
J
266
Basic Principles
(Continued)
The migration of MS$ contents ("n" tera-bytu) to new media
is not a trivial operation
One-time migrations:
• Run for long periods of tlme (years)
• Are dlfflcult to guarantee
• Are costiy
• Shouldn't be done
_k | _.Mq ,Scle_Wk: ComD_Ir_a [)N_s_ J
Basic Principles
(Continued)
Data OOZE preferred over migration
Data OOZE Is a continuous movement of data within the system:
• Data movement acres:
- The storage hierarchy
- The changing media types
l!
J
267
4480
Optical
Disk
Worm
Memory
Solid State
Disk
High-Speed Disk
Disk Arrays or Farms
Disk
MO
.1/2" tape
Optical Other
Disk Distributed
ROM Systems
| NCAR Sctentmo Com_t_ OM_
CONCLUSIONS
New components and media types are Integrated according to
these rules:
• Standards (real or de facto)
- For channels and Interfaces (IBM, IPI, HIPPI, SCSD
- For media
• Long-term viability of vendor
• Multiple source availability for media (drives?)
J
288
MSS Integration
• Access _eeds (sometimes)
• Ease of expandabillty
• Multiple heterogeneous host access
• Maintenance costs
• Media costs
• System cost
f
FUTURE GROWTH
ISSUES
IN THE
NCAR MASS STORAGE SYSTEM
Functional Diagram
Network Mainframe
Servers Computers
F.tpath I
CRAY
.Mass Y-MPTJ216
F--- storage "-] ,1
Export ] Output Connection
Servers Machine
Local Data
Network (LDN)
Commuakatlon
Systems
I
CRAY
Y-MPa/864
I
I
IBM
Front-end
Local
E AccemRemote
Access
NCARScillalilicCompulingDivision
Mainframe and Server
Network (MASnet)
j
Functional Diagram of the NCAR Computing Complex
[ " ..... .UCAR.EDU" l
l Special Network Services l
l "B!g Iron" Servlc.....__..._ .rid Access /
[ f-Y-MP8 % . Gateways _ Foothills [
// _/-_s.... i -"--'-- L"I
IFastpath I--Y'MP2 _ _, / _ /
I for / _ I ......... Me,* l
[ i--TAGS Servers ---'t-'-_VVlde Ares Net_-'-- Universities l
l - Emall _ l
l - Math IAIN I
i
27O
f FY93-95 Functional Diagram
Data Big
)pen Shelves _-- Shard Mem°ry --_F| ,_ | mum-processor
r _ _ supercomputers
I /
Online dm -- D
[ S [ -dlskarrays
High capacity, ] W ] - robotic HI_
possibly _ | l D
_._ ,c.ceu I x _xlg_r Var,Uel --
tecnuo/ogy i _ [ - Tightly Coupled I
- pied J
i:i_,_'_,_._.. -' _._-
Network Services
and Acceu
Gateways_ _ Foothllla
IRJE Lab
Mess
Lab
Servers -- - Universities
- NFS
-Data
. Documents
- Math Llhs
- Emall
. Output
f
NCAR MSS
Near Term Upgrades
1. Purchase (IBM) 3490E drives for double density capability
2. Automatic double density migration takes place for sheff archive
3. Hope Is STK furnishes double density for drives on ACS In
< 6 months.
ffi | NCAR Scle_tif._ C_ln_ O k_...k___
271
NCAR MSS
The Issues of Future Growth are dependent upon:
1. Future MSS Software
a. Distflbuted MSS
b. Large archives 0Peta-Byte)
2. Future Data Storage
a. The media
b. The drives
e. The robotics
3. The Network and Channels
a. HIPPI
b. Fibre channel standards
c. fabrk (switch)
1. Future MSS Software
a. Distributed MSS
• UNITREE (DISCOS)
• Infinite Storage Architecture (EPOCH)
• Distributed Physical Volume Repository (EPOCH & STK)
• EMASS (E-SYSTEMS)
• NAStore (NASA, Ames)
• NETARC and AWBUS (CDC)
• SWlVI" (IBM)
DataMef, h (Hewlett Packard)
M (DS) ZNASA Goddard
b. Peta-Byte Archives
• How do we build them?
_,_rl _._.p.,_ o..._... _.. J
E
272
f
2. Future Data Storage
a. THE MEDIA
• The 3M National Media Laboratory (Media Database)
Government funds and goals
Private sector participation
• Standards being developed
b. THE DRIVES
Being developed for the medialO-yesr life span
• Attachable to various robotics
c. THE ROBOTICS
StorageTek is the leader
ODETICS
• EXABYTE and others
__]_J NCAR SctomlflcC_lna Dlvb_
_a._._,.,_,.; O_
f
3. The Network and Channels
a. Standards moving fast for HIPPI
• The HIPPI switch
b. Fibrs Channel Advantages
• Length to 10 kilometers
• General Protocol
. HIPPI
- SCSI
- IPI
- Others
• Security
• Immune to Electrkal Dlsturbancs
c. Fabrk Switch
/..K_....AS_..__C._Ion.__O_.. J
273
2"74
[_..! t,_AR Scte.twtcCom_ iot,.4.k,.
(From: ltatrk v-map) j
27e;


Mass storage system experiences and future needs at the National Center for Atmospheric Research

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server