This presentation is designed to relate some of the experiences of the Scientific Computing Division at NCAR dealing with the 'data problem'. A brief history and a development of some basic Mass Storage System (MSS) principles are given. An attempt is made to show how these principles apply to the integration of various components into NCAR's MSS. There is discussion of future MSS needs for future computing environments

Olear, Bernard T.

English

NASA Technical Reports Server

N93-18138
MASS STORAGE SYSTEM
EXPERIENCES AND FUTURE NEEDS
AT
THE NATIONAL CENTER FOR ATMOSPHERIC
RESEARCH
Summary of the Presentation to the Conference on
Mass Storage Systems and Technologies for
Space and Earth Science Applications
July 23-25, 1991
by
Bernard T. O'Lear
Manager, Systems Programming
Scientific Computing Division
National Center for Atmospheric Research
1-I63
https://ntrs.nasa.gov/search.jsp?R=19930003950 2020-03-17T10:15:12+00:00Z
3/5/91
This is a summary of the presentation given at the Conference on Mass Storage
Systems and Technologies for Space and Earth Science Applications. The presen-
tation was compiled at the National Center for Atmospheric Research (NCAR),
Boulder, Colorado. NCAR is operated by the University Corporation for
Atmospheric Research and is sponsored by the National Science Foundation. Any
opinions, findings, conclusions, or recommendations expressed in this paper are
those of the author and do not necessarily reflect the views of the National Science
Foundation.
1-164
This presentation is designed to
relate some of the experiencesof the
Scientific Computing Division at NCAR
dealing with the "data problem." A brief
history and a developmentof some basic
Mass Storage System (MSS) principles are
given. An attempt is made to show how
these principles apply to the integration
of various componentsinto NCAR's MSS.
There is discussionof future MSS needs
for future computing environments.
NCAR provides supercomputingand
data processing for atmospheric, oceanic
and related sciences. This service is
provided for university scientists and for
scientistslocated at NCAR. There is a total
of about 1200 users.
The data problem for this
community can briefly be summarizedas
follows; Historical atmosphericdata is
archived, programs are saved and the data
which model the atmosphere,oceans and
sun are saved. The NCAR storage
experience is based upon current
supercomputing megaflop rates which
produce a number of terabytes archived
on a yearly basis. There is a history of
data growth and file growth. The NCAR
data storage experience has been as
follows; There are about 500 bytes of
information archived for each megaflop
of computing. When NCAR had an X-
MP/48, the archive rate for the utilized
megaflop compute rate was 3 terabytes
per year. The installation of a Y-MP8/864
increased the archival rate to 6 terabytes
per year. Forecasting future computing
configurations and atmospheric models
being planned we are now approximating
a 30-50 terabyte archive per year rate by
the year 1993 or 1994.
Data has been saved in many forms
over NCAR's existence and then migrated
to machine-readable media. Some of the
data has come from handwritten logs,
from punch cards, half-inch tape. All of
this has been collected and is now
archived on IBM 3480 cartridge tape. One
of the basic principles for archiving this
data is to identify certain classes of data.
Archive data is kept forever. Long-term
data is kept for 10 to 15 years. Near-term
data is kept for 1 month to 1 year and a
category called scratch data is killed after
1 month and cannot be recovered
automatically by the system.
One of the other basic principles
that has been identified is that dataset
sizes continue to grow as a function of
supercomputing sizing. The amount of
data that can be saved is bound in storage
by media capacities. That is, these criteria
are established for determining which
data will be saved and for how long
because there is not an infinite media
capacity at this time. Our experience has
shown that every 10 to 15 years the data
in the MSS will need to be migrated to a
new media base because of changing
systems and obsolescence of existing
media. Usually the media or the drives
cannot be purchased anymore. This
migration takes place not because the data
is bad on the media, but because the drives
will not be available.
Another problem is that a number
of companies have provided the capability
for this massive storage, but the small
companies tend to disappear within five
years. The drive components that have
been furnished for mass data storage
disappear in five to eight years no matter
what company they come from.
The next basic principle is that the
migration of the mass storage system data
to a new media base, which is now several
ten's of terabytes, is not a trivial
operation. The migration does not take
place in a short amount of time. For
instance, one-time migrations can run for
long periods of time, necessarily years to
move terabytes data. It is very difficult to
guarantee that the data is migrated
absolutely without reading it back, which
is time consuming. These migrations are
very costly and in my opinion shouldn't
be done. We have developed the concept
of "DATA OOZE," and we prefer this
technique over migration right now. The
way DATA OOZE works is that it is a
continuous movement of data within the
system. The data is moving across the
1-165
storage hierarchy and across the
changing media types under the control
of the MSS. The migration path for this
data in the hierarchy can be from
memory to solid state disk to high speed
disk to disk arrays or farms, and from
there out to some kind of tape. Later on as
new data storage media become available,
the data is migrated onto these media in
real time, since every day some amount of
the data is migrated as it is being used.
Our conclusions from these
experiences have been that new
components and media types are
integrated according to the following
rules; Use standard components. The
standards may be real or de facto and
apply in the areas of channels, interfaces,
operating systems, media, etc. We look for
media that is easy to obtain and is cost
effective. We look for the long-term
viability of the vendor and multiple
sources for the many system components.
In the area of mass storage system
integration we look at access speeds, ease
of expandability, heterogeneous host
access, maintenance costs, media costs and
systems costs.
There are a number of future
growth issues for the NCAR MSS. The
Scientific Computing Division (SCD)
continues to develop future configuration
scenarios. These scenarios try to
anticipate the functional requirements
we anticipate providing for our scientific
community. There are three key
components we need to address: network
services and access, the large scale
computing (Big Iron), and the data
archives. Of course, these all play within
the context of distributed computing.
The near-term issues for the NCAR
MSS focus on some immediate upgrades
which will deal with the MSS growth for a
couple of years. The entire archive will
be migrated onto double density 3490 and
3490-compatible media. The mid-90s to
late 90s became more interesting because
of the expanding interest in archiving
vast data collections.
The issues of future growth will be
centered in three areas of ongoing
development: the various MSS software
packages, the data storage components
and the networks.
The questions then become how all
of these components get assembled and
which ones do we plan to use. Will SCD be
able to construct on effective peta-byte
MSS by the end of the decade? Which of
our basic principles can we apply to
insure that such a system can be built?
###
1-166
_m
Mass Storage System
Experiences and Future Needs
at
The National Center for Atmospheric Research
Conference on Mass Storage Systems
and Technologies for
Space and Earth Science Applications
July 23-25, 1991
Bernard T. O'Lear
Manager, Systems Programming
Scientific Computing Division
National Center for Atmospheric Research
! NCAR_l=fi¢_
J
The following presentation was compiled at the
National Center for Atmospheric Research
Boulder, Colorado
• The National Center for Atmospheric Research Is operated by the
University Corporation for Atmospheric Research and Is sponsored by the
National Science Foundation.
• Any opinions, findings, conclusions, or recommendations expressed
in this talk are those of the author(s) and do not necessarily reflect the views
of the National Science Foundation.
r"_ _._. c_=_=,. _,,.
1-167
Introduction
The experiences of a scientific center dealing with
"The Data Problem"
• Brief history
• The current computing environment
• Development of some basic principles
• How the principles apply
• Future needs for future computing environments
J
History
NCAR provides supercomputlng and data processing for atmospheric,
oceanic, and related sciences:
• At universities
• At NCAR
- Totals about 1200 users
The data problem for this community
• Save and archive historical atmospheric data
• Save programs and data which model the atmosphere, oceans, and sun
I ..._.._.n..__._tff_ _ ._L_.__.
4
J
1-168
=
The NCAR Storage Experience
• 500 Bytes per million flop
• Archival rate for model output
- 4 TBytes/year with X-MP/48
- 8 TBytes/year with Y-MPS/864
- 40 TBytes for climate simulation
J
f
t_l MU-D_a Or_Q
lmM-l_l
147345_t
T
b
I
t
|
1-169
f 7
NCAI MS/-PJs
| NCAR Scientific Computl_ DN4=ion J
NCAR Mass Storage Systems
(MSS)
Usage Data ]
101,700 tape cartridges in use
Over 18.5 Tbytes of data stored
Over 710,000 flies
Average file length 26.2 MB
Fast Path ]
< 2 minute delivery
_r_l s,.w_p,_, c_,,,,_, cw J
1-170
fHistory
(Continued)
Data saved In many forms -- then migrated to machine readable media:
• Handwritten logs ...... • Punched cards
• Punched cards ...... • One-half Inch tape
• One-half Inch tape ...... • AMPEX TBM tape
• AMPEX TBM tape ...... • IBM 3480 tape
• IBM 3480 tape ...... • IBM 3490-E tape
• IBM 3490-E tape ..... • ? ? ?
| NCAR Scientific Cotnl_in_l Divest, J
Basic Principles
• Identification of data classes:
• Archive data = keep forever
• Long.term data = keep 10-15 years
• Near-term data = keep 1 month to I year
• Scratch data = kill after I month
1-171
fBasic Principles
(Continued)
• Dataset sizes continue to grow as a function of supercomputer sizing
• Dataset sizes are constrained in Storage by media capacities
• Every ten to fifteen years, the data in the MSS will need to be migrated
to a new media base
Basic Principles
(Continued)
Migration:
• Not because the data Is bad on the media...
• But because the drives wiU not be there
- Half life of a start-up company = 5 years
- Half life of drive electronics = 5-8 years
/ NCA_ Sc_tmc C_pl_.
I"D _.,_,,_•_m,..,_,.._,,,
1-172 i
_z
Basic Principles
(Continued)
The migration of MSS contents (" n" tera.bytes) to new media
is not a trivial operation
One.time migrations:
• Run for long periods of time (years)
• Are difficult to guarantee
• Are costly
• Shouldn't be done
_ NCAR Scientific C utin Division
Basic Principles
(Continued)
Data OOZE preferred over migration
Data OOZE is a continuous movement of data within the system:
• Data movement across:
- The storage hierarchy
- The changing media types
_, / ___N____ff._.._..L_ C.._._t P_....
J
1-173
Memory
State
Disk
High-SpeedDisk
Disk Arrays or Farms
4480 .1/2" tape
Optical Optical Other
Disk Disk Disk Distributed
Worm MO ROM Systems
ABYTE)
I NCAR Scientlf_ComputlngDiv_slon
f
CONCLUSIONS
New components and medin types are Integrated according to
these rules:
• Standards (real or de facto)
- For channels and Interfaces (IBM, IPI, HIPPI, SCSI)
- For medin
• Long-term vlablUty of vendor
• Multiple source avallablUty for media (drives?)
| ..__.._A_n__lf_ Corr_l_ O___
J
1-174
!
i
MSS Integration
• Access speeds (sometimes)
• Ease of expandabillty
• Multiple heterogeneous host access
• Maintenance costs
• Media costs
• System cost
J
FUTURE GROWTH
ISSUES
IN THE
NCAR MASS STORAGE SYSTEM
1-175
Functional Diagram
Network Mainframe Communication
Servers Computers Systems
[ [ [ I [
r V=tpath ] I
CRAY CRAY
Mass Y-MP2/216 Y-MPS/864 Local
I Exp°rt I Output }Remote
Connection IBM "- Access
Servers Machine Front-end
Local Data
Network (LDN)
Mainframe and Server
Network (MASnet)
i_ NCARSoentificComputingDivision
J
F
Functional Diagram of the NCAR Computing Complex
" .UCAR.EDU"
Special Network Services
"Big Iron" Services and Access
F Y -MP8 _ . G ateways--7
/ :_GS / _F°°thlusLab
Data L::: _ervers .,._
S --_w=ue _rea _'_e_r Universities
. Emaii
- Math Llbs
. Documentation
_4_ | NCAR Scientific C utin Oivi$1orl
1-176
f FY93-95 Functional Diagram
Data Big
Archives Iron
A
Open Shelves----_n _Shared Memory _ --
I,f P multi-processor F
High capacity,
possibly j
SlOW access
technology
supercomputers
P
I
Online data
S - disk arrays
W - robotic llbs
I
T |_Highly Parallel
C / - Tightly Coupled
- Loosely Coupled
_ | NCAR SclenllflcC k_ DM$1on
[_ _q_ceeml_ll _ • Gommuc_c.a_l • _la
D
D
Network Services
and Access
Gateways_ _ Foothills
IRJE Lab
Mesa
Lab
Servers -- - Universities
I - NFS
- Data
-Documents
U - Math Llbs- Emall
- Output
J
NCAR MSS
Near Term Upgrades
1. Purchase (IBM) 3490E drives for double density capability
2. Automatic double density migration takes place for shelf archive
3. Hope is STK furnishes double density for drives on ACS in
< 6 months.
1-177
NCAR MSS
The Issues of Future Growth are dependent upon:
1. Future MSS Software
a. Distributed MSS
b. Large archives (Peta-Byte)
2. Future Data Storage
a. The media
b. The drives
c. The robotics
3. The Network and Channels
a. HIPPI
b. Fibre channel standards
c. fabric (switch)
| NCAR ,e_ctentffic Computing Dlvll.lon
1. Future MSS Software
a. Distributed MSS
• UNITREE (DISCOS)
• Infinite Storage Architecture (EPOCH)
• Distributed Physical Volume Repository (EPOCH & STK)
• EMASS (E-SYSTEMS)
• NAStore (NASA, Ames)
• NETARC and AWBUS (CDC)
• SWIFI' (IBM)
DataMe.sh (Uewlett Packard)
M (DS) :_NASA Goddard
b. Peta-Byte Archives
• How do we build them?
1-178
_2. Future Data Storage
a. THE MEDIA
• The 3M National Media Laboratory (Media Database)
• Government funds and goals
• Private sector participation
• Standards being developed
b. THE DRIVES
• Being developed for the media
• 10-year life span
• Attachable to various robotics
c. THE ROBOTICS
• StorageTek is the leader
• ODETICS
• EXABYTE and others
| NCAR Scientific
U
J
f
3. The Network and Channels
a. Standards moving fast for HIPPI
• The HIPPI switch
b. Fibre Channel Advantages
• Length to 10 kilometers
• General Protocol
- HIPPI
- SCSI
- IPI
- Others
• Security
• Immune to Electrical Dlsturbance
c. Fabric Switch
J
1-179
..-q
[COMPUTER?
!
DEVICE DEVICE
| NCAFI ,_lent_ric C(x_u_lna Oivislon
(]From: l_Izlc .$a_)
f 25 MB/s 25 MB/s 25 MB/s 25 MB/s
[DEVICE] [ DEVICE ] [ DEVICE ] f DEVICE
\\
_[ CONCENTRATOR 1
It
DEVICE DEVICE
_%_ __. | NCAFI 5ctenlfftc Co_uling Ot_ti,m_
(From: Patrk Sa_,qe)
1-180
i
=_
f -'x"
[ WORK] [ ER ER I [tILE ]
[ STATION J t ] " t SERVER J
-i,' _">"P// ._LE --I,[ SOFTCOPY ] _ _ ] SERVER
It " ._
[SPECIALIZED_ "'_ I COMPUTE?SERVER
_ w
I ER I
| NCAR Scientific CornDutina Divt$1on
(From: Pmtrk _vap) t
1-181



Mass storage system experiences and future needs at the National Center for Atmospheric Research

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server