Data management for Mesospheric-Stratospheric-Tropospheric, (MST) radars is addressed. An incoherent-scatter radar data base is discussed in terms of purpose, centralization, scope, and nature of the data base management system

Wickwar, V. B.

English

NASA Technical Reports Server

573 
9. MST RADAR DATA-BASE MANAGEMENT 
(Keynote Paper 
V. B. Wickwar 
SBI Internat ional  
Menlo Park, CA 94025 
The management of data  i s  an important topic and one tha t  has had con- 
s iderable  a t t en t ion  during the l a s t  few years (e.g., CODMAC, 1982; LIDE, 1981; 
JFXNE, 1981; ROEDERER, 1981; USRA, 1978). Indeed, fo r  MST radars,  there  are 
several  reasons f o r  considering it. 
t ion,  the l a rge  amount of data t h a t  are acquired by an MST radar, whether 
operating continuously or i n  campaign mode. 
data among s c i e n t i s t s  a t  the several  radars located around the world. 
the MST technique i s  maturing, there  i s  a l s o  a growing des i r e  by researchers 
using other techniques t o  acquire co r re l a t ive  radar data and a des i r e  by 
theoret ic ians  t o  compare theory and observation. 
appropriate t i m e  t o  discuss the management of MST data. 
The f i r s t  i s  a very p rac t i ca l  considera- 
Another is the des i r e  t o  exchange 
Now t h a t  
Thus I believe t h a t  t h i s  i s  an 
However, I do not want t o  s tep forward a s  an expert i n  the area of data  
management. I w a s ,  nevertheless,  great ly  involved as a ca t a lys t  i n  s e t t i ng  up 
an incoherent-scatter radar data  base a t  NCAR. 
"catalyst" because se t t i ng  up such a data base has t o  be a co l l ec t ive  e f f o r t  of 
s c i e n t i s t s  involved i n  obtaining and using the data  i n  question. 
L e t  me emphasize the word 
What I would l i k e  t o  do I n  t h i s  a r t i c l e  i s  review some o t  the questions 
and concerns t h a t  were involved i n  se t t i ng  up t h a t  data base. That experience 
should be relevant t o  the present concerns because of the s imi l a r i t y  of the 
techniques and the overlap i n  the s c i e n t i f i c  community interested i n  data from 
the two techniques. 
The f i r s t  question i s  the purpose of the data base. Is it LO bring 
together a l l  the data from one type of instrument, or i s  t o  bring together a l l  
the data from any source t h a t  per ta ins  t o  a given s c i e n t i f i c  problem? 
solution w e  chose was a blend of both, but it tended t o  emphasize the former. 
We f e l t  that  c e r t a i n  portions of the radar data could make pa r t i cu la r ly  valuable 
contributions to  the s c i e n t i f i c  f i e l d ,  and therefore we recommended including 
them. 
should a l so  be included. While we did not recommend the inclusion of other 
major data se t s ,  we did recommend t h a t  t h a t  option be l e f t  open and, s imilar ly ,  
t h a t  the option t o  obtain data from other data  bases not be precluded. 
The 
But €or the data t o  be useful,  we f e l t  ce r t a in  geophysical parameters 
The r a t iona le  was t h a t  keeping the data  base l imited addid t o  the l i k e l i -  
hood of having a responsive f a c i l i t y  and one t h a t  would great ly  complement the 
radars  over the years. 
excellence fo r  using the radar data. 
t h i s  type of data base could coatr ibute  eas i ly  and great ly  t o  short-term CDAWS- 
l i k e  exercises where many v a r i e t i e s  of data a r e  brought together t o  study a 
pa r t i cu la r  event or problem. 
We even hoped t h a t  the f a c i l i t y  would become a center of 
In another direct ion,  w e  believed tha t  
The next question w e  considered was the nature of the data  base. Should i t  
be a d i s t r ibu ted  system o r  a central ized one? 
decentralization, we chose a central ized f a c i l i t y .  
two r e a l i t i e s :  
d i f f e ren t  countries or ,  even worse, on d i f f e ren t  continents, and (2) the expense 
of having a l l  the data on l i n e  a l l  of the t i m e  or even portions of the data  
sh i f t ed  around on a f ixed schedule. 
We f e l t  there would be more uniform qua l i ty  control i f  a l l  the data passed 
I n  t h i s  age of networking and 
This came about because of 
(1) the present cost  of having a network l inking radars i n  
It a l s o  came about for  pos i t i ve  reasons. 
https://ntrs.nasa.gov/search.jsp?R=19840019107 2020-03-22T09:25:19+00:00Z
574 
through a common gateway. 
posi t ion t o  t r a i n  and help data-base users,  t o  be able  t o  assemble catalogs of 
observations, e tc .  
We f e l t  tha t  a cen t r a l  f a c i l i t y  would be i n  a be t t e r  
Another aspect of the nature  of the data base i s  whether it should be on 
l i n e  or not. 
and i t  was not clear tha t  i t  was necessary. The data  could be kept on magnetic 
tape and then promoted t o  disks when requested. 
f o r  a few'days i n  case it w a s  looked a t  again. The delay i n  promoting a tape 
would not a f f e c t  most users. 
useful fo r  s e l ec t ing  data ,  such as a catalog, should be kept on l i n e  fo r  easy 
browsing. 
While having the data  on l i n e  would be very nice,  it i s  expensive; 
It could then be l e f t  on disk 
However, i t  w a s  f e l t  t ha t  something s m a l l  and very 
Y e t  another aspect of the nature  of the data  base was the nature of the 
data-base management system. Almost a l l  computer systems have data-base manage- 
ment systems. However, while they may be excellent for  keeping t rack of mailing 
lists., inventory control , or customer accounts i n  f inanc ia l  i n s t i t u t ions ,  they 
a re  not w e l l  adapted t o  s c i e n t i f i c  data. Among other reasons, the data 
formatting may vary very great ly  from radar t o  radar,  or even fo r  a single  radar 
Another i s  t h a t  re la t ionships  may ex i s t  with a 
series of' points instead of on a one-to-one basis. However, the data can be 
handled simply with a f a r  simpler f i l e  management system. 
*from one experiment t o  the next. 
Having examined the nature  of the data base, the next question was what 
data t o  iaclude. 
data. 
data. 
radar s i t e .  Thus, w e  f e l t  t h a t  they should be kept a t  the radar locat ion and 
not,  a t  least i n i t i a l l y ,  made par t  of the data  base. However, magnetic tapes 
do de te r io ra t e  with t i m e ,  and so it seems advisable tha t  those tapes should be 
copied o p  t o  new tapes o r  i n t o  new storage media i n  the fu tu re  i f  they are t o  
be saved. 
d i r ec t ly  from the power and spec t r a l  shape as w e l l  as from combining line-of- 
s ight  v e l o c i t i e s  i n t o  vectors. 
independent of the radars and are used d i r ec t ly  i n  s c i e n t i f i c  studies o r  used t o  
der ive a higher l eve l  of parameter, they are the ones submitted t o  the data  
base. Final ly ,  the next higher l eve l  of data could then be derived i n  a uniform 
manner from data from any of the radars. They would be best  derived by the 
users  with t h e i r  own programs or those avai lable  a t  the cen t r a l  f a c i l i t y .  So, 
essen t i a l ly ,  our r a t iona le  i n  deciding how t o  treat the various l eve l s  of data 
w a s  based on what could best  be done a t  the most appropriate location. 
This had two major aspects. The f i r s t  was the level  of the 
F q r  example, there  are the raw data  tha t  are recorded on tape f o r  
However, these data  can only be reduced t o  geophysical parameters a t  the 
. subsequent reduction. They a r e  very radar-specific and comprise the bulk of the 
The next l eve l  of data  consists of geophysical parameters derived 
Because a t  t h i s  l eve l  the parameters a r e  
The second aspect of what data t o  include had t o  do with whether the data 
base should include data from a l l  experiments or from only a subset. 
decision was t o  include a l l  long experiments t h a t  observed standard ionospheric 
parameters. 
usable by those other  than the o r ig ina l  observers. 
appropriate t o  include s tudies  of special  parameters or of new a l t i t u d e  regions. 
Often such s tudies  are experimental i n  nature, i.e., aimed a t  extending the  
technique, or are of l imited i n t e r e s t  t o  the wider s c i e n t i f i c  community. 
The 
The r a t iona le  w a s  t h a t  these longer experiments would be the most 
Similarly,  it w a s  f e l t  not 
Having decided on what data  t o  include, our next area of concern w a s  with 
what tape format t o  use for  t r ans fe r r ing  data.  
t h a t  w a s  w e l l  specif ied y e t  v e r s a t i l e ,  one t h a t  would handle any type of radar 
data -- present or fu tu re  -- plus other types of co r re l a t ive  data. To do tha t  
the format a l so  had to  be largely self-documented. 
three types of records. The f i r s t  i s  a protocol record t h a t  uses ASCII 
characters t o  describe the experiment and data. 
t h a t  i d e n t i f i e s  the radar,  date ,  t i m e ,  pointing direct ions,  e tc .  Following the 
The a i m  was t o  adopt a format 
The r e s u l t  w a s  a format with 
The second i s  a header record 
5 75 
header is the t h i r d  type, a da ta  matrix of values and iden t i f i e r s .  The second 
and t h i r d  records are wr i t t en  i n  twos-complement 16-bit integers.  
machinery was moving forward t o  e s t ab l i sh  the data  base, the th ree  radars 
involved i n  Project  MITHRAS (Chatanika, Millstone H i l l ,  and EISCAT) implemented 
the tape format and used it fo r  exchanging data. 
While the 
Although the foregoing considerations have largely been concerned with 
technical aspects of the data base, there  are some very important human con- 
s iderat ions t h a t  must not be forgotten.  For the data  base t o  be useful,  the 
users  must have confidence i n  the qua l i t y  of the data or have a mechanism for  
assuring themselves of t h a t  qual i ty .  Similarly,  i f  they do not understand how 
some of the geophysical parameters were derived or what t h e i r  l imitat ions are ,  
they need a mechanism for  finding t h a t  out. 
with the radars need a Compelling reason f o r  reformatting t h e i r  data  and sub- 
mit t ing it  i n  a timely fashion t o  the data base. Although data  acquired with 
publ ic  funds usually have to  be made avai lable ,  it was not f e l t  t h a t  coercion 
w a s  the best method to  ensure data  qual i ty .  Instead, i n  recognition of the 
e f f o r t  put i n  by the s c i e n t i s t s  working with the radars,  i t  was f e l t  t h a t  the 
"carrot" was the best  approach. 
pioneered by the Atmospheric Explorer team, was developed t o  set out the 
r e spons ib i l i t i e s  of the providers and the users. The c r i t i c a l  point was tha t ,  
i n  exchange f o r  providing the data  i n  a timely fashion and f o r  answering 
questions about qua l i t y  and derivation, the appropriate s c i e n t i s t  working with 
the radar could be offered coauthorship on r e su l t i ng  papers and reports.  
While it is implied i n  the foregoing paragraphs tha t  there  would be 
appropriate people a t  t he  f a c i l i t y  housing the data base who would be entering 
the data,  the committee f e l t  that  fo r  the da t a  base t o  prosper and evolve, there  
had t o  be a t  l e a s t  one s c i e n t i s t  associated with it who would, among other 
things, use the data. H e  or she would therefore  be intimately aware of the 
programs to access the data,  plot ,  derive higher l eve l  parameters, and compare 
them t o  advanced models. 
those types of software. 
outside s c i e n t i s t s  and v i s i t o r s  contributing addi t ional  software. 
Analogously, the s c i e n t i s t s  working 
A s e t  of "Rules of the Road", such as  those 
He or she would a l so  pa r t i c ipa t e  i n  the development of 
The e f f o r t s  of t h i s  person would be supplemented by 
The del iberat ions t h a t  led t o  the establishment of the data base were 
conducted by a committee careful ly  selected t o  include representatives of each 
radar,  the user community, and experts i n  data-base systems. It was strongly 
f e l t  t ha t  some such committee had t o  be establ ished on a permanent bas i s  t o  
assess how the data  base w a s  developing and t o  guide i t s  evolution. 
I n  conclusion, the management of data i s  exceedingly important, 
par t icular ly  i n  those d i sc ip l ines  t h a t  have t o  r e ly  on extensive observations 
instead of controlled experiments. 
base" i s  a very appealing one. 
issues  t h a t  came up during the meeting on the formation of an incoherent-scatter 
data base have convinced you t h a t  the doctor cannot w r i t e  a simple prescr ipt ion 
f o r  "one data  base". Considerable thought, discussion, and ac t ive  involvement 
are required of the whole community. L e t  us continue t o  think, discuss,  and 
become involved during the remainder of t h i s  workshop and beyond. 
ACKNOWLEDGMENTS 
The solut ion of put t ing data  i n  a "data 
However, I hope my descriptions of some of the 
The other par t ic ipants  i n  the Incoherent-Scatter Radar Data Base Workshop 
were D. Alcayde, J. Baron, R. A. Behnke, 0. de l a  Beaujardiere, B. A. Emery, 
C. Gonzales, T. Hagfors, J. M. Holt, B. L. Jenne, D. H. Joseph, M. C. K e l l y ,  
L. A. Lee, W. M. MacIntyre, E. S. Oran, A. D. Richmond, R. G. Roble, 
R. W. Schmk, P. Shames, J. Silen,  W. Swartz, and R. F. Woodman. Support fo r  
pa r t i c ipa t ion  i n  t h i s  workshop was provided by SRI Internat ional .  
576 
REFERENCES 
CODMAC (Committee on Management and Computation) (19821, Data Management and 
Computation, Volume 1 : Issues and Recommendations, Space Science Board, 
National Academy Press, Washington. 
Jenne, R. L. (1981), Strategies  t o  develop and access l a rge  sets of s c i e n t i f i c  
data,  I n  report  of conference on "Frontiers i n  Data Storage Retrieval and 
Display", National Geophysics and Solar-Terrestrial  Data Center, EDIS/NOAA, 
Boulder. 
Lide, D. R., Jr. (19811, C r i t i c a l  data for c r i t i c a l  needs, Science, 212, 
13 43 -13 49. 
Roederer, J. G. (19811, Considerations i n  the development of a national 
geophysical data  policy, EOS, 62, 569-570. 
USRA (Universit ies Space Research Association) (19781, Proposal f o r  a national 
data center  t o  promote data analysis  and theoret ical  research i n  solar  
terrestrial physics, Final  Report, USRA Data Analysis Panel, Houston. 


MST radar data-base management

Wickwar, Vincent B.

DigitalCommons@USU

Undetermined

MST Radar Data-Base Management

https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=1353&amp;context=physics_facpub

MST radar data-base management

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server

DigitalCommons@USU