In order to meet the data research requirements of the Safety, Reliability & Quality Assurance activities at Kennedy Space Center (KSC), a new computer search method for technical data documents was developed. By their very nature, technical documents are partially encrypted because of the author's use of acronyms, abbreviations, and shortcut notations. This problem of computerized searching is compounded at KSC by the volume of documentation that is produced during normal Space Shuttle operations. The Centralized Document Database (CDD) is designed to solve this problem. It provides a common interface to an unlimited number of files of various sizes, with the capability to perform any diversified types and levels of data searches. The heart of the CDD is the nature and capability of its search algorithms. The most complex form of search that the program uses is with the use of a domain-specific database of acronyms, abbreviations, synonyms, and word frequency tables. This database, along with basic sentence parsing, is used to convert a request for information into a relational network. This network is used as a filter on the original document file to determine the most likely locations for the data requested. This type of search will locate information that traditional techniques, (i.e., Boolean structured key-word searching), would not find

Gross, David L.

English

NASA Technical Reports Server

METHODS AND MEANS USED IN PROGRAMMING INTELLIGENT
SEARCHES OF TECHNICAL DOCUMENTS N9 5 5
David L. Gross
Computer Engineer
Analex Space Systems, Inc.
NASA Kennedy Space Center
Post Office Box 21206
Kennedy Space Center, FL 32815-0206
Phone: 407/861-5716
Fax: 407/861-5T/4
/
/
ABSTRACT
In orde_ to meet the data research requirements of the Safety, Reliability & Quality Assurance a_tivities at
Kennedy Space Center (KSC), a new computer search method for technical data documents was developed. By their
very nature, technical documents _ partially encrypted because of the author's use of acronyms, abbreviations, and
shortcut notations. This l_oblem of computeaiz_! searching is compounded at KSC by the volume of documentation
that is produced during normal Space Shuttle operations. The Centralized Document Database (CDD) is designed
to solvethis problem. It provides a common interface to an unlimited number of files of various sizes, with the
capability to perform many diversified types and levels of data searches. The heart of the CDD is the nature and
capability of its se.azch algorithms. The most complex form of search that the program uses is with the use of a
domain-stgcific database of acronyms, abbreviations, synonyms, and word frequency tables. This database, along
with basic sentence parsing, is used to convert a request for information into a relational network. This network is
used as a filter on the original document file to determine the most likely locations for the data requested. This type
of search will locate information that traditional techniques, (i.e., Boolean structured key-word searching), would not
fred.
INTRODUCTION
The need to search technical documentation for desired information is a labor intensive activity. In the past,
data se.m,ches have been restricted to human effort with limited compute_"searching, (generally Boolean key-word
searching). This is primarily due to the type of information that is being searched and referenced. Technical
documents are partially encrypted by the author's use of acronyms, abbreviations, and shortcut notations. At
Kennedy Space Center (KS(::), this problem is magnified further. A researcher who is searching for information
based on an engineer's or a technician's notes is faced with notes that are usually more encrypted and/or abbreviated
than those which are contained in the actual document. The problem is further compounded by the volume and
dispersal of documentation that is IX_luc_ during normal shuttle operations. The CDD addresses these problems.
The commercial potential of this system is evident from the savings in man-hours alone. Any profession that devotes
time to specific subject review and research would benefit greatly from this time-saving system, (e.g. legal, medical,
information specialist, etc.).
BACKGROUND
In 1990 NASA funded a project to improve the data retrieval and dissimilation methods used by the Safety,
Reliability & Quality Assurance (SR&QA) directorate. Systems and quality asstrance reviews were identified as
likely candidates for improvement. This procedure requires accessing a large number of technical documents and
uses a large percentage of available man-hours. A project was initiated to develop a more time-efficient method of
doing these searches. Several commercial package, were evaluated, but none met SR&QA's needs. Finally, a
decision was made to develop custom software.
55
https://ntrs.nasa.gov/search.jsp?R=19930012966 2020-03-17T06:36:26+00:00Z
Software algorithms f_3m the Artificial Intelligence (AD field were used in an attempt to duplicate human
sem'ch methods. The tlaee methods that showed promise were:
1. Sentence parsing used in natural language processing
2. Confidence factors or weights from hem'istic searching
3. Network connection and propagation from connectionism
Parsing analyzes the syntactic structure of sentences. To adapt this technique to technical data queries,
parsing is used to identify word and phrase relationships such as subject-verb, verb-object, and noun-modifier (Figure
1). The parser uses knowledge of language syntax, morphology, and semantics. In technical document searches,
sentence parsing is used to identify word types, (i.e. noun, verb, adjective) based on the context in which the
abbreviation or acronym is used.
motm_ian
0smm_)
todw_lolm't
Irmpected the crane
Figure 1. Sentence Parsing
Confidence factors or weights are normally used to measure the confidence level in a rule-based system.
These factors combine to usefully measure uncerlamty. In the CDD they are used to measure the probability that
a given search parameter is correcL For instance, if a search query uses an acronym that has two possible meanings,
a seaxch for both would be performed using a lower weighing value than if the acronym had only one possible
meaning.
Network connection and propagation refers to the construction of multi-layer networks and how the weights
of the nodes ate paUemed. In this search technique, the nodes of the network are words (or plLrases)with the pattern
of the weighing determined by a set of hemstic rules. This network then can be used to develop a set of conditions
that evaluate each area of a document.
METHODOLOGY
In any highly developed field, especially a highly technical one, there are a number of words, phrases, and
acronyms that have specific meanings. These can be considered a spec_ knowledge base for that particular
field. Developing an intelligent search system for a specialized field must utilize that knowledge base, along with
more general informationof the English language.
In developing this knowledge base for NASA operations, a general database of acronyms, abbreviations,
and synonyms was used as a st_ing point. Specialized acronyms and abbreviations used in normal shuttle
operations we='eadded to this database. In addition, word frequency tables were developed to identify"the most
commonly used words.
56
The first step in processing a query is to break down the sentence structure. Initially, the sentence or
sentences are separated into individual word objects. These prime words form the first level nodes of the filter, with
the order of the words maintained through the links between nodes (see Figure 2). The node object includes
weighing variables for the word and for the links between nodes. The weighing variables for the prime nodes are
set to a benchmark reference value of 100.
Wllat the ehuldo ixoc. for HF to the 8RB7
I I I
I f I
l ! t
I I I
I I I
I I I
I f I
I ! I
! I l
I I I
I I I
I # I
I I I
I I II
I/ ,
! I
I I
t I
I !
I I
I I
I I
f _
T-N
I I I I
I I ! I
I ! I I
I I I I
I I ! I
I I I !
I I I I
I ! I !
I I I I
I I I I
I I I I
I I I
I I II !
I I
I I
I I
I I
I !
I f
I I
t
I
I
t
I
t
t
I
t
I
t
I
I
t
t
t
t
I
Figure 2. Generation of Prime Nodes
The knowledge base is used to expand the single level of prime nodes into a multilevel node network. Each
word in the first level is referenced in the knowledge base. If a match is found, the reference values from the
knowledge base become new nodes at a lower level. For example, in Figure 3, the node with the value "SRB" is
matched in the acronym table with the value "Solid Rocket Booster". This value then becomes a new sub-node with
links to the same nodes that "SRB" has. If more than one value is found, then more than one sub-node is created
for each prime node.
In Figure 3, the parsing function identifies the prime node "PROC." as being used as a noun. A sub-node
of this produced from abbreviation tables is "PROCEED," a verb. Since the word types do not match, the sub-node,
"PROCEED" can be eliminated along with any synonyms produced from iL Eliminating the node this way would
require assuming that the original query was structured syntactically correct. An alternate method is to reduce the
weight of that node to indicate a much lower probability.
57
SHUTDOWN l _-R-O-_ J--'H-_
• .v . .v • _
, .,-'''° °'°" o," "'.
A'.'.'.'... _ ..... A._ _ _ _ _. "4
• " ........................ ,I
STOPPAGE ]W-eO ]
"r--N J
T-N |
PROCEDURE !)
:,
3
,!
:l
3
.!
"!
3
'1
L _j :,
3
SEQUENCE
I w.m
I T,-,N
METHOD I
W-80
TmN
HYDRAUUCFLUIDI
T,mN
JSOUO w-loo I
nOC_[T BOO6TER
T,,,N
L ...........
Figure 3. Generation of Sub-Nodes
After this multilevel network is created, a set of heuristi._crules Is used fo_ setting the weights for each node.
If the conditional part of the rule tests true, then the rule sets the weights of Ihatsub-node, and can adjust the weight
for the prime node(s) of the sub-node. Table 1 fists some of the heuristic rules used to set these values.
A Boolean search condition is then generated for every node in the neL This se_,ch condiuon is of the type:
if <node> exists then VALUE = VALUE + WEIGHT
The list of these conditions forms the filter function. The filter function geneames a value for different areas
of the document.
IMPLEMENTATION
The search technique described in the proceeding section was implemented, along with standard document
handling techniques, into a system called the Cenwalized Document Database (CDD). This system has an extensive
database of technical documents supported on a Local Area Network (LAN). The system provides a single point
for accessing and sea_hing technical documentation. The system is designed to access ASCII formatted files of
various sizes and types and different physical storage locations.
58
#1
Condition
Word is a prime.
Word is unique (not in any table of knowledge base).
Word is a prime.
Word is not in frequency table.
Word frequency level > 10
Word frequency level > 5
Word (or phrase) is a sub-node.
Phrase is only possible acronym meaning.
Word (or phrase) is a sub-node.
Phrase is one of several possible acronym meanings.
Word is a sub-node.
Word is a synonym.
Word (or phrase) is a sub-node.
Phrase is only possible abbreviation meaning.
Word (or phrase) is a sub-node.
Phrase is one of several possible abbreviation meanings.
Change
Value = 400
Value = 200
Value = 0
Value = Value/2
Value = Value of prime
node
Value = (Value of prime
node) / (number of acronym
meanings- 1)
Value = (value of prime
node) / (number of
synonyms)
Value = Valueof prime
node
Value -- (Value of prime
node) / (number of
abbreviation meanings - 1)
Table 1
Weighing Conditions
A basic menu system is used to call up and display all of the available document files (see Figure 4). It
uses a number of filters for common word processors and mainframe printer formats. These are simple filtezs
designed to mask the command codes used by the different application programs that produced the documenL The
end result is that a document in almost any format, (e.g. Word PerfecL Displaywrite, or mainframe redirected printer
output), can be displayed (somewhat distorted) and used by the system.
The program provides immediate access to any part of a document through the use of special pointer fries.
The CDD program uses these pointer files to speed-up direct location access. These pointer files can be, used for
pages, sections, record numbers, or any string value. When a document is selected, the softwm-e will check for all
available pointe, files and add the options to the option menu. These pointer files are creawMoutside of the CDD
to fulfill specific needs within the SR&QA community. The CDD requires the pointer files to be in a particular
format and location, but any programming language can be used to create them.
The CDD has the capability to perform several different types and levels of data searches. The simplest
type is a basic Boolean key-word search. This type of sem'ch is a useful and fairly common type of search that can
locate a specific string using standard AND/OR logic. The program provides an improvement to this type of search
by expanding the Boolean logic to include any acronyms and abbreviationsofthesearch strings from its built-in
database.
The program has a fully operational version of the intelligence searching technique explained previously
(Figure 5). The initial query is broken down into its related components (words and phrases). Network nodes are
established and expanded through the methods described previously. A set of heuristic roles are used to assign
weights for the nodes in the new levels.
59
_II 81AIICI IIIDIE iilt'_IQII _ Gl"lqmld IIILIP "rA+i I
ATAZF,JI_ DOmmgllTl
Om-OOiI3IX
O,_-e04(i)z
_(3ll
l-oot(ill
lHiilll
Ol-eoi(ilK
l-oil(Ill
_Oiilili
l-Illllll
tll4111i
tllSllii
_illili
l-lilllll
lll31i
/ _ IJIBllll11111 1151J08]IPT ITII
INI_]ICYIOll IPOI]IM
_ ill P_/wtl_m _ Pl, a[JKm
/ I m l_i I i I/
i ! r-I.l'll/gT.ml _ _ • i
81C /I]lm AlllmUmm"
l_:]_ql/laZirPUlO _ ol L,_arx t _T
Ul_iLz Da_ _ (_J3k) arM_IOll
msru6,ZOl/Itm m,Oor._
Figure 4. Document Selection in The CDD
i.t I
n ill l_ i_ (Yl I r][_Jmlo)
_t'l t L* liu_ i_*oe. _ I_ to ths sn?
• i
Ii
Figure 5. Intelligent Search Query Screen
This network can identify and rank key mreas of the document that am likely to contain the information
requested. The program does this by filtering the document through the network. This relalional network relmms
a weight value for the section of the docum¢nt dial is currently passing through the netwoit. The user is appraised
of lh¢ search siaUis as die document is being processed (Figure 6). A list of pointers, to the sections of the document
that had die highest values, is the final result of the filtering. The software will immediately display the area of the
document that had the highest weighing (Figure 7). If the usei does not find the needed informalion, the software
will move to the next highest weighted area of the document.
6O
_I,Ij ° mmm_"llekef JLu
. unmtor Cbu_ neoo_S (N_]iazc 1]9 u)
. mqmo_l/XamtaJLlatJ_ ]_t=Lt Job _ (Cempvt_g gm_ntil)
3. rJFP,ZB Xnd4s (IIC 4 184)
J 4. _a3mdmt _m
S. _reet. r_mtLo _ i_raTm;
3.0 lumqmmlolm DOCImM
37t L'OIOS,aI_D
1. _ O& 04113)Ke Lain
2. SIPI I_ SO4(3)S, oeaeLa4
3. _ ImY S00(2)g, _ J_ Card I_/etam
4.0 IlnrJmrrzc01s
Ik_ sppl JLcebJ.e
S.O gsmn,kL it,oG4_lmmln, s
1. 'the ,_ral/£mmwtAlll&£om i_mm,m,m fo_ 5¢8 Oqmlp_tm
l_vRv af'q'_mNi GO_O CW_'__ _ s.l_; 15 "_tJm _ to tmghofl=e snd _,_.l'ol tlm ._.lt amd
.m-
Is
Figure 6. Search Status Screen
The results of any search are displayed on the terminal and can be exported to other applications. The
program has the option to copy part or all of a document to a fde or printer. The data is copied in standard ASCII
format that can be imported into most word processor and database applications. The data also can be printed to
any network printer.
Modifying the program to work in other fields (legal, medical, and business) would n:quire the c_ation of
a specialized database of words, acronyms, and abbreviations used in that field. In most cases this database already
exists in the reference documentation used in that field.
km_ m_ lC?ll mtu • m_ of tin _ *ti_iylq
mulie f_[m_l to _ Idtgt Ira. Itywt_o • II_z4ml_Lo _ Vires _4k
tk4 fl_lA lova,.]L 18 1_o z_e_vol_ emo_Nbl _0.0 pocomet.
13.17.2 I_ll_lrCIIIM. 9_11_1
_ify l_bst 4_N_lr_344J_ 6604 _IOZT _ _, C_/X21_I,4B>
'S6OS gM_r pOllBIt AlrJ_IdUIf, B, ¢_Y_r_.I431D, /A184 NKZI _ ON 111_1_
4Lind 4_STX264q_> _|$ MP_M _ GI8 Z]_ZCi_.XQI8 lu[o O_Y _ t_'_L_lto.
Jet t.blt £ollovLl_J aaS _ Irg"s as follmmt
_1fl(02:lell_ 6683 _ NO • _ (li_l_al_t_X)
_GInl(r44oI_. f,_83 _ It) 3 II_&IT (MOI88TJ4_) QgT
_o=mm_ s6s] l,I]m _o _t _ (NmiUl_.l[) GII
,r.GR_(2_54'B)' 6684 m_ _ lllG_ _ G85SII 4_
4_r_25_lm) $605 KO_l_ _ zlmL _v otqm (_D GFF
Figure 7. Results of _earch
61
[1]
[2]
[3]
[4]
REFERENCES
Luger, G. and Stubblefield, W. : Artificial Intelligence and the Design of Expert Systems,
Benjamin/Cummings Publishing Company, Inc. 1989.
Minsky, M. : Semantic Information Processing, The Massachusetts Institute of Technology Press, 1968.
Sombe, L. : Reasoning Unde_r Incomplete Information in Artificial Intelligence, John Wiley & Sons, Inc.
1990.
Whittington, ILl'. : Database Systems Engineering, Oxford University Press, 1988.
62


Methods and means used in programming intelligent searches of technical documents

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server