A real-time autonomous expert system has been developed to carry out traffic management in the Southern Californian telephone network. The system has been working on live data since September 1991 and generates rerouting advice that agrees with that generated by the present network management procedures. A modular software design was adopted to allow for evolution. A graphics interface allows the user to easily navigate through the display of exception conditions and advice. Exceptions are shown highlighted on a map of Southern California. A severity measure is calculated for each exception and is used to prioritize the display of information

Ambrose, Barry

Finnell, Sandee

Goodman, Rodney M.

Latin, Hayes

Caltech Authors - Main

NETWORK OPERATORS ADVICE AND ASSISTANCE (NOAA): A REAL-TIME TRAFFIC 
REROUTING EXPERT SYSTEM 
Rodney M. Goodman*, Barry Ambrose*, Hayes Latin** and Sandee Finnell*** 
* California Institute of Technology, ** Systems Technology Pacific Bell, *** Southern California Network 
Management Center Pacific Bell 
Abstract: A real-time autonomous expert system has 
been developed to cany out traffic management in the 
Southern Californian telephone network. The system 
has been working on live data since September 1991 
and generates rerouting advice that agrees with that 
generated by the present network management 
procedures. A modular software design was adopted 
to allow for evolution. A graphics interface allows 
the user to easily navigate through the display of 
exception conditions and advice. Exceptions are 
shown highlighted on a map of Southern California. 
A severity measure is calculated for each exception 
and is used to prioritize the display of information. 
Introduction 
In September 1990, a project was started by CalTech 
and Pacific Bell to investigate the implementation of a 
real-time autonomous expert system which would aid 
network management of the Pacific Bell Southern 
Californian telephone network. The ultimate aim of 
the project was to produce and automatically 
implement recommendations for network controls. 
To do this, the knowledge in everyday use by the 
network management staff had to be written down, 
and coded for use in a computer system while 
retaining flexibility and openness to change. 
The situations that the network management staff are 
exposed to are complex. Every day new problems 
arise. Examples are single random overloads of 
traffic on a given route, high volume phone-in 
conditions, disasters such as earthquakes, unusual 
calling patterns such as occw on Mother’s Day, killer 
trunks and other events. The introduction of new 
facilities, such as advanced signalling systems, adds 
additional problems that have to be diagnosed. 
To date, a number of software modules have been 
developed which use data gathered in real time from 
the network. The software can produce a graphic 
analysis of the network, describe how the network is 
functioning in terms of exception conditions and 
recommend controls. The controls are given in the 
format that the network manager is accustomed to. 
Each module has been extensively tested. 
0-7803-0608-2/92/$3.00 0 1992 IEEE 1240 
The rest of the paper gives a description of the 
Pacific Bell telephone network and the architecture of 
the Network Operators Advice and Assistance 
(NOAA) system. This is followed by the details of 
how NOAA works presented logically in the order (i) 
input information available to NOAA (ii) decision 
making and decision support for NOAA (iii) controls 
recommended by NOAA. The map based Graphics 
Interface developed for NOAA is described 
separately and finally an indication is given of future 
work. 
The Pacific Bell Network 
In order to gain some appreciation of the network 
management tasks, one must have a description of 
the network to be managed. The Pacific Bell 
Southern Californian telephone network provides 
service to at least 1 million subscribers and has a 
hierarchical structure. For administrative and legal 
purposes it is divided into Local Access and 
Transport Areas (LATAs) [BELL86a] and the 
network management center in Pasadena manages 
LATA’s 5 and 6. Geographically LATA’s 5 and 6 
extend to San Diego in the south and include all the 
Los Angeles urban area. 
The network is hierarchical. End offices are the 
exchanges that serve customers, and tandems are the 
exchanges used for traffic between end offices that 
are not directly connected [BELL86a]. There are 6 
tandems to be managed and over 200 end offices. 
There are two types of trunk groups. High usage 
trunk groups are dimensioned to be lossy, i.e. during 
the busy hour they are not guaranteed to have enough 
capacity to carry all offered traffic. Traffic will 
therefore overflow onto the Final trunk groups which 
are dimensioned to provide a good Grade of Service. 
In general there will be a final route between each 
end office and its parent tandem. It is these final 
routes that provide the backbone of the network. The 
final routes are therefore closely monitored by the 
network managers. 
Architecture of NOAA Advice (AD) 
The Architecture of NOAA is shown in Figure 1. 
The Pacific Bell network management system is 
called EADAS [BELL86b]. NOAA is connected over 
a 4800 baud data link and appears as an ordinary 
operators terminal to EADAS. NOAA then runs on a 
Sun workstation. 
Given the statistics on the various trunk groups, 
exception situations are noted and a severity measure 
associated with each exception. A diagnosis based on 
observed symptoms is carried out according to rules 
supplied by network management staff. Finally 
advice is given about the response to the exception. 
EADAS 
Engineering and 
Administrative 
Data Acquisition 
System 
RS-232 
NOAA 
Sun Workstation 
Figure 1 Architecture of NOAA 
There are four software modules that make up 
NOAA, each made up of about 3000 lines of C code. 
These are: 
data gathering (DG). 
This module is responsible for logging into the 
EADAS system and maintaining the basic dialog with 
EADAS. The UNIX tip command is used for this 
purpose. A significant effort was put into making 
this module robust in the face of communication 
difficulties, such as loss of synchronization in the 
EADAS-NOAA dialogue. 
tandem calculations (TC). 
This modules takes the data which has been 
downloaded from EADAS and carries out the 
calculations needed to transform the data into usable 
statistics. The data is downloaded in register format. 
These registers, also known as peg counts, are 
integer counts of events such as trunk seizures, that 
have happened in a five minute interval. The 
calculations that are carried out are well defined and 
produce more useful statistics such as seizures per 
circuit per hour. 
If a con601 is advised, it isgiven 
in the format expected by the 
network operator. This can be 
then approved by the operator 
and implemented through 
EADAS. 
Graphics Interface (GI) 
The graphics interface uses a 
map of Southern California. 
Exceptions are indicated by red, 
suggested controls by green and 
actual controls already in place 
by blue. Additional information 
is given when the operator clicks 
on the map. This information 
includes calculated statistics on the route and any 
advice generated for that route. 
Statistics 
NOAA makes its diagnosis on the basis of available 
information about the trunk groups. This information 
is available to NOAA in the form of counts of certain 
events for each trunk group during a 5 minute 
period. These events are for instance (i) a trunk being 
seized (ii) a seizure that is not successful i.e. 
overflow (iii) a survey of trunk occupancy at a given 
instant. On the basis of these counts, the following 
standard statistics are calculated: 
ACH Attempts per circuit per hour 
CCH Connections per circuit per hour 
OFL Percentage of attempts that overflowed 
USG Percentage of trunks occupied on 
average 
HT Holding Time of Calls 
The 5 minute period is a compromise. The shorter 
the period between data collections, the faster the 
response to any potential problems, but the larger the 
amount of information to be processed, requiring 
more expensive hardware. The 5 minute period 
means that problems can be cleared relatively rapidly. 
It is known that mean traffic increases do occur on a 
time scale of the holding time whereas traffic 
1241 
fluctuations occur on a time scale of the inter-arrival 
rate of calls [FILI90]. 
Since a large amount of information is available, 
filtering is carried out. Firstly exceptions on high 
usage routes are ignored and only final route data are 
examined. Secondly exceptions on special purpose 
final routes are ignored. For example, we expect a 
trunk group for the time announcement to have a 
short holding time. Thirdly overflows are treated as 
having higher priority than other exceptions, since an 
overflow on a final may represent lost traffic. 
Decisions 
When an exception condition has been noted on a 
trunk route, there could be many possible 
explanations for it. Typically phone-ins to radio 
stations and TV stations may generate excess call 
attempts. Facilities failures may mean that overflow 
shoots up on related trunk groups. Occasionally the 
data gathering may interfere with maintenance 
operations and unreliable data is returned. Finally 
random overflows can occur on individual trunk 
groups. 
For any reroutes, a search for spare capacity is 
carried out. If there is an overflow problem between 
A and B where A is an end office and B is a tandem, 
then the program first looks for capacity on other 
trunk groups going from A to By and then carries out 
a search for A to B via C possibilities. Here C has to 
be one of the 6 tandem exchanges, because of the 
hierarchical nature of the network. The candidate 
reroutes are then sorted according to available 
capacity and controls are suggested that make use of 
the minimum number of reroutes that achieves the 
required capacity. 
A severity measure is attached to each exception. 
This measure is biased to give overflow exceptions a 
high severity figure, since overflows can result in 
lost traffic. The severity measure is also made 
proportional to the size of the trunk group and the 
amount by which the statistic is above threshold. 
Thus a 10% overflow on 10 circuits is considered 
equivalent to 1% overflow on 100 circuits. A log 
scale is then used to scale the figure between 0 and 
100. This seventy measure has proved useful for 
sorting exceptions prior to displaying them. In a full 
scale emergency with the loss of a lot of capacity, a 
sort by severity would indicate which trunk groups 
to deal with first. Note that 91 1 calls have their own 
reserved trunks from end offices and are not 
supervised by NOAA. 
The rules used in the program are of three separate 
types: 
rules which indicate which exceptions can be 
safely ignored. For example overflow on 
high usage routes is ignored; 
rules which indicate which routes can be use 
as candidate reroutes; 
rules that map a suggested reroute into a list 
of controls to effect the reroutes. E.g. certain 
other routes may have to be "finalized" first 
to prevent a round-robin situation. 
Some of the above rules were already written down 
in operators handbooks. Others were supplied by the 
network management staff. Examples of the rules are 
given in Table 1. 
DISREGARD ANY EXCEPTIONS ON TRUNK GROUP 
COMMON LANGUAGES (CLLIS) ENDING WITH "MD" 
(EX: LSANCA02AMD; LSANFDRCCMD) . EXCLUDE 
SAME WHEN SEARCHING FOR VIA ROUTE 
CANDIDATES. 
DISREGARD ANY EXCEPTIONS ON CLLIS 
IND IC AT I NG " P B " IN THE STATE DESIGNATION 
(EX: OKLDPB0349T). EXCLUDE SAME WHEN 
SEARCHING FOR VIA ROUTE CANDIDATES. 
DISREGARD ANY EXCEPTIONS ON THE FOLLOWING 
HIGH VOLUME CALL-IN CLLIS: 
HLWDCA01520 SNANCA01977 COTNCA1143A 
SIMICA11629 SNDGCA0157X 
Table 1 Typical Network Management Rules 
As with any rule based system, including the rules in 
a computer program has the advantage that any 
special cases are immediately recognized and the 
operators handbook does not have to be referred to. 
However for complete trust in the system, the rule 
base has to be extensively tested and compared with 
the experts analysis in a wide range of cases. 
Controls 
It was mentioned above that there are many causes of 
exceptions on the trunk groups. Similarly there are 
many possible controls that can be put in to improve 
the network throughput. In general, controls can be 
divided into protective controls and expansive 
controls [FILI90, BELL881. 
1242 
Protective controls are used to cut down the amount 
of call attempts entering the network. For example in 
the event of a phone-in, controls can be put into each 
end office to block call-attempts to that telephone 
number. Alternatively a certain percentage can be let 
through. This limits the amount of congestion in the 
network. Clearly the calls destined for a phone-in are 
“bad” traffic as there is a high possibility of non 
completion, and therefore they should be blocked at 
source and not rerouted. 
Graphics Interface 
It was found that this application was eminently 
suitable for graphical interfacing. Initially a scrolling 
list of exception conditions was presented to the 
operator in a window. Buttons allowed the operator 
to pull up information about the statistics on a trunk 
group and the recommended advice. 
This was then modified to allow the operator to 
choose from a number of lists of exceptions, each list 
-\ 
1 
1 
Figure 2 Graphical Interface for NOAA 
On the other hand, expansive controls allow trunk 
groups that are normally separate to share their 
capacity. If there is a single random overload, then 
expansive controls are called for. Two alternatives 
here are ovefflow rerouting, which reroutes ovefflow 
traffic only, and an immediate reroute which puts all 
call attempts onto the alternative route first. Each is 
appropriate for certain cases. Finally spray rerouting 
is possible, whereby a number of alternatives are 
given and each is tried for a given call. Clearly 
todays telephone exchanges have a lot more 
flexibility than the hardwired mechanical exchanges 
that were common thirty years ago. 
corresponding to a 
different statistic over 
threshold. 
Finally a map interface 
was developed (see 
Figure 2). Without doubt, 
this was the easiest way 
to (i) display an analysis 
of the network, showing 
the exceptions clearly (ii) 
pull up information about 
individu a1 exceptions. 
The map, with its 
translation and zoom 
features, could be 
mastered by any user in 
minutes and added a 
“fun” element to 
navigation through the 
program. Coordinates of 
most of the cities in 
Southern California were 
available from a public 
database. 
Future Work 
Over the past 18 months, much work has been done 
in interfacing NOAA to the Pacific Bell network 
management computer and building the infrastructure 
for an expert system. Over the coming months the 
rules implemented in the program will be tested by 
running the program on live data. When sufficient 
cases have been tested, it is intended to close the loop 
and allow NOAA to carry out controls 
autonomously. Clearly considerations of reliability 
and robustness have also to be taken into account 
when this step is to be carried out. 
The ability of NOAA to diagnose problems correctly 
and to take the correct actions will be enhanced if the 
system has other information sources besides 
EADAS. Two other sources being considered at 
present are NSCS which provides information about 
1243 
seizures of trunks, and a separate system which 
provides information about the SS#7 signalling 
network. 
Looking at the long term future for NOAA, the 
definition of a standard data format for exceptions 
and for statistical information about trunk group 
performance would help in minimizing the cost of 
upgrade of NOAA, as new versions of EADAS 
become available. As in any network management 
application, standardization of data formats between 
applications that share the data is an important 
requirement. The I S 0  standards work on fault 
management and performance management should 
go some way to filling this gap. 
To summarize, network management advice is 
currently being generated for the Southern California 
telephone network. As the rules that generate this 
advice are tuned, a robust network management 
application will have been developed that relieves 
network management staff of some of the need to 
supervise the day to day running of the telephone 
network. 
References 
BELL86a Bellcore, Network Management Intra- 
LATA Network Fundamentals, BR 780- 
150-122, Issue 1, December 1986. 
BELL86b Bellcore, Network Management 
Technical Overview, BR 780-150-130, 
Issue 2, December 1986. 
BELL88 Bellcore, Network Management Control 
Descriptions, BR 780-150-128, Issue 3, 
June 1988. 
FILI90 Filipiak, J., Real Time Network 
Management, Draft Copy, April, 1990. 
1244 


Network Operators Advice and Assistance (NOAA): a real-time traffic rerouting expert system

https://authors.library.caltech.edu/93852/1/00276590.pdf

Network Operators Advice and Assistance (NOAA): a real-time traffic rerouting expert system

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main