In this paper we describe how SNMP (Simple Network Management Protocol) can be extended to do control system diagnostics. Our solution consists of a SNMP agent for LynxOS and a configurable MIB (Management Information Base) browser. We have reused diagnostic modules from the existing diagnostic system and integrated our development into a commercial network management product

Christiansen, H P

Reisacher, O

Ribeiro, P

CERN Document Server

Using SNMP for creatingDistributed Diagnostic ToolsO. Reisacher*, P. Ribeiro, H. P. Christiansen**SL Controls GroupCERN, Geneva, Switzerland* Currently at Sun France** Currently at UI Design AB, Linköping, SwedenAbstractIn this paper we describe how SNMP (Simple Network Management Protocol) can be extended to docontrol system diagnostics. Our solution consists of a SNMP agent for LynxOS and a configurable MIB(Management Information Base) browser. We have reused diagnostic modules from the existing diagnosticsystem and integrated our development into a commercial network management product.IntroductionThe control system  for the LEP and SPS accelerators, at CERN, is a distributed computer system of over300 nodes. This system must be operational during long periods of the year on a 24 hours a day basis. Anumber of diagnostic tools have been developed over the years to support the intervention of the on-callteam, but they were also integrated into the Control Room operations environment. For the management ofthe large Local Area Network, covering over 200 sq. Km, a commercial product, HP OpenView is used. In1993 aiming at integrating these different tools into a common framework the project described in thispaper was started.This paper is divided into five parts. We start with the description of the existing distributed diagnosticarchitecture and motivations for this project. Follows a brief description of the current network managementstandards. The description of an SNMP agent for LynxOS along with its enterprise MIB is followed by thepresentation of a configurable MIB browser. The last part covers the integration of this framework into theHP OpenView environment.1.  Existing architecture and motivations for this projectThe LEP and SPS control system is a distributed system that can be divided into three interconnectedlogical layers :1. Back-EndsAt this level run the applications for the daily operations of the accelerators. This includes all theapplication software related to normal machine operation but also all the work related to anticipating control2system failures and bringing the system back to a normal state after a breakdown. At this level, for systemmanagement, we have a home-made tool called xcluc browses an information repository where all theperiodic reports from the agents arrive. Xcluc presents the state of the different network nodes, enables theaccess to the report information stored in the repository and also provides a number of primitives to interactwith the different nodes. These systems are mainly X-Terminals running of HP 9000-7xx servers.2. Front-EndsThe front-ends run a number of applications controlling the activity of a certain number of entities describedin 3. These entities also act as proxies for the message oriented acquisition and control scheme. A home-made diagnostic tool, clic (from the sound made when one takes a photograph), runs periodically acquiring acertain number of states pertaining to important sub-systems in this environment. These systems are PC orVME boards running the LynxOS operating system.3. Equipment Control Assemblies (ECA)These systems are doing I/O multiplexing acquisition and control. They are connected to the upper layer(the front-ends) by a message oriented protocol on top of a field bus. The status of the different ECAstations on the bus is periodically controlled by the clic application mentioned on 2.The main motivation for this project was to develop an integrated system and network tool for operationsand diagnostics. Also to reuse existing code from the clic application but standardizing the systeminformation model and transport protocol aspects. As HP OpenView (OV) is used for the networkmanagement aspects the basic idea was to transform the existing tools so that they could be easily interfacedto this package. Accordingly we took a number of decisions :- Use SNMPv1 with MIB II, supported by OV- Use the HP SNMP library- Develop a number of applications to support the integration2. StandardsA number of Network Management standards have been developed over the past few years. These standardshave a scope that goes beyond managing networks into areas like the one explored by this paper, systemmanagement. All the different architectures and standards for designing network management share the  basicmodel that splits the Managing System and the Managed System into two separate sub-systems. Each oneof these sub-systems communicates with a Manager Kernel or an Agent and these last two layerscommunicate with each other in a client-server relationship via a Management Protocol. The applications,their environment, and the system they run on are called a managing system. The network equipment andsoftware is represented as a collection of managed objects. Those managed objects are supported by anagent. The managed objects, their agent and the computer system they run on are called the managedsystem. This management framework provides application programming interfaces (APIs) to theapplications and objects. Of the three main management families, SNMP, CMIP (OSI) and DME (OSF) wewill focus on SNMP used as framework for this project.The management information model (SMI) is the most important part of a network management structureas it defines the kinds of interactions between management applications and managed objects supported bythe model. The managed objects are often active entities their behavior being independent of anymanagement system.3The SNMP familyThe following refers to what is termed as SNMP version 1 as described in [RFC 1155, 1157, 1212]. Thereare three distinct sets in this family of standards :- Management Information Base (MIB)Specifies the variables maintained by the framework (this is the information that can be queried and set bythe manager (MIB II, [RFC 1213]). This is in practice a text file describing the variables supported by aSNMP agent. For the sake of simplicity a limited number of syntactical elements, of the base syntaxASN.1 (Abstract Syntax Notation 1), are used. To avoid variations on the possible syntactical constructsdefining the same object a set of macros in ASN.1 notation is used [RFC 1212].- Structure of Management Information (SMI)A set of common structures and an identification scheme used to reference the variables in  the MIB,described in [RFC 1155].- The Management ProtocolA protocol layer on top of the User Datagram Protocol (UDP) implementing five basic types oftransactionsSNMP GETSNMP GETNEXTSNMP SETSNMP RESPONSESNMP TRAP The philosophy behind SNMP is that the agent and the object implementation should be simple. In thiscontext the SNMP SMI only supports a limited set of interactions between the application and the object.Operations to Get and Set attributes (called MIB variables) are supported. Attributes can be gathered intotables and groups, but operations on tables or groups are not directly supported. Create, Delete and variousActions are all implemented as a side effect of setting a MIB variable. An individual MIB variable is namedwithin an agent by an object identifier. Object identifiers are a sequence of integers, representing a paththrough a tree. Individual MIB variables in a system are represented by a leaf in this tree. The nodes above,the leaf level, represent tables, groups or MIB variable types. The GetNext operation is the only means tomake an inventory, much like a directory tree listing, of all the MIB variables supported by an agent.43. SNMP agent for LynxOS and its enterprise MIBThe SNMP agents available for commercial systems are often closed in design. The extension capability,although available most of the time, is often of limited scope for the needs. The first version of the  SNMPagent for LynxOS was based on the Carnegie-Mellon University (CMU) source code.Along with the extensible SNMP agent prototype, compliant to the MIB II specification, a ASN.1 parserhas been developed. Each extension, new object, is described as a MIB object with the ASN.1 syntax andentered in the database. This description is run through the parser which generates the hooks to the C codeto be called whenever an access to the new object is requested. The new code should be linked with the otherSNMP agent modules to produce the extended agent (see figure 2). Using this method the SNMP agent wasfirst made to support the network management MIB. The private part of the MIB was then integrated in astepwise fashion.Figure 1 shows the main building blocks of the described SNMP agent. The system surveillance isperformed every trap_interval seconds. If an error condition is detected a TRAP is sent to the manager.Trap_interval is a MIB object that can be set. Another agent module handles the SNMP requests originatedby the manager. The extensions to the MIB describe the system variables that should be monitored and alsothe variables specifying the limits against which the first are tested. These variables are specific to thesystem to be managed.While loopSNMP packet requestevery trap_interval secondsDo the testIn case of failureSNMP TRAPParse SNMPpacket requestCheck community/type of requestAre the variables supported by the MIBCheck variable access/type of requestCall get/set C functionBuild SNMP packetresponseSNMP PACKET RESPONSEFigure 1The instrumentation of operating system parameters is not a trivial task. Moreover one of the mainconcerns of this project was to provide this information using a minimum of system resources. As our5LynxOS systems run diskless we had to implement and incorporate into the SNMP agent functionalitysimilar to the one provided by system applications like df, ps and netstat. The agent is not forking,doing dynamic memory allocations or interacting with files once it is up and running. The reason is thatthis is the only way to be efficient when the node is in a bad state and needs diagnostic.What follows is an example off adding an object to the private MIB subtree. This example code is writtenfor LynxOS where many system parameters can be obtained using the info system call.The ASN.1 definition for the MIB :max_inode OBJECT-TYPESYNTAX INTEGERACCESS read-onlyDESCRIPTION“Read-only integer value corresponding to the system file table size”::= { filesystem 1 }The C function that implements the described object :#include “snmp_impl.h”#include “pdu.h” /* VarBind */#include “error.h” /* ERROR_MALLOC, NO_ERROR */int get_file_table_size ( var )VarBind *var;{var->value.integer = ( long *) malloc (sizeof (long));if (var->value.integer == NULL) return (ERROR_MALLOC);* (var->value.integer) = info (I_NFILES);var->value.len = 1;return (NO_ERROR);}6MIB.txtASN.1 PARSER C structuresC functionsSNMP enginec c SNMPAGENTFigure 24. The development of a configurable MIB browserThe development of this product was decided at a late stage of the project. Apparently in contradiction withthe project aim of integrating applications at the top level this product is justified by the need of asimplified and user configurable view of a limited number of variables and tables. The OpenView browser has a lot of very powerful capabilities like the possibility to dynamicallyload/unload a MIB file. It also has a menu based application builder but it  suffers from a number ofdrawbacks like being too close to SNMP theory, producing almost unreadable output of table variables andproducing a complex set of windows where the destination entity for the manager queries is not easilyknown all the time.The described browser was developed using the Hewlett-Packard SNMP library and a Motif user interface.Following the requirement specifications, the browser, is capable of reproducing a session, by saving theprevious session layout  in a configuration file, on a per user basis. This enables a user to create areproducible session profile by adding to the current session the objects pertaining to the  required statusview. A clear distinction is made between columns in a table object themselves and plain objects so that thepresentation follows the object type.All the SNMP related information is kept away from the user interface.The browser can be configured to run in a single window mode.The type of the data displayed can be changed with a “translator” function.75. The integration of this framework into HP OpenView environmentThe HP OpenView system allows the creation of submaps and different views. This has been exploited inthis project. In addition to the view that IP Map offers, a secondary view, close to the existing repositorybrowsing tool xcluc has been created.The usage of the IP Map application can continue as before. This view will reflect the combined networkand system states of the nodes as reported by the extended SNMP agents.With the API provided with HP OpenView a simple interface to display parts of the extended MIB can bequickly built. The experience acquired by doing this will make extensions to similar interfaces concerningother objects in the MIB an even easier job.The API also provides interfaces to the HP OpenView database to get hold of the received traps and alsoOSF/Motif functions that allow several applications to work with and control the visual aspects of acommon submap hierarchy, thus cooperating with the IP Map application.ConclusionsThis project proved the feasibility of replacing the home-made distributed diagnostic tools by a SNMPbased framework. Among other benefits we will be able to integrate the view of network and systemdiagnostics, use a standard protocol for the manager-agent communications, provide a standard hierarchy forthe managed data and reduce the maintenance effort to one agent.We stepped into some limitations and this were,  short support at the data types level, further studies to bedone on security and reliability issues, very slow access time under SNMPv1 for table objects retrieval.ReferencesManaging networks and systems with SNMP. O. Reisacher 1995,http://www.cern.ch/CERN/Divisions/SL/SpsLepControls/snmp_sysman.htmlInstallation and Management of the SPS and LEP control system computers, Alastair Bland, CERN, paperpresented at ICALEPCS’93, Berlin, GermanySNMP, SNMPv2 and CMIP, William Stallings, Addison Wesley 1993

Using SNMP for creating distributed diagnostic tools

Abstract

Similar works

Full text

Available Versions

CERN Document Server