Exposing MIB Data to a Web-based Interface

A lthough the Simple Network Management Protocol (SNMP) and Web-based management (WBM) are vastly different management paradigms, an SNMP MIB inheritance library, along with a backplane-based architecture, allows them to be combined in such a way that the advantages native to each methodology can be fully exploited. Furthermore, if a product is designed thoughtfully, using a backplane, SNMP code can be leveraged across multiple management paradigms for comprehensive access and control capabilities. In addition to WBM, for example, management via Java applications/applets, JavaScript, XML, and command line interfaces (CLI) can be quickly and easily achieved. Lastly, and most importantly, with a backplane-based architecture in place, no new development effort for a given device is required. Developers can fully leverage hundreds, and potentially thousands, of hours of pre-existing work. This expedites product time-to-market and provides the ability to safely migrate to next-generation network management protocols.

To show how a backplane-based architecture along with an SNMP inheritance library can provide these benefits, I will first explain the general principles behind both SNMP and WBM, and then explain how those principles can be seamlessly integrated to provide a robust, extensible Web-based management solution using previously defined SNMP MIB variables.

SNMP operation
SNMP is a well-established protocol framework that has been in published form since 1990. It has been widely adopted in the datacom industry, and close to 100% of shipping network equipment is manageable via SNMP. Furthermore, its rapid adoption in the telecom industry ensures that SNMP will continue to be a de facto management standard in all packet-based networks.

SNMP enables a system administrator to simultaneously manage a vast number of heterogeneous network elements (NEs), or devices, through an external management console such as HP's OpenView. This management is accomplished through the manipulation of Management Information Base (MIB) objects, which consist of a series of variables that reside on each device, and describe the state of the NE. The administrator can direct the SNMP console to monitor and alter the MIB objects on any device, thereby controlling its behavior. (Throughout the remainder of this discussion, I will use the terms "MIB variables" and "MIB objects" interchangeably.)

The console performs this manipulation of a device's MIB objects by sending out SNMP GET Protocol Data Units (PDUs) and SNMP SET PDUs. Each PDU encodes within it a list of the relevant MIB variables and values, if appropriate. This behavior is shown in Figure 1.

The SNMP agent, which resides on the NE itself, receives those PDUs, parses out the list of variables, and consults its management information base about those variables. The management information base, likewise, is found on the NE itself.

This parsing of the MIB objects within the PDUs is achieved through the use of the Structure of Management Information (SMI), which is a published, freely available, standard, and universal method of specifying a MIB object. The MIB objects in the PDUs are encoded both by the console and by the agent, according to the rules of SMI. And since SMI is a recognized standard, it is known both by the agent and by the console a priori to any NE management via SNMP.

Figure 2 illustrates how SMI works with respect to SNMP MIB variables. All MIB objects are laid out first according to the SMI hierarchy shown. That is, all MIB objects in the world of SNMP have the following prefix:

iso.org.dod.internet or, in numerical form:

1.3.6.1

The other groups shown in Figure 2, ccitt and iso-ccitt, have never had any MIBs specified under them that have gained any kind of meaningful acceptance. Likewise, there have never been any other standard subgroups defined under the iso, org, or dod groups (so you will not be seeing any MIB objects starting with 2.1.4.2, for example, anytime soon).

Continuing on, suppose you wanted to specify a particular SNMP MIB object, say sysName. sysName is part of the system group, which, in turn, is part of MIB II. MIB II, which is part of the mgmt group shown in Figure 2, is a standard set of MIB objects, published in RFC form and implemented on virtually every packet-based NE that has SNMP. Both the console and the agent have a priori knowledge of the sysName variable.

To specify sysName, the SMI hierarchy shown in Figure 2 is merely taken out to deeper levels, and is shown in Figure 3. Thus, sysName has the following full specification:

iso.org.dod.internet.mgmt.mib-2.system.sysName

or, in numerical form:

1.3.6.1.2.1.1.5

This specification is called an object identifier (OID). It consists of a base OID, 1.3.6.1.2.1.5. In addition, it has an instance of 0 appended onto it, for a total specification of 1.3.6.1.2.1.1.5.0. The 0 enumerates that the first and only instance of sysName should be returned, since sysName is known to be a scalar SNMP variable.

Thus, when an SNMP console wants to "get" or "set" the value for sysName, it sends out an SNMP GET or SET PDU with the OID 1.3.6.1.2.1.1.5.0 encoded within it.

The NE, in turn, will parse through the OID and consult its management information base. The management information base is a set of mappings between MIB variable OIDs and appropriate GET and SET routines. These routines provide read and write access to that data. An example of the mapping for sysName is shown in Figure 4.

Depending on the nature of the request, the SNMP agent then calls either the GET or SET routine associated with the relevant variable passes the instance variable 0 to the routine, and from there is able to monitor or update the state of its host's system name, as shown in Figure 5.

As these diagrams indicate, in order for an NE to be manageable by SNMP, it must have an SNMP agent that is capable of receiving, parsing, building, and sending SNMP PDUs, and it must have a management information base. The management information base, in turn, must have a significant number of MIB objects, with their corresponding GET and SET routines, in order for the console to monitor and control the device in a meaningful fashion.

Commercial quality SNMP agents are provided by a number of vendors, including SNMP Research and Integrated Systems (ISI)/Epilogue (now part of Wind River Systems). In addition, a number of free SNMP agents have been developed, primarily at CMU or MIT.

However, the real "meat" of SNMP-the design, definition, and description of a device's MIB objects-must be done for each device, in almost a unique fashion. The reason for this is that the MIB objects need to access real data, which means there must be GET and SET routines written that access real variables used in the firmware. And since there are as many different ways of representing data as there are engineers, these GET and SET routines for each MIB variable are (relatively) unique to each device code line.

For example, the ISI/Epilogue agent requires a variation of the following system call to construct the MIB object for sysName:

LEAF_ASYS(sysName_leaf, "sysName",
VT_STRING, RW, SCALAR, it_exists, 
get_sysName, set_sysName, std_
next, (char*)sysName, 0x0000, 
0xFF, 0xFF);

Because of this, the developers at the network equipment vendor must themselves perform the task of writing the GET and SET routines (the get_sysName() and set_sysName() functions shown above)-there is no easy way out and no code that you can buy for MIB variable design and implementation.

Furthermore, that task is not limited to five or 10 MIB variables-NEs have encoded in them, at an absolute minimum, the roughly 200 MIB objects described in MIB II. In addition, many of these devices have literally hundreds of other MIB objects encoded within them, representing a development effort measured in the thousands of hours and hundreds of thousands of dollars.

However, even this does not fully describe the task. Most of the other MIB objects in the device are proprietary, which means two things:

• You have to have a knowledgeable engineer-a MIB-smith-to design and create the proper SNMP MIB objects
• The MIB-smith must generate a text document, written in a pseudo-programming language called Abstract Syntax Notation 1 (ASN.1), that fully describes the proprietary MIB on the NE

The first item is necessary because you want to have MIB objects that describe the state of your device in a meaningful way. The second item is necessary because you have to provide knowledge of the MIB objects in your device to the system administrators and their SNMP consoles. Otherwise, the SNMP console will not know the proper encoding for the proprietary MIB data, and how it fits in with the SMI. While the console will know the encoding for MIB II items, such as sysName, it will not know the encoding for anything unique to your device, and the a priori SMI knowledge mentioned earlier will not, in fact, exist. This will make the proprietary items on your device unmanageable.

So while SNMP is popular and likely to remain the de facto management paradigm for all packet-based networks into the foreseeable future, a non-trivial amount of engineering resources need to go into developing and maintaining a viable SNMP implementation on a NE.

Web-based management operation
While WBM is a new management paradigm (it has only been around since 1995), it has gained remarkable mindshare and is rapidly becoming a standard mechanism by which NEs are managed.

WBM allows a system administrator to manage an NE by simply "pointing" the Web browser at the device. The Web browser then makes a HyperText Transfer Protocol (HTTP) GET request to the device. From there, the device sends-over the same HTTP connection-a series of HyperText Markup Language (HTML) pages. These pages, which are nothing more than simple text files, tell the Web browser how to create the user interface (UI) of the device.

With the explosion of the World Wide Web (WWW) in the last four years, a huge number of new technologies have been developed, including Dynamic HTML, eXtensible Markup Language (XML), JavaScript, and Java. All of these technologies serve greatly to enhance the capabilities of the "Web experience," both on the browser side and the server side. As a result, HTML files can now describe a UI that effectively mirrors, and often surpasses, the UIs found in conventional management applications.

Regardless of the technologies employed in the device's UI, those HTML pages have to be linked, somehow, to the underlying device data. That is, those pages have to be able to monitor and alter the host-specific dynamic data.

Thus, in order to implement a WBM solution, a vendor needs to have two things: an HTTP server embedded in his device-which is the main "packet engines" of WBM-and a mechanism for linking the HTML pages to the underlying data.

As with SNMP agents, commercial HTTP servers can be purchased or free versions can be ported. However, again like the SNMP paradigm, the real "meat" of WBM comes in linking the data in the device to its UI.

In the case of SNMP, the "meat" consists of linking the device data to the MIB objects. In the case of WBM, the "meat" consists of linking the device data to HTML pages.

A backplane-based architecture allows a developer to link an HTML page to underlying data in a fashion that completely separates the HTML from the back-end C code. By completely separating HTML from C code, the developer can hand off GUI design to the marketing department or an outside design firm to develop the device's UI independently, without affecting the back-end code. This allows for professional quality HTML UIs to be developed rapidly without tying up the embedded software developers. Furthermore, this it easy to modify the UI for OEM channels, localize it for foreign markets, and so on.

Lastly, by coupling a backplane-based architecture with an SNMP MIB inheritance library, a company can, in WBM, reuse all of the GET and SET routines written for its SNMP MIB variables without writing a line of code. This architecture allows device vendors to leverage the thousands of hours of work already spent in writing MIB variables and code, in a WBM scheme that is both flexible and scaleable for future business development.

How a backplane and SNMP MIB inheritance library work together
Figure 6 shows an architecture that provides a complete separation of the HTML UI from the back-end C code, and which allows a developer to leverage pre-existing SNMP MIB variable code without affecting embedded proprietary code.

A backplane-based architecture fits into a given embedded system as show in Figure 6. We assume that the typical embedded system has two primary parts: A real-time operating system (RTOS) and, more importantly, a vendor's proprietary code.

The RTOS provides the system with low-level services such as memory allocation, network services (TCP/IP), and semaphores.

The vendor's proprietary code provides the system with its unique value, the device's "reason for being." In the vendor's code, there is assumed to be an SNMP agent-either developed in-house or purchased commercially. In addition, there is a set of MIB objects, which access the device data for management via SNMP.

The glue code consists of a series of "Read" and "Write" wrapper routines (access methods) for the data found in the vendor's proprietary code, data that is not accessible via the SNMP MIB.

The glue code is a necessary part of many WBM implementations. As you will see in the example detailed below, the backplane works in a very general sense-it is not limited to just one presentation media. As such, there is much data that the backplane works with that is wholly inappropriate to SNMP, such as dynamic data to specify HTML pages. Also, there is data that is not conveyed in a meaningful sense via SNMP, such as a dynamic log of entries.

Lastly, since there are as many different ways of representing data as there are individual engineers, a viable Web-based management solution needs the data access written in a special format that it can understand.

This Web-based management solution consists of three parts:

• A backplane. This is a database of mappings between special HTML tags and function pointers to the relevant "Read" and "Write" routines found in the glue code. The backplane and the special HTML tags are discussed at length below
• The Web server. This module consists of a fully functional HTTP 1.1/1.0 server plus a set of common gateway interface (CGI) handlers. CGI is a standard approach to handling data exchanges between an HTTP server and ancillary programs. The Web server architecture is shown in Figure 7
• The SNMP MIB Inheritance Technology. This module is a general-purpose code library that handle SNMP specific queries from the backplane. It consists of an SNMP abstraction layer (so that it can effectively mate with a wide variety of different SNMP stacks), conversion routines, and logic to work with advanced SNMP principles, such as SNMP tables. The architecture is shown in Figure 8

Basic operation
The easiest way to describe how the backplane achieves the complete abstraction of HTML from the underlying C code while leveraging pre-existing SNMP MIB variables, is through a pair of examples. The examples are simplified for the sake of this discussion, but the concepts they illustrate are complete.

To show how this set of technologies works as a whole, we will first go through an example where the backplane accesses some data that is only available via a glue code routine. Next month, we will go through a parallel example where the data is available from a pre-existing SNMP MIB object.

Backplane operation with glue code routines
Assume that the following HTML is located in a file named MYFILE, and MYFILE is stored on an embedded network device using such a backplane architecture:

<INPUT TYPE="text" SIZE="20" NAME="userName" VALUE="$%userName#$">

In this instance, the HTML referenced above is a FORM element that is used to describe a one-line text box. A text box is a common element in Web-based management because it allows an end-user to read and modify a particular string (be it a device name, an IP address, and so on) through the Web browser. The Web-based management solution works with any and all types of HTML FORM elements.

Note also that this is nothing more than standard HTML (albeit with an interesting VALUE field), and as such it can be written or modified by any ordinary HTML editor.

When a Web browser asks for (that is, makes an HTTP GET request for) the URL that corresponds to MYFILE, the HTML in that file is first loaded into the GET CGI handler, as shown in Figure 9.

The GET handler then steps through the HTML character-by-character until the handler runs into "$%," the notation for the beginning of the special HTML tag. This pair of symbols, which does not appear in ordinary HTML, signifies to the GET handler that the following text is not standard HTML. In this particular example, "$%userName#$" is the special tag.

At this stage, the GET handler stops parsing the HTML and looks up userName in the backplane, which is essentially an online database of the special tags. The backplane returns a database entry to the GET handler, as shown in Figure 10.

The GET handler now has a pointer to a "Read" glue code routine, as shown in Figure 11, which it calls. The "Read" routine accesses the vendor's proprietary code, finds the relevant data, and copies it into the HTML that is streamed out to the remote browser. Assuming that the "Read" routine retrieves and copies in the data "Wendy Rosenthal" in lieu of "$%userName#$" the following HTML is constructed:

< INPUT TYPE="text" SIZE="20" NAME="userName" VALUE=" Wendy Rosenthal" CHECKED >

The GET handler, after making any other tag substitutions that might be in the HTML file, then sends MYFILE out to the Web browser via the HTTP server. The Web browser then displays the HTML in the example as:

As mentioned earlier, this text box is an HTML FORM element. HTML FORM elements are essentially variables, where the name of the variable is specified in a NAME field, and the value of the variable is specified in a VALUE field. In the example above, the NAME of the text box is userName, and the VALUE is "Wendy Rosenthal."

Now, when an end-user wants to change the value in the text box, he or she simply highlights "Wendy Rosenthal" in the Web browser, hits the delete button, and then types "Gayle Semrad." After that, the end-user hits the submit button.

At this stage, the Web browser sends back a message to the Web server with a stream of data in it. The message is called an HTTP POST message, and the data is called a CGI stream. For the above example, the CGI stream that is sent back looks like the following:

. . . &userName=Gayle+Semrad& . . .

Notice how the name-value pair of the CGI stream is identified by userName, which is the name of the one-line textbox as shown in the HTML listing. The value part of the name-value pair is just what the end-user typed in. This is why the HTML FORM elements have a NAME-so that the appropriate element and its value can be readily identified in the CGI stream.

This CGI stream is received by the embedded Web server and passed to the Web-based management system's POST CGI handler. This is shown in Figure 12.

From there, the POST handler parses through the CGI stream. When it gets to userName=Gayle+Semrad, the POST handler then looks up userName in the backplane. The following entry is returned to the POST handler:

Now, the POST Handler has a pointer to a "Write" routine in the glue code, which it calls. The "Write" routine takes the "Gayle+Semrad" data, accesses the vendor's proprietary code, and makes the appropriate update. (The '+' sign is appropriately dealt with by the post-CGI handler code.)

By designing the special tag "$%userName#$" in the HTML file to match the NAME of the FORM element, userName, the same database entry in the backplane can be used bi-directionally, or both in the "outbound" direction and in the "inbound" direction.

This is the essence of how a backplane-based Web-based management system operates. Note that the HTML in MYFILE is completely unchanged. If the end-user hits "reload" on his/her Web browser, or if another Web browser were to access it, the GET operation would be performed again. This time, instead of copying in "Wendy Rosenthal," the GET handler would copy in "Gayle Semrad."

This model essentially takes the concept of a form letter-namely, a form template and a database that are merged to create the illusion of a personalized letter-and moves it into the realm of HTML/HTTP/CGI. The original template, in this case, is simply the HTML file MYFILE. And just as a form letter template can be easily modified with any word processor, so, too, can the HTML template file be easily modified with any HTML editor.

However, as demonstrated above, the HTML/HTTP/CGI mechanism takes the form letter model to a new level. By choosing the special tags so that they match HTML FORM element NAMEs, one can essentially have a bi-directional form letter.

With this architecture, if a company wants to change the UI for rapid prototyping, for localizing to foreign markets, or for setting up new OEM channels, the only thing that needs to be modified is the HTML-and that can be modified by using any number of standard tools. The rest of the system image can remain completely unchanged. Likewise, if file-based uploads are supported, then the UI of the device can be dynamically altered and engineering doesn't have to be involved in the process at all.

Next month, I'll show you how to make examples like these a reality for any SNMP-enabled system. We'll look at some even more powerful advantages of adding WBM-support this way.