SNMPv3 Implementation and Interoperability Report
Reported by Russ Mundy on behalf of the SNMPv3 Working Group. Received on February 10, 1999.
This memo on SNMPv3 implementation and interoperability has been prepared to meet one of the requirements for advancing the SNMPv3 specifications from Proposed to Draft state on the IETF standards track as described in RFC-2026.
The SNMP architecture lends itself to a client/server type of approach with relatively powerful systems that are commonly referred to as a Management Station that communicates with generally less powerful systems that are commonly called agents. The architecture associated with the SNMPv3 specifications is intended to be very modular and flexible with identified interfaces so that "simple" devices can remain simple while "powerful" devices with a wide range of capabilities also fit the architecture. Although the modular structure and interfaces are used to illustrate the architecture, software developers may choose whether or not to implement the modules and abstract service interfaces in their software.
As a result of this flexible architecture, implementations are expected to provide different specific capabilities. The common functional element associated with all SNMPv3 entities is the SNMP engine and one (or more) of several types of applications. The currently identified types of applications include; command generators, which monitor and manipulate management data; command responders, which provide access to management data; notification originators, which initiate asynchronous messages; notification receivers, which process asynchronous messages; and proxy forwarders, which forward messages between entities. Although each implementation need only provide a single application, multiple applications are provided by most implementations.
SNMPv3 implementations are available as both commercial products and as freely available software. Some of the earliest implementations supporting many of the features were available prior to SNMPv3 reaching the Proposed state on the standards track. SNMPv3 development kits and products have been available from commercial vendors since early in 1998. Some major SNMP technology users are now requiring their suppliers to provide SNMPv3 products or a viable plan to provide SNMPv3 in the near future.
There have been a number of SNMPv3 interoperability testing activities between various implementations. A number of these tests have been done at trade shows, during IETF meetings, over the Internet and during an SNMPv3 Interoperability Testing event (a report is attached to this memo). Each of the interoperability activities have included a different set of implementations. Implementations from the following organizations are known to have participated in one or more of the interoperability testing activities:
There are additional implementations which are expected to meet at least portions of the SNMPv3 specifications but, at this point, may not have participated in interoperability testing:
Interoperability Testing Summary:
The flexible and modular nature of the SNMPv3 specifications result in a very large quantity of what could be considered specific features. Rather than providing an exhaustive enumeration of details, a more useful approach to documenting interoperability is identifying the significant features and capabilities of the SNMPv3 specifications. The following list constitutes those features and capabilities. The list was drawn from the five core specifications for SNMPv3. With few exceptions, each of the implementations participating in interoperability testing supports all of these features and capabilities. The Proxy Forwarder Functions and SNMP-PROXY-MIB are the only known exceptions; these have only been implemented and tested in the SNMP Research and University of Quebec products. Testing methodology generally used a stimulus - response approach where a certain input (stimulus) initiated an action that ended with an expected result (response). In some cases, success of the test meant that some action failed, e.g., failure due to an incorrect key or a requested action not being permitted by the access control. RFC 2438 provides BCP guidance for testing of MIBs. The MIBs in the specifications have generally been tested as described in the Basic Object Comparison or Stimulus/Response sections of Appendix A to RFC 2438.
The following significant capabilities of the SNMPv3 specifications have been implemented and interoperability demonstrated by at least two (in most cases several) of the implementations identified above:
General Architecture and Framework
Message Processing and Dispatching Subsystem
Security Subsystem, User-based Security Model
Access Control Subsystem, View-based Access Control Model (VACM)
Report on 21-22 July 98 Interoperability Test
From owner-snmpv3 Tue Aug 25 01:11:57 1998
Attached is the test report from the SNMPv3 interoperability testing conducted in July 1998.
SNMPv3 Testing Report
An SNMPv3 Interoperability Testing event was held on July 21-22, 1998. Six implementations of "manager" features were tested against four implementations of "agent" features. All major aspects of the protocol and supporting MIB objects were successfully exercised. Although not all implementations were complete and correct, there were at least two independent and interoperable implementations of each tested feature. Many minor aspects of the protocol and MIB objects received only cursory testing or no testing. In at least one case (proxy forwarding) there was only one independent agent application.
Several implementation errors were identified (and often corrected in real-time) but there were no new problems with the specification that had not already been identified and discussed on the mailing list. The problematic areas of the specification were tested thoroughly and recommended text was generated. (Bert "understands the edits.") All of the systems which had implemented these features were tested and it was found that these problem areas were implemented in the same way and they successfully interoperated.
The specifications and implementations appear to be reaching a level of maturity and interoperability which justifies promotion to Draft Standard after relatively minor editorial changes, should the Working Group wish to forward a recommendation for advancement.
The testing event was held at BMC facilities in Sunnyvale, California on Tuesday, July 21 and 22, 1998.
There were five independent implementations of "manager" features (which in several cases were primarily command-line testing tools):
There were four independent implementations of "agent" features (which in several cases included run-time extensibility capabilities:
The testing was performed using a sets of matrices, with one page devoted to each set of features tested.
Basic Packet Interchange
We began we basic packet interchange testing, proceeding through noAuthNoPriv, md5AuthNoPriv, md5AuthDESPriv, shaAuthNoPriv, and shaAuthDESPriv. All worked ok, probably because this had been tested before during testing in April (IETF LA), May (Interop), and June (HP OpenView Forum). Two implementations did not support DESPriv; the others supported all flavors.
In the process of performing this testing, we found that:
the error reporting is useful for diagnosing incorrect configurations; and
when managers are not configured, or secrets are incorrect, transactions are properly blocked and/or views are properly limited
One manager implementation was equipped to send incorrectly formatted messages as well as to test boundary conditions with regard to model numbers, securityLevels, etc. This test tool was used to probe weaknesses in the agent implementations.
Most implementations were found to have relaxed range checking on received values for msgID and msgMaxSize, although this is not particularly significant for very large msgMaxSize values. Further testing should probably be performed to determine if the implementations handle, at the very least, the case where msgMaxSize is smaller than the response being generated. msgMaxSize values below the valid range were tested, but were not compared with the size of the response.
The implementations exhibited different behaviors when receiving bad and unknown values for msgSecurityModel. This area is known to need some clarification or enhancement such as the addition of a noSec security model or other behavioral definitions. The results were as follows:
For an unknown msgSecurityModel (99), three implementations dropped the request and incremented snmpUnknownSecurityModels. Of these only one returned a REPORT. The fourth implementation dropped the packet and incremented snmpInASNParseErrs instead. These results were consistent for both null and arbitrary msgSecurityParameters.
For a bad msgSecurityModel (-2), three implementations incremented snmpUnknownSecurityModels. The fourth implementation returned a REPORT, probably also after incrementing snmpUnknownSecurityModels. The expected result in this case was for snmpInASNParseErrs to be incremented.
For another bad msgSecurityModel (2147483650), two implementations incremented snmpUnknownSecurityModels, one implementation incremented snmpInASNParseErrs, and one implementation returned a REPORT, probably also after incrementing snmpUnknownSecurityModels. The expected result in this case was for snmpInASNParseErrs to be incremented.
For msgSecurityModel encoded with the wrong tag, three implementations incremented snmpInASNParseErrs and one implementation returned a REPORT, probably after incrementing snmpUnknownSecurityModels. The expected result in this case was for snmpInASNParseErrs to be incremented.
Most implementations were able to perform snmpEngineID discovery under the various circumstances tested. These circumstances involved different values for msgUserName and contextEngineID. There is some ambiguity in the RFC's as to whether or not the user must be "initial" in order to perform snmpEngineID discovery and, if not, whether it is valid for msgUserName to be a zero-length string.
Three implementations performed snmpEngineID discovery using all combinations of msgUserName and contextEngineID tested. Additionally, all three of these allowed msgUserName to be a zero-length string, although msgUserName is defined with a minimum size of 1 octet. One implementation required the user to be "initial" for snmpEngineID discovery to be performed.
The support for the MIB modules was examined. IWL and SNMP Research used Tcl/Tk scripts to examine the quantity and [data] type of information returned. All agent implementations fully implemented the objects found in the SNMP-FRAMEWORK-MIB and the SNMP-VIEW-BASED-ACM-MIB although one did not support non-volatile storage for the "boots" object [and the right thing happened when managers discovered that time had gone backwards]. All but one fully implemented the SNMP-MPD-MIB objects.
All implemented the SNMP-USER-BASED-SM-MIB, but an implementation which did not support DES Priv did not implement MIB objects related to security keys for use with privacy. Not all systems supported non-volatile storage yet.
Two systems supported the SNMP-TARGET-MIB and SNMP-NOTIFICATION-MIB modules and one implementation supported the SNMP-PROXY-MIB definitions.
All objects were found to be of the correct data type.
User Cloning and Key Change
The MIB objects which support user cloning and key change were tested extensively. Some overall testing was performed, followed by some more focused tests.
One manager tool was successfully used to create (clone), use, and delete users and views successfully. The created views were intentionally made to be different from noAuth, authNoPriv, and authPriv securityLevels (using both the included and excluded features) and the results were found to be as expected. Another manager was used to create (clone) users and to associate them with existing vacmTable entries and views.
It was found that these newly created users could be used successfully by independently implemented managers, i.e., that the clone-from and key change operations produced the expected results.
However, these tools did not test some specific aspects of the design and additional tests were performed to exercise these areas: create/clone user using MD5 and MD5&DES and use newly created user to walk MIB using noAuth, authNoPriv, and authPriv; have another manager use cloned user create/clone user using SHA and SHA&DES and use newly created user to walk MIB using noAuth, authNoPriv, and authPriv; have another manager use cloned user use created user to perform "own" key change on the auth and priv keys using first MD5, then SHA and then use the result by an independent implementation use key change without clone (a "root" key change) and test as above
It was a pleasant surprise that these tests worked *_very_* well, even when using "unfriendly" command-line tools. [Command recall to avoid typos was a blessing.]
There was an "ah ha" moment when conducting this testing. We often used the initial configurations from the Networld+Interop testing (usernames initial and initial2) to clone users with new passwords: "bertbert" for the auth key and "bertbertbert" for the priv key (before password to key transformation and localization). Because one command-line implementation uses the same passphrase to generate the auth key and priv key, it was learned serendipitously that passphrases of "bertbert" and "bertbertbert" result in the same key. Now that we know it, it is obvious, but it was not obvious before we knew it. It was generally felt that it would be a good idea to add a note to the security considerations section of the USM document which states why this is true and that one should not attempt to make passphrases longer by repeating the same pattern.
The group then divided to focus on the remaining goals; in part because not all implementations had all features, and in part because we needed to use our time effectively. One group (IBM and SNMP Research) tested traps and informs extensively. The other group (primarily BMC and ISI) tested view-based access control and views extensively.
Traps and Informs
Two managers and two agent implementations were used to test virtually every combination of user, securityModel, securityLevel, and trap versus inform:
In the process of investigating why some messages were not emitted, we found (unintentionally) that view-based access control on notifications works as designed. Once the tables were configured properly, all messages were emitted as expected.
View-based Access Control and Views
Prior to intentional testing of access control and views, there was some amount of testing as a byproduct of testing other features. For example, one tool which tested adding user cloning and key changes also created and used views.
When we attempted to specifically test VACM, we added several entries to the vacmAccessTable and had managers try and use these new entries. Unfortunately we were able to update the AccessTable for only one agent at that time (other agents either were testing other items, or couldn't have their AccessTable changed).
We had three managers try to get information from one agent and found the agent to be choosing the proper view given the group, prefix, match and security level information. The one oddity that showed up in the testing was the effects of the ordering of entries (step 2 of the description of the vacmAccessTable). Step (b) can cause a more restrictive view associated with a lower security level to be chosen over a less restrictive view from a higher security level but with a shorter (non-identical) prefix.
It is believed that this is not a problem when writing code to implement the VACM tables. It may need some explanation during deployment in order to get the tables to do what the user desires.
It was noted as the result of testing one implementation that a correct implementation of VACM is not always an efficient one. When processing getnext requests to subagents, master agents need to be smart enough to "skip" past VACM excluded regions instead of blindly walking through them and testing for isAccessAllowed() at each interim result. Actual bakeoff tests with one version of an un-optimized (but otherwise correct) VACM implementation showed that VACM can severely degrade performance under certain conditions and if not implemented in a way to maximize efficiency. One suggestion is that the RFC should probably provide some discussion as to this design consideration issue.