]> Johns Hopkins University

3400 North Charles Street Baltimore MD 20723 USA +1.443.910.4420 rgcole01@comcast.net http://www.cs.jhu/~rgcole/

Avaya

Atidim Technology Park, Bldg. #3 Tel Aviv 61131 Israel dromasca@avaya.com

InterWorking Labs

303 Potrero Street, Suite 52 Santa Cruz, CA 95060-2760 USA andyb@iwl.com

General Internet Engineering Task Force configuration management, verification, manets, dtns This document extends the capabilities of the NETCONF configuration management protocol in order to standardize mechanisms to perform sets of active tests (i.e., verification) against servers' running configuration over a period of time to afford the client and server a more robust and resilient configuration management capability. This is of value to commercial enterprise and public networks as well as wireless emergency and military networks. We accomplish this through the definition of the new verify.yang module. Servers supporting this module will advertise this capability according to the YANG specification. We also explore the future alternatives for developing these capabilities within the context of the existing NETCONF protocol, the YANG modeling language and existing related IETF, IEEE and ITU-T standards.

This document defines a new verify.yang module to achieve a more robust model of configuration management for future IETF systems. Most network management systems which are required to provide a highly robust network service rely upon some form of out-of-band access for configuration management. This provides an alternative management entry into devices in the event that in-band access is unavailable due to mis-configuration. However, not all network deployments can afford the luxury of alternative networks for management access to all networking devices, nor should this be necessary. Examples include Mobile Ad-Hoc Wireless Networks (MANETs) and other forms of Disruption Tolerant Networks (DTNs). All managed networks, as well, would benefit from a more robust and extensive configuration management capability from the IETF, e.g., to provide equivalent network reliability at reduced infrastructure costs. Towards this goal, we define a new verify.yang module to manage active tests and assess success, i.e., verification, (from both the client and the servers) involving server-side running configuration. Servers supporting this module will advertise this capability according to the YANG specification YANG. As an example, we envision a NETCONF client-server interaction model shown in the below figure. Here, the client issues a <commit> operation with the confirming option. As part of testing prior to issuing the confirming <commit> the client wishes to execute a set of verification tests from the server. It issues the new <verify> operation to manage this aspect of verification testing. The client passes a reference to the server indicating instances of specific pre-configured test sets that define the verification test suite. The server executes these as part of the NETCONF <verify> testing process. Simultaneously, the client may also run a set of tests to gain confidence in the proposed configuration changes to the server. Once the server completes its test execution, it indicates success through notification messages. Once the client is comfortable with its own tests and those of the server, it issues the confirming <commit> to the server which forces the server to commit to the proposed configuration change.

Sets up config +------------------------------> Sets up test control --- +------------------------------> | Sends (set - timeout timeout) - confirm option | | | +------------------------------> | Sends | - timeout | - test-template:instanceIDs | (running (running client-side server-side tests) tests) +--------+ | | | | | <--------+ | (server-side tests | complete) | <-----------------------------+ | notification | | | +-----------------------------> | Sends | | --- ]]>

This, and other use cases for the verify.yang module are discussed further in the 'Framework' section below. NETCONF defines the term 'validation' as the set of checks performed on proposed configuration code up to the point that the server places it into its running-configuration. We use the term 'verification' as the act of performing active tests against configuration code in the running-configuration on the server. (Note: strictly, verification should also cover the act of loading new configuration into the <running> configuration as this may fail, e.g., due to undocumented configuration constraints. However, here we focus on aspects of running active tests to measure network behavior as a form of verification testing.) Verification tests can be executed from either the NETCONF client or the NETCONF server, or from a NETCONF server(a) against running configuration code on a NETCONF server(b), or all combinations. Within the new verify.yang module, we define a set of stand-alone operations, notifications, and requirements on the definition of future test modules for the purpose of managing verification testing on remote servers through standardized mechanisms. This allows for extensible verification testing of configuration across the base of IETF compliant devices. This leads to more resilient configuration management for operators manging multi-vendor networks of devices. This capability may promote future integrated network management capabilities as opposed to device management capabilities. A more detailed presentation of the stand-alone operation of the proposed verification testing is given in the below figure. Here the client issues the <verify> operation indicating the timeout period, the test sets which comprise the overall verification test suite, and the nature of the reporting from the server using the associated notification messages. The 'verifyStatus=true' indicates that the server should send intermediate status reports following completion of each test set in the suite. At the completion of the entire verification test suite, the server always sends the final <verifyComplete> notification to the client unless explicitly canceled by the client. The optional <verifyStatus> and mandatory <verifyComplete> notifications carry indications of test success or failure based upon pre-configured thresholds and metrics defined within the test module(s) resident on the server. Further, the <verify> operation carries test instance identifiers and switches for various types of reporting, i.e., summary or extended. In total, these place requirements on the definition of future interoperable test modules to be developed in support of the verification testing managed through the verify.yang module. We give an example of a small test module in Appendix B. In the 'Framework' section we discuss the set of requirements on the test modules necessary for inter-operation with the verify.yang verification testing.

Sends - timeout - test-template:instanceID=1, test-template:instanceID=263, test-template:instanceID=51 - verifyStatus=true - extendedResults=false --- | +-------+ | | tests 1 (set | timeout) <-------+ | <-----------------------------+ | notification | | +-------+ | | tests 263 | | | <-------+ | <-----------------------------+ | notification | | +-------+ | | tests 51 | | | <-------+ | <-----------------------------+ | notification | <-----------------------------+ | notification | | | --- ]]>

Our objective is the development of a robust and resilient network configuration capability, building upon the improvements afforded by the NETCONF protocol and it's associated modeling language, YANG. The envisioned benefits of a standardized set of mechanisms and capabilities for verification testing include: Minimize faulty configuration. Support for these capabilities in large multi-vendor networks. Minimize long term disconnects in networks with no 'out-of-band' access, e.g., wired-networks, wireless MANETs or DTNs. Appendix C presents a set of example situational cases which illustrate benefits of these enhanced NETCONF capabilities. Provide opportunity for device modelers to associate/recommend tests tied to specific configuration items. Improve efficiency of coordinated network upgrades (See the below discussion in Appendix D). (Note: should we place the presentation of use cases for the verification testing here or leave that discussion in the 'Framework' section below?)

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The outline of the remainder of this document follows. We next give a set of definitions to be adhered to for the remainder of this discussion. We then more formally present the new verify.yang module which acheives initial aspects of the Robust-NETCONF capabilities. We examine in the Framework section the verify mechanisms, their relationship to test modules and definitions of metrics and success criteria. The 'Framework' section also covers use cases of the verify mechanisms. Then 'Acknowledgments' and 'IANA Considerations' are presented. A section on 'Security Considerations' is provided concluding the main body of the document. Various appendices are provided to compliment the text of the main body. Prominent appendices are 'Appendix A: verify.yang' which documents the verify.yang module and 'Appendix B: ping.yang' which presents a simple test module which complies with the requirements of the verify.yang. Other appendices cover motivational cases, potential applications to network-wide upgrade procedures, and related capabilities in existing IETF protocols and systems.

In this section we provide definitions strictly adhered to throughout this document. The NETCONF specification maintains the following terms: NETCONF Client (or client) - this is the management application responsible for the configuration management of network devices. NETCONF Server (or server) - this is the networked device being managed by the clients. We maintain the following distinction between validation checks and verification tests: Validation checks - checking non-running configuration code against a set of rules, constraints or other requirements. This addresses the total set of checks performed prior to the server placing the code into its running-configuration. Verification tests - measuring behavior of running configuration code against a set of expectations or success criteria. This is generally performed through active testing and comparison of results against expectations. This could also be accomplished (given enough time) through observation of state information related to the running configuration, although here we focus on running active measurements. Active measurements perform verification while rule-based checks perform validation. We maintain the following definitions in the descriptions of the :verification testing: Verification test (or test set) - a set of identical measurements identified through a single instance identifier and defined in a test module which is pre-configured on the server. E.g., a verification test could be a set of five pings sent to the same destination address. Verification test suite - the total set of verification tests identified by a set of instance identifiers passed to the server as parameters in the same <verify> message. E.g., the verification test suite could be composed of five ping test sets, each sending ten pings to an IP address which differ for each test set.

In this section we describe operations and notifications, which we collectively refer to as the verify mechanisms. The Yang module for these mechanisms, i.e., verify.yang, is listed in Appendix A below. Servers supporting this module must advertise it according to the YANG specification . An example test module, which complies with the requirements for test modules as spelled out in this document, is the ping.yang module in Appendix B below.

The verification test mechanisms provide a set of standard tools allowing the client to direct verifications tests from remote servers and to collect verification test reports related to the success or failure of the tests. Note: This verify.yang module has several prerequisites, including support for secondary modules on servers and notifications. There are advantages to supporting other capabilities, such as <candidate> configuration and :commit capability as discussed in the use cases in the Framework section. However, the verify.yang module provides stand-alone <verify> and other operations. Secondary modules are required for the definition of specific verification tests. We present an example in terms of a ping.yang module sketched out in Appendix B below. So, a typical client/server interaction follows: Client sets up the <candidate> configuration on all relevant agents. Client sets up all the relevant test control configuration needed for the verification tests on all relevant agents. Client sends <verify> to all agents with parameters (timeout:seconds, test-template:instance-identifier,verifyStatus:true, extendedStatus:false), i.e.,

60 /at:ping/at:pingEntry[at:pingControlIndex=21] /at:ping/at:pingEntry[at:pingControlIndex=42] /at:ping/at:pingEntry[at:pingControlIndex=48] true false ]]>

Server returns <ok/>. The server runs the tests with the specified (e.g., pingControlEntry) configuration subtree. For each completed test set, the server sends a report in a <verifyStatus> notification. At the completion of the entire verification test suite, the server sends a summary report in a <verifyComplete> notification message. The client can adjust the nature of the reporting through the 'verifyStatus' and the 'extendedResults' parameters of the &ly;verify> operation. The former determines whether or not <verifyStatus> notifications are sent from the server following the completion of each test set. The later determines whether or not the <verifyStatus> notifications carry raw test data (as defined within the test modules). Further, the client can decide to immediately cancel all ongoing verification testing by issuing the <cancel-verify> operation. Or the client can decide to gracefully cut short the testing by issuing the <complete-verify> operation which instructs the server to complete only the in-progress test set, to follow up with an optional <verifyStatus> notification for that completed test set if the client had required this notification message and to wrap up the verification process by sending the manditory <verifyComplete> notification.

The verification mechanism requires the existence of test modules resident on servers which comply with the following requirements: Must contain a 'list' keyed by a controlIndex which defines a verification test set. The 'list' must define an unequivocal means to determine the success or failure of the specific verification test set. This information is passed in the <verifyStatus> notification. The 'list' must define a 'leaf-list' of raw results which may be passed to the client through the <verifyStatus> notification.

The server must advertise the verify.yang module capability URI as follows: <capability> file:///draft-cole-netwconf-robust-config-02.txt?module=verify&revision=2010-03-02 </capability>

The verify.yang module defines a number of operations, as described in this section.

The <verify> operation starts the verification tests on the server. The <verify> operation has four parameters: timeout - the timeout period associated with the verification test suite. If the timeout expires, the server should complete the in-progress test, send the <verifyStatus> notification for the test if the 'verifyStatus' parameter is set to 'true' and send the <verifyComplete> notification. test-template - this 'leaf-list' parameter uniquely identifies the suite of verification tests to be performed by the server. Each verification test is identified by an 'instance-identifier' indexing a test definition on a test module resident on the server. verifyStatus - this parameter is a switch which defaults to 'false' indicating no <verifyStatus> notifications are to be sent from the server to the client. When set to 'true' the server should send a <verifyStatus> notification following the completion of each verification test. extendedResults - this parameter is a switch which defaults to 'false' indicating no 'anyxml extendedResults' are to be sent in the <verifyStatus> notification. This parameter can only be set to 'true' if the 'verifyStatus' parameter is also set to 'true'. This indicates that the <verifyStatus> notifications should include the raw measurement results carried in the 'anyxml extendResults'.

The <cancel-verify> operation immediately cancels a verify test suite in progress on the server. The server terminates in-progress tests immediately and is not required to send any followup notification messages carrying test results.

The <complete-verify> operation tells the server to complete the in-progress verification test set and to send any required followup notifications carrying test results. This affords a more graceful shutdown of the ongoing verification test suite by terminating tests following a completion and associated notification of the currently active test set.

The <verifyStatus> notification carries the results for each verification test set comprising the entire verification test suite. This notification may also carry raw extended results per test set. This notification is optional and is requested explicitly by the client sending the 'verifyStatus=true' parameter in the <verify> operation.

The <verifyComplete> notification is mandatory (unless canceled by the <cancel-verify> operation) and carries a summary result covering the entire verification test suite.

None.

This section discusses the relationship between verification mechanisms and supporting test.yang modules, the way metrics and thresholds are defined in order to assess test 'pass/failure' decisions, and presents use cases for the verification testing.

The verification mechanism requires the existence of test modules resident on servers which advertise verify.yang module. For proper interoperation, these test modules must comply with the following requirements: Must contain a 'list' keyed by a controlIndex which defines a verification test set. Specifically, this identifies a control table row of the test template to be executed, which represents the verification test set to be performed on the server as part of the verify operation The 'list' must contain a 'threshold leaf' for the definition of an unequivocal means to determine the success or failure of the specific verification test set. This information is passed in the <verifyStatus> notification. The 'list' identifies a pass/fail measurement based upon a metric for each individual test comprising a test set. The 'threshold' defines the minimum number of successful test measurements for a test set to be declared a success. The 'list' must define a 'leaf-list rawResults' which may be passed to the client through the <verifyStatus> notification. The raw results record the individual transaction history of the test set measurements. The values conform to the units defined for the 'leaf metric'. If requested, this information is passed to the client through the 'verifyStatus' notification's 'anyxml extendedStatus'.

The use cases for the <verify> operation are discussed here. These include: Used within a <commit> operation with the 'confirmed' parameter to enhance the confidence of the client in the current running configuration loaded from the candidate configuration. Used to verify the configuration of server(a) following the re-configuration of server(b). Here, a configuration change on one device could impact the interoperability with another device. Hence, it may be desirable to execute a verification test suite against the modified device. Used to verify the <running> configuration prior to copying it into the <startup> configuration. It is desirable to perform additional checks of the configuration prior to moving into the <startup> configuration.

The authors would like to acknowledge the useful comments and suggestions of M. Bjorklund. He had suggested developing the verification mechanisms as stand alone operations and suggested several of the use cases discussed in the Framework section.

This memo includes no request to IANA. All drafts are required to have an IANA considerations section (see the update of RFC 2434 for a guide). If the draft does not require IANA to do anything, the section contains an explicit statement that this is the case (as above). If there are no requirements for IANA, the section will be removed during conversion into an RFC by the RFC Editor.

This section presents the required security considerations for all IETF protocols and capabilities. This section was developed following guidelines within . This section addresses the security concerns and objectives for the verify.yang module and the associated set of issues tied to overall verification test mechanisms within the YANG modeling language and NETCONF protocol. This section is currently TBD. Security issues related to the verification tests should address issues specific to the proposed operations and notifications. They should also address security issues associated with the development of associated test modules for the purpose of running verification tests. Here is an initial list of potential considerations (in no particular order): Verification requires server-side tests that require that packets to be injected into the network for the purpose of measuring some performance characteristics. As such, associated test modules will contain sensitive network and application data; e.g., user IDs and passwords. Further, if security is compromised, this capability could provide a source for denial-of-service, and potential other, attacks. The configuration of verification tests may require passing sensitive network information. For this reason, this configuration information should be encrypted prior to transport over the network. Some test attributes configure username and password information for some application-level protocols as indicated above. Access to these attributes may provide unauthorized use of resources. Some test attributes configure the size and rate of traffic flows for the purpose of performance measurements. Access to these attributes may exacerbate the use of this capability in denial-of-service attacks. It is recommended that test modules define a maximum packet rate on the device and to indicate this rate. Other objects that control aspects of the test packets related to packet size and rate are will exist in test modules and bounds on these should be set. Test module objects will exist which set the source and destination addresses on the packet headers. The server should not allow the setting of source addresses on the test packets other than those that are administratively configured onto the server.

&RFC2119; &RFC2021; &RFC2074; &RFC3552; &RFC3577; &RFC3729; &RFC4149; &RFC4150; &RFC4377; &RFC4378; &RFC4656; &RFC4687; &RFC4741; &RFC5357; &I-D.narten-iana-considerations-rfc2434bis; draft-ietf-netmod-yang-11 IEEE 802.1 IEEE 802.3 ITU-T Study Group 13 ITU-T Study Group 13 ITU-T Study Group 13

In this appendix we list the verify.yang model for use in conjunction with the robust-netconf capabilities.

operation. The entire collection of test sets is refered to as the verification test suite. * the agent will cancel the verify procedure if the operation is invoked. * the agent will complete the current verification test set and generate the (if requested) and notifications if the operation is invoked. * the agent will start, monitor, and report the verification test(s) during the time interval after this operation, and before the 'timeout' interval has expired. * if indicated by the client in the operation, the agent will generate the notification for each verification test set specified in the 'test-template leaf-list', indicating the result of each verification test. * the agent will generate the notification at the completion of the entire verified commit procedure, indicating the final verify procedure status. * the definition of this capability places requirements on the development of test.yang modules to provide the following set of features: - test suites identified by 'instanceId's, - test suites identify: metric and target, (pass/fail) threshold, (optional) raw data capability. - optional rawResults are to be stored and passed, if indicated by the client. These requirements are defined in section X.X.. * is sent follow each verification test set, if requested by the client, and indicates pass/fail status of test based upon (metric, target, threshold) triplet. It may also carry raw data values from the 'rawResults' node carried within the 's 'anyxml extendedStatus'. "; input { leaf timeout { description "The time interval the agent has to perform the verify operation. If not complete at timeout, then server must issue indicating partial test results and that verification tests are being terminated."; type uint32; units seconds; default 60; } leaf-list test-template { description "Identifies a verification test control entry or entries for the agent to use for the verification procedure. The verification test control entry must conform to the requirements specified in section X.X, and the agent must be capable of starting, monitoring, and reporting the results of the verification test, as required. The agent will also generate the notification, as specified for each verification test control entry indicated by this parameter."; ordered-by user; type instance-identifier; min-elements 1; } leaf verifyStatus { description "A switch indicating the use of the notification associated with each verification test in the verification test suite. Instead, it only wants to receive the final notification which contains a summarized pass/fail result for the verification test suite. If 'true', then the client is requesting that the server generates notifications for each verification test in the verification test suite."; type boolean; default false; } leaf extendedResults { description "A switch indicating that the client is requesting raw test results through the 'anyxml extendedResults'. This defaults to 'false'. This can only be set to 'true' if the proceeding 'verifyStatus' leaf is set to 'true'. Else, the server should generate an error response to this request."; type boolean; default false; } } } rpc cancel-verify { description "Cancel a verify procedure already in progress. If no verify procedure is currently in progress, then an 'operation-failed' error is generated, and the value 'no-verify' is used for the error-app-tag field. If the verify procedure in progress cannot be canceled for any reason, then an 'operation-failed' error is returned, and the value 'cancel-failed' is used in the error-app-tag field. If any verification tests associated with this verify procedure are still in progress, they will be immediately canceled by this operation. If the verify procedure in progress is canceled, then the agent will return . "; } rpc complete-verify { description "Complete a verify procedure already in progress. If no verify procedure is currently in progress, then an 'operation-failed' error is generated, and the value 'no-verify' is used for the error-app-tag field. If the verify procedure in progress cannot be completed for any reason, then an 'operation-failed' error is returned, and the value 'complete-failed' is used in the error-app-tag field. If any verification test sets associated with this verify procedure are still in progress, the current test set will be completed and any associated notifications will be sent. Test sets following the current in-progress test set will not be executed. The agent will return if it is able to initiate the4 completion of the current test set. This does not indicate that the current test set has completed. This is indicated when the server issues the notification. "; } notification verifyStatus { description "Contains the current or final status of a verification test being invoked on behalf of the current verify procedure."; list eachTest { key "testIdentifier"; leaf testIdentifier { description "Indicates which verification test this status report is associated with. This value will identify the same node as specified in a 'test-template' parameter instance provided in the operation."; type instance-identifier; mandatory true; } leaf statusType { description "Indicates the type of status report that this notification contains."; type enumeration { enum partial { description "Indicates this is a partial status result for this verification test which is still in progress."; } enum final { description "Indicates this is the final status result and this verification test which completed or canceled."; } } mandatory true; } leaf status { description "Indicates the NETCONF error-tag value most closely associated with the test status. The string 'ok' is used to indicate that the pass threshold for the test has been exceeded."; type string; reference "RFC 4741bis, Appendix A"; mandatory true; } anyxml extendedStatus { description "Indicates verification test-specific status data. The requirements for verification tests (section X.X) describes how the semantics of this structure are determined."; } } } notification verifyComplete { description "Contains the final status of the current verify test suite."; leaf status { description "Indicates the NETCONF error-tag value most closely associated with the test status. The string 'ok' is used to indicate that the pass thresholds were exceeded for all tests in the verification test suite."; type string; reference "RFC 4741bis, Appendix A"; mandatory true; } } } ]]>

In this appendix we list an example ping.yang model for use in conjunction with the verification test management mechanisms identified in the verify.yang module. Specifically, the <verify> operation passes the instance-identifiers in the 'test-template' parameter. Each instance-identifier identifies a specific ping test. The <verify> operation manages the identification, execution and reporting of multiple tests within a single verification test procedure.

3600 /at:ping/at:pingEntry[at:pingControlIndex=21] /at:ping/at:pingEntry[at:pingControlIndex=42] /at:ping/at:pingEntry[at:pingControlIndex=48] true false ]]>

In this appendix we motivate the need for more robust configuration management through a set of example cases and failure situations. We solicit other cases from readers. One note, not all of these cases currently apply to the application of NETCONF configuration management for various reasons not of interest here. But we do believe that future implementations and versions of NETCONF will be applied to all these use cases; so we include them here.

This section discusses a potential failure in configuration management in the case of a multi-frequency, multi-domain wireless Mobile Ad-hoc Network (MANET) scenario. Here there is a single NETCONF client connected to both MANET domains. The MANET domains are operating on different wireless frequencies. MANET_1 operates on freq_1 while MANET_2 operates on freq_2. In MANET_2 is the Server in question, which is indicated with an 'X'. Other nodes in the MANETs are indicated with a 'O'. The following sequence of events follow. The Client issues a <commit> operation with the :confirmed capability. Part of the new configuration pushed to the Server, i.e., 'X', includes inadvertently changing its operating frequency from freq_2 to freq_1. However, the Server maintains connectivity back to the Client through the MANET node indicated as '@' which sits on the border of MANET_1 within radio range of the Server. This allows the Client to confirm its connectivity tests to the Server and then finally issue a confirming-commit. The Server then moves deeper into MANET_2 and becomes disconnected from the Client and all other nodes within MANET_2 do to the erroneous change in its operating frequency. The Client has no means at this point to reconnected to the Server and fix its configuration.

O | | | | O O | | O O O O | | O O | | O | | O | | O O | | O O O| | O | | O | | OO | | O | +---------------+ +----------------+ MANET_1 MANET_2 CLIENT/SERVER INTERACTIONS: --------------------------- Client Server(X) ------ --------- w/confirm (changing to freq_1) +-------------------------> +Changes configuration. +Sets timer. | +Executes | ping tests. | | +Connectivity | confirmed. | | --- +-------------------------> +Verifies configuration. +Stops timer. +Wanders off into MANET_2 and looses connectivity to client. ]]>

Our proposed solution is to have the server perform its own connectivity tests to a set of critical neighbor or peer nodes. This would allow the server to realize the incorrect frequency setting. It would then need a means to indicate back to the Client that a configuration error has occurred. Then the client would not issue the confirming commit operation and the server would back out into its previous configuration.

This section is TBD.

|X---> O | | | | O O O O | | O | | O O | | O | | OO | +---------------+ MANET_1 CLIENT/SERVER INTERACTIONS: --------------------------- Client Server(X) ------ --------- w/confirm (changes to ipTables) +-------------------------> +Changes configuration (looses connectivity to all neighbors but client). +Sets timer. | +Executes | connectivity tests. | | +Connectivity | confirmed. | | --- +-------------------------> +Verifies configuration. +Stops timer. +Wanders off into MANET_1 and looses connectivity to client. ]]>

This is a rather extreme use case, but one which is of interest to address within the Disruption Tolerant Network (DTN) development community. DTNs are characterized by large and/or intermittent delays between network systems. Clearly there are numerous issues to be worked in order to achieve NETCONF configuration management over DTNs. This use case illustrates just one example issue. Here, the NETCONF Client issues a commit with the confirm capability to the DTN's Bundle delivery protocol. By the time the configuration change request reaches the distant, remote server the client and server have no immediate connectivity. Hence, any testing performed by the client to verify the proposed configuration changes on the server are bound to fail. If this is the only means to perform verification of running configurations then this form of management over DTNs is bound to always fail.

|O O | | | | O O O O | | O | | O O | | O O|------>server | OO | +---------------+ DTN CLIENT/SERVER INTERACTIONS: --------------------------- Client Bundle Delivery Server(X) ------ --------------- --------- w/confirm (long delivery delay) +-------------------------> +Changes configuration (but has no current communication to Client). +Sets timer. | +Cannot execute | connectivity tests. | | +Cannot confirm changes, | will always fail. | | --- +Stops timer. +Backs out of configuration change. ]]>

In this use case, the Server is dual homed over two different ISPs, A and B. The link to ISP B is currently the primary router path between the server and client. The two ISPs are very protective of the specifics of their internal networks and block all attempts of external devices to probe the internals of their network, e.g., pings, traceroutes, etc are blocked. The client issues a configuration change to the server via the commit with confirm capability. The new configuration is flawed and causes the server to loose connectivity over the backup link_a path. The client performs connectivity tests to the Server, which succeed due to the presence of the primary path over link_b. The client issues the confirming commit and the server commits to the current configuration. Sometime later, link_b fails and the server becomes totally disconnected and the client cannot access the server to fix it.

w/confirm (changes cause server to loose connectivity over backup link_a) +-------------------------> +Changes configuration (looses connectivity over link_a, but not link_b). +Sets timer. | +Executes | connectivity tests | (running over link_b) | +Connectivity | confirmed. | | --- +-------------------------> +Verifies configuration. +Stops timer. +Link_b fails and server looses all connectivity. ]]>

One further point regarding network versus device management and the utility of more extensive verification mechanisms within NETCONF and YANG. The NETCONF protocol is currently defined to provide a set of operations and optional capabilities which afford management applications a configuration framework which improves previous capabilities. Specifically, as described in Appendix D of NETCONF RFC 4741, the following client to server procedure is possible within NETCONF: Acquire a configuration 'lock' - prevent other applications from simultaneously modifying the same sections of the device configuration. Load configuration update - move the desired new configuration to the managed device. Verify the configuration (syntax) - perform a syntax check on the new configuration code. Checkpoint the <running> configuration - save the old configuration in case the device needs to back out of the desired changes. Change the <running> configuration - move the proposed configuration changes over to the <running> configuration using, e.g., the ':confirmed-commit' capability. Validate the new configuration - within the time limits set in the ':confirmed-commit' the application can perform a set of tests, e.g., 'ping', or inferential checks, e.g., pull routing information from the device or peers, to build some confidence in the proposed configuration changes. If the application is not satisfied with the tests and checks available to it, it can withhold the 'confirming-commit' forcing the device to back out of the desired configuration changes. Make the changes permanent (if desired) - Release the configuration 'lock' - This represents an significant step forward from a reliance upon SNMP for configuration management. However, further improvements are desirable, specifically in the definition and automation of tests associated with Step 6 above. Herein lies our interests and the focus of the framework discussion outlined in this document. With respect to the above procedure, extensions to network-wide configuration changes are limited to a serial repetition of the above procedure for each network device. This may prove awkward for large numbers of devices; if one device fails to upgrade its configuration the client has to back out of all previous device upgrades serially. Whereas, an enhanced validation and, specifically, an enhanced verification capability may result in improved methods and procedures for network-wide configuration updates. As an example, the following network upgrade procedure may be feasible. Configure N devices with appropriate configuration changes in candidate configuration files. Regular YANG 'static' file checking used to make sure first <commit> will work on each device. Issue <commit> with confirmation parameter to move new configuration into running. Issue <verify> (#1) to all devices with extra parameters identifying the master test template to run on each device (if needed). Run all the tests according to the template(s) and report the results to the client with internal code. Servers will issue a pass/failed notification and save a detailed report as well. Client issuing all these tests waits for notifications or polls the agents for the pass/fail (i.e., done) flag NMS can let all tests finish or cancel all tests/commits on first failure reported (with <cancel-verify> RPC operation). All agents report OK; issue all confirming <commit> (#2) operations to finish robust configuration change, or Analyze detailed reports from agents that failed to see what network/device/bug/other conditions are preventing the test(s) from passing. This is an opportunity to do some network management, not just device management. Clearly this is an area for further study.

In this appendix we identify existing protocol capabilities which may play a role in extending NETCONF verification mechanisms and specifications for improved configuration management. This is by no means meant to be an exhaustive, all-inclusive list. It is merely intended to better reinforce this proposal and give an appreciation of its potential mechanisms currently available in other contexts.

Here we highlight existing NETCONF mechanisms associated with validation checking and verification testing configuration changes prior to committing to those changes. We conclude this section with a potential list of extensions to NETCONF which may be necessary to accomplish improved configuration management. The NETCONF protocol is a new tool for configuration management over IP networks. The NETCONF protocol current supports a set of configuration operations, including: <get-config>, <edit-config>, <copy-config>, ...., <commit>. NETCONF servers can advertise capabilities upon initial session establishments. One capability is the ':validate' capability. When implementing the ':validate' capability, the server ``checks at least for syntax error ...'' (reference NETCONF). This level of checking can be tied directly to the <edit-config> operation through the operation test-option: 'test-then-set' if the server advertises :validate capability (NETCONF sect 8.6). This forces the server to perform syntax checking during the <edit-config> operation. We describe this as validation checking made against non-running configuration code. However, NETCONF and YANG do not fully define this 'validation' capability. Currently only limited syntax checking is defined. Yang proposes to extend this capability by adding 'constraints' checking through the definition of XPATH relationships within the server management model. We propose that further and useful extensions should be included to cover more general cross-management model relationships, a ka, 'validation' statements. The 'writable-running' capability allows the <copy-config> operation to define the <running> configuration to be the target. However, in this case, we believe that the checks are to be performed prior to copying the proposed configuration to the <running> configuration. Hence, we still maintain that this is validation. A further NETCONF capability is the ':confirmed-commit' capability. This allows the client to instruct the server through the optional <commit> operation's parameters, 'confirmed' and 'confirmed-timeout', to run the desired configuration changes for a period of time, until it either receives a 'confirming commit' from the client and commits, or times out and reverts back to the prior configuration. This gives the client time to perform an unspecified set of verification tests to build confidence in the desired changes prior to instructing to commit. However, NETCONF does not specify or recommend the tests to be performed, nor the success criteria for the tests, nor does it specify how the server can actively participate in the test phase of the 'commit' and 'confirmed-commit' procedure. The following enhancements are in consideration for improved verification testing and validation checking of proposed configuration code: Enhance the <validate> operation to include a greater set of validation checks on the proposed configuration. These may include specifying tests through reference, i.e., URL, or through explicit device models, e.g, constraint checks defined through YANG. This would allow for improved validation. Define a <verify&ge', and associated operations, to pass specific tests for the server to run (in addition to any tests that the client may be running) prior to allowing the complete commit operation to occur. This would also require a method to specify the success criteria associated with the specified tests. The tests and their success criteria could be specified by reference, e.g., URI, or by explicit definitions, e.g., YANG, or by other means. This would allow for verification of the configuration.

In this section we discuss relevant aspects of the YANG modeling language. We conclude this sections by identifying some areas for potential enhancements to YANG or new applications of the existing YANG language. The YANG 'must' statement extends the ':validate' capability beyond simple syntax checking by including checking of 'must' constraints specified in the device model through YANG. (YANG sect 7.5.2) ``.. when a configuration data-store is validated, all 'must' constraints (defining necessary relationships between device configuration parameter values) are conceptually evaluated ...''. Further, (YANG sect 7.5.2) ``.. all such constraints must evaluate to true ..'' prior to copying the new configuration. This allows for a richer set of validation checks. YANG, as a device modeling language, could be used to define an extensible set of tests through a specific test device model akin to the SSPM-MIB RFC 4149 defined within RMON RFC 2021. This would allow specific tests to be indicated from the application to the device associated with specific configuration changes. The SSPM-MIB relies upon extensible methods (described below) to define broad sets of network tests. Our simple ping.yang module is an example of this approach. This notion can be extended to other network capabilities, such as the tests afforded through recent Operations and Management (OAM) enhancements to 'Carrier Class Ethernet' services or MPLS services. Specifically, YANG models can be defined to exercise 'Continuity', 'Fault', 'Performance' and 'SLA' monitoring provided by these subnet technologies as defined in 802.1ag, 802.3ah, Y.1730, and Y.1731, and identified in RFC 4377, RFC 4378, RFC 4687, and VIGO, Clearly, this can be extended to other active test capabilities not explicitly identified in this document. YANG, as a device modeling language, could be used to associate within each device model itself, tests explicitly associated with configuration objects. This would afford the device modelers the ability to recommend associated (optional) tests tied to desired object changes. Clearly, associated success criteria parameters would need to be modeled as well. YANG, as a device modeling language, could be used to specify references to associated tests, or test scripts, and test success criteria, e.g., through URIs. See for more information. The following enhancements or work items are in consideration: There may be additional relevant YANG extensions (YANG sect 7.17) to define further the NETCONF ':validate' procedure similar to the already defined constraints checking. YANG models may be required to define extensible network tests and associated success metric parameters. Future YANG device models may contain test definitions and success criteria or references to these.

RMON defined a set of capabilities related to definition and execution of network tests which may be valuable to this discussion. Refer to , , , , and for further information. The RMON 'protocol-ID' and AppLocalIndex (APM-MIB) define an extensible method to specify application/protocol network transactions. These have proven useful in the definition of network monitoring and reporting and in the specification of specific network-level active tests. These constructs can be moved forward into YANG models to provide similar benefits in the context of NETCONF configuration management. These could be used to specify a rich set of network tests, e.g., the SSPM-MIB. RMON Synthetic Sources for Performance Management MIB (SSPM-MIB) uses AppLocalIndex to define network measurements and remotely instrument such measurements. SSPM-MIB does not specify success criteria, i.e., ``What constitutes success''. This model can be defined within YANG and enhanced to potentially incorporate success criteria along with the test specification. SSPM-MIB does not report measurements; these are collected via APM-MIB and TPM-MIB . Collectively, this capability set may need to be carried forward. RMON developed the control table and report table constructs. These allow a management application to instruct a remote device to monitor specific performance objects and to construct reports which can be collected in bulk at a later time. This capability may be desirable in the development of tests and reports to influence a new Robust-NETCONF capability.

Carriers are actively deploying new metropolitan data services based upon 'Carrier Class Ethernet Services'. In order to support robust and reliable data services new Operations and Management (OAM) capabilities are being defined in IEEE and ITU-T standards. Of particular interest to our discussion are the active test capabilities defined within the IEEE 802 set of standards, i.e., 802.1ag, 802.3ah, Y.1730, and Y.1731. These define OAM active test capabilities providing for continuity testing and fault isolation measurements, as well as end-to-end performance measurements of delay, delay variation and loss. For these technologies, these active measurement capabilities can be exercised by the new <verify> operation.

Carriers are actively deploying new metropolitan data services based upon 'MPLS Services'. As with Carrier Class Ethernet deployments, new OAM capabilities need to be defined. Current work to date primarily involves the definitions of requirements for these capabilities. These are discussed in , , , Y.1710, and VIGO. Once defined, we can envision exercising active tests to Verify proposed configuration changes to these MPLS-based carrier services. Automatically coupling proposed configuration changes to verification tests relying upon defined OAM active measurements of the resulting MPLS service instance will provide a robust configuration management capability for carriers while simplifying their configuration management Manual Methods and Procedures (MMPs).

The IPPM Working Group has developed several measurement protocols for active measurements of metrics defined in various IPPM WG documents. Specifically, the One-way Active Measurement Protocol (OWAMP) and the Two-way Active Measurement Protocol (TWAMP) are defined in , and . These allow for the generation of active test measurements for precise performance measurements across IP networks. These specify the nature of the traffic generation, the collection process and the data reduction methods to achieve precise performance metrics. The measurement protocols define their own packet formats; hence these protocols are not intended for broad continuity tests such as obtainable through the SSPM-MIB. Instead they are developed for precise performance measurements. In applications where concern with the impact of configuration changes on fine grained network performance is important, then methods to automatically invoke these types of tests through the NETCONF protocol and YANG models become interesting.