idnits 2.17.1 draft-verdt-netmod-yang-versioning-reqs-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 1, 2018) is 2116 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 8049 (Obsoleted by RFC 8299) Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Clarke, Ed. 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational July 1, 2018 5 Expires: January 2, 2019 7 YANG Module Versioning Requirements 8 draft-verdt-netmod-yang-versioning-reqs-00 10 Abstract 12 This document describes the problems that can arise because of the 13 YANG language module update rules, that require all updates to YANG 14 module preserve strict backward compatibility. It also defines the 15 requirements on any solution designed to solve the stated problems. 16 This document does not consider possible solutions, nor endorse any 17 particular solution. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on January 2, 2019. 36 Copyright Notice 38 Copyright (c) 2018 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2.1. Striving for model perfection . . . . . . . . . . . . . . 3 56 2.2. Some YANG Modules Are Not Backward Compatible . . . . . . 3 57 2.3. Non-Backward Compatible Errors . . . . . . . . . . . . . 4 58 2.4. No way to easily decide whether a change is Backward 59 Compatible . . . . . . . . . . . . . . . . . . . . . . . 4 60 2.5. No good way to specify which module revision to import . 5 61 2.6. Early Warning about Removal . . . . . . . . . . . . . . . 6 62 2.7. Clear Indication of Node Support . . . . . . . . . . . . 6 63 3. Terminology and Conventions . . . . . . . . . . . . . . . . . 7 64 4. The Problem Statement . . . . . . . . . . . . . . . . . . . . 7 65 5. Requirements of a YANG Versioning Solution . . . . . . . . . 9 66 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 10 67 7. Security Considerations . . . . . . . . . . . . . . . . . . . 11 68 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 69 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 70 9.1. Normative References . . . . . . . . . . . . . . . . . . 11 71 9.2. Informative References . . . . . . . . . . . . . . . . . 12 72 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 74 1. Introduction 76 This requirements document initially considers some of the existing 77 YANG module update rules, then describes the problems that arise due 78 to those rules embracing strict backward compatibility, and finally 79 defines requirements on any solution that may be designed to solve 80 these problems by providing an alternative YANG versioning strategy. 82 2. Background 84 The YANG data modeling language [RFC7950] specifies strict rules for 85 updating YANG modules (see section 11 "Updating a Module"). Citing a 86 few of the relevant rules: 88 1. "As experience is gained with a module, it may be desirable to 89 revise that module. However, changes to published modules are 90 not allowed if they have any potential to cause interoperability 91 problems between a client using an original specification and a 92 server using an updated specification." 94 2. "Note that definitions contained in a module are available to be 95 imported by any other module and are referenced in "import" 96 statements via the module name. Thus, a module name MUST NOT be 97 changed. Furthermore, the "namespace" statement MUST NOT be 98 changed, since all XML elements are qualified by the namespace." 100 3. "Otherwise, if the semantics of any previous definition are 101 changed (i.e., if a non-editorial change is made to any 102 definition other than those specifically allowed above), then 103 this MUST be achieved by a new definition with a new identifier." 105 4. "deprecated indicates an obsolete definition, but it permits new/ 106 continued implementation in order to foster interoperability with 107 older/existing implementations." 109 The rules described above, along with other similar rules, causes 110 various problems, as described in the following sections: 112 2.1. Striving for model perfection 114 The points made above lead to the logical conclusion that the 115 standardized YANG modules have to be perfect on day one (at least the 116 structure and meaning), which in turn might explain why IETF YANG 117 modules take so long to standardize. Shooting for perfection is 118 obviously a noble goal, but if the perfect standard comes too late, 119 it doesn't help the industry. 121 2.2. Some YANG Modules Are Not Backward Compatible 123 As we learn from our mistakes, we're going to face more and more 124 backward-incompatible YANG modules. An example is the YANG data 125 model for L3VPN service delivery [RFC8049], which, based on 126 implementation experience, has been updated in a backward- 127 incompatible way by [RFC8299]. 129 While Standards Development Organization (SDO) YANG modules are 130 obviously better for the industry, we must recognize that many YANG 131 modules are actually generated YANG modules (for example, from 132 internal databases), which is sometimes the case for vendor modules 133 [RFC8199]. From time to time, the new YANG modules are not backward- 134 compatible. 136 Old module parts that are no longer needed, no longer supported, or 137 are not used by consumers need to be removed from modules. It is 138 often hard to decide which parts are no longer needed/used; still the 139 need and practice of removing old parts exist. While it is rare in 140 standard modules it is more common in vendor YANG modules where the 141 usage of modules is more controlled. 143 The problems described in Section 2.7 may also result in incompatible 144 changes. 146 In such cases, it would be better to indicate how backward-compatible 147 a given YANG module actually is. 149 As modules are sometimes updated in an incompatible way the current 150 assumption that once a YANG module is defined all further revisions 151 can be freely used as they are compatible is not valid. 153 2.3. Non-Backward Compatible Errors 155 Sometimes small errors force us to make non-backward compatible 156 updates. As an example imagine that we have a string with a complex 157 pattern (e.g., an IP address). Let's assume the initial pattern 158 incorrectly allows IP addresses to start with 355. In the next 159 version this is corrected to disallow addresses starting with 355. 160 Formally this is a non-backward compatible change as the value space 161 of the string is decreased. In reality an IP address and the 162 implementation behind it was never capable of handling an address 163 starting with 355. So practically this is a backward compatible 164 change, just like a correction of the description statement. Current 165 YANG rules are ambiguous as to whether non-backward compatible bug 166 fixes are allowed without also requiring a module name change. 168 2.4. No way to easily decide whether a change is Backward Compatible 170 A management system, SDN controller, or any other user of a module 171 should be capable of easily determining the compatibility between two 172 module versions. Higher level logic for a network function, 173 something that cannot be implemented in a purely model driven way, is 174 always dependent on a specific version of the module. If the client 175 finds that the module has been updated on the network node, it has to 176 decide if it tries to handle it as it handled the previous version of 177 the model or if it just stops, to avoid problems. To make this 178 decision the client needs to know if the module was updated in a 179 backward compatible way or not. 181 This is not possible to decide today because of the following: 183 o It is sometimes necessary to change the semantic behavior of a 184 data node, action or rpc while the YANG definition does not change 185 (with the possible exception of the description statement). In 186 such a case it is impossible to determine whether the change is 187 backward compatible just by looking at the YANG statements. It's 188 only the human model designer who can decide. 190 o Problems with the deprecated and obsolete status statement, 191 Section 2.7 193 o Modelers might decide to violate YANG 1.1 update rules for some of 194 the reasons above. 196 Finding status changes or violations of update rules need a line-by- 197 line comparison of the old and new modules is a tedious task. 199 2.5. No good way to specify which module revision to import 201 If a module (MOD-A) is imported by another one (MOD-B) the importer 202 may specify which revision must be imported. Even if MOD-A is 203 developed in backward-compatible way not all revisions will be 204 suitable, e.g., a new MOD-B might need the newest MOD-A. However, 205 both specifying or omitting the revision date for import leads to 206 problems. 208 If the import by revision-date is specified 210 o If corrections are made to MOD-A these would not have any effect 211 as the import's revision date would still point to the un- 212 corrected earlier YANG module revision. 214 o If MOD-A is developed in a backward-compatible way because another 215 importer (MOD-C) needs some functionality, the new MOD-A could be 216 used by MOD-B, but specifying the exact import revision-date 217 prevents this. This will force the implementers to import two 218 different revisions of MOD-A, forcing them to maintain old MOD-A 219 revisions unnecessarily. 221 o If multiple modules import different revisions of MOD-A the human 222 user will need to understand the subtle differences between the 223 different revisions. Small differences would easily lead to 224 operator mistakes as the operator will rarely check the 225 documentation. 227 o Tooling/SW is often not prepared to handle multiple revisions of 228 the same YANG module. 230 If the import revision-date is not specified 232 o any revision of MOD-A may be used including unsuitable ones. 233 Older revisions may be lacking functionality MOD-B needs. Newer 234 MOD-A revisions may obsolete definitions used by MOD-B in which 235 case these must not be used by MOD-B anymore. 237 o As it is not specified which revisions of MOD-A are suitable for 238 MOD-B. The problem has to be solved on a case by case basis 239 studying all the details of MOD-A and MOD-B which is considerable 240 work. 242 2.6. Early Warning about Removal 244 If a schema part is considered old/bad we need to be able to give 245 advance warning that it will be removed. As this is an advance 246 warning the part must still be present and usable in the current 247 revision; however, it will be removed in one of the next revisions. 248 The deprecated statement cannot be reliably used for this purpose 249 both because deprecated nodes may not be implemented and also there 250 is no mandate that text be provided explaining the deprecation. 252 We need the advance warning to allow users of the module time to 253 plan/execute migration away from the deprecated functionality. 254 Deprecation should be accompanied by information whether the 255 functionality will just disappear or that there is an alternative, 256 possibly more advanced solution that should be used. 258 Vendors use such warnings often, but the NMDA related redesign of 259 IETF modules is also an example where it would be useful for IETF. 260 As another example, see the usage of deprecated in the Java 261 programming language. 263 2.7. Clear Indication of Node Support 265 The current definition of deprecated and obsolete in [RFC7950] (as 266 quoted below) is problematic and should be corrected. 268 o "deprecated" indicates an obsolete definition, but it permits new/ 269 continued implementation in order to foster interoperability with 270 older/existing implementations. 272 o "obsolete" means that the definition is obsolete and SHOULD NOT be 273 implemented and/or can be removed from implementations. 275 YANG is considered an interface contract between the server and the 276 client. The current definitions of deprecated and obsolete mean that 277 a schema node that is either deprecated or obsolete may or may not be 278 implemented. The client has no way to find out which is the case 279 except for by trying to write or read data at the leaf in question. 280 This probing would need to be done for each separate data-node, which 281 is not a trivial thing to do. This "may or may not" is unacceptable 282 in a contract. In effect, this works as if there would be an if- 283 feature statement on each deprecated schema node where the server 284 does not advertise whether the feature is supported or not. Why is 285 it not advertised? 287 3. Terminology and Conventions 289 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 290 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 291 document are to be interpreted as described in [RFC2119]. 293 In addition, this document uses the following terminology: 295 o YANG module revision: An instance of a YANG module, with no 296 implied ordering or backward compatibility between different 297 revisions of the same module." 299 o YANG module version: A YANG module revision, but also with an 300 implied partial ordering relationship between other versions of 301 the same module. Each module version must be uniquely 302 identifiable. 304 4. The Problem Statement 306 Considering the issues described in the background, the problem 307 definition can be summarized as follows. 309 Development of data models for a large collection of communication 310 protocols and system components is difficult and typically only 311 manageable with an iterative development process. Agile development 312 approaches advocate evolutionary development, early delivery, and 313 continual improvement. They are designed to support rapid and 314 flexible response to change. Agile development has been found to be 315 very successful in a world where the objects being modeled undergo 316 constant changes. 318 The current module versioning scheme relies on the fundamental idea 319 that a definition, once published, never changes its semantics. As a 320 consequence, if a new definition is needed with different non- 321 backward compatible semantics, then a new definition must be created 322 to replace the old definition. The advantage of this versioning 323 scheme is that a definition identified by a module name and a path 324 has fixed semantics that never change. (The details are a bit more 325 nuanced but we simplify things here a bit in order to get the 326 problems worked out clearly.) 328 There are two main disadvantages of the current YANG versioning 329 scheme: 331 o Any non-backward compatible change of a definition requires either 332 a new module name or a new path. This has been found costly to 333 support in implementations, in particular on the client side. 335 o Since non-backward compatible changes require either a new module 336 name or a new path, such changes will impact other modules that 337 import definitions. In fact, with the current module versioning 338 scheme other modules have to opt-in in order to use the new 339 version. This essentially leads to a ripple effect where a non- 340 backward compatible change of a core module causes updates on a 341 potentially large number of dependent modules. 343 Other problems experienced with the current YANG versioning scheme 344 are the following: 346 o YANG has a mechanism to mark definitions deprecated but it leaves 347 it open whether implementations are expected to implement 348 deprecated definitions and there is no way (other than trial and 349 error) for a client to find out whether deprecated definitions are 350 supported by a given implementation. 352 o YANG does not have a robust mechanism to document which data 353 definitions have changed and to provide guidance how 354 implementations should deal with the change. While it is possible 355 to have this described in general description statements, having 356 these details embedded in general description statements does not 357 make this information accessible to tools. 359 o YANG data models often do not exist in isolation and they interact 360 with other software systems or data models that often do allow 361 (controlled) non-backward compatible changes. In some cases, YANG 362 models are mechanically derived from other data models that do 363 allow (controlled) non-backward compatible changes. In such 364 situations, a robust mapping to YANG requires to have version 365 numbers exposed as part of the module name or a path definition, 366 which has been found to be expensive on the client side (see 367 above). 369 Given the need to support agile development processes and the 370 disadvantages and problems of the current YANG versioning scheme 371 described above, it is necessary to develop requirements and 372 solutions for a future YANG versioning scheme that better supports 373 agile development processes, whilst retaining the ability for servers 374 to handle clients using older versions of YANG modules. 376 5. Requirements of a YANG Versioning Solution 378 The following is a list of requirements that a solution to the 379 problems mentioned above MUST or SHOULD have. The list is grouped by 380 similar requirements but is not presented in a set priority order. 382 1. Requirements related to making non-backward compatible updates to 383 modules: 385 1.1 A mechanism is REQUIRED to update a module in a non-backward 386 compatible way without forcing all modules with import 387 dependencies on the updated module from being updated at the 388 same time (e.g. to change its import to use a new module 389 name). 391 1.2 A mechanism is REQUIRED to update a module in a non-backward 392 compatible way without forcing all clients/servers to access 393 data nodes in the model on new paths, or in a new module 394 namespace. Specifically, if a particular data node is 395 updated in a non-backward compatible way then it may be 396 desirable for it to be available on the same path and in the 397 same module namespace. 399 1.3 A refined form of YANG's 'import' statement MUST be provided 400 that is more restrictive than "import any revision" and less 401 restrictive than "import a specific revision". Once non- 402 backward compatible changes to modules are allowed, the 403 refined import statement is used to express the correct 404 dependency between modules. 406 2. Requirements related to identifying changes between different 407 module revisions: 409 2.1 Readers of modules, and tools that use modules, MUST be able 410 to determine whether changes between two revisions of a 411 module constitute a backward compatible or non-backward 412 compatible version change. In addition, it MAY be helpful 413 to identify whether changes represent bug fixes, new 414 functionality, or both. 416 2.2 A mechanism SHOULD be defined to determine whether data 417 nodes between two arbitrary YANG module revisions have (i) 418 not changed, (ii) changed in a backward compatible way, 419 (iii) changed in a non-backward compatible way. 421 3. Requirements related to supporting existing clients in a backward 422 compatible way: 424 3.1 The solution MUST provide a mechanism to allow servers to 425 support existing clients in a backward compatible way. 427 3.2 The solution MUST provide a mechanism to allow servers to 428 simultaneously support clients using different revisions of 429 modules. A client's choice of particular revision of one or 430 more modules may restrict the particular revision of other 431 modules that may be used in the same request or session. 433 4. Requirements related to managing and documenting the life cycle 434 of data nodes: 436 4.1 A mechanism is REQUIRED to allow a client to determine 437 whether deprecated nodes are implemented by the server. 439 4.2 If a data node is deprecated or obsolete then it MUST be 440 possible to document in the YANG module what alternatives 441 exist, the reason for the status change, or any other status 442 related information. 444 4.3 A mechanism is REQUIRED to indicate that certain definitions 445 in a YANG module will become status obsolete in future 446 revisions but definitions marked as such MUST still be 447 implemented by compliant servers. 449 4.4 If multiple revisions of a YANG module are published, then 450 the solution SHOULD allow for bug fixes to be made to an 451 older revision of the module. 453 5. Requirements related to documentation and education: 455 5.1 The solution MUST provide guidance to model authors and 456 clients on how to use the new YANG versioning scheme. 458 5.2 The solution is REQUIRED to describe how to transition from 459 the existing YANG 1.0/1.1 versioning scheme to the new 460 scheme. 462 5.3 The solution MUST describe how the versioning scheme affects 463 the interpretation of instance data and references to 464 instance data, for which the schema definition has been 465 updated in a non backward compatible way. 467 6. Contributors 469 This document grew out of the YANG module versioning design team that 470 started after IETF 101. The following people are members of that 471 design team and have contributed to defining the problem and 472 specifying the requirements: 474 o Balazs Lengyel 476 o Benoit Claise 478 o Ebben Aries 480 o Joe Clarke 482 o Juergen Schoenwaelder 484 o Mahesh Jethanandani 486 o Michael (Wangzitao) 488 o Qin Wu 490 o Reshad Rahman 492 o Rob Wilton 494 7. Security Considerations 496 The document does not define any new protocol or data model. There 497 is no security impact. 499 8. IANA Considerations 501 None 503 9. References 505 9.1. Normative References 507 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 508 Requirement Levels", BCP 14, RFC 2119, 509 DOI 10.17487/RFC2119, March 1997, 510 . 512 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 513 RFC 7950, DOI 10.17487/RFC7950, August 2016, 514 . 516 9.2. Informative References 518 [RFC8049] Litkowski, S., Tomotaki, L., and K. Ogaki, "YANG Data 519 Model for L3VPN Service Delivery", RFC 8049, 520 DOI 10.17487/RFC8049, February 2017, 521 . 523 [RFC8199] Bogdanovic, D., Claise, B., and C. Moberg, "YANG Module 524 Classification", RFC 8199, DOI 10.17487/RFC8199, July 525 2017, . 527 [RFC8299] Wu, Q., Ed., Litkowski, S., Tomotaki, L., and K. Ogaki, 528 "YANG Data Model for L3VPN Service Delivery", RFC 8299, 529 DOI 10.17487/RFC8299, January 2018, 530 . 532 Author's Address 534 Joe Clarke (editor) 535 Cisco Systems, Inc. 536 7200-12 Kit Creek Rd 537 Research Triangle Park, North Carolina 538 United States of America 540 Phone: +1-919-392-2867 541 Email: jclarke@cisco.com