idnits 2.17.1 draft-ietf-netmod-yang-versioning-reqs-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: 1.2 Non-backwards-compatible updates of a module MUST not impact clients that only access data nodes of the module that have either not been updated or have been updated in backwards-compatible ways. -- The document date (December 30, 2019) is 1573 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Obsolete informational reference (is this intentional?): RFC 8049 (Obsoleted by RFC 8299) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Clarke, Ed. 3 Internet-Draft Cisco Systems, Inc. 4 Intended status: Informational December 30, 2019 5 Expires: July 2, 2020 7 YANG Module Versioning Requirements 8 draft-ietf-netmod-yang-versioning-reqs-02 10 Abstract 12 This document describes the problems that can arise because of the 13 YANG language module update rules, that require all updates to YANG 14 module preserve strict backwards compatibility. It also defines the 15 requirements on any solution designed to solve the stated problems. 16 This document does not consider possible solutions, nor endorse any 17 particular solution. 19 Status of This Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at https://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on July 2, 2020. 36 Copyright Notice 38 Copyright (c) 2019 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (https://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 54 2. Background . . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2.1. Striving for model perfection . . . . . . . . . . . . . . 3 56 2.2. Some YANG Modules Are Not Backwards-Compatible . . . . . 3 57 2.3. Non-Backwards-Compatible Errors . . . . . . . . . . . . . 4 58 2.4. No way to easily decide whether a change is Backwards- 59 Compatible . . . . . . . . . . . . . . . . . . . . . . . 4 60 2.5. No good way to specify which module revision to import . 5 61 2.6. Early Warning about Removal . . . . . . . . . . . . . . . 6 62 2.7. Clear Indication of Node Support . . . . . . . . . . . . 6 63 3. Terminology and Conventions . . . . . . . . . . . . . . . . . 7 64 4. The Problem Statement . . . . . . . . . . . . . . . . . . . . 7 65 5. Requirements of a YANG Versioning Solution . . . . . . . . . 9 66 6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 11 67 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 11 68 8. Security Considerations . . . . . . . . . . . . . . . . . . . 11 69 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 70 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 71 10.1. Normative References . . . . . . . . . . . . . . . . . . 12 72 10.2. Informative References . . . . . . . . . . . . . . . . . 12 73 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 75 1. Introduction 77 This requirements document initially considers some of the existing 78 YANG module update rules, then describes the problems that arise due 79 to those rules embracing strict backwards compatibility, and finally 80 defines requirements on any solution that may be designed to solve 81 these problems by providing an alternative YANG versioning strategy. 83 2. Background 85 The YANG data modeling language [RFC7950] specifies strict rules for 86 updating YANG modules (see section 11 "Updating a Module"). Citing a 87 few of the relevant rules: 89 1. "As experience is gained with a module, it may be desirable to 90 revise that module. However, changes to published modules are 91 not allowed if they have any potential to cause interoperability 92 problems between a client using an original specification and a 93 server using an updated specification." 95 2. "Note that definitions contained in a module are available to be 96 imported by any other module and are referenced in "import" 97 statements via the module name. Thus, a module name MUST NOT be 98 changed. Furthermore, the "namespace" statement MUST NOT be 99 changed, since all XML elements are qualified by the namespace." 101 3. "Otherwise, if the semantics of any previous definition are 102 changed (i.e., if a non-editorial change is made to any 103 definition other than those specifically allowed above), then 104 this MUST be achieved by a new definition with a new identifier." 106 4. "deprecated indicates an obsolete definition, but it permits new/ 107 continued implementation in order to foster interoperability with 108 older/existing implementations." 110 The rules described above, along with other similar rules, causes 111 various problems, as described in the following sections: 113 2.1. Striving for model perfection 115 The points made above lead to the logical conclusion that the 116 standardized YANG modules have to be perfect on day one (at least the 117 structure and meaning), which in turn might explain why IETF YANG 118 modules take so long to standardize. Shooting for perfection is 119 obviously a noble goal, but if the perfect standard comes too late, 120 it doesn't help the industry. 122 2.2. Some YANG Modules Are Not Backwards-Compatible 124 As we learn from our mistakes, we're going to face more and more non- 125 backwards-compatible YANG modules. An example is the YANG data model 126 for L3VPN service delivery [RFC8049], which, based on implementation 127 experience, has been updated in a non-backwards-compatible way by 128 [RFC8299]. 130 While Standards Development Organization (SDO) YANG modules are 131 obviously better for the industry, we must recognize that many YANG 132 modules are actually generated YANG modules (for example, from 133 internal databases), which is sometimes the case for vendor modules 134 [RFC8199]. From time to time, the new YANG modules are not 135 backwards-compatible. 137 Old module parts that are no longer needed, no longer supported, or 138 are not used by consumers need to be removed from modules. It is 139 often hard to decide which parts are no longer needed/used; still the 140 need and practice of removing old parts exist. While it is rare in 141 standard modules it is more common in vendor YANG modules where the 142 usage of modules is more controlled. 144 The problems described in Section 2.7 may also result in incompatible 145 changes. 147 In such cases, it would be better to indicate how backwards- 148 compatible a given YANG module actually is. 150 As modules are sometimes updated in an incompatible way the current 151 assumption that once a YANG module is defined all further revisions 152 can be freely used as they are compatible is not valid. 154 2.3. Non-Backwards-Compatible Errors 156 Sometimes small errors force us to make non-backwards-compatible 157 updates. As an example imagine that we have a string with a complex 158 pattern (e.g., an IP address). Let's assume the initial pattern 159 incorrectly allows IP addresses to start with 355. In the next 160 version this is corrected to disallow addresses starting with 355. 161 Formally this is a non-backwards-compatible change as the value space 162 of the string is decreased. In reality an IP address and the 163 implementation behind it was never capable of handling an address 164 starting with 355. So practically this is a backwards-compatible 165 change, just like a correction of the description statement. Current 166 YANG rules are ambiguous as to whether non-backwards-compatible bug 167 fixes are allowed without also requiring a module name change. 169 2.4. No way to easily decide whether a change is Backwards-Compatible 171 A management system, SDN controller, or any other user of a module 172 should be capable of easily determining the compatibility between two 173 module versions. Higher level logic for a network function, 174 something that cannot be implemented in a purely model driven way, is 175 always dependent on a specific version of the module. If the client 176 finds that the module has been updated on the network node, it has to 177 decide if it tries to handle it as it handled the previous version of 178 the model or if it just stops, to avoid problems. To make this 179 decision the client needs to know if the module was updated in a 180 backwards-compatible way or not. 182 This is not possible to decide today because of the following: 184 o It is sometimes necessary to change the semantic behavior of a 185 data node, action or rpc while the YANG definition does not change 186 (with the possible exception of the description statement). In 187 such a case it is impossible to determine whether the change is 188 backwards-compatible just by looking at the YANG statements. It's 189 only the human model designer who can decide. 191 o Problems with the deprecated and obsolete status statement, 192 Section 2.7 194 o YANG module authors might decide to violate YANG 1.1 update rules 195 for some of the reasons above. 197 Finding status changes or violations of update rules need a line-by- 198 line comparison of the old and new modules is a tedious task. 200 2.5. No good way to specify which module revision to import 202 If a module (MOD-A) is imported by another one (MOD-B) the importer 203 may specify which revision must be imported. Even if MOD-A is 204 updated in a backwards-compatible way not all revisions will be 205 suitable, e.g., a new MOD-B might need the newest MOD-A. However, 206 both specifying or omitting the revision date for import leads to 207 problems. 209 If the import by revision-date is specified 211 o If corrections are made to MOD-A these would not have any effect 212 as the import's revision date would still point to the un- 213 corrected earlier YANG module revision. 215 o If MOD-A is updated in a backwards-compatible way because another 216 importer (MOD-C) needs some functionality, the new MOD-A could be 217 used by MOD-B, but specifying the exact import revision-date 218 prevents this. This will force the implementers to import two 219 different revisions of MOD-A, forcing them to maintain old MOD-A 220 revisions unnecessarily. 222 o If multiple modules import different revisions of MOD-A the human 223 user will need to understand the subtle differences between the 224 different revisions. Small differences would easily lead to 225 operator mistakes as the operator will rarely check the 226 documentation. 228 o Tooling/SW is often not prepared to handle multiple revisions of 229 the same YANG module. 231 If the import revision-date is not specified 233 o any revision of MOD-A may be used including unsuitable ones. 234 Older revisions may be lacking functionality MOD-B needs. Newer 235 MOD-A revisions may obsolete definitions used by MOD-B in which 236 case these must not be used by MOD-B anymore. 238 o As it is not specified which revisions of MOD-A are suitable for 239 MOD-B. The problem has to be solved on a case by case basis 240 studying all the details of MOD-A and MOD-B which is considerable 241 work. 243 2.6. Early Warning about Removal 245 If a schema part is considered old/bad we need to be able to give 246 advance warning that it will be removed. As this is an advance 247 warning the part must still be present and usable in the current 248 revision; however, it will be removed in one of the next revisions. 249 The deprecated statement cannot be reliably used for this purpose 250 both because deprecated nodes may not be implemented and also there 251 is no mandate that text be provided explaining the deprecation. 253 We need the advance warning to allow users of the module time to 254 plan/execute migration away from the deprecated functionality. 255 Deprecation should be accompanied by information whether the 256 functionality will just disappear or that there is an alternative, 257 possibly more advanced solution that should be used. 259 Vendors use such warnings often, but the NMDA related redesign of 260 IETF modules is also an example where it would be useful for IETF. 261 As another example, see the usage of deprecated in the Java 262 programming language. 264 2.7. Clear Indication of Node Support 266 The current definition of deprecated and obsolete in [RFC7950] (as 267 quoted below) is problematic and should be corrected. 269 o "deprecated" indicates an obsolete definition, but it permits new/ 270 continued implementation in order to foster interoperability with 271 older/existing implementations. 273 o "obsolete" means that the definition is obsolete and SHOULD NOT be 274 implemented and/or can be removed from implementations. 276 YANG is considered an interface contract between the server and the 277 client. The current definitions of deprecated and obsolete mean that 278 a schema node that is either deprecated or obsolete may or may not be 279 implemented. The client has no way to find out which is the case 280 except for by trying to write or read data at the leaf in question. 281 This probing would need to be done for each separate data-node, which 282 is not a trivial thing to do. This "may or may not" is unacceptable 283 in a contract. In effect, this works as if there would be an if- 284 feature statement on each deprecated schema node where the server 285 does not advertise whether the feature is supported or not. Why is 286 it not advertised? 288 3. Terminology and Conventions 290 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 291 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 292 document are to be interpreted as described in [RFC2119]. 294 In addition, this document uses the following terminology: 296 o YANG module revision: An instance of a YANG module, with no 297 implied ordering or backwards compatibility between different 298 revisions of the same module." 300 o YANG module version: A YANG module revision, but also with an 301 implied partial ordering relationship between other versions of 302 the same module. Each module version must be uniquely 303 identifiable. 305 o Non-backwards-compatible (NBC): In the context of this document, 306 the term 'non-backwards-compatible' refers to a change or set of 307 changes between two YANG module revisions that do not adhere to 308 the list of allowable changes specified in Section 11 "Updating a 309 Module" of [RFC7950], with the following additional clarification: 311 * Any addition of, or change to, a "status" statement that allows 312 a server to remove support for a schema node is considered a 313 non-backwards-compatible change 315 4. The Problem Statement 317 Considering the issues described in the background, the problem 318 definition can be summarized as follows. 320 Development of data models for a large collection of communication 321 protocols and system components is difficult and typically only 322 manageable with an iterative development process. Agile development 323 approaches advocate evolutionary development, early delivery, and 324 continual improvement. They are designed to support rapid and 325 flexible response to change. Agile development has been found to be 326 very successful in a world where the objects being modeled undergo 327 constant changes. 329 The current module versioning scheme relies on the fundamental idea 330 that a definition, once published, never changes its semantics. As a 331 consequence, if a new definition is needed with different non- 332 backwards-compatible semantics, then a new definition must be created 333 to replace the old definition. The advantage of this versioning 334 scheme is that a definition identified by a module name and a path 335 has fixed semantics that never change. (The details are a bit more 336 nuanced but we simplify things here a bit in order to get the 337 problems worked out clearly.) 339 There are two main disadvantages of the current YANG versioning 340 scheme: 342 o Any non-backwards-compatible change of a definition requires 343 either a new module name or a new path. This has been found 344 costly to support in implementations, in particular on the client 345 side. 347 o Since non-backwards-compatible changes require either a new module 348 name or a new path, such changes will impact other modules that 349 import definitions. In fact, with the current module versioning 350 scheme other modules have to opt-in in order to use the new 351 version. This essentially leads to a ripple effect where a non- 352 backwards-compatible change of a core module causes updates on a 353 potentially large number of dependent modules. 355 Other problems experienced with the current YANG versioning scheme 356 are the following: 358 o YANG has a mechanism to mark definitions deprecated but it leaves 359 it open whether implementations are expected to implement 360 deprecated definitions and there is no way (other than trial and 361 error) for a client to find out whether deprecated definitions are 362 supported by a given implementation. 364 o YANG does not have a robust mechanism to document which data 365 definitions have changed and to provide guidance how 366 implementations should deal with the change. While it is possible 367 to have this described in general description statements, having 368 these details embedded in general description statements does not 369 make this information accessible to tools. 371 o YANG data models often do not exist in isolation and they interact 372 with other software systems or data models that often do allow 373 (controlled) non-backwards-compatible changes. In some cases, 374 YANG models are mechanically derived from other data models that 375 do allow (controlled) non-backwards-compatible changes. In such 376 situations, a robust mapping to YANG requires to have version 377 numbers exposed as part of the module name or a path definition, 378 which has been found to be expensive on the client side (see 379 above). 381 Given the need to support agile development processes and the 382 disadvantages and problems of the current YANG versioning scheme 383 described above, it is necessary to develop requirements and 384 solutions for a future YANG versioning scheme that better supports 385 agile development processes, whilst retaining the ability for servers 386 to handle clients using older versions of YANG modules. 388 5. Requirements of a YANG Versioning Solution 390 The following is a list of requirements that a solution to the 391 problems mentioned above MUST or SHOULD have. The list is grouped by 392 similar requirements but is not presented in a set priority order. 394 1. Requirements related to making non-backwards-compatible updates 395 to modules: 397 1.1 A mechanism is REQUIRED to update a module in a non- 398 backwards-compatible way without forcing all modules with 399 import dependencies on the updated module from being updated 400 at the same time (e.g. to change its import to use a new 401 module name). 403 1.2 Non-backwards-compatible updates of a module MUST not impact 404 clients that only access data nodes of the module that have 405 either not been updated or have been updated in backwards- 406 compatible ways. 408 1.3 A refined form of YANG's 'import' statement MUST be provided 409 that is more restrictive than "import any revision" and less 410 restrictive than "import a specific revision". Once non- 411 backwards-compatible changes to modules are allowed, the 412 refined import statement is used to express the correct 413 dependency between modules. 415 1.4 The solution MUST be able to express when non-backwards- 416 compatible changes have occurred between two revisions of a 417 given YANG module. 419 2. Requirements related to identifying changes between different 420 module revisions: 422 2.1 Readers of modules, and tools that use modules, MUST be able 423 to determine whether changes between two revisions of a 424 module constitute a backwards-compatible or non-backwards- 425 compatible version change. In addition, it MAY be helpful 426 to identify whether changes represent bug fixes, new 427 functionality, or both. 429 2.2 A mechanism SHOULD be defined to determine whether data 430 nodes between two arbitrary YANG module revisions have (i) 431 not changed, (ii) changed in a backwards-compatible way, 432 (iii) changed in a non-backwards-compatible way. 434 3. Requirements related to supporting existing clients in a 435 backwards-compatible way: 437 3.1 The solution MUST provide a mechanism to allow servers to 438 support existing clients in a backwards-compatible way. 440 3.2 The solution MUST provide a mechanism to support clients 441 that expect an older version of a given module when the 442 current version has had non-backwards-compatible changes. 444 3.3 Clients are expected to be able to handle unexpected 445 instance data resulting from backwards-compatible changes. 447 4. Requirements related to managing and documenting the life cycle 448 of data nodes: 450 4.1 A mechanism is REQUIRED to allow a client to determine 451 whether deprecated nodes are implemented by the server. 453 4.2 If a data node is deprecated or obsolete then it MUST be 454 possible to document in the YANG module what alternatives 455 exist, the reason for the status change, or any other status 456 related information. 458 4.3 A mechanism is REQUIRED to indicate that certain definitions 459 in a YANG module will become status obsolete in future 460 revisions but definitions marked as such MUST still be 461 implemented by compliant servers. 463 5. Requirements related to documentation and education: 465 5.1 The solution MUST provide guidance to model authors and 466 clients on how to use the new YANG versioning scheme. 468 5.2 The solution is REQUIRED to describe how to transition from 469 the existing YANG 1.0/1.1 versioning scheme to the new 470 scheme. 472 5.3 The solution MUST describe how the versioning scheme affects 473 the interpretation of instance data and references to 474 instance data, for which the schema definition has been 475 updated in a non-backwards-compatible way. 477 6. Contributors 479 This document grew out of the YANG module versioning design team that 480 started after IETF 101. The following people are members of that 481 design team and have contributed to defining the problem and 482 specifying the requirements: 484 o Balazs Lengyel 486 o Benoit Claise 488 o Ebben Aries 490 o Jason Sterne 492 o Joe Clarke 494 o Juergen Schoenwaelder 496 o Mahesh Jethanandani 498 o Michael (Wangzitao) 500 o Qin Wu 502 o Reshad Rahman 504 o Rob Wilton 506 7. Acknowledgments 508 The design team would like to thank Christian Hopps and Vladimir 509 Vassilev for their feedback and perspectives in shaping and fine 510 tuning the versioning requirements. 512 One of the inspirations for solving the YANG module versioning comes 513 from OpenConfig. The authors would like to thank Anees Shaikh and 514 Rob Shakir for their helpful input. 516 8. Security Considerations 518 The document does not define any new protocol or data model. There 519 is no security impact. 521 9. IANA Considerations 523 None 525 10. References 527 10.1. Normative References 529 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 530 Requirement Levels", BCP 14, RFC 2119, 531 DOI 10.17487/RFC2119, March 1997, 532 . 534 [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", 535 RFC 7950, DOI 10.17487/RFC7950, August 2016, 536 . 538 10.2. Informative References 540 [RFC8049] Litkowski, S., Tomotaki, L., and K. Ogaki, "YANG Data 541 Model for L3VPN Service Delivery", RFC 8049, 542 DOI 10.17487/RFC8049, February 2017, 543 . 545 [RFC8199] Bogdanovic, D., Claise, B., and C. Moberg, "YANG Module 546 Classification", RFC 8199, DOI 10.17487/RFC8199, July 547 2017, . 549 [RFC8299] Wu, Q., Ed., Litkowski, S., Tomotaki, L., and K. Ogaki, 550 "YANG Data Model for L3VPN Service Delivery", RFC 8299, 551 DOI 10.17487/RFC8299, January 2018, 552 . 554 Author's Address 556 Joe Clarke (editor) 557 Cisco Systems, Inc. 558 7200-12 Kit Creek Rd 559 Research Triangle Park, North Carolina 560 United States of America 562 Phone: +1-919-392-2867 563 Email: jclarke@cisco.com