idnits 2.17.1 draft-freytag-lager-variant-rules-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 13, 2017) is 2600 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Freytag 3 Internet-Draft March 13, 2017 4 Intended status: Informational 5 Expires: September 14, 2017 7 Variant Rules 8 draft-freytag-lager-variant-rules-04 10 Abstract 12 Rules for validating identifier labels and alternate representations 13 of those labels (variants) are known as "Label Generation Rulesets" 14 (LGRs); they are used for the implementation of identifier systems 15 such as Internationalized Domain Names (IDNs). This document 16 describes ways of designing Label Generation Rulesets (LGRs) that 17 support variant labels. In designing LGRs, it is important to ensure 18 that the label generation rules are consistent and well-behaved in 19 the presence of variants. The design decisions can then be expressed 20 using the an XML representation of LGRs that is defined in RFC7940. 22 Status of This Memo 24 This Internet-Draft is submitted in full conformance with the 25 provisions of BCP 78 and BCP 79. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF). Note that other groups may also distribute 29 working documents as Internet-Drafts. The list of current Internet- 30 Drafts is at http://datatracker.ietf.org/drafts/current/. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 This Internet-Draft will expire on September 14, 2017. 39 Copyright Notice 41 Copyright (c) 2017 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with respect 49 to this document. Code Components extracted from this document must 50 include Simplified BSD License text as described in Section 4.e of 51 the Trust Legal Provisions and are provided without warranty as 52 described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 57 2. Variant Relationships . . . . . . . . . . . . . . . . . . . . 4 58 3. Symmetry and Transitivity . . . . . . . . . . . . . . . . . . 4 59 4. A Word on Notation . . . . . . . . . . . . . . . . . . . . . 5 60 5. Variant Mappings . . . . . . . . . . . . . . . . . . . . . . 5 61 6. Variant Labels . . . . . . . . . . . . . . . . . . . . . . . 6 62 7. Variant Types and Label Dispositions . . . . . . . . . . . . 6 63 8. Allocatable Variants . . . . . . . . . . . . . . . . . . . . 7 64 9. Blocked Variants . . . . . . . . . . . . . . . . . . . . . . 8 65 10. Pure Variant Labels . . . . . . . . . . . . . . . . . . . . . 9 66 11. Reflexive Variants . . . . . . . . . . . . . . . . . . . . . 10 67 12. Limiting Allocatable Variants by Subtyping . . . . . . . . . 10 68 13. Allowing Mixed Originals . . . . . . . . . . . . . . . . . . 13 69 14. Handling Out-of-Repertoire Variants . . . . . . . . . . . . . 14 70 15. Conditional Variants . . . . . . . . . . . . . . . . . . . . 15 71 16. Making Conditional Variants Well-Behaved . . . . . . . . . . 16 72 17. Variants for Sequences . . . . . . . . . . . . . . . . . . . 18 73 18. Corresponding XML Notation . . . . . . . . . . . . . . . . . 19 74 19. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 75 20. Security Considerations . . . . . . . . . . . . . . . . . . . 21 76 21. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 77 21.1. Normative References . . . . . . . . . . . . . . . . . . 22 78 21.2. Informative References . . . . . . . . . . . . . . . . . 22 79 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 22 80 Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 22 81 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 22 83 1. Introduction 85 Label Generation Rulesets (LGRs) that define the set of permissible 86 labels, may be applied to a variety of identifier systems, although 87 to date, the most common use is to define policies for implementing 88 Internationalized Domain Names (IDNs) for some zone of the Domain 89 Name System (DNS). Without restricting any of the more general 90 applications of this document, the explanations and examples in this 91 document may be stated in terms of IDNs. 93 In addition to determining whether a given label is eligible, LGRs 94 may also define the condition under which alternate representations 95 of these labels, so called variant labels, may exist and also define 96 their status (disposition). In the most general sense, variant 97 labels are typically labels that are either visually or semantically 98 indistinguishable from an applied for label in the context of the 99 writing system or script supported by the LGR. Unlike merely similar 100 labels, where there may be a measurable degree of similarity, variant 101 labels considered here represent a form of equivalence in meaning or 102 appearance. What constitutes an appropriate variant in any writing 103 system or given context, particular in the DNS, is assumed to have 104 been determined ahead of time, and therefore is not subject of this 105 document. 107 Once identified, variant labels are typically delegated to some 108 entity together with the applied-for label, or permanently reserved, 109 based on the disposition derived from the LGR. Correctly defined, 110 variant labels can improve the security of an LGR yet successfully 111 defining variant rules for an LGR so that the result is well-behaved 112 is not always trivial. This document describes the basic 113 considerations and constraints that must be taken into account and 114 gives examples of what might be use cases for different types of 115 variant specifications in an LGR. 117 This document does not address the question whether variants are an 118 appropriate means to solve any given issue, nor on what basis they 119 should be defined. It is intended to explain in more detail the 120 effects of various declarations and the tradeoffs in making design 121 choices. It tacitly assumes that any LGR will be expressed using the 122 XML representation defined in [RFC7940] and therefore conform to any 123 requirements stated therein. Purely for clarity of exposition, 124 examples in this document are using a more compact notation than the 125 XML syntax defined in [RFC7940]. However, the reader is expected to 126 have some familiarity with the concepts described in that RFC (see 127 Section 4). 129 The user of any identifier system, such as the DNS, interacts with it 130 in the context of labels; variants are experienced as variant labels: 131 two (or more) labels that are functionally "the same" under the 132 conventions of the writing system used, even though their code point 133 sequences are different. An LGR specification, on the other hand, 134 defines variant mappings between code points, and only in a secondary 135 step, derives the variant labels from these mappings. For a 136 discussion of this process, see [RFC7940]. 138 By assigning a "type" to the variant mappings and carefully 139 constructing the derivation of variant label dispositions from these 140 types, the designer of an LGR can control whether some or all of the 141 variant labels created from an original label should be allocatable, 142 that is available for allocation (to the original applicant) or 143 whether some or all of these labels should be blocked instead, that 144 is, remain not allocatable (to anyone). 146 The choice of desired label disposition would be based on the 147 expectations of the users of the particular zone, and is not the 148 subject of this document. Likewise, this document does not address 149 the possibility of an LGR defining custom label dispositions. 150 Instead, this document suggests ways of desiging an LGR to achieve 151 the selected design choice for handling variants in the context of 152 the two standard label dispositions "allocatable" and "blocked". 154 2. Variant Relationships 156 A variant relationship is fundamentally a "same as", in other words, 157 it is an equivalence relationship. Now the strictest sense of "same 158 as" would be equality, and for any equality, we have both symmetry 160 A = B => B = A 162 and transitivity 164 A = B and B = C => A = C 166 The variant relationship with its functional sense of "same as" must 167 really satisfy the same constraint. Once we say A is the "same as" 168 B, we also assert that B is the "same as" A. In this document, the 169 symbol "~" means "has a variant relationship with". Thus, we get 171 A ~ B => B ~ A 173 Likewise, if we make the same claim for B and C (B ~ C) then we do 174 get A ~ C, because if B is "the same" as both A and C then A must be 175 "the same as" C: 177 A ~ B and B ~ C => A ~ C 179 3. Symmetry and Transitivity 181 Not all relationships between labels constitute equivalence and those 182 that do not are not transitive and may not be symmetric. For 183 example, the degree to which labels are confusable is not transitive: 184 two labels can be confusingly similar to a third without necessarily 185 being confusable with each other, such as when the third one has a 186 shape that is "in between" the other two. A variant relation based 187 on identical or effectively identical appearance would meet the 188 criterion of transitivity, as would other forms of equivalence, such 189 as semantic equivalence. 191 From the perspective of [RFC7940], an LGR could be specified that is 192 neither symmetric or transitive, and such an LGR would be a valid 193 specification. However, from an implementation point of view there 194 are certain benefits from an LGR that is symmetric and transitive. 195 It greatly simplifies the check for collisions between labels with 196 variants. For this reason, we will limit the discussion of variants 197 in this document to those are symmetric and transitive. 198 Incidentally, it is often straightforward to verify mechanically 199 whether an LGR is symmetric and or transitive, and to compute any 200 mappings required to make it so (but see Section 15). 202 4. A Word on Notation 204 [RFC7940] defines an XML schema for Label Generation Rulesets in 205 general, and variant code points and sequences in particular, see 206 Section 18. That notation is rather verbose and can easily obscure 207 salient features to anyone not trained to read XML. For this reason, 208 this document uses a symbolic shorthand notation in presenting the 209 examples for discussion. This shorthand is merely a didactic tool 210 for presentation and not intended as alternative to or replacement 211 for the XML syntax that is used in formally specifying an LGR under 212 [RFC7940]. 214 When it comes time to capture the LGR in a formal definition, the 215 notation used for any of examples in this document can be converted 216 to the XML format as described in Section 18. 218 5. Variant Mappings 220 So far, we have treated variant relationships as simple "same as" 221 ignoring that each relationship representing an equivalence would 222 consist of a symmetric pair of reciprocal mappings. In this 223 document, the symbol "-->" means "maps to". 225 A ~ B => A --> B, B --> A 227 In an LGR, these mappings are not defined directly between labels, 228 but between code points (or code point sequences, see Section 17). 229 In the transitive case, given 231 A ~ B => A --> B, B --> A 233 A ~ C => A --> C, C --> A 235 we also get 237 B ~ C => B --> C, C --> B 239 for a total of six possible mappings. Conventionally, these are 240 listed in tables in order of the source code point, like so 241 A --> B 242 A --> C 243 B --> A 244 B --> C 245 C --> A 246 C --> B 248 As we can see, each of A, B and C can be mapped two ways. 250 6. Variant Labels 252 To create a variant label, each code point in the original label is 253 successively replaced by all variant code points defined by a mapping 254 from the original code point. For a label AAA (the letter "A" three 255 times), the variant labels (given the mappings from transitive 256 example above) would be 258 AAB 259 ABA 260 ABB 261 BAA 262 BAB 263 BBA 264 BBB 265 AAC 266 ... 267 CCC 269 So far, we have mere defined what the variant labels are, but we have 270 not considered their possible dispositions. In the next section we 271 discuss how to set up the variant mappings so that some variant 272 labels are mutually exclusive (blocked), but some may be allocated to 273 the same applicant as the original label (allocatable). 275 7. Variant Types and Label Dispositions 277 Assume we wanted to allow a variant relation between some code points 278 O and A, and perhaps also between O and B as well as O and C. By 279 transitivity we would have 281 O ~ A ~ B ~ C 283 However, we would like to distinguish the case where someone applies 284 for OOO from the case where someone applies for the label ABC. In 285 the former case we would like to allocate only the label OOO, but in 286 the latter case, we would like to also allow the allocation of either 287 the original label OOO or the variant label ABC, or both, but not of 288 any of the other possible variant labels, like OAO, BCO or the like. 290 (A real-world example might be the case where O represents an 291 unaccented letter, while A, B and C might represent various accented 292 forms of the same letter. Because unaccented letters are a common 293 fallback, there might be a desire to allocate an unaccented label as 294 a variant, but not the other way around.) 296 How do we make that distinction? 298 The answer lies in labeling the mappings A --> O, B --> O, and C --> 299 O with the type "allocatable" and the mappings O --> A, O --> B, and 300 O --> C with the type "blocked". In this document, the symbol "x-->" 301 means "maps with type blocked" and the symbol "a-->" means "maps with 302 type allocatable". Thus: 304 O x--> A 305 O x--> B 306 O x--> C 307 A a--> O 308 B a--> O 309 C a--> O 311 When we generate all permutations of labels, we use mappings with 312 different types depending from which code points we start. 314 In creating an LGR with variants, all variant mappings should always 315 be labeled with a type ([RFC7940] does not formally require a type, 316 but any well-behaved LGR would be fully typed). By default, these 317 types correspond directly to the dispositions for variant labels, 318 with the most restrictive type determining the disposition of the 319 variant label. However, as we shall see later, it is sometimes 320 useful to assign types from a wider array of values than the final 321 dispositions for the labels and then define explicitly how to derive 322 label dispositions from them. 324 8. Allocatable Variants 326 If we start with AAA, the permutation OOO will have been the result 327 of applying the mapping A a--> O at each code point. That is, only 328 mappings with type "a" (allocatable) were used. To know whether we 329 can allocate both the label OOO and the original label AAA we track 330 the types of the mappings used in generating the label. 332 We record the variant types for each of the variant mappings used in 333 creating the permutation in an ordered list. Such an ordered list of 334 variant types is called a "variant type list". In running text we 335 often show it enclosed in square brackets. For example [a x -] means 336 the variant label was derived from a variant mapping with the "a" 337 variant type in the first code point position, "x" in the second code 338 point position, and that the third position is the original code 339 point ("-" means "no variant mapping"). 341 For our example permutation we get the following variant type list 342 (brackets dropped): 344 AAA --> OOO : a a a 346 From the variant type list we derive a "variant type set", denoted by 347 curly braces, that contains an unordered set of unique variant types 348 in the variant type list. For the variant type list for the given 349 permutation, [a a a], the variant type set is { a }, which has a 350 single element "a". 352 Deciding whether to allow the allocation of a variant label then 353 amounts to deriving a disposition for the variant label from the 354 variant type set created from the variant mappings that were used to 355 create the label. For example the derivation 357 if "all variants" = "a" => set label disposition to "allocatable" 359 would allow OOO to be allocated, because the types of all variants 360 mappings used to create that variant label from AAA are "a". 362 The "all-variants" condition is tolerant of an extra "-" in the 363 variant set (unlike the "only-variants" condition described below). 364 So, had we started with AOA, OAA or AAO, the variant set for the 365 permuted variant OOO would have been { a - } because in each case one 366 of the code points remains the same as the original. The "-" means 367 that because of the absence of a mapping O --> O there is no variant 368 type for the O in each of these labels. 370 The "all-variants" = "a" condition ignores the "-", so using the 371 derivation from above, we find that OOO is an allocatable variant for 372 each of the labels AOA, OAA or AAO. 374 Allocatable variant labels, especially large numbers of allocatable 375 variants per label, incur a certain cost to users of the LGR. A 376 well-behaved LGR will minimize the number allocatable variants. 378 9. Blocked Variants 380 Blocked variants are not available to another registrant. They 381 therefore protect the applicant of the original label from someone 382 else registering a label that is "the same as" under some user- 383 perceived metric. Blocked variants can be a useful tool even for 384 scripts for which no allocatable labels are ever defined. 386 If we start with OOO, the permutation AAA will have been the result 387 of applying only mappings with type "blocked" and we cannot allocate 388 the label AAA, only the original label OOO. This corresponds to the 389 following derivation: 391 if "any variants" = "x" => set label disposition to "blocked" 393 To additionally prevent allocating ABO as a variant label for AAA we 394 further need to make sure that the mapping A --> B has been defined 395 with type "blocked" as in 397 A x--> B 399 so that 401 AAA --> ABO: - x a. 403 Thus the set {x a} contains at least one "x" and satisfies the 404 derivation of a blocked disposition for ABO when AAA is applied for. 406 If an LGR results in a symmetric and transitive set of variant 407 labels, then the task of determining whether a label or its variants 408 collide with another label or its variants can be implemented very 409 efficiently. Symmetry and transitivity implies that each set of 410 labels that are mutually variants of each other is disjoint from all 411 other such sets. Only labels within the same set can be variants of 412 each other. Identifying the variant set can be an O(1) operation, 413 and enumerating all variants is not necessary. 415 10. Pure Variant Labels 417 Now, if we wanted to prevent allocation of AOA when we start from 418 AAA, we would need a rule disallowing a mix of original code points 419 and variant code points, which is easily accomplished by use of the 420 "only-variants" qualifier, which requires that the label consist 421 entirely of variants and all the variants are from the same set of 422 types. 424 if "only-variants" = "a" => set label disposition to "allocatable" 426 The two code points A in AOA are not arrived at by variant mappings, 427 because the code points are unchanged and no variant mappings are 428 defined for A --> A. So, in our example, the set of variant mapping 429 types is 431 AAA --> AOA: - a - 433 but unlike the "all-variants" condition, "only-variants" requires a 434 variant type set { a } corresponding to a variant type list [a a a] 435 (no - allowed). By adding a final derivation 437 else if "any-variants" = "a" => set label disposition to "blocked" 439 and executing that derivation only on any remaining labels, we 440 disallow AOA when starting from AAA, but still allow OOO. 442 Derivation conditions are always applied in order, with later 443 derivations only applying to labels that did not match any earlier 444 conditions, as indicated by the use of "else" in the last example. 445 In other words, they form a cascade. 447 11. Reflexive Variants 449 But what if we started from AOA? We would expect OOO to be 450 allocatable, but the variant type set would be 452 OOO --> OOO: a - a 454 because the O is the original code point. Here is where we use a 455 reflexive mapping, by realizing that O is "the same as" O, which is 456 normally redundant, but allows us to specify a disposition on the 457 mapping 459 O a--> O 461 with that, the variant type list for OOO --> OOO becomes: 463 AOA --> OOO: a a a 465 and the label OOO again passes the derivation condition 467 if "only-variants" = "a" => set label disposition to "allocatable" 469 as desired. This use of reflexive variants is typical whenever 470 derivations with the "only-variants" qualifier are used. If any code 471 point uses a reflexive variant, a well-behaved LGR would specify an 472 appropriate reflexive variant for all code points. 474 12. Limiting Allocatable Variants by Subtyping 476 As we have seen, the number of variant labels can potentially be 477 large, due to combinatorics. Sometimes it is possible to divide 478 variants into categories and to stipulate that only variant labels 479 with variants from the same category should be allocatable. For some 480 LGRs this constraint can be implemented by a rule that disallows code 481 points from different categories to occur in the same allocatable 482 label. For other LGRs the appropriate mechanism may be dividing the 483 allocatable variants into subtypes. 485 To recap, in the standard case a code point C can have (up to) two 486 types of variant mappings 488 C x--> X 489 C a--> A 491 where a--> means a variant mapping with type "allocatable", and x--> 492 means "blocked". For the purpose of the following discussion, we 493 name the target code point with the corresponding uppercase letter. 495 Subtyping allows us to distinguish among different types of 496 allocatable variants. For example, we can define three new types: 497 "s", "t" and "b". Of these, "s" and "t" are mutually incompatible, 498 but "b" is compatible with either "s" or "t" (in this case, "b" 499 stands for "both"). A real-world example for this might be variant 500 mappings appropriate for "simplified" or "traditional" Chinese 501 variants, or appropriate for both. 503 With subtypes defined as above, a code point C might have (up to) 504 four types of variant mappings 506 C x--> X 507 C s--> S 508 C t--> T 509 C b--> B 511 and explicit reflexive mappings of one of these types 513 C s--> C 514 C t--> C 515 C b--> C 517 As before, all mappings must have one and only one type, but each 518 code point may map to any number of other code points. 520 We define the compatibility of "b" with "t" or "s" by our choice of 521 derivation conditions as follows 523 if "any-variants" = "x" => blocked 524 else if "only-variants" = "s" or "b" => allocatable 525 else if "only-variants" = "t" or "b" => allocatable 526 else if "any-variants" = "s" or "t" or "b" => blocked 528 An original label of four code points 529 CCCC 531 may have many variant labels such as this example listed with its 532 corresponding variant type list: 534 CCCC --> XSTB : x s t b 536 This variant label is blocked because to get from C to B required 537 x-->. (Because variant mappings are defined for specific source code 538 points, we need to show the starting label for each of these 539 examples, not merely the code points in the variant label.) The 540 variant label 542 CCCC --> SSBB : s s b b 544 is allocatable, because the variant type list contains only 545 allocatable mappings of subtype "s" or "b", which we have defined as 546 being compatible by our choice of derivations. The actual set of 547 variant types {s, b} has only two members, but the examples are 548 easier to follow if we list each type. The label 550 CCCC --> TTBB : t t b b 552 is again allocatable, because the variant type set {t, b} contains 553 only allocatable mappings of the mutually compatible allocatable 554 subtypes "t" or "b". In contrast, 556 CCCC --> SSTT : s s t t 558 is not allocatable, because the type set contains incompatible 559 subtypes "t" and "s" and thus would be blocked by the final 560 derivation. 562 The variant labels 564 CCCC --> CSBB : c s b b 565 CCCC --> CTBB : c t b b 567 are only allocatable based on the subtype for the C --> C mapping, 568 which is denoted here by c and (depending on what was chosen for the 569 type of the reflexive mapping) could correspond to "s", "t", or "b". 571 If it is "s", the first of these two labels is allocatable; if it is 572 "t", the second of these two labels is allocatable; if it is b, both 573 labels are allocatable. 575 So far, the scheme does not seem to have brought any huge reduction 576 in allocatable variant labels, but that is because we tacitly assumed 577 that C could have all three types of allocatable variants "s", "t", 578 and "b" at the same time. 580 In a real world example, the types "s", "t" and "b" are assigned so 581 that each code point C normally has at most one non-reflexive variant 582 mapping labeled with one of these subtypes, and all other mappings 583 would be assigned type "x" (blocked). This holds true for most code 584 points in existing tables (such as those used in current IDN TLDs), 585 although certain code points have exceptionally complex variant 586 relations and may have an extra mapping. 588 13. Allowing Mixed Originals 590 If the desire is to allow original labels (but not variant labels) 591 that are s/t mixed, then the scheme needs to be slightly refined to 592 distinguish between reflexive and non-reflexive variants. In this 593 document, the symbol "r-n" means "a reflexive (identity) mapping of 594 type 'n'". The reflexive mappings of the preceding section thus 595 become: 597 C r-s--> C 598 C r-t--> C 599 C r-b--> C 601 With this convention, and redefining the derivations 603 if "any-variants" = "x" => blocked 604 else if "only-variants" = "s" or "r-s" or "b" or "r-b" => allocatable 605 else if "only-variants" = "t" or "r-t" or "b" or "r-b" => allocatable 606 else if "any-variants" = "s" or "t" or "b" => blocked 607 else => allocatable 609 any labels that contain only reflexive mappings of otherwise mixed 610 type (in other words, any mixed original label) now fall through and 611 their disposition is set to "allocatable" in the final derivation. 613 In a well-behaved LGR, it is preferable to explicitly define the 614 derivation for allocatable labels, instead of using a fall-through. 615 In the derivation above, code points without any variant mappings 616 fall through and become allocatable by default if they are part of an 617 original label. Especially in a large repertoire it can be difficult 618 to identify which code points are affected. Instead, it is 619 preferable to mark them with their own reflexive mapping type 620 "neither" or "r-n". 622 C r-n--> C 624 With that we can change 625 else => allocatable 627 to 629 else if "only-variants" = "r-s" or "r-t" or "r-b" or "r-n" 630 => allocatable 631 else => invalid 633 This makes the intent more explicit and by ensuring that all code 634 points in the LGR have a reflexive mapping of some kind, it is easier 635 to verify the correct assignment of their types. 637 14. Handling Out-of-Repertoire Variants 639 At first it may seem counterintuitive to define variants that map to 640 code points not part of the repertoire. However, for zones for which 641 multiple LGRs are defined, there may be situations where labels valid 642 under one LGR should be blocked if a label under another LGR is 643 already delegated. This situation can arise whether or not the 644 repertoires of the affected LGRs overlap, and, where repertoires 645 overlap, whether or not the labels are both restricted to the common 646 subset. 648 In order to handle this exclusion relation through definition of 649 variants, it is necessary to be able to specify variant mappings to 650 some code point X that is outside an LGR's repertoire, R: 652 C x--> X : where C = elementOf(R) and X != elementOf(R) 654 Because of symmetry, it is necessary to also specify the inverse 655 mapping in the LGR: 657 X x--> C : where X != elementOf(R) and C = elementOf(R) 659 This makes X a source of variant mappings and it becomes necessary to 660 identify X as being outside the repertoire, so that any attempt to 661 apply for a label containing X will lead to a disposition of 662 "invalid" - just as if X had never been listed in the LGR. The 663 mechanism to do this, again uses reflexive variants, but with a new 664 type of reflexive mapping of "out-of-repertoire-var", shown as 665 "r-o-->": 667 X r-o--> X 669 This indicates X != elementOf(R), as long as the LGR is provided with 670 a suitable derivation, so that any label containing "r-o-->" is 671 assigned a disposition of "invalid", just as if X was any other code 672 point not part of the repertoire. The derivation used is: 674 if "any-variant" = "out-of-repertoire-var" => invalid 676 It is inserted ahead of any other derivation of the "any-variant" 677 kind in the chain of derivations. As a result, instead of the 678 minimum two symmetric variants, for any out-of repertoire variants 679 there are a minimum of three variant mappings defined: 681 C x--> X 682 X x--> C 683 X r-o--> X 685 where C = elementOf(R) and X != elementOf(R). 687 Because no variant label with any code point outside the repertoire 688 could ever be allocated, the only logical choice for the non- 689 reflexive mappings to out-of-repertoire code points is "blocked". 691 15. Conditional Variants 693 Variant mappings are based on whether code points are "the same" to 694 the user. In some writing systems, code points change shape based on 695 where they occur in the word (positional forms). Some code points 696 have matching shapes in some positions, but not in others. In such 697 cases, the variant mapping only exists for some possible positions, 698 or more general, only for some contexts. For other contexts, the 699 variant mapping does not exist. 701 For example, take two code points, that have the same shape at the 702 end of a label (or in final position) but not in any other position. 703 In that case, they are variants only when they occur in the final 704 position, something we indicate like this: 706 final: C --> D 708 In cursively connected scripts, like Arabic, a code point may take 709 its final form when next to any following code point that interrupts 710 the cursive connection, not just at the end of a label. (We ignore 711 the isolated form to keep the discussion simple, if it was included, 712 "final" might be "final-or-isolate", for example). 714 From symmetry, we expect that the mapping D --> C should also exist 715 only when the code point D is in final position. (Similar 716 considerations apply to transitivity). 718 Sometimes a code point has a final form that is practically the same 719 as that of some code point while sharing initial and medial forms 720 with another. 722 final: C --> D 723 !final: C --> E 725 Here the case where the condition is the opposite of final is shown 726 as "!final". 728 Because shapes differ by position, when a context is applied to a 729 variant mapping, it is treated independently from the same mapping in 730 other contexts. This extends to the assignment of types. For 731 example, the mapping C --> F may be "allocatable" in final position, 732 but "blocked" in any other context: 734 final: C a--> F 735 !final: C x--> F 737 Now, the type assigned to the forward mapping is independent of the 738 reverse symmetric mapping, or any transitive mappings. Imagine a 739 situation where the symmetric mapping is defined as F a--> C, that 740 is, all mappings from F to C are "allocatable": 742 final: F a--> C 743 !final: F a-->C 745 Why not simply write F a--> C? Because the forward mapping is 746 divided by context. Adding a context makes the two forward variant 747 mappings distinct and that needs to be accounted for explicitly in 748 the reverse mappings so that human and machine readers can easily 749 verify symmetry and transitivity of the variant mappings in the LGR. 750 (This is true even though the two opposite contexts "final" and 751 "!final" should together cover all possible cases). 753 16. Making Conditional Variants Well-Behaved 755 To ensure that LGR with contextual variants is well-behaved it is 756 best to always use "fully qualified" variant mappings that always 757 agree in the names of the context rules for forward and reverse 758 mappings. It also necessary to ensure that no label can match more 759 than one context for the same mapping. Using mutually exclusive 760 contexts, such as "final" and "!final" is an easy way to ensure that. 762 However, it is not always necessary to define dual or multiple 763 contexts that together cover all possible cases. For example, here 764 are two contexts that do not cover all possible positional contexts: 766 final: C --> D 767 initial: C --> D. 769 A well-behaved LGR using these two contexts, would define all 770 symmetric and transitive mappings involving C, D and their variants 771 consistently in terms of the two conditions "final" and "initial" and 772 ensure both cannot be satisfied at the same time by some label. 774 In addition to never defining the same mapping with two contexts that 775 may be satisfied by the same label, a well-behaved LGR never combines 776 a variant mapping with context with the same variant mapping without 777 a context: 779 context: C --> D 780 C --> D 782 Inadvertent mixing of conditional and unconditional variants can be 783 detected and flagged by a parser, but verifying that two formally 784 distinct contexts are never satisfied by the same label would depend 785 on the interaction between labels and context rules, which means that 786 it will be up to the LGR designer to ensure the LGR is well-behaved. 788 A well-behaved LGR never assigns conditions on a reflexive variant, 789 as that is effectively no different from having a context on the code 790 point itself; the latter is preferred. 792 Finally, for symmetry to work as expected, the context must be 793 defined such that it is satisfied for both the original code point in 794 the context of the original label and for the variant code point in 795 the variant label. In other words the context should be "stable 796 under variant substitution" anywhere in the label. 798 Positional contexts usually satisfy this last condition; for example, 799 a code point that interrupts a cursive connection would likely share 800 this property with any of its variants. However, as it is in 801 principle possible to define other kinds of contexts, it is necessary 802 to make sure that the LGR is well behaved in this aspect at the time 803 the LGR is designed. 805 Due to the difficulty in verifying these constraints mechanically, it 806 is essential that an LGR designer document the reasons why the LGR 807 can be expected to meet them, and the details of the techniques used 808 to ensure that outcome. This information should be found in the 809 description element of the LGR. 811 In summary, conditional contexts can be an essential tool, but some 812 additional care must be taken to ensure that an LGR containing 813 conditional contexts is well behaved. 815 17. Variants for Sequences 817 Variants mappings can be defined between sequences, or between a code 818 point and a sequence. For example one might define a "blocked" 819 variant between the sequence "rn" and the code point "m" because they 820 are practically indistinguishable in common UI fonts. 822 Such variants are no different from variants defined between single 823 code points, except if a sequence is defined such that there is a 824 code point or shorter sequence that is a prefix (initial subsequence) 825 and both it and the remainder are also part of the repertoire. In 826 that case, it is possible to create duplicate variants with 827 conflicting dispositions. 829 The following shows such an example resulting in conflicting 830 reflexive variants: 832 A a--> C 833 AB x--> CD 835 where AB is a sequence with an initial subsequence of A. For 836 example, B might be a combining code point used in sequence AB. If B 837 only occurs in the sequence, there is no issue, but if B also occurs 838 by itself, for example: 840 B a--> D 842 then a label "AB" might correspond to either {A}{B}, that is the two 843 code points, or {AB}, the sequence, where the curly braces show the 844 sequence boundaries as they would be applied during label validation 845 and variant mapping. 847 A label AB would then generate the "allocatable" variant label {C}{D} 848 and the "blocked" variant label {CD} thus creating two variant labels 849 with conflicting dispositions. 851 For the example of a blocked variant between "m" and "rn" (and vice 852 versa) there is no issue as long as "r" and "n" do not have variant 853 mappings of their own, so that there cannot be multiple variant 854 labels for the same input. However, it is preferable to avoid 855 ambiguities altogether, where possible. 857 The easiest way to avoid an ambiguous segmentation into sequences is 858 by never allowing both a sequence and all of its constituent parts 859 simultaneously as independent parts of the repertoire, for example, 860 by not defining B by itself as a member of the repertoire. 862 Sequences are often used for combining sequences, which consist of a 863 base character B followed by one or more combining marks C. By 864 enumerating all sequences in which a certain combining mark is 865 expected, and by not listing the combining mark by itself in the LGR, 866 the mark cannot occur outside of these specifically enumerated 867 contexts. In cases where enumeration is not possible or practicable, 868 other techniques can be used to prevent ambiguous segmentation, for 869 example, a context rule on code points that disallows B preceding C 870 in any label except as part of a predefined sequence or class of 871 sequences. The details of such techniques are outside the scope of 872 this document (see [RFC7940] for information on context rules for 873 code points). 875 18. Corresponding XML Notation 877 The XML format defined in [RFC7940] corresponds fairly directly to 878 the notation used for variant mappings in this document. (There is 879 no notation in the RFC for variant type sets). In an LGR document, a 880 simple member of a repertoire that does not have any variants is 881 listed as: 883 885 where nnnn is the [Unicode9] code point value in the standard 886 uppercase hexadecimal notation padded to at least 4 digits and 887 without leading "U+". For a code point sequence of length two, the 888 XML notation becomes: 890 892 Variant mappings are defined by nesting elements inside the 893 element. For example, a variant relation of type "blocked" 895 C x--> X 897 is expressed as 899 900 901 903 where "x-->" identifies a "blocked" type. (Other types include 904 "a-->" for "allocatable", for example). Here, nnnn and mmmm are the 905 [Unicode9] code point values for C and X, respectively. Either C or 906 X could be a code point sequence or a single code point. 908 A reflexive mapping is specified the same way, except that it always 909 uses the same code point value for both the and element, 910 for example 912 X r-o--> X 914 would correspond to 916 918 Multiple elements may be nested inside a single element, 919 but their "cp" values must be distinct (unless attributes for context 920 rules are present and the combination of "cp" value and context 921 attributes are distinct). 923 924 925 926 928 A set of conditional variants like 930 final: C a--> K 931 !final: C x--> K 933 would correspond to 935 936 938 where the string "final" references a name of a context rule. 939 Context rules are defined in [RFC7940] and conceptually correspond to 940 regular expressions. The details of how to create and define these 941 rules are outside the scope of this document. If the label matches 942 the context defined in the rule, the variant mapping is valid and 943 takes part in further processing. Otherwise it is invalid and 944 ignored. Using the "not-when" attribute inverts the sense of the 945 match. The two attributes are mutually exclusive. 947 A derivation of a variant label disposition 949 if "only-variants" = "s" or "b" => allocatable 951 is expressed as 953 955 Instead of using "if" and "else if" the elements implicitly 956 form a cascade, where the first action triggered defines the 957 disposition of the label. The order of action elements is thus 958 significant. 960 For the full specification of the XML format see [RFC7940]. 962 19. IANA Considerations 964 This document does not specify any IANA actions. 966 20. Security Considerations 968 As described in [RFC7940], variants may be used as a tool to reduce 969 certain avenues of attack in security-relevant identifiers by 970 allowing certain labels to be "mutually exclusive or registered only 971 to the same user". However, variants, if indiscriminately designed, 972 may themselves contribute to risks to the security or usability of 973 the identifiers, whether resulting from an ambiguous definition or 974 from allowing too many allocatable variants per label. 976 The information in this document is intended to allow the reader to 977 design a specification of an LGR that is "well-behaved" with respect 978 to variants; as used here, this term refers to an LGR that is 979 predictable in its effects to the LGR-author (and reviewer) and more 980 reliable in its implementation. 982 A well-behaved LGR is not merely one that can be expressed in 983 [RFC7940] but in addition, it actively avoids certain edge cases not 984 prevented by the schema, such as those that would result in 985 ambiguities in the specification of the intended disposition for some 986 variants. By applying the additional considerations introduced in 987 this document, including adding certain declarations that are 988 optional under the schema and may not alter the results of processing 989 a label, such an LGR becomes easier to review and its implementations 990 easier to verify. 992 It should be noted, that variants are an important part, but only a 993 part of an LGR design. There are many other features of an LGR that 994 this document does not touch upon. Also, the question of whether to 995 define variants are all, or what labels are to be considered variants 996 of each other is not addressed here. 998 21. References 999 21.1. Normative References 1001 [RFC7940] Davies, K. and A. Freytag, "Representing Label Generation 1002 Rulesets Using XML", RFC 7940, DOI 10.17487/RFC7940, 1003 August 2016, . 1005 21.2. Informative References 1007 [Unicode9] 1008 The Unicode Consortium, "The Unicode Standard, Version 1009 9.0.0", ISBN 978-1-936213-13-9, 2016, 1010 . 1012 Preferred Citation: The Unicode Consortium. The Unicode 1013 Standard, Version 9.0.0, (Mountain View, CA: The Unicode 1014 Consortium, 2016. ISBN 978-1-936213-13-9) 1016 Appendix A. Acknowledgments 1018 Contributions that have shaped this document have been provided by 1019 Marc Blanchet, Patrik Faltstrom, Sarmad Hussain, John Klensin, 1020 Nicholas Ostler, Michel Suignard, Wil Tan and Suzanne Woolf. 1022 Appendix B. Change Log 1024 RFC Editor: Please remove this appendix before publication. 1026 -00 Initial draft. 1028 -01 Minor fix to references. 1030 -02 Some formatting and grammar issues as well as typos fixed. 1031 Added a few real-world examples where required for context. 1032 Added "r-n" to description of subtyping. 1034 -03 Fix ID nits and other typos. Expanded security section. Minor 1035 tweaks. 1037 -04 Additional context. Added to introduction. Introduced sections 1038 on notation and symmetry and transititivy. Expanded the section 1039 on XML notation. 1041 Author's Address 1043 Asmus Freytag 1045 Email: asmus@unicode.org