idnits 2.17.1
draft-freytag-lager-variant-rules-05.txt:
Checking boilerplate required by RFC 5378 and the IETF Trust (see
https://trustee.ietf.org/license-info):
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
----------------------------------------------------------------------------
No issues found here.
Checking nits according to https://www.ietf.org/id-info/checklist :
----------------------------------------------------------------------------
No issues found here.
Miscellaneous warnings:
----------------------------------------------------------------------------
== The copyright year in the IETF Trust and authors Copyright Line does not
match the current year
-- The document date (April 13, 2017) is 2541 days in the past. Is this
intentional?
Checking references for intended status: Informational
----------------------------------------------------------------------------
No issues found here.
Summary: 0 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--).
Run idnits with the --verbose option for more detailed information about
the items above.
--------------------------------------------------------------------------------
2 Network Working Group A. Freytag
3 Internet-Draft April 13, 2017
4 Intended status: Informational
5 Expires: October 15, 2017
7 Variant Rules
8 draft-freytag-lager-variant-rules-05
10 Abstract
12 Rules for validating identifier labels and alternate representations
13 of those labels (variants) are known as "Label Generation Rulesets"
14 (LGRs); they are used for the implementation of identifier systems
15 such as Internationalized Domain Names (IDNs). This document
16 describes ways of designing Label Generation Rulesets (LGRs) that
17 support variant labels. In designing LGRs, it is important to ensure
18 that the label generation rules are consistent and well-behaved in
19 the presence of variants. The design decisions can then be expressed
20 using the XML representation of LGRs that is defined in RFC7940.
22 Status of This Memo
24 This Internet-Draft is submitted in full conformance with the
25 provisions of BCP 78 and BCP 79.
27 Internet-Drafts are working documents of the Internet Engineering
28 Task Force (IETF). Note that other groups may also distribute
29 working documents as Internet-Drafts. The list of current Internet-
30 Drafts is at http://datatracker.ietf.org/drafts/current/.
32 Internet-Drafts are draft documents valid for a maximum of six months
33 and may be updated, replaced, or obsoleted by other documents at any
34 time. It is inappropriate to use Internet-Drafts as reference
35 material or to cite them other than as "work in progress."
37 This Internet-Draft will expire on October 15, 2017.
39 Copyright Notice
41 Copyright (c) 2017 IETF Trust and the persons identified as the
42 document authors. All rights reserved.
44 This document is subject to BCP 78 and the IETF Trust's Legal
45 Provisions Relating to IETF Documents
46 (http://trustee.ietf.org/license-info) in effect on the date of
47 publication of this document. Please review these documents
48 carefully, as they describe your rights and restrictions with respect
49 to this document. Code Components extracted from this document must
50 include Simplified BSD License text as described in Section 4.e of
51 the Trust Legal Provisions and are provided without warranty as
52 described in the Simplified BSD License.
54 Table of Contents
56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
57 2. Variant Relationships . . . . . . . . . . . . . . . . . . . . 4
58 3. Symmetry and Transitivity . . . . . . . . . . . . . . . . . . 4
59 4. A Word on Notation . . . . . . . . . . . . . . . . . . . . . 5
60 5. Variant Mappings . . . . . . . . . . . . . . . . . . . . . . 5
61 6. Variant Labels . . . . . . . . . . . . . . . . . . . . . . . 6
62 7. Variant Types and Label Dispositions . . . . . . . . . . . . 6
63 8. Allocatable Variants . . . . . . . . . . . . . . . . . . . . 7
64 9. Blocked Variants . . . . . . . . . . . . . . . . . . . . . . 9
65 10. Pure Variant Labels . . . . . . . . . . . . . . . . . . . . . 9
66 11. Reflexive Variants . . . . . . . . . . . . . . . . . . . . . 10
67 12. Limiting Allocatable Variants by Subtyping . . . . . . . . . 11
68 13. Allowing Mixed Originals . . . . . . . . . . . . . . . . . . 13
69 14. Handling Out-of-Repertoire Variants . . . . . . . . . . . . . 14
70 15. Conditional Variants . . . . . . . . . . . . . . . . . . . . 15
71 16. Making Conditional Variants Well-Behaved . . . . . . . . . . 16
72 17. Variants for Sequences . . . . . . . . . . . . . . . . . . . 18
73 18. Corresponding XML Notation . . . . . . . . . . . . . . . . . 19
74 19. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21
75 20. Security Considerations . . . . . . . . . . . . . . . . . . . 21
76 21. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
77 21.1. Normative References . . . . . . . . . . . . . . . . . . 22
78 21.2. Informative References . . . . . . . . . . . . . . . . . 22
79 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 22
80 Appendix B. Change Log . . . . . . . . . . . . . . . . . . . . . 22
81 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 23
83 1. Introduction
85 Label Generation Rulesets (LGRs) that define the set of permissible
86 labels, may be applied to a variety of identifier systems, although
87 to date, the most common use is to define policies for implementing
88 Internationalized Domain Names (IDNs) for some zone of the Domain
89 Name System (DNS). Without restricting any of the more general
90 applications of this document, the explanations and examples in this
91 document may be stated in terms of IDNs.
93 In addition to determining whether a given label is eligible, LGRs
94 may also define the condition under which alternate representations
95 of these labels, so called variant labels, may exist and also define
96 their status (disposition). In the most general sense, variant
97 labels are typically labels that are either visually or semantically
98 indistinguishable from an applied for label in the context of the
99 writing system or script supported by the LGR. Unlike merely similar
100 labels, where there may be a measurable degree of similarity, variant
101 labels considered here represent a form of equivalence in meaning or
102 appearance. What constitutes an appropriate variant in any writing
103 system or given context, particular in the DNS, is assumed to have
104 been determined ahead of time, and therefore is not subject of this
105 document.
107 Once identified, variant labels are typically delegated to some
108 entity together with the applied-for label, or permanently reserved,
109 based on the disposition derived from the LGR. Correctly defined,
110 variant labels can improve the security of an LGR yet successfully
111 defining variant rules for an LGR so that the result is well-behaved
112 is not always trivial. This document describes the basic
113 considerations and constraints that must be taken into account and
114 gives examples of what might be use cases for different types of
115 variant specifications in an LGR.
117 This document does not address the question whether variants are an
118 appropriate means to solve any given issue, nor on what basis they
119 should be defined. It is intended to explain in more detail the
120 effects of various declarations and the tradeoffs in making design
121 choices. It tacitly assumes that any LGR will be expressed using the
122 XML representation defined in [RFC7940] and therefore conform to any
123 requirements stated therein. Purely for clarity of exposition,
124 examples in this document are using a more compact notation than the
125 XML syntax defined in [RFC7940]. However, the reader is expected to
126 have some familiarity with the concepts described in that RFC (see
127 Section 4).
129 The user of any identifier system, such as the DNS, interacts with it
130 in the context of labels; variants are experienced as variant labels:
131 two (or more) labels that are functionally "the same" under the
132 conventions of the writing system used, even though their code point
133 sequences are different. An LGR specification, on the other hand,
134 defines variant mappings between code points, and only in a secondary
135 step, derives the variant labels from these mappings. For a
136 discussion of this process, see [RFC7940].
138 The designer of an LGR can control whether some or all of the variant
139 labels created from an original label should be allocatable, that is
140 available for allocation (to the original applicant) or whether some
141 or all of these labels should be blocked instead, that is, remain not
142 allocatable (to anyone). The document describes how this choice of
143 label disposition is accomplished (see Section Section 7).
145 The choice of desired label disposition would be based on the
146 expectations of the users of the particular zone; it is not the
147 subject of this document. Likewise, this document does not address
148 the possibility of an LGR defining custom label dispositions.
149 Instead, this document suggests ways of designing an LGR to achieve
150 the selected design choice for handling variants in the context of
151 the two standard label dispositions "allocatable" and "blocked".
153 The information in this document is based on operational experience
154 gained in developing LGRs for a wide number of languages and scripts
155 using RFC 7940. This information is provided here as a benefit to
156 the wider community. It does not alter or change the specification
157 found in RFC 7940 in any way.
159 2. Variant Relationships
161 A variant relationship is fundamentally a "same as", in other words,
162 it is an equivalence relationship. Now the strictest sense of "same
163 as" would be equality, and for any equality, we have both symmetry
165 A = B => B = A
167 and transitivity
169 A = B and B = C => A = C
171 The variant relationship with its functional sense of "same as" must
172 really satisfy the same constraint. Once we say A is the "same as"
173 B, we also assert that B is the "same as" A. In this document, the
174 symbol "~" means "has a variant relationship with". Thus, we get
176 A ~ B => B ~ A
178 Likewise, if we make the same claim for B and C (B ~ C) then we do
179 get A ~ C, because if B is "the same" as both A and C then A must be
180 "the same as" C:
182 A ~ B and B ~ C => A ~ C
184 3. Symmetry and Transitivity
186 Not all relationships between labels constitute equivalence and those
187 that do not are not transitive and may not be symmetric. For
188 example, the degree to which labels are confusable is not transitive:
189 two labels can be confusingly similar to a third without necessarily
190 being confusable with each other, such as when the third one has a
191 shape that is "in between" the other two. A variant relation based
192 on identical or effectively identical appearance would meet the
193 criterion of transitivity, as would other forms of equivalence, such
194 as semantic equivalence.
196 From the perspective of [RFC7940], an LGR could be specified that is
197 neither symmetric or transitive, and such an LGR would be a valid
198 specification. However, from an implementation point of view there
199 are certain benefits from an LGR that is symmetric and transitive.
200 It greatly simplifies the check for collisions between labels with
201 variants. For this reason, we will limit the discussion of variants
202 in this document to those are symmetric and transitive.
203 Incidentally, it is often straightforward to verify mechanically
204 whether an LGR is symmetric and or transitive, and to compute any
205 mappings required to make it so (but see Section 15).
207 4. A Word on Notation
209 [RFC7940] defines an XML schema for Label Generation Rulesets in
210 general, and variant code points and sequences in particular, see
211 Section 18. That notation is rather verbose and can easily obscure
212 salient features to anyone not trained to read XML. For this reason,
213 this document uses a symbolic shorthand notation in presenting the
214 examples for discussion. This shorthand is merely a didactic tool
215 for presentation and not intended as alternative to or replacement
216 for the XML syntax that is used in formally specifying an LGR under
217 [RFC7940].
219 When it comes time to capture the LGR in a formal definition, the
220 notation used for any of examples in this document can be converted
221 to the XML format as described in Section 18.
223 5. Variant Mappings
225 So far, we have treated variant relationships as simple "same as"
226 ignoring that each relationship representing an equivalence would
227 consist of a symmetric pair of reciprocal mappings. In this
228 document, the symbol "-->" means "maps to".
230 A ~ B => A --> B, B --> A
232 In an LGR, these mappings are not defined directly between labels,
233 but between code points (or code point sequences, see Section 17).
234 In the transitive case, given
236 A ~ B => A --> B, B --> A
238 A ~ C => A --> C, C --> A
240 we also get
241 B ~ C => B --> C, C --> B
243 for a total of six possible mappings. Conventionally, these are
244 listed in tables in order of the source code point, like so
246 A --> B
247 A --> C
248 B --> A
249 B --> C
250 C --> A
251 C --> B
253 As we can see, each of A, B and C can be mapped two ways.
255 6. Variant Labels
257 To create a variant label, each code point in the original label is
258 successively replaced by all variant code points defined by a mapping
259 from the original code point. For a label AAA (the letter "A" three
260 times), the variant labels (given the mappings from transitive
261 example above) would be
263 AAB
264 ABA
265 ABB
266 BAA
267 BAB
268 BBA
269 BBB
270 AAC
271 ...
272 CCC
274 So far, we have mere defined what the variant labels are, but we have
275 not considered their possible dispositions. In the next section we
276 discuss how to set up the variant mappings so that some variant
277 labels are mutually exclusive (blocked), but some may be allocated to
278 the same applicant as the original label (allocatable).
280 7. Variant Types and Label Dispositions
282 Assume we wanted to allow a variant relation between some code points
283 O and A, and perhaps also between O and B as well as O and C. By
284 transitivity we would have
286 O ~ A ~ B ~ C
288 However, we would like to distinguish the case where someone applies
289 for OOO from the case where someone applies for the label ABC. In
290 the former case we would like to allocate only the label OOO, but in
291 the latter case, we would like to also allow the allocation of either
292 the original label OOO or the variant label ABC, or both, but not of
293 any of the other possible variant labels, like OAO, BCO or the like.
294 (A real-world example might be the case where O represents an
295 unaccented letter, while A, B and C might represent various accented
296 forms of the same letter. Because unaccented letters are a common
297 fallback, there might be a desire to allocate an unaccented label as
298 a variant, but not the other way around.)
300 How do we make that distinction?
302 The answer lies in labeling the mappings A --> O, B --> O, and C -->
303 O with the type "allocatable" and the mappings O --> A, O --> B, and
304 O --> C with the type "blocked". In this document, the symbol "x-->"
305 means "maps with type blocked" and the symbol "a-->" means "maps with
306 type allocatable". Thus:
308 O x--> A
309 O x--> B
310 O x--> C
311 A a--> O
312 B a--> O
313 C a--> O
315 When we generate all permutations of labels, we use mappings with
316 different types depending from which code points we start.
318 In creating an LGR with variants, all variant mappings should always
319 be labeled with a type ([RFC7940] does not formally require a type,
320 but any well-behaved LGR would be fully typed). By default, these
321 types correspond directly to the dispositions for variant labels,
322 with the most restrictive type determining the disposition of the
323 variant label. However, as we shall see later, it is sometimes
324 useful to assign types from a wider array of values than the final
325 dispositions for the labels and then define explicitly how to derive
326 label dispositions from them.
328 8. Allocatable Variants
330 If we start with AAA, the permutation OOO will have been the result
331 of applying the mapping A a--> O at each code point. That is, only
332 mappings with type "a" (allocatable) were used. To know whether we
333 can allocate both the label OOO and the original label AAA we track
334 the types of the mappings used in generating the label.
336 We record the variant types for each of the variant mappings used in
337 creating the permutation in an ordered list. Such an ordered list of
338 variant types is called a "variant type list". In running text we
339 often show it enclosed in square brackets. For example [a x -] means
340 the variant label was derived from a variant mapping with the "a"
341 variant type in the first code point position, "x" in the second code
342 point position, and that the third position is the original code
343 point ("-" means "no variant mapping").
345 For our example permutation we get the following variant type list
346 (brackets dropped):
348 AAA --> OOO : a a a
350 From the variant type list we derive a "variant type set", denoted by
351 curly braces, that contains an unordered set of unique variant types
352 in the variant type list. For the variant type list for the given
353 permutation, [a a a], the variant type set is { a }, which has a
354 single element "a".
356 Deciding whether to allow the allocation of a variant label then
357 amounts to deriving a disposition for the variant label from the
358 variant type set created from the variant mappings that were used to
359 create the label. For example the derivation
361 if "all variants" = "a" => set label disposition to "allocatable"
363 would allow OOO to be allocated, because the types of all variants
364 mappings used to create that variant label from AAA are "a".
366 The "all-variants" condition is tolerant of an extra "-" in the
367 variant set (unlike the "only-variants" condition described below).
368 So, had we started with AOA, OAA or AAO, the variant set for the
369 permuted variant OOO would have been { a - } because in each case one
370 of the code points remains the same as the original. The "-" means
371 that because of the absence of a mapping O --> O there is no variant
372 type for the O in each of these labels.
374 The "all-variants" = "a" condition ignores the "-", so using the
375 derivation from above, we find that OOO is an allocatable variant for
376 each of the labels AOA, OAA or AAO.
378 Allocatable variant labels, especially large numbers of allocatable
379 variants per label, incur a certain cost to users of the LGR. A
380 well-behaved LGR will minimize the number allocatable variants.
382 9. Blocked Variants
384 Blocked variants are not available to another registrant. They
385 therefore protect the applicant of the original label from someone
386 else registering a label that is "the same as" under some user-
387 perceived metric. Blocked variants can be a useful tool even for
388 scripts for which no allocatable labels are ever defined.
390 If we start with OOO, the permutation AAA will have been the result
391 of applying only mappings with type "blocked" and we cannot allocate
392 the label AAA, only the original label OOO. This corresponds to the
393 following derivation:
395 if "any variants" = "x" => set label disposition to "blocked"
397 To additionally prevent allocating ABO as a variant label for AAA we
398 further need to make sure that the mapping A --> B has been defined
399 with type "blocked" as in
401 A x--> B
403 so that
405 AAA --> ABO: - x a.
407 Thus the set {x a} contains at least one "x" and satisfies the
408 derivation of a blocked disposition for ABO when AAA is applied for.
410 If an LGR results in a symmetric and transitive set of variant
411 labels, then the task of determining whether a label or its variants
412 collide with another label or its variants can be implemented very
413 efficiently. Symmetry and transitivity implies that each set of
414 labels that are mutually variants of each other is disjoint from all
415 other such sets. Only labels within the same set can be variants of
416 each other. Identifying the variant set can be an O(1) operation,
417 and enumerating all variants is not necessary.
419 10. Pure Variant Labels
421 Now, if we wanted to prevent allocation of AOA when we start from
422 AAA, we would need a rule disallowing a mix of original code points
423 and variant code points, which is easily accomplished by use of the
424 "only-variants" qualifier, which requires that the label consist
425 entirely of variants and all the variants are from the same set of
426 types.
428 if "only-variants" = "a" => set label disposition to "allocatable"
430 The two code points A in AOA are not arrived at by variant mappings,
431 because the code points are unchanged and no variant mappings are
432 defined for A --> A. So, in our example, the set of variant mapping
433 types is
435 AAA --> AOA: - a -
437 but unlike the "all-variants" condition, "only-variants" requires a
438 variant type set { a } corresponding to a variant type list [a a a]
439 (no - allowed). By adding a final derivation
441 else if "any-variants" = "a" => set label disposition to "blocked"
443 and executing that derivation only on any remaining labels, we
444 disallow AOA when starting from AAA, but still allow OOO.
446 Derivation conditions are always applied in order, with later
447 derivations only applying to labels that did not match any earlier
448 conditions, as indicated by the use of "else" in the last example.
449 In other words, they form a cascade.
451 11. Reflexive Variants
453 But what if we started from AOA? We would expect OOO to be
454 allocatable, but the variant type set would be
456 OOO --> OOO: a - a
458 because the O is the original code point. Here is where we use a
459 reflexive mapping, by realizing that O is "the same as" O, which is
460 normally redundant, but allows us to specify a disposition on the
461 mapping
463 O a--> O
465 with that, the variant type list for OOO --> OOO becomes:
467 AOA --> OOO: a a a
469 and the label OOO again passes the derivation condition
471 if "only-variants" = "a" => set label disposition to "allocatable"
473 as desired. This use of reflexive variants is typical whenever
474 derivations with the "only-variants" qualifier are used. If any code
475 point uses a reflexive variant, a well-behaved LGR would specify an
476 appropriate reflexive variant for all code points.
478 12. Limiting Allocatable Variants by Subtyping
480 As we have seen, the number of variant labels can potentially be
481 large, due to combinatorics. Sometimes it is possible to divide
482 variants into categories and to stipulate that only variant labels
483 with variants from the same category should be allocatable. For some
484 LGRs this constraint can be implemented by a rule that disallows code
485 points from different categories to occur in the same allocatable
486 label. For other LGRs the appropriate mechanism may be dividing the
487 allocatable variants into subtypes.
489 To recap, in the standard case a code point C can have (up to) two
490 types of variant mappings
492 C x--> X
493 C a--> A
495 where a--> means a variant mapping with type "allocatable", and x-->
496 means "blocked". For the purpose of the following discussion, we
497 name the target code point with the corresponding uppercase letter.
499 Subtyping allows us to distinguish among different types of
500 allocatable variants. For example, we can define three new types:
501 "s", "t" and "b". Of these, "s" and "t" are mutually incompatible,
502 but "b" is compatible with either "s" or "t" (in this case, "b"
503 stands for "both"). A real-world example for this might be variant
504 mappings appropriate for "simplified" or "traditional" Chinese
505 variants, or appropriate for both.
507 With subtypes defined as above, a code point C might have (up to)
508 four types of variant mappings
510 C x--> X
511 C s--> S
512 C t--> T
513 C b--> B
515 and explicit reflexive mappings of one of these types
517 C s--> C
518 C t--> C
519 C b--> C
521 As before, all mappings must have one and only one type, but each
522 code point may map to any number of other code points.
524 We define the compatibility of "b" with "t" or "s" by our choice of
525 derivation conditions as follows
526 if "any-variants" = "x" => blocked
527 else if "only-variants" = "s" or "b" => allocatable
528 else if "only-variants" = "t" or "b" => allocatable
529 else if "any-variants" = "s" or "t" or "b" => blocked
531 An original label of four code points
533 CCCC
535 may have many variant labels such as this example listed with its
536 corresponding variant type list:
538 CCCC --> XSTB : x s t b
540 This variant label is blocked because to get from C to B required
541 x-->. (Because variant mappings are defined for specific source code
542 points, we need to show the starting label for each of these
543 examples, not merely the code points in the variant label.) The
544 variant label
546 CCCC --> SSBB : s s b b
548 is allocatable, because the variant type list contains only
549 allocatable mappings of subtype "s" or "b", which we have defined as
550 being compatible by our choice of derivations. The actual set of
551 variant types {s, b} has only two members, but the examples are
552 easier to follow if we list each type. The label
554 CCCC --> TTBB : t t b b
556 is again allocatable, because the variant type set {t, b} contains
557 only allocatable mappings of the mutually compatible allocatable
558 subtypes "t" or "b". In contrast,
560 CCCC --> SSTT : s s t t
562 is not allocatable, because the type set contains incompatible
563 subtypes "t" and "s" and thus would be blocked by the final
564 derivation.
566 The variant labels
568 CCCC --> CSBB : c s b b
569 CCCC --> CTBB : c t b b
571 are only allocatable based on the subtype for the C --> C mapping,
572 which is denoted here by c and (depending on what was chosen for the
573 type of the reflexive mapping) could correspond to "s", "t", or "b".
575 If it is "s", the first of these two labels is allocatable; if it is
576 "t", the second of these two labels is allocatable; if it is b, both
577 labels are allocatable.
579 So far, the scheme does not seem to have brought any huge reduction
580 in allocatable variant labels, but that is because we tacitly assumed
581 that C could have all three types of allocatable variants "s", "t",
582 and "b" at the same time.
584 In a real world example, the types "s", "t" and "b" are assigned so
585 that each code point C normally has at most one non-reflexive variant
586 mapping labeled with one of these subtypes, and all other mappings
587 would be assigned type "x" (blocked). This holds true for most code
588 points in existing tables (such as those used in current IDN TLDs),
589 although certain code points have exceptionally complex variant
590 relations and may have an extra mapping.
592 13. Allowing Mixed Originals
594 If the desire is to allow original labels (but not variant labels)
595 that are s/t mixed, then the scheme needs to be slightly refined to
596 distinguish between reflexive and non-reflexive variants. In this
597 document, the symbol "r-n" means "a reflexive (identity) mapping of
598 type 'n'". The reflexive mappings of the preceding section thus
599 become:
601 C r-s--> C
602 C r-t--> C
603 C r-b--> C
605 With this convention, and redefining the derivations
607 if "any-variants" = "x" => blocked
608 else if "only-variants" = "s" or "r-s" or "b" or "r-b" => allocatable
609 else if "only-variants" = "t" or "r-t" or "b" or "r-b" => allocatable
610 else if "any-variants" = "s" or "t" or "b" => blocked
611 else => allocatable
613 any labels that contain only reflexive mappings of otherwise mixed
614 type (in other words, any mixed original label) now fall through and
615 their disposition is set to "allocatable" in the final derivation.
617 In a well-behaved LGR, it is preferable to explicitly define the
618 derivation for allocatable labels, instead of using a fall-through.
619 In the derivation above, code points without any variant mappings
620 fall through and become allocatable by default if they are part of an
621 original label. Especially in a large repertoire it can be difficult
622 to identify which code points are affected. Instead, it is
623 preferable to mark them with their own reflexive mapping type
624 "neither" or "r-n".
626 C r-n--> C
628 With that we can change
630 else => allocatable
632 to
634 else if "only-variants" = "r-s" or "r-t" or "r-b" or "r-n"
635 => allocatable
636 else => invalid
638 This makes the intent more explicit and by ensuring that all code
639 points in the LGR have a reflexive mapping of some kind, it is easier
640 to verify the correct assignment of their types.
642 14. Handling Out-of-Repertoire Variants
644 At first it may seem counterintuitive to define variants that map to
645 code points not part of the repertoire. However, for zones for which
646 multiple LGRs are defined, there may be situations where labels valid
647 under one LGR should be blocked if a label under another LGR is
648 already delegated. This situation can arise whether or not the
649 repertoires of the affected LGRs overlap, and, where repertoires
650 overlap, whether or not the labels are both restricted to the common
651 subset.
653 In order to handle this exclusion relation through definition of
654 variants, it is necessary to be able to specify variant mappings to
655 some code point X that is outside an LGR's repertoire, R:
657 C x--> X : where C = elementOf(R) and X != elementOf(R)
659 Because of symmetry, it is necessary to also specify the inverse
660 mapping in the LGR:
662 X x--> C : where X != elementOf(R) and C = elementOf(R)
664 This makes X a source of variant mappings and it becomes necessary to
665 identify X as being outside the repertoire, so that any attempt to
666 apply for a label containing X will lead to a disposition of
667 "invalid" - just as if X had never been listed in the LGR. The
668 mechanism to do this, again uses reflexive variants, but with a new
669 type of reflexive mapping of "out-of-repertoire-var", shown as
670 "r-o-->":
672 X r-o--> X
674 This indicates X != elementOf(R), as long as the LGR is provided with
675 a suitable derivation, so that any label containing "r-o-->" is
676 assigned a disposition of "invalid", just as if X was any other code
677 point not part of the repertoire. The derivation used is:
679 if "any-variant" = "out-of-repertoire-var" => invalid
681 It is inserted ahead of any other derivation of the "any-variant"
682 kind in the chain of derivations. As a result, instead of the
683 minimum two symmetric variants, for any out-of repertoire variants
684 there are a minimum of three variant mappings defined:
686 C x--> X
687 X x--> C
688 X r-o--> X
690 where C = elementOf(R) and X != elementOf(R).
692 Because no variant label with any code point outside the repertoire
693 could ever be allocated, the only logical choice for the non-
694 reflexive mappings to out-of-repertoire code points is "blocked".
696 15. Conditional Variants
698 Variant mappings are based on whether code points are "the same" to
699 the user. In some writing systems, code points change shape based on
700 where they occur in the word (positional forms). Some code points
701 have matching shapes in some positions, but not in others. In such
702 cases, the variant mapping only exists for some possible positions,
703 or more general, only for some contexts. For other contexts, the
704 variant mapping does not exist.
706 For example, take two code points, that have the same shape at the
707 end of a label (or in final position) but not in any other position.
708 In that case, they are variants only when they occur in the final
709 position, something we indicate like this:
711 final: C --> D
713 In cursively connected scripts, like Arabic, a code point may take
714 its final form when next to any following code point that interrupts
715 the cursive connection, not just at the end of a label. (We ignore
716 the isolated form to keep the discussion simple, if it was included,
717 "final" might be "final-or-isolate", for example).
719 From symmetry, we expect that the mapping D --> C should also exist
720 only when the code point D is in final position. (Similar
721 considerations apply to transitivity).
723 Sometimes a code point has a final form that is practically the same
724 as that of some code point while sharing initial and medial forms
725 with another.
727 final: C --> D
728 !final: C --> E
730 Here the case where the condition is the opposite of final is shown
731 as "!final".
733 Because shapes differ by position, when a context is applied to a
734 variant mapping, it is treated independently from the same mapping in
735 other contexts. This extends to the assignment of types. For
736 example, the mapping C --> F may be "allocatable" in final position,
737 but "blocked" in any other context:
739 final: C a--> F
740 !final: C x--> F
742 Now, the type assigned to the forward mapping is independent of the
743 reverse symmetric mapping, or any transitive mappings. Imagine a
744 situation where the symmetric mapping is defined as F a--> C, that
745 is, all mappings from F to C are "allocatable":
747 final: F a--> C
748 !final: F a-->C
750 Why not simply write F a--> C? Because the forward mapping is
751 divided by context. Adding a context makes the two forward variant
752 mappings distinct and that needs to be accounted for explicitly in
753 the reverse mappings so that human and machine readers can easily
754 verify symmetry and transitivity of the variant mappings in the LGR.
755 (This is true even though the two opposite contexts "final" and
756 "!final" should together cover all possible cases).
758 16. Making Conditional Variants Well-Behaved
760 To ensure that LGR with contextual variants is well-behaved it is
761 best to always use "fully qualified" variant mappings that always
762 agree in the names of the context rules for forward and reverse
763 mappings. It also necessary to ensure that no label can match more
764 than one context for the same mapping. Using mutually exclusive
765 contexts, such as "final" and "!final" is an easy way to ensure that.
767 However, it is not always necessary to define dual or multiple
768 contexts that together cover all possible cases. For example, here
769 are two contexts that do not cover all possible positional contexts:
771 final: C --> D
772 initial: C --> D.
774 A well-behaved LGR using these two contexts, would define all
775 symmetric and transitive mappings involving C, D and their variants
776 consistently in terms of the two conditions "final" and "initial" and
777 ensure both cannot be satisfied at the same time by some label.
779 In addition to never defining the same mapping with two contexts that
780 may be satisfied by the same label, a well-behaved LGR never combines
781 a variant mapping with context with the same variant mapping without
782 a context:
784 context: C --> D
785 C --> D
787 Inadvertent mixing of conditional and unconditional variants can be
788 detected and flagged by a parser, but verifying that two formally
789 distinct contexts are never satisfied by the same label would depend
790 on the interaction between labels and context rules, which means that
791 it will be up to the LGR designer to ensure the LGR is well-behaved.
793 A well-behaved LGR never assigns conditions on a reflexive variant,
794 as that is effectively no different from having a context on the code
795 point itself; the latter is preferred.
797 Finally, for symmetry to work as expected, the context must be
798 defined such that it is satisfied for both the original code point in
799 the context of the original label and for the variant code point in
800 the variant label. In other words the context should be "stable
801 under variant substitution" anywhere in the label.
803 Positional contexts usually satisfy this last condition; for example,
804 a code point that interrupts a cursive connection would likely share
805 this property with any of its variants. However, as it is in
806 principle possible to define other kinds of contexts, it is necessary
807 to make sure that the LGR is well behaved in this aspect at the time
808 the LGR is designed.
810 Due to the difficulty in verifying these constraints mechanically, it
811 is essential that an LGR designer document the reasons why the LGR
812 can be expected to meet them, and the details of the techniques used
813 to ensure that outcome. This information should be found in the
814 description element of the LGR.
816 In summary, conditional contexts can be an essential tool, but some
817 additional care must be taken to ensure that an LGR containing
818 conditional contexts is well behaved.
820 17. Variants for Sequences
822 Variants mappings can be defined between sequences, or between a code
823 point and a sequence. For example one might define a "blocked"
824 variant between the sequence "rn" and the code point "m" because they
825 are practically indistinguishable in common UI fonts.
827 Such variants are no different from variants defined between single
828 code points, except if a sequence is defined such that there is a
829 code point or shorter sequence that is a prefix (initial subsequence)
830 and both it and the remainder are also part of the repertoire. In
831 that case, it is possible to create duplicate variants with
832 conflicting dispositions.
834 The following shows such an example resulting in conflicting
835 reflexive variants:
837 A a--> C
838 AB x--> CD
840 where AB is a sequence with an initial subsequence of A. For
841 example, B might be a combining code point used in sequence AB. If B
842 only occurs in the sequence, there is no issue, but if B also occurs
843 by itself, for example:
845 B a--> D
847 then a label "AB" might correspond to either {A}{B}, that is the two
848 code points, or {AB}, the sequence, where the curly braces show the
849 sequence boundaries as they would be applied during label validation
850 and variant mapping.
852 A label AB would then generate the "allocatable" variant label {C}{D}
853 and the "blocked" variant label {CD} thus creating two variant labels
854 with conflicting dispositions.
856 For the example of a blocked variant between "m" and "rn" (and vice
857 versa) there is no issue as long as "r" and "n" do not have variant
858 mappings of their own, so that there cannot be multiple variant
859 labels for the same input. However, it is preferable to avoid
860 ambiguities altogether, where possible.
862 The easiest way to avoid an ambiguous segmentation into sequences is
863 by never allowing both a sequence and all of its constituent parts
864 simultaneously as independent parts of the repertoire, for example,
865 by not defining B by itself as a member of the repertoire.
867 Sequences are often used for combining sequences, which consist of a
868 base character B followed by one or more combining marks C. By
869 enumerating all sequences in which a certain combining mark is
870 expected, and by not listing the combining mark by itself in the LGR,
871 the mark cannot occur outside of these specifically enumerated
872 contexts. In cases where enumeration is not possible or practicable,
873 other techniques can be used to prevent ambiguous segmentation, for
874 example, a context rule on code points that disallows B preceding C
875 in any label except as part of a predefined sequence or class of
876 sequences. The details of such techniques are outside the scope of
877 this document (see [RFC7940] for information on context rules for
878 code points).
880 18. Corresponding XML Notation
882 The XML format defined in [RFC7940] corresponds fairly directly to
883 the notation used for variant mappings in this document. (There is
884 no notation in the RFC for variant type sets). In an LGR document, a
885 simple member of a repertoire that does not have any variants is
886 listed as:
888
890 where nnnn is the [Unicode] code point value in the standard
891 uppercase hexadecimal notation padded to at least 4 digits and
892 without leading "U+". For a code point sequence of length two, the
893 XML notation becomes:
895
897 Variant mappings are defined by nesting elements inside the
898 element. For example, a variant relation of type "blocked"
900 C x--> X
902 is expressed as
904
905
906
908 where "x-->" identifies a "blocked" type. (Other types include
909 "a-->" for "allocatable", for example). Here, nnnn and mmmm are the
911 [Unicode] code point values for C and X, respectively. Either C or X
912 could be a code point sequence or a single code point.
914 A reflexive mapping is specified the same way, except that it always
915 uses the same code point value for both the and element,
916 for example
918 X r-o--> X
920 would correspond to
922
924 Multiple elements may be nested inside a single element,
925 but their "cp" values must be distinct (unless attributes for context
926 rules are present and the combination of "cp" value and context
927 attributes are distinct).
929
930
931
932
934 A set of conditional variants like
936 final: C a--> K
937 !final: C x--> K
939 would correspond to
941
942
944 where the string "final" references a name of a context rule.
945 Context rules are defined in [RFC7940]; they conceptually correspond
946 to regular expressions. The details of how to create and define
947 these rules are outside the scope of this document. If the label
948 matches the context defined in the rule, the variant mapping is valid
949 and takes part in further processing. Otherwise, it is invalid and
950 ignored. Using the "not-when" attribute inverts the sense of the
951 match. The two attributes are mutually exclusive.
953 A derivation of a variant label disposition
955 if "only-variants" = "s" or "b" => allocatable
957 is expressed as
958
960 Instead of using "if" and "else if" the elements implicitly
961 form a cascade, where the first action triggered defines the
962 disposition of the label. The order of action elements is thus
963 significant.
965 For the full specification of the XML format see [RFC7940].
967 19. IANA Considerations
969 This document does not specify any IANA actions.
971 20. Security Considerations
973 As described in [RFC7940], variants may be used as a tool to reduce
974 certain avenues of attack in security-relevant identifiers by
975 allowing certain labels to be "mutually exclusive or registered only
976 to the same user". However, variants, if indiscriminately designed,
977 may themselves contribute to risks to the security or usability of
978 the identifiers, whether resulting from an ambiguous definition or
979 from allowing too many allocatable variants per label.
981 The information in this document is intended to allow the reader to
982 design a specification of an LGR that is "well-behaved" with respect
983 to variants; as used here, this term refers to an LGR that is
984 predictable in its effects to the LGR-author (and reviewer) and more
985 reliable in its implementation.
987 A well-behaved LGR is not merely one that can be expressed in
988 [RFC7940] but in addition, it actively avoids certain edge cases not
989 prevented by the schema, such as those that would result in
990 ambiguities in the specification of the intended disposition for some
991 variant labels. By applying the additional considerations introduced
992 in this document, including adding certain declarations that are
993 optional under the schema and may not alter the results of processing
994 a label, such an LGR becomes easier to review and its implementations
995 easier to verify.
997 It should be noted that variants are an important part, but only a
998 part of an LGR design. There are many other features of an LGR that
999 this document does not touch upon. Also, the question of whether to
1000 define variants are all, or what labels are to be considered variants
1001 of each other is not addressed here.
1003 21. References
1005 21.1. Normative References
1007 [RFC7940] Davies, K. and A. Freytag, "Representing Label Generation
1008 Rulesets Using XML", RFC 7940, DOI 10.17487/RFC7940,
1009 August 2016, .
1011 21.2. Informative References
1013 [Unicode] The Unicode Consortium, "The Unicode Standard, Version
1014 9.0.0", ISBN 978-1-936213-13-9, 2016,
1015 .
1017 Preferred Citation: The Unicode Consortium. The Unicode
1018 Standard, Version 9.0.0, (Mountain View, CA: The Unicode
1019 Consortium, 2016. ISBN 978-1-936213-13-9)
1021 Appendix A. Acknowledgments
1023 Contributions that have shaped this document have been provided by
1024 Marc Blanchet, Patrik Faltstrom, Scott Hollenbeck, Sarmad Hussain,
1025 John Klensin, Alexey Melnikov, Nicholas Ostler, Michel Suignard, Wil
1026 Tan and Suzanne Woolf.
1028 Appendix B. Change Log
1030 RFC Editor: Please remove this appendix before publication.
1032 -00 Initial draft.
1034 -01 Minor fix to references.
1036 -02 Some formatting and grammar issues as well as typos fixed.
1037 Added a few real-world examples where required for context.
1038 Added "r-n" to description of subtyping.
1040 -03 Fix ID nits and other typos. Expanded security section. Minor
1041 tweaks.
1043 -04 Additional context. Added to introduction. Introduced sections
1044 on notation and symmetry and transitivity. Expanded the section
1045 on XML notation.
1047 -05 Additional change to introduction.
1049 Author's Address
1051 Asmus Freytag
1053 Email: asmus@unicode.org