idnits 2.17.1 draft-ietf-precis-7564bis-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 25, 2017) is 2465 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'Unicode' -- Obsolete informational reference (is this intentional?): RFC 7564 (ref. 'Err4568') (Obsoleted by RFC 8264) == Outdated reference: A later version (-05) exists of draft-klensin-idna-5892upd-unicode70-04 -- Obsolete informational reference (is this intentional?): RFC 3454 (Obsoleted by RFC 7564) -- Obsolete informational reference (is this intentional?): RFC 3490 (Obsoleted by RFC 5890, RFC 5891) -- Obsolete informational reference (is this intentional?): RFC 3491 (Obsoleted by RFC 5891) -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) -- Obsolete informational reference (is this intentional?): RFC 5246 (Obsoleted by RFC 8446) -- Duplicate reference: RFC7564, mentioned in 'RFC7564', was also mentioned in 'Err4568'. -- Obsolete informational reference (is this intentional?): RFC 7564 (Obsoleted by RFC 8264) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Saint-Andre 3 Internet-Draft Filament 4 Obsoletes: 7564 (if approved) M. Blanchet 5 Intended status: Standards Track Viagenie 6 Expires: January 26, 2018 July 25, 2017 8 PRECIS Framework: Preparation, Enforcement, and Comparison of 9 Internationalized Strings in Application Protocols 10 draft-ietf-precis-7564bis-10 12 Abstract 14 Application protocols using Unicode code points in protocol strings 15 need to properly handle such strings in order to enforce 16 internationalization rules for strings placed in various protocol 17 slots (such as addresses and identifiers) and to perform valid 18 comparison operations (e.g., for purposes of authentication or 19 authorization). This document defines a framework enabling 20 application protocols to perform the preparation, enforcement, and 21 comparison of internationalized strings ("PRECIS") in a way that 22 depends on the properties of Unicode code points and thus is more 23 agile with respect to versions of Unicode. As a result, this 24 framework provides a more sustainable approach to the handling of 25 internationalized strings than the previous framework, known as 26 Stringprep (RFC 3454). This document obsoletes RFC 7564. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at http://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on January 26, 2018. 45 Copyright Notice 47 Copyright (c) 2017 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 64 3. Preparation, Enforcement, and Comparison . . . . . . . . . . 7 65 4. String Classes . . . . . . . . . . . . . . . . . . . . . . . 8 66 4.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 8 67 4.2. IdentifierClass . . . . . . . . . . . . . . . . . . . . . 9 68 4.2.1. Valid . . . . . . . . . . . . . . . . . . . . . . . . 9 69 4.2.2. Contextual Rule Required . . . . . . . . . . . . . . 10 70 4.2.3. Disallowed . . . . . . . . . . . . . . . . . . . . . 10 71 4.2.4. Unassigned . . . . . . . . . . . . . . . . . . . . . 11 72 4.2.5. Examples . . . . . . . . . . . . . . . . . . . . . . 11 73 4.3. FreeformClass . . . . . . . . . . . . . . . . . . . . . . 11 74 4.3.1. Valid . . . . . . . . . . . . . . . . . . . . . . . . 11 75 4.3.2. Contextual Rule Required . . . . . . . . . . . . . . 12 76 4.3.3. Disallowed . . . . . . . . . . . . . . . . . . . . . 12 77 4.3.4. Unassigned . . . . . . . . . . . . . . . . . . . . . 12 78 4.3.5. Examples . . . . . . . . . . . . . . . . . . . . . . 12 79 4.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 13 80 5. Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . 15 81 5.1. Profiles Must Not Be Multiplied beyond Necessity . . . . 15 82 5.2. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16 83 5.2.1. Width Mapping Rule . . . . . . . . . . . . . . . . . 16 84 5.2.2. Additional Mapping Rule . . . . . . . . . . . . . . . 16 85 5.2.3. Case Mapping Rule . . . . . . . . . . . . . . . . . . 17 86 5.2.4. Normalization Rule . . . . . . . . . . . . . . . . . 17 87 5.2.5. Directionality Rule . . . . . . . . . . . . . . . . . 18 88 5.3. A Note about Spaces . . . . . . . . . . . . . . . . . . . 19 89 6. Applications . . . . . . . . . . . . . . . . . . . . . . . . 19 90 6.1. How to Use PRECIS in Applications . . . . . . . . . . . . 19 91 6.2. Further Excluded Characters . . . . . . . . . . . . . . . 21 92 6.3. Building Application-Layer Constructs . . . . . . . . . . 21 94 7. Order of Operations . . . . . . . . . . . . . . . . . . . . . 22 95 8. Code Point Properties . . . . . . . . . . . . . . . . . . . . 22 96 9. Category Definitions Used to Calculate Derived Property . . . 25 97 9.1. LetterDigits (A) . . . . . . . . . . . . . . . . . . . . 26 98 9.2. Unstable (B) . . . . . . . . . . . . . . . . . . . . . . 26 99 9.3. IgnorableProperties (C) . . . . . . . . . . . . . . . . . 26 100 9.4. IgnorableBlocks (D) . . . . . . . . . . . . . . . . . . . 26 101 9.5. LDH (E) . . . . . . . . . . . . . . . . . . . . . . . . . 26 102 9.6. Exceptions (F) . . . . . . . . . . . . . . . . . . . . . 26 103 9.7. BackwardCompatible (G) . . . . . . . . . . . . . . . . . 26 104 9.8. JoinControl (H) . . . . . . . . . . . . . . . . . . . . . 27 105 9.9. OldHangulJamo (I) . . . . . . . . . . . . . . . . . . . . 27 106 9.10. Unassigned (J) . . . . . . . . . . . . . . . . . . . . . 27 107 9.11. ASCII7 (K) . . . . . . . . . . . . . . . . . . . . . . . 27 108 9.12. Controls (L) . . . . . . . . . . . . . . . . . . . . . . 27 109 9.13. PrecisIgnorableProperties (M) . . . . . . . . . . . . . . 28 110 9.14. Spaces (N) . . . . . . . . . . . . . . . . . . . . . . . 28 111 9.15. Symbols (O) . . . . . . . . . . . . . . . . . . . . . . . 28 112 9.16. Punctuation (P) . . . . . . . . . . . . . . . . . . . . . 28 113 9.17. HasCompat (Q) . . . . . . . . . . . . . . . . . . . . . . 28 114 9.18. OtherLetterDigits (R) . . . . . . . . . . . . . . . . . . 29 115 10. Guidelines for Designated Experts . . . . . . . . . . . . . . 29 116 11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30 117 11.1. PRECIS Derived Property Value Registry . . . . . . . . . 30 118 11.2. PRECIS Base Classes Registry . . . . . . . . . . . . . . 30 119 11.3. PRECIS Profiles Registry . . . . . . . . . . . . . . . . 31 120 12. Security Considerations . . . . . . . . . . . . . . . . . . . 33 121 12.1. General Issues . . . . . . . . . . . . . . . . . . . . . 33 122 12.2. Use of the IdentifierClass . . . . . . . . . . . . . . . 33 123 12.3. Use of the FreeformClass . . . . . . . . . . . . . . . . 34 124 12.4. Local Character Set Issues . . . . . . . . . . . . . . . 34 125 12.5. Visually Similar Characters . . . . . . . . . . . . . . 34 126 12.6. Security of Passwords . . . . . . . . . . . . . . . . . 36 127 13. Interoperability Considerations . . . . . . . . . . . . . . . 37 128 13.1. Coded Character Sets . . . . . . . . . . . . . . . . . . 37 129 13.2. Dependency on Unicode . . . . . . . . . . . . . . . . . 37 130 13.3. Encoding . . . . . . . . . . . . . . . . . . . . . . . . 38 131 13.4. Unicode Versions . . . . . . . . . . . . . . . . . . . . 38 132 13.5. Potential Changes to Handling of Certain Unicode Code 133 Points . . . . . . . . . . . . . . . . . . . . . . . . . 38 134 14. References . . . . . . . . . . . . . . . . . . . . . . . . . 39 135 14.1. Normative References . . . . . . . . . . . . . . . . . . 39 136 14.2. Informative References . . . . . . . . . . . . . . . . . 40 137 Appendix A. Changes from RFC 7564 . . . . . . . . . . . . . . . 44 138 Appendix B. Acknowledgements . . . . . . . . . . . . . . . . . . 44 139 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 44 141 1. Introduction 143 Application protocols using Unicode code points [Unicode] in protocol 144 strings need to properly handle such strings in order to enforce 145 internationalization rules for strings placed in various protocol 146 slots (such as addresses and identifiers) and to perform valid 147 comparison operations (e.g., for purposes of authentication or 148 authorization). This document defines a framework enabling 149 application protocols to perform the preparation, enforcement, and 150 comparison of internationalized strings ("PRECIS") in a way that 151 depends on the properties of Unicode code points and thus is more 152 agile with respect to versions of Unicode. (PRECIS is restricted to 153 Unicode and does not support any other coded character set 154 [RFC6365].) 156 As described in the PRECIS problem statement [RFC6885], many IETF 157 protocols have used the Stringprep framework [RFC3454] as the basis 158 for preparing, enforcing, and comparing protocol strings that contain 159 Unicode code points, especially code points outside the ASCII range 160 [RFC20]. The Stringprep framework was developed during work on the 161 original technology for internationalized domain names (IDNs), here 162 called "IDNA2003" [RFC3490], and Nameprep [RFC3491] was the 163 Stringprep profile for IDNs. At the time, Stringprep was designed as 164 a general framework so that other application protocols could define 165 their own Stringprep profiles. Indeed, a number of application 166 protocols defined such profiles. 168 After the publication of [RFC3454] in 2002, several significant 169 issues arose with the use of Stringprep in the IDN case, as 170 documented in the IAB's recommendations regarding IDNs [RFC4690] 171 (most significantly, Stringprep was tied to Unicode version 3.2). 172 Therefore, the newer IDNA specifications, here called "IDNA2008" 173 ([RFC5890], [RFC5891], [RFC5892], [RFC5893], [RFC5894]), no longer 174 use Stringprep and Nameprep. This migration away from Stringprep for 175 IDNs prompted other "customers" of Stringprep to consider new 176 approaches to the preparation, enforcement, and comparison of 177 internationalized strings, as described in [RFC6885]. 179 This document defines a framework for a post-Stringprep approach to 180 the preparation, enforcement, and comparison of internationalized 181 strings in application protocols, based on several principles: 183 1. Define a small set of string classes that specify the Unicode 184 code points appropriate for common application protocol 185 constructs (where possible, maintaining compatibility with 186 IDNA2008 to help ensure a more consistent user experience). 188 2. Define each PRECIS string class in terms of Unicode code points 189 and their properties so that an algorithm can be used to 190 determine whether each code point or character category is 191 (a) valid, (b) allowed in certain contexts, (c) disallowed, or 192 (d) unassigned. 194 3. Use an "inclusion model" such that a string class consists only 195 of code points that are explicitly allowed, with the result that 196 any code point not explicitly allowed is forbidden. 198 4. Enable application protocols to define profiles of the PRECIS 199 string classes if necessary (addressing matters such as width 200 mapping, case mapping, Unicode normalization, and directionality) 201 but strongly discourage the multiplication of profiles beyond 202 necessity in order to avoid violations of the "Principle of Least 203 Astonishment". 205 It is expected that this framework will yield the following benefits: 207 o Application protocols will be more agile with regard to Unicode 208 versions (recognizing that complete agility cannot be realized in 209 practice). 211 o Implementers will be able to share code point tables and software 212 code across application protocols, most likely by means of 213 software libraries. 215 o End users will be able to acquire more accurate expectations about 216 the code points that are acceptable in various contexts. Given 217 this more uniform set of string classes, it is also expected that 218 copy/paste operations between software implementing different 219 application protocols will be more predictable and coherent. 221 Whereas the string classes define the "baseline" code points for a 222 range of applications, profiling enables application protocols to 223 apply the string classes in ways that are appropriate for common 224 constructs such as usernames [I-D.ietf-precis-7613bis], opaque 225 strings such as passwords [I-D.ietf-precis-7613bis], and nicknames 226 [I-D.ietf-precis-7700bis]. Profiles are responsible for defining the 227 handling of right-to-left code points as well as various mapping 228 operations of the kind also discussed for IDNs in [RFC5895], such as 229 case preservation or lowercasing, Unicode normalization, mapping of 230 certain code points to other code points or to nothing, and mapping 231 of fullwidth and halfwidth code points. 233 When an application applies a profile of a PRECIS string class, it 234 transforms an input string (which might or might not be conforming) 235 into an output string that definitively conforms to the profile. In 236 particular, this document focuses on the resulting ability to achieve 237 the following objectives: 239 a. Enforcing all the rules of a profile for a single output string 240 (e.g., to determine if a string can be included in a protocol 241 slot, communicated to another entity within a protocol, stored in 242 a retrieval system, etc.) to check whether the output string 243 conforms to the rules of the profile. 245 b. Comparing two output strings to determine if they are equivalent, 246 typically through octet-for-octet matching to test for 247 "bit-string identity" (e.g., to make an access decision for 248 purposes of authentication or authorization as further described 249 in [RFC6943]). 251 The opportunity to define profiles naturally introduces the 252 possibility of a proliferation of profiles, thus potentially 253 mitigating the benefits of common code and violating user 254 expectations. See Section 5 for a discussion of this important 255 topic. 257 In addition, it is extremely important for protocol designers and 258 application developers to understand that the transformation of an 259 input string to an output string is rarely reversible. As one 260 relatively simple example, case mapping would transform an input 261 string of "StPeter" to "stpeter", and information about the 262 capitalization of the first and third characters would be lost. 263 Similar considerations apply to other forms of mapping and 264 normalization. 266 Although this framework is similar to IDNA2008 and includes by 267 reference some of the character categories defined in [RFC5892], it 268 defines additional character categories to meet the needs of common 269 application protocols other than DNS. 271 The character categories and calculation rules defined under 272 Sections 8 and 9 are normative and apply to all Unicode code points. 273 The code point table that results from applying the character 274 categories and calculation rules to the latest version of Unicode can 275 be found in an IANA registry. 277 2. Terminology 279 Many important terms used in this document are defined in [RFC5890], 280 [RFC6365], [RFC6885], and [Unicode]. The terms "left-to-right" (LTR) 281 and "right-to-left" (RTL) are defined in Unicode Standard Annex #9 282 [UAX9]. 284 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 285 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 286 "OPTIONAL" in this document are to be interpreted as described in 287 [RFC2119]. 289 3. Preparation, Enforcement, and Comparison 291 This document distinguishes between three different actions that an 292 entity can take with regard to a string: 294 o Enforcement entails applying all of the rules specified for a 295 particular string class or profile thereof to a single input 296 string, for the purpose of checking whether the string conforms to 297 all of the rules and thus determining if the string can be used in 298 a given protocol slot. 300 o Comparison entails applying all of the rules specified for a 301 particular string class or profile thereof to two separate input 302 strings, for the purpose of determining if the two strings are 303 equivalent. 305 o Preparation primarily entails ensuring that the code points in a 306 single input string are allowed by the underlying PRECIS string 307 class, and sometimes also entails applying one or more of the 308 rules specified for a particular string class or profile thereof. 309 Preparation can be appropriate for constrained devices that can to 310 some extent restrict the code points in a string to a limited 311 repertoire of characters but that do not have the processing power 312 or onboard memory to perform operations such as Unicode 313 normalization. However, preparation does not ensure that an input 314 string conforms to all of the rules for a string class or profile 315 thereof. 317 Note: The term "preparation" as used in this specification and 318 related documents has a much more limited scope than it did in 319 Stringprep; it essentially refers to a kind of preprocessing of 320 an input string, not the actual operations that apply 321 internationalization rules to produce an output string (here 322 termed "enforcement") or to compare two output strings (here 323 termed "comparison"). 325 In most cases, authoritative entities such as servers are responsible 326 for enforcement, whereas subsidiary entities such as clients are 327 responsible only for preparation. The rationale for this distinction 328 is that clients might not have the facilities (in terms of device 329 memory and processing power) to enforce all the rules regarding 330 internationalized strings (such as width mapping and Unicode 331 normalization), although they can more easily limit the repertoire of 332 characters they offer to an end user. By contrast, it is assumed 333 that a server would have more capacity to enforce the rules, and in 334 any case acts as an authority regarding allowable strings in protocol 335 slots such as addresses and endpoint identifiers. In addition, a 336 client cannot necessarily be trusted to properly generate such 337 strings, especially for security-sensitive contexts such as 338 authentication and authorization. 340 4. String Classes 342 4.1. Overview 344 Starting in 2010, various "customers" of Stringprep began to discuss 345 the need to define a post-Stringprep approach to the preparation and 346 comparison of internationalized strings other than IDNs. This 347 community analyzed the existing Stringprep profiles and also weighed 348 the costs and benefits of defining a relatively small set of Unicode 349 code points that would minimize the potential for user confusion 350 caused by visually similar code points (and thus be relatively 351 "safe") vs. defining a much larger set of Unicode code points that 352 would maximize the potential for user creativity (and thus be 353 relatively "expressive"). As a result, the community concluded that 354 most existing uses could be addressed by two string classes: 356 IdentifierClass: a sequence of letters, numbers, and some symbols 357 that is used to identify or address a network entity such as a 358 user account, a venue (e.g., a chatroom), an information source 359 (e.g., a data feed), or a collection of data (e.g., a file); the 360 intent is that this class will minimize user confusion in a wide 361 variety of application protocols, with the result that safety has 362 been prioritized over expressiveness for this class. 364 FreeformClass: a sequence of letters, numbers, symbols, spaces, and 365 other code points that is used for free-form strings, including 366 passwords as well as display elements such as human-friendly 367 nicknames for devices or for participants in a chatroom; the 368 intent is that this class will allow nearly any Unicode code 369 point, with the result that expressiveness has been prioritized 370 over safety for this class. Note well that protocol designers, 371 application developers, service providers, and end users might not 372 understand or be able to enter all of the code points that can be 373 included in the FreeformClass -- see Section 12.3 for details. 375 Future specifications might define additional PRECIS string classes, 376 such as a class that falls somewhere between the IdentifierClass and 377 the FreeformClass. At this time, it is not clear how useful such a 378 class would be. In any case, because application developers are able 379 to define profiles of PRECIS string classes, a protocol needing a 380 construct between the IdentifierClass and the FreeformClass could 381 define a restricted profile of the FreeformClass if needed. 383 The following subsections discuss the IdentifierClass and 384 FreeformClass in more detail, with reference to the dimensions 385 described in Section 5 of [RFC6885]. Each string class is defined by 386 the following behavioral rules: 388 Valid: Defines which code points are treated as valid for the 389 string. 391 Contextual Rule Required: Defines which code points are treated as 392 allowed only if the requirements of a contextual rule are met 393 (i.e., either CONTEXTJ or CONTEXTO as originally defined in the 394 IDNA2008 specifications). 396 Disallowed: Defines which code points need to be excluded from the 397 string. 399 Unassigned: Defines application behavior in the presence of code 400 points that are unknown (i.e., not yet designated) for the version 401 of Unicode used by the application. 403 This document defines the valid, contextual rule required, 404 disallowed, and unassigned rules for the IdentifierClass and 405 FreeformClass. As described under Section 5, profiles of these 406 string classes are responsible for defining the width mapping, 407 additional mappings, case mapping, normalization, and directionality 408 rules. 410 4.2. IdentifierClass 412 Most application technologies need strings that can be used to refer 413 to, include, or communicate protocol strings like usernames, 414 filenames, data feed identifiers, and chatroom names. We group such 415 strings into a class called "IdentifierClass" having the following 416 features. 418 4.2.1. Valid 420 o Code points traditionally used as letters and numbers in writing 421 systems, i.e., the LetterDigits ("A") category first defined in 422 [RFC5892] and listed here under Section 9.1. 424 o Code points in the range U+0021 through U+007E, i.e., the 425 (printable) ASCII7 ("K") category defined under Section 9.11. 426 These code points are "grandfathered" into PRECIS and thus are 427 valid even if they would otherwise be disallowed according to the 428 property-based rules specified in the next section. 430 Note: Although the PRECIS IdentifierClass reuses the LetterDigits 431 category from IDNA2008, the range of code points allowed in the 432 IdentifierClass is wider than the range of code points allowed in 433 IDNA2008. The main reason is that IDNA2008 applies the Unstable 434 category before the LetterDigits category, thus disallowing 435 uppercase code points, whereas the IdentifierClass does not apply 436 the Unstable category. 438 4.2.2. Contextual Rule Required 440 o A number of code points from the Exceptions ("F") category defined 441 under Section 9.6 (see Section 9.6 for a full list). 443 o Joining code points, i.e., the JoinControl ("H") category defined 444 under Section 9.8. 446 4.2.3. Disallowed 448 o Old Hangul Jamo code points, i.e., the OldHangulJamo ("I") 449 category defined under Section 9.9. 451 o Control code points, i.e., the Controls ("L") category defined 452 under Section 9.12. 454 o Ignorable code points, i.e., the PrecisIgnorableProperties ("M") 455 category defined under Section 9.13. 457 o Space code points, i.e., the Spaces ("N") category defined under 458 Section 9.14. 460 o Symbol code points, i.e., the Symbols ("O") category defined under 461 Section 9.15. 463 o Punctuation code points, i.e., the Punctuation ("P") category 464 defined under Section 9.16. 466 o Any code point that is decomposed and recomposed into something 467 other than itself under Unicode normalization form KC, i.e., the 468 HasCompat ("Q") category defined under Section 9.17. These code 469 points are disallowed even if they would otherwise be valid 470 according to the property-based rules specified in the previous 471 section. 473 o Letters and digits other than the "traditional" letters and digits 474 allowed in IDNs, i.e., the OtherLetterDigits ("R") category 475 defined under Section 9.18. 477 4.2.4. Unassigned 479 Any code points that are not yet designated in the Unicode coded 480 character set are considered unassigned for purposes of the 481 IdentifierClass, and such code points are to be treated as 482 disallowed. See Section 9.10. 484 4.2.5. Examples 486 As described in the Introduction to this document, the string classes 487 do not handle all issues related to string preparation and comparison 488 (such as case mapping); instead, such issues are handled at the level 489 of profiles. Examples for profiles of the IdentifierClass can be 490 found in [I-D.ietf-precis-7613bis] (the UsernameCaseMapped and 491 UsernameCasePreserved profiles). 493 4.3. FreeformClass 495 Some application technologies need strings that can be used in a 496 free-form way, e.g., as a password in an authentication exchange (see 497 [I-D.ietf-precis-7613bis]) or a nickname in a chatroom (see 498 [I-D.ietf-precis-7700bis]). We group such things into a class called 499 "FreeformClass" having the following features. 501 Security Warning: As mentioned, the FreeformClass prioritizes 502 expressiveness over safety; Section 12.3 describes some of the 503 security hazards involved with using or profiling the 504 FreeformClass. 506 Security Warning: Consult Section 12.6 for relevant security 507 considerations when strings conforming to the FreeformClass, or a 508 profile thereof, are used as passwords. 510 4.3.1. Valid 512 o Traditional letters and numbers, i.e., the LetterDigits ("A") 513 category first defined in [RFC5892] and listed here under 514 Section 9.1. 516 o Letters and digits other than the "traditional" letters and digits 517 allowed in IDNs, i.e., the OtherLetterDigits ("R") category 518 defined under Section 9.18. 520 o Code points in the range U+0021 through U+007E, i.e., the 521 (printable) ASCII7 ("K") category defined under Section 9.11. 523 o Any code point that is decomposed and recomposed into something 524 other than itself under Unicode normalization form KC, i.e., the 525 HasCompat ("Q") category defined under Section 9.17. 527 o Space code points, i.e., the Spaces ("N") category defined under 528 Section 9.14. 530 o Symbol code points, i.e., the Symbols ("O") category defined under 531 Section 9.15. 533 o Punctuation code points, i.e., the Punctuation ("P") category 534 defined under Section 9.16. 536 4.3.2. Contextual Rule Required 538 o A number of code points from the Exceptions ("F") category defined 539 under Section 9.6 (see Section 9.6 for a full list). 541 o Joining code points, i.e., the JoinControl ("H") category defined 542 under Section 9.8. 544 4.3.3. Disallowed 546 o Old Hangul Jamo code points, i.e., the OldHangulJamo ("I") 547 category defined under Section 9.9. 549 o Control code points, i.e., the Controls ("L") category defined 550 under Section 9.12. 552 o Ignorable code points, i.e., the PrecisIgnorableProperties ("M") 553 category defined under Section 9.13. 555 4.3.4. Unassigned 557 Any code points that are not yet designated in the Unicode coded 558 character set are considered unassigned for purposes of the 559 FreeformClass, and such code points are to be treated as disallowed. 561 4.3.5. Examples 563 As described in the Introduction to this document, the string classes 564 do not handle all issues related to string preparation and comparison 565 (such as case mapping); instead, such issues are handled at the level 566 of profiles. Examples for profiles of the FreeformClass can be found 567 in [I-D.ietf-precis-7613bis] (the OpaqueString profile) and 568 [I-D.ietf-precis-7700bis] (the Nickname profile). 570 4.4. Summary 572 The following table summarizes the differences between the 573 IdentifierClass and the FreeformClass, i.e., the disposition of a 574 code point as valid, contextual rule required, disallowed, or 575 unassigned) depending on its PRECIS category. 577 +===============================+=================+===============+ 578 | CATEGORY | IDENTIFIERCLASS | FREEFORMCLASS | 579 +===============================+=================+===============+ 580 | (A) LetterDigits | Valid | Valid | 581 +-------------------------------+-----------------+---------------+ 582 | (B) Unstable | [N/A (unused)] | 583 +-------------------------------+-----------------+---------------+ 584 | (C) IgnorableProperties | [N/A (unused)] | 585 +-------------------------------+-----------------+---------------+ 586 | (D) IgnorableBlocks | [N/A (unused)] | 587 +-------------------------------+-----------------+---------------+ 588 | (E) LDH | [N/A (unused)] | 589 +-------------------------------+-----------------+---------------+ 590 | (F) Exceptions | Contextual | Contextual | 591 | | Rule Required | Rule Required | 592 +-------------------------------+-----------------+---------------+ 593 | (G) BackwardCompatible | [Handled by IDNA Rules] | 594 +-------------------------------+-----------------+---------------+ 595 | (H) JoinControl | Contextual | Contextual | 596 | | Rule Required | Required | 597 +-------------------------------+-----------------+---------------+ 598 | (I) OldHangulJamo | Disallowed | Disallowed | 599 +-------------------------------+-----------------+---------------+ 600 | (J) Unassigned | Unassigned | Unassigned | 601 +-------------------------------+-----------------+---------------+ 602 | (K) ASCII7 | Valid | Valid | 603 +-------------------------------+-----------------+---------------+ 604 | (L) Controls | Disallowed | Disallowed | 605 +-------------------------------+-----------------+---------------+ 606 | (M) PrecisIgnorableProperties | Disallowed | Disallowed | 607 +-------------------------------+-----------------+---------------+ 608 | (N) Spaces | Disallowed | Valid | 609 +-------------------------------+-----------------+---------------+ 610 | (O) Symbols | Disallowed | Valid | 611 +-------------------------------+-----------------+---------------+ 612 | (P) Punctuation | Disallowed | Valid | 613 +-------------------------------+-----------------+---------------+ 614 | (Q) HasCompat | Disallowed | Valid | 615 +-------------------------------+-----------------+---------------+ 616 | (R) OtherLetterDigits | Disallowed | Valid | 617 +-------------------------------+-----------------+---------------+ 619 Table 1: Comparative Disposition of Code Points 621 5. Profiles 623 This framework document defines the valid, contextual-rule-required, 624 disallowed, and unassigned rules for the IdentifierClass and the 625 FreeformClass. A profile of a PRECIS string class MUST define the 626 width mapping, additional mappings (if any), case mapping, 627 normalization, and directionality rules. A profile MAY also restrict 628 the allowable code points above and beyond the definition of the 629 relevant PRECIS string class (but MUST NOT add as valid any code 630 points that are disallowed by the relevant PRECIS string class). 631 These matters are discussed in the following subsections. 633 Profiles of the PRECIS string classes are registered with the IANA as 634 described under Section 11.3. Profile names use the following 635 convention: they are of the form "Profilename of BaseClass", where 636 the "Profilename" string is a differentiator and "BaseClass" is the 637 name of the PRECIS string class being profiled; for example, the 638 profile of the FreeformClass used for opaque strings such as 639 passwords is the OpaqueString profile [I-D.ietf-precis-7613bis]. 641 5.1. Profiles Must Not Be Multiplied beyond Necessity 643 The risk of profile proliferation is significant because having too 644 many profiles will result in different behavior across various 645 applications, thus violating what is known in user interface design 646 as the "Principle of Least Astonishment". 648 Indeed, we already have too many profiles. Ideally we would have at 649 most two or three profiles. Unfortunately, numerous application 650 protocols exist with their own quirks regarding protocol strings. 651 Domain names, email addresses, instant messaging addresses, chatroom 652 nicknames, filenames, authentication identifiers, passwords, and 653 other strings are already out there in the wild and need to be 654 supported in existing application protocols such as DNS, SMTP, the 655 Extensible Messaging and Presence Protocol (XMPP), Internet Relay 656 Chat (IRC), NFS, the Internet Small Computer System Interface 657 (iSCSI), the Extensible Authentication Protocol (EAP), and the Simple 658 Authentication and Security Layer (SASL), among others. 660 Nevertheless, profiles must not be multiplied beyond necessity. 662 To help prevent profile proliferation, this document recommends 663 sensible defaults for the various options offered to profile creators 664 (such as width mapping and Unicode normalization). In addition, the 665 guidelines for designated experts provided under Section 10 are meant 666 to encourage a high level of due diligence regarding new profiles. 668 5.2. Rules 670 5.2.1. Width Mapping Rule 672 The width mapping rule of a profile specifies whether width mapping 673 is performed on a string, and how the mapping is done. Typically, 674 such mapping consists of mapping fullwidth and halfwidth code points, 675 i.e., code points with a Decomposition Type of Wide or Narrow, to 676 their decomposition mappings; as an example, FULLWIDTH DIGIT ZERO 677 (U+FF10) would be mapped to DIGIT ZERO (U+0030). 679 The normalization form specified by a profile (see below) has an 680 impact on the need for width mapping. Because width mapping is 681 performed as a part of compatibility decomposition, a profile 682 employing either normalization form KD (NFKD) or normalization form 683 KC (NFKC) does not need to specify width mapping. However, if 684 Unicode normalization form C (NFC) is used (as is recommended) then 685 the profile needs to specify whether to apply width mapping; in this 686 case, width mapping is in general RECOMMENDED because allowing 687 fullwidth and halfwidth code points to remain unmapped to their 688 compatibility variants would violate the "Principle of Least 689 Astonishment". For more information about the concept of width in 690 East Asian scripts within Unicode, see Unicode Standard Annex #11 691 [UAX11]. 693 Note: Because the East Asian width property is not guaranteed to 694 be stable by the Unicode Standard (see 695 for details), 696 the results of applying a given width mapping rule might not be 697 consistent across different versions of Unicode. 699 5.2.2. Additional Mapping Rule 701 The additional mapping rule of a profile specifies whether additional 702 mappings are performed on a string, such as: 704 Mapping of delimiter code points (such as '@', ':', '/', '+', and 705 '-') 707 Mapping of special code points (e.g., non-ASCII space code points 708 to ASCII space or control code points to nothing). 710 The PRECIS mappings document [RFC7790] describes such mappings in 711 more detail. 713 5.2.3. Case Mapping Rule 715 The case mapping rule of a profile specifies whether case mapping 716 (instead of case preservation) is performed on a string, and how the 717 mapping is applied (e.g., mapping uppercase and titlecase code points 718 to their lowercase equivalents). 720 If case mapping is desired (instead of case preservation), it is 721 RECOMMENDED to use the Unicode toLowerCase() operation defined in the 722 Unicode Standard [Unicode]. In contrast to the Unicode toCaseFold() 723 operation, the toLowerCase() operation is less likely to violate the 724 "Principle of Least Astonishment", especially when an application 725 merely wishes to convert uppercase and titlecase code points to the 726 lowercase equivalents while preserving lowercase code points. 727 Although the toCaseFold() operation can be appropriate when an 728 application needs to compare two strings (such as in search 729 operations), in general few application developers and even fewer 730 users understand its implications, so toLowerCase() is almost always 731 the safer choice. 733 Note: Neither toLowerCase() nor toCaseFold() is designed to handle 734 various language-specific issues (such as so-called "dotless i" in 735 several Turkic languages). The reader is referred to the PRECIS 736 mappings document [RFC7790], which describes these issues in 737 greater detail. 739 In order to maximize entropy and minimize the potential for false 740 accepts, it is NOT RECOMMENDED for application protocols to map 741 uppercase and titlecase code points to their lowercase equivalents 742 when strings conforming to the FreeformClass, or a profile thereof, 743 are used in passwords; instead, it is RECOMMENDED to preserve the 744 case of all code points contained in such strings and then perform 745 case-sensitive comparison. See also the related discussion in 746 Section 12.6 and in [I-D.ietf-precis-7613bis]. 748 5.2.4. Normalization Rule 750 The normalization rule of a profile specifies which Unicode 751 normalization form (D, KD, C, or KC) is to be applied (see Unicode 752 Standard Annex #15 [UAX15] for background information). 754 In accordance with [RFC5198], normalization form C (NFC) is 755 RECOMMENDED. 757 Protocol designers and application developers need to understand that 758 use certain Unicode normalization forms, especially NFKC and NFKD, 759 can result in significant loss of information in various 760 circumstances, and that these circumstances can vary depending on the 761 language and script of the strings to which the normalization forms 762 are applied. Extreme care should be taken when specifying the use of 763 these normalization forms. 765 5.2.5. Directionality Rule 767 The directionality rule of a profile specifies how to treat strings 768 containing what are often called "right-to-left" (RTL) code points 769 (see Unicode Standard Annex #9 [UAX9]). RTL code points come from 770 scripts that are normally written from right to left and are 771 considered by Unicode to, themselves, have right-to-left 772 directionality. Some strings containing RTL code points also contain 773 "left-to-right" (LTR) code points, such as ASCII numerals, as well as 774 code points without directional properties. Consequently, such 775 strings are known as "bidirectional strings". 777 Presenting bidirectional strings in different layout systems (e.g., a 778 user interface that is configured to handle primarily an RTL script 779 vs. an interface that is configured to handle primarily an LTR 780 script) can yield display results that, while predictable to those 781 who understand the display rules, are counter-intuitive to casual 782 users. In particular, the same bidirectional string (in PRECIS 783 terms) might not be presented in the same way to users of those 784 different layout systems, even though the presentation is consistent 785 within any particular layout system. In some applications, these 786 presentation differences might be considered problematic and thus the 787 application designers might wish to restrict the use of bidirectional 788 strings by specifying a directionality rule. In other applications, 789 these presentation differences might not be considered problematic 790 (this especially tends to be true of more "free-form" strings) and 791 thus no directionality rule is needed. 793 The PRECIS framework does not directly address how to deal with 794 bidirectional strings across all string classes and profiles, and 795 does not define any new directionality rules, because at present 796 there is no widely accepted and implemented solution for the safe 797 display of arbitrary bidirectional strings beyond the Unicode 798 bidirectional algorithm [UAX9]. Although rules for management and 799 display of bidirectional strings have been defined for domain name 800 labels and similar identifiers through the "Bidi Rule" specified in 801 the IDNA2008 specification on right-to-left scripts [RFC5893], those 802 rules are quite restrictive and are not necessarily applicable to all 803 bidirectional strings. 805 The authors of a PRECIS profile might believe that they need to 806 define a new directionality rule of their own. Because of the 807 complexity of the issues involved, such a belief is almost always 808 misguided, even if the authors have done a great deal of careful 809 research into the challenges of displaying bidirectional strings. 810 This document strongly suggests that profile authors who are thinking 811 about defining a new directionality rule think again, and instead 812 consider using the "Bidi Rule" [RFC5893] (for profiles based on the 813 IdentifierClass) or following the Unicode bidirectional algorithm 814 [UAX9] (for profiles based on the FreeformClass or in situations 815 where the IdentifierClass is not appropriate). 817 5.3. A Note about Spaces 819 With regard to the IdentifierClass, the consensus of the PRECIS 820 Working Group was that spaces are problematic for many reasons, 821 including the following: 823 o Many Unicode code points are confusable with ASCII space. 825 o Even if non-ASCII space code points are mapped to ASCII space 826 (U+0020), space code points are often not rendered in user 827 interfaces, leading to the possibility that a human user might 828 consider a string containing spaces to be equivalent to the same 829 string without spaces. 831 o In some locales, some devices are known to generate a code point 832 other than ASCII space (such as ZERO WIDTH JOINER, U+200D) when a 833 user performs an action like hitting the space bar on a keyboard. 835 One consequence of disallowing space code points in the 836 IdentifierClass might be to effectively discourage their use within 837 identifiers created in newer application protocols; given the 838 challenges involved with properly handling space code points 839 (especially non-ASCII space code points) in identifiers and other 840 protocol strings, the PRECIS Working Group considered this to be a 841 feature, not a bug. 843 However, the FreeformClass does allow spaces, which enables 844 application protocols to define profiles of the FreeformClass that 845 are more flexible than any profiles of the IdentifierClass. In 846 addition, as explained in Section 6.3, application protocols can also 847 define application-layer constructs containing spaces. 849 6. Applications 851 6.1. How to Use PRECIS in Applications 853 Although PRECIS has been designed with applications in mind, 854 internationalization is not suddenly made easy through the use of 855 PRECIS. Indeed, because it is extremely difficult for protocol 856 designers and application developers to do the right thing for all 857 users when supporting internationalized strings, often the safest 858 option is to support only the ASCII range [RFC20] in various protocol 859 slots. This state of affairs is unfortunate but is the direct result 860 of the complexities involved with human languages (e.g., the vast 861 number of code points, scripts, user communities, and rules with 862 their inevitable exceptions), which kinds of strings application 863 developers and their users wish to support, the wide range of devices 864 that users employ to access services enabled by various Internet 865 protocols, and so on. 867 Despite these significant challenges, application and protocol 868 developers sometimes persevere in attempting to support 869 internationalized strings in their systems. These developers need to 870 think carefully about how they will use the PRECIS string classes, or 871 profiles thereof, in their applications. This section provides some 872 guidelines to application developers (and to expert reviewers of 873 application protocol specifications). 875 o Don't define your own profile unless absolutely necessary (see 876 Section 5.1). Existing profiles have been designed for wide 877 reuse. It is highly likely that an existing profile will meet 878 your needs, especially given the ability to specify further 879 excluded code points (Section 6.2) and to build application-layer 880 constructs (see Section 6.3). 882 o Do specify: 884 * Exactly which entities are responsible for preparation, 885 enforcement, and comparison of internationalized strings (e.g., 886 servers or clients). 888 * Exactly when those entities need to complete their tasks (e.g., 889 a server might need to enforce the rules of a profile before 890 allowing a client to gain network access). 892 * Exactly which protocol slots need to be checked against which 893 profiles (e.g., checking the address of a message's intended 894 recipient against the UsernameCaseMapped profile 895 [I-D.ietf-precis-7613bis] of the IdentifierClass, or checking 896 the password of a user against the OpaqueString profile 897 [I-D.ietf-precis-7613bis] of the FreeformClass). 899 See [I-D.ietf-precis-7613bis] and [RFC7622] for definitions of 900 these matters for several applications. 902 6.2. Further Excluded Characters 904 An application protocol that uses a profile MAY specify particular 905 code points that are not allowed in relevant slots within that 906 application protocol, above and beyond those excluded by the string 907 class or profile. 909 That is, an application protocol MAY do either of the following: 911 1. Exclude specific code points that are allowed by the relevant 912 string class. 914 2. Exclude code points matching certain Unicode properties (e.g., 915 math symbols) that are included in the relevant PRECIS string 916 class. 918 As a result of such exclusions, code points that are defined as valid 919 for the PRECIS string class or profile will be defined as disallowed 920 for the relevant protocol slot. 922 Typically, such exclusions are defined for the purpose of backward 923 compatibility with legacy formats within an application protocol. 924 These are defined for application protocols, not profiles, in order 925 to prevent multiplication of profiles beyond necessity (see 926 Section 5.1). 928 6.3. Building Application-Layer Constructs 930 Sometimes, an application-layer construct does not map in a 931 straightforward manner to one of the base string classes or a profile 932 thereof. Consider, for example, the "simple user name" construct in 933 the Simple Authentication and Security Layer (SASL) [RFC4422]. 934 Depending on the deployment, a simple user name might take the form 935 of a user's full name (e.g., the user's personal name followed by a 936 space and then the user's family name). Such a simple user name 937 cannot be defined as an instance of the IdentifierClass or a profile 938 thereof, because space code points are not allowed in the 939 IdentifierClass; however, it could be defined using a space-separated 940 sequence of IdentifierClass instances, as in the following ABNF 941 [RFC5234] from [I-D.ietf-precis-7613bis]: 943 username = userpart *(1*SP userpart) 944 userpart = 1*(idpoint) 945 ; 946 ; an "idpoint" is a Unicode code point that 947 ; can be contained in a string conforming to 948 ; the PRECIS IdentifierClass 949 ; 951 Similar techniques could be used to define many application-layer 952 constructs, say of the form "user@domain" or "/path/to/file". 954 7. Order of Operations 956 To ensure proper comparison, the rules specified for a particular 957 string class or profile MUST be applied in the following order: 959 1. Width Mapping Rule 961 2. Additional Mapping Rule 963 3. Case Mapping Rule 965 4. Normalization Rule 967 5. Directionality Rule 969 6. Behavioral rules for determining whether a code point is valid, 970 allowed under a contextual rule, disallowed, or unassigned 972 As already described, the width mapping, additional mapping, case 973 mapping, normalization, and directionality rules are specified for 974 each profile, whereas the behavioral rules are specified for each 975 string class. Some of the logic behind this order is provided under 976 Section 5.2.1 (see also the PRECIS mappings document [RFC7790]). In 977 addition, this order is consistent with IDNA2008, and with both 978 IDNA2003 and Stringprep before then, for the purpose of enabling code 979 reuse and of ensuring as much continuity as possible with the 980 Stringprep profiles that are obsoleted by several PRECIS profiles. 982 Because of the order of operations specified here, applying the rules 983 for any given PRECIS profile is not necessarily an idempotent 984 procedure (e.g., under certain circumstances, such as when Unicode 985 normalization form KC is used, performing Unicode normalization after 986 case mapping can still yield uppercase characters for certain code 987 points). Therefore, an implementation SHOULD apply the rules 988 repeatedly until the output string is stable; if the output string 989 does not stabilize after reapplying the rules three (3) additional 990 times, the implementation SHOULD terminate application of the rules 991 and reject the input string as invalid. 993 8. Code Point Properties 995 In order to implement the string classes described above, this 996 document does the following: 998 1. Reviews and classifies the collections of code points in the 999 Unicode coded character set by examining various code point 1000 properties. 1002 2. Defines an algorithm for determining a derived property value, 1003 which can vary depending on the string class being used by the 1004 relevant application protocol. 1006 This document is not intended to specify precisely how derived 1007 property values are to be applied in protocol strings. That 1008 information is the responsibility of the protocol specification that 1009 uses or profiles a PRECIS string class from this document. The value 1010 of the property is to be interpreted as follows. 1012 PROTOCOL VALID Those code points that are allowed to be used in any 1013 PRECIS string class (currently, IdentifierClass and 1014 FreeformClass). The abbreviated term "PVALID" is used to refer to 1015 this value in the remainder of this document. 1017 SPECIFIC CLASS PROTOCOL VALID Those code points that are allowed to 1018 be used in specific string classes. In the remainder of this 1019 document, the abbreviated term *_PVAL is used, where * = (ID | 1020 FREE), i.e., either "FREE_PVAL" or "ID_PVAL". In practice, the 1021 derived property ID_PVAL is not used in this specification, 1022 because every ID_PVAL code point is PVALID. 1024 CONTEXTUAL RULE REQUIRED Some characteristics of the code point, 1025 such as its being invisible in certain contexts or problematic in 1026 others, require that it not be used in a string unless specific 1027 other code points or properties are present in the string. As in 1028 IDNA2008, there are two subdivisions of CONTEXTUAL RULE REQUIRED 1029 -- the first for Join_controls (called "CONTEXTJ") and the second 1030 for other code points (called "CONTEXTO"). A string MUST NOT 1031 contain any characters whose validity is context-dependent, unless 1032 the validity is positively confirmed by a contextual rule. To 1033 check this, each code point identified as CONTEXTJ or CONTEXTO in 1034 the PRECIS Derived Property Value registry MUST have a non-null 1035 rule. If such a code point is missing a rule, the string is 1036 invalid. If the rule exists but the result of applying the rule 1037 is negative or inconclusive, the proposed string is invalid. The 1038 most notable of the CONTEXTUAL RULE REQUIRED code points are the 1039 Join Control code points U+200D ZERO WIDTH JOINER and U+200C ZERO 1040 WIDTH NON-JOINER, which have a derived property value of CONTEXTJ. 1041 See Appendix A of [RFC5892] for more information. 1043 DISALLOWED Those code points that are not permitted in any PRECIS 1044 string class. 1046 SPECIFIC CLASS DISALLOWED Those code points that are not to be 1047 included in one of the string classes but that might be permitted 1048 in others. In the remainder of this document, the abbreviated 1049 term *_DIS is used, where * = (ID | FREE), i.e., either "FREE_DIS" 1050 or "ID_DIS". In practice, the derived property FREE_DIS is not 1051 used in this specification, because every FREE_DIS code point is 1052 DISALLOWED. 1054 UNASSIGNED Those code points that are not designated (i.e., are 1055 unassigned) in the Unicode Standard. 1057 The algorithm to calculate the value of the derived property is as 1058 follows (implementations MUST NOT modify the order of operations 1059 within this algorithm, because doing so would cause inconsistent 1060 results across implementations): 1062 If .cp. .in. Exceptions Then Exceptions(cp); 1063 Else If .cp. .in. BackwardCompatible Then BackwardCompatible(cp); 1064 Else If .cp. .in. Unassigned Then UNASSIGNED; 1065 Else If .cp. .in. ASCII7 Then PVALID; 1066 Else If .cp. .in. JoinControl Then CONTEXTJ; 1067 Else If .cp. .in. OldHangulJamo Then DISALLOWED; 1068 Else If .cp. .in. PrecisIgnorableProperties Then DISALLOWED; 1069 Else If .cp. .in. Controls Then DISALLOWED; 1070 Else If .cp. .in. HasCompat Then ID_DIS or FREE_PVAL; 1071 Else If .cp. .in. LetterDigits Then PVALID; 1072 Else If .cp. .in. OtherLetterDigits Then ID_DIS or FREE_PVAL; 1073 Else If .cp. .in. Spaces Then ID_DIS or FREE_PVAL; 1074 Else If .cp. .in. Symbols Then ID_DIS or FREE_PVAL; 1075 Else If .cp. .in. Punctuation Then ID_DIS or FREE_PVAL; 1076 Else DISALLOWED; 1078 The value of the derived property calculated can depend on the string 1079 class; for example, if an identifier used in an application protocol 1080 is defined as profiling the PRECIS IdentifierClass then a space 1081 character such as U+0020 would be assigned to ID_DIS, whereas if an 1082 identifier is defined as profiling the PRECIS FreeformClass then the 1083 character would be assigned to FREE_PVAL. For the sake of brevity, 1084 the designation "FREE_PVAL" is used herein, instead of the longer 1085 designation "ID_DIS or FREE_PVAL". In practice, the derived 1086 properties ID_PVAL and FREE_DIS are not used in this specification, 1087 because every ID_PVAL code point is PVALID and every FREE_DIS code 1088 point is DISALLOWED. 1090 Use of the name of a rule (such as "Exceptions") implies the set of 1091 code points that the rule defines, whereas the same name as a 1092 function call (such as "Exceptions(cp)") implies the value that the 1093 code point has in the Exceptions table. 1095 The mechanisms described here allow determination of the value of the 1096 property for future versions of Unicode (including code points added 1097 after Unicode 5.2 or 7.0 depending on the category, because some 1098 categories mentioned in this document are simply pointers to IDNA2008 1099 and therefore were defined at the time of Unicode 5.2). Changes in 1100 Unicode properties that do not affect the outcome of this process 1101 therefore do not affect this framework. For example, a code point 1102 can have its Unicode General_Category value change from So to Sm, or 1103 from Lo to Ll, without affecting the algorithm results. Moreover, 1104 even if such changes were to result, the BackwardCompatible list 1105 (Section 9.7) can be adjusted to ensure the stability of the results. 1107 9. Category Definitions Used to Calculate Derived Property 1109 The derived property obtains its value based on a two-step procedure: 1111 1. Code points are placed in one or more character categories either 1112 (1) based on core properties defined by the Unicode Standard or 1113 (2) by treating the code point as an exception and addressing the 1114 code point based on its code point value. These categories are 1115 not mutually exclusive. 1117 2. Set operations are used with these categories to determine the 1118 values for a property specific to a given string class. These 1119 operations are specified under Section 8. 1121 Note: Unicode property names and property value names might have 1122 short abbreviations, such as "gc" for the General_Category 1123 property and "Ll" for the Lowercase_Letter property value of the 1124 gc property. 1126 In the following specification of character categories, the operation 1127 that returns the value of a particular Unicode code point property 1128 for a code point is designated by using the formal name of that 1129 property (from the Unicode PropertyAliases.txt file [PropertyAliases] 1130 followed by "(cp)" for "code point". For example, the value of the 1131 General_Category property for a code point is indicated by 1132 General_Category(cp). 1134 The first ten categories (A-J) shown below were previously defined 1135 for IDNA2008 and are referenced from [RFC5892] to ease the 1136 understanding of how PRECIS handles various code points. Some of 1137 these categories are reused in PRECIS, and some of them are not; 1138 however, the lettering of categories is retained to prevent overlap 1139 and to ease implementation of both IDNA2008 and PRECIS in a single 1140 software application. The next eight categories (K-R) are specific 1141 to PRECIS. 1143 9.1. LetterDigits (A) 1145 This category is defined in Section 2.1 of [RFC5892] and is included 1146 by reference for use in PRECIS. 1148 9.2. Unstable (B) 1150 This category is defined in Section 2.2 of [RFC5892]. However, it is 1151 not used in PRECIS. 1153 9.3. IgnorableProperties (C) 1155 This category is defined in Section 2.3 of [RFC5892]. However, it is 1156 not used in PRECIS. 1158 Note: See the PrecisIgnorableProperties ("M") category below for a 1159 more inclusive category used in PRECIS identifiers. 1161 9.4. IgnorableBlocks (D) 1163 This category is defined in Section 2.4 of [RFC5892]. However, it is 1164 not used in PRECIS. 1166 9.5. LDH (E) 1168 This category is defined in Section 2.5 of [RFC5892]. However, it is 1169 not used in PRECIS. 1171 Note: See the ASCII7 ("K") category below for a more inclusive 1172 category used in PRECIS identifiers. 1174 9.6. Exceptions (F) 1176 This category is defined in Section 2.6 of [RFC5892] and is included 1177 by reference for use in PRECIS. 1179 9.7. BackwardCompatible (G) 1181 This category is defined in Section 2.7 of [RFC5892] and is included 1182 by reference for use in PRECIS. 1184 Note: Management of this category is handled via the processes 1185 specified in [RFC5892]. At the time of this writing (and also at the 1186 time that RFC 5892 was published), this category consisted of the 1187 empty set; however, that is subject to change as described in 1188 RFC 5892. 1190 9.8. JoinControl (H) 1192 This category is defined in Section 2.8 of [RFC5892] and is included 1193 by reference for use in PRECIS. 1195 Note: In particular, the code points ZERO WIDTH JOINER (U+200D) and 1196 ZERO WIDTH NON-JOINER (U+200C) are necessary to produce certain 1197 combinations of characters in certain scripts (e.g., Arabic, Persian, 1198 and Indic scripts), but if used in other contexts can have 1199 consequences that violate the principle of least user astonishment. 1200 Therefore these code points are allowed only in contexts where they 1201 are appropriate, specifically where the relevant rule (CONTEXTJ or 1202 CONTEXTO) has been defined. See [RFC5892] and [RFC5894] for further 1203 discussion. 1205 9.9. OldHangulJamo (I) 1207 This category is defined in Section 2.9 of [RFC5892] and is included 1208 by reference for use in PRECIS. 1210 Note: Exclusion of these code points results in disallowing certain 1211 archaic Korean syllables and of restricting supported Korean 1212 syllables to preformed, modern Hangul characters. 1214 9.10. Unassigned (J) 1216 This category is defined in Section 2.10 of [RFC5892] and is included 1217 by reference for use in PRECIS. 1219 9.11. ASCII7 (K) 1221 This PRECIS-specific category consists of all printable, non-space 1222 code points from the 7-bit ASCII range. By applying this category, 1223 the algorithm specified under Section 8 exempts these code points 1224 from other rules that might be applied during PRECIS processing, on 1225 the assumption that these code points are in such wide use that 1226 disallowing them would be counter-productive. 1228 K: cp is in {0021..007E} 1230 9.12. Controls (L) 1232 This PRECIS-specific category consists of all control code points. 1234 L: Control(cp) = True 1236 9.13. PrecisIgnorableProperties (M) 1238 This PRECIS-specific category is used to group code points that are 1239 discouraged from use in PRECIS string classes. 1241 M: Default_Ignorable_Code_Point(cp) = True or 1242 Noncharacter_Code_Point(cp) = True 1244 The definition for Default_Ignorable_Code_Point can be found in the 1245 DerivedCoreProperties.txt file [DerivedCoreProperties]. 1247 Note: In general, these code points are constructs such as so-called 1248 soft hypens, certain joining code points, various specialized code 1249 points for use within Unicode itself (e.g., language tags and 1250 variation selectors), and so on. Disallowing these code points in 1251 PRECIS reduces the potential for unexpected results in the use of 1252 internationalized strings. 1254 9.14. Spaces (N) 1256 This PRECIS-specific category is used to group code points that are 1257 space code points. 1259 N: General_Category(cp) is in {Zs} 1261 9.15. Symbols (O) 1263 This PRECIS-specific category is used to group code points that are 1264 symbols. 1266 O: General_Category(cp) is in {Sm, Sc, Sk, So} 1268 9.16. Punctuation (P) 1270 This PRECIS-specific category is used to group code points that are 1271 punctuation code points. 1273 P: General_Category(cp) is in {Pc, Pd, Ps, Pe, Pi, Pf, Po} 1275 9.17. HasCompat (Q) 1277 This PRECIS-specific category is used to group any code point that is 1278 decomposed and recomposed into something other than itself under 1279 Unicode normalization form KC. 1281 Q: toNFKC(cp) != cp 1282 Typically this category is true of code points that are 1283 "compatibility decomposable characters" as defined in the Unicode 1284 Standard. 1286 The toNFKC() operation returns the code point in normalization form 1287 KC. For more information, see Section 5 of Unicode Standard Annex 1288 #15 [UAX15]. 1290 9.18. OtherLetterDigits (R) 1292 This PRECIS-specific category is used to group code points that are 1293 letters and digits other than the "traditional" letters and digits 1294 grouped under the LetterDigits (A) class (see Section 9.1). 1296 R: General_Category(cp) is in {Lt, Nl, No, Me} 1298 10. Guidelines for Designated Experts 1300 Experience with internationalization in application protocols has 1301 shown that protocol designers and application developers usually do 1302 not understand the subtleties and tradeoffs involved with 1303 internationalization and that they need considerable guidance in 1304 making reasonable decisions with regard to the options before them. 1306 Therefore: 1308 o Protocol designers are strongly encouraged to question the 1309 assumption that they need to define new profiles, because existing 1310 profiles are designed for wide reuse (see Section 5 for further 1311 discussion). 1313 o Those who persist in defining new profiles are strongly encouraged 1314 to clearly explain a strong justification for doing so, and to 1315 publish a stable specification that provides all of the 1316 information described under Section 11.3. 1318 o The designated experts for profile registration requests ought to 1319 seek answers to all of the questions provided under Section 11.3 1320 and to encourage applicants to provide a stable specification 1321 documenting the profile (even though the registration policy for 1322 PRECIS profiles is Expert Review and a stable specification is not 1323 strictly required). 1325 o Developers of applications that use PRECIS are strongly encouraged 1326 to apply the guidelines provided under Section 6 and to seek out 1327 the advice of the designated experts or other knowledgeable 1328 individuals in doing so. 1330 o All parties are strongly encouraged to help prevent the 1331 multiplication of profiles beyond necessity, as described under 1332 Section 5.1, and to use PRECIS in ways that will minimize user 1333 confusion and insecure application behavior. 1335 Internationalization can be difficult and contentious; designated 1336 experts, profile registrants, and application developers are strongly 1337 encouraged to work together in a spirit of good faith and mutual 1338 understanding to achieve rough consensus on profile registration 1339 requests and the use of PRECIS in particular applications. They are 1340 also encouraged to bring additional expertise into the discussion if 1341 that would be helpful in adding perspective or otherwise resolving 1342 issues. 1344 11. IANA Considerations 1346 11.1. PRECIS Derived Property Value Registry 1348 IANA has created and now maintains the "PRECIS Derived Property 1349 Value" registry that records the derived properties for the versions 1350 of Unicode that are released after (and including) version 7.0. The 1351 derived property value is to be calculated in cooperation with a 1352 designated expert [RFC5226] according to the rules specified under 1353 Sections 8 and 9. 1355 The IESG is to be notified if backward-incompatible changes to the 1356 table of derived properties are discovered or if other problems arise 1357 during the process of creating the table of derived property values 1358 or during expert review. Changes to the rules defined under 1359 Sections 8 and 9 require IETF Review. 1361 Note: IANA is requested to not make further updates to this registry 1362 until it receives notice from the IESG that the issues described in 1363 [IAB-Statement] and Section 13.5 of this document have been settled. 1365 11.2. PRECIS Base Classes Registry 1367 IANA has created the "PRECIS Base Classes" registry. In accordance 1368 with [RFC5226], the registration policy is "RFC Required". 1370 The registration template is as follows: 1372 Base Class: [the name of the PRECIS string class] 1374 Description: [a brief description of the PRECIS string class and its 1375 intended use, e.g., "A sequence of letters, numbers, and symbols 1376 that is used to identify or address a network entity."] 1378 Specification: [the RFC number] 1380 The initial registrations are as follows: 1382 Base Class: FreeformClass. 1383 Description: A sequence of letters, numbers, symbols, spaces, and 1384 other code points that is used for free-form strings. 1385 Specification: Section 4.3 of [[this document]]. 1387 Base Class: IdentifierClass. 1388 Description: A sequence of letters, numbers, and symbols that is 1389 used to identify or address a network entity. 1390 Specification: Section 4.2 of [[this document]]. 1392 11.3. PRECIS Profiles Registry 1394 IANA has created the "PRECIS Profiles" registry to identify profiles 1395 that use the PRECIS string classes. In accordance with [RFC5226], 1396 the registration policy is "Expert Review". This policy was chosen 1397 in order to ease the burden of registration while ensuring that 1398 "customers" of PRECIS receive appropriate guidance regarding the 1399 sometimes complex and subtle internationalization issues related to 1400 profiles of PRECIS string classes. 1402 The registration template is as follows: 1404 Name: [the name of the profile] 1406 Base Class: [which PRECIS string class is being profiled] 1408 Applicability: [the specific protocol elements to which this profile 1409 applies, e.g., "Localparts in XMPP addresses."] 1411 Replaces: [the Stringprep profile that this PRECIS profile replaces, 1412 if any] 1414 Width Mapping Rule: [the behavioral rule for handling of width, 1415 e.g., "Map fullwidth and halfwidth code points to their 1416 compatibility variants."] 1418 Additional Mapping Rule: [any additional mappings that are required 1419 or recommended, e.g., "Map non-ASCII space code points to ASCII 1420 space."] 1422 Case Mapping Rule: [the behavioral rule for handling of case, e.g., 1423 "apply the Unicode toLowerCase() operation"] 1425 Normalization Rule: [which Unicode normalization form is applied, 1426 e.g., "NFC"] 1428 Directionality Rule: [the behavioral rule for handling of right-to- 1429 left code points, e.g., "The 'Bidi Rule' defined in RFC 5893 1430 applies."] 1432 Enforcement: [which entities enforce the rules, and when that 1433 enforcement occurs during protocol operations] 1435 Specification: [a pointer to relevant documentation, such as an RFC 1436 or Internet-Draft] 1438 In order to request a review, the registrant shall send a completed 1439 template to the precis@ietf.org list or its designated successor. 1441 Factors to focus on while defining profiles and reviewing profile 1442 registrations include the following: 1444 o Would an existing PRECIS string class or profile solve the 1445 problem? If not, why not? (See Section 5.1 for related 1446 considerations.) 1448 o Is the problem being addressed by this profile well defined? 1450 o Does the specification define what kinds of applications are 1451 involved and the protocol elements to which this profile applies? 1453 o Is the profile clearly defined? 1455 o Is the profile based on an appropriate dividing line between user 1456 interface (culture, context, intent, locale, device limitations, 1457 etc.) and the use of conformant strings in protocol elements? 1459 o Are the width mapping, case mapping, additional mappings, 1460 normalization, and directionality rules appropriate for the 1461 intended use? 1463 o Does the profile explain which entities enforce the rules, and 1464 when such enforcement occurs during protocol operations? 1466 o Does the profile reduce the degree to which human users could be 1467 surprised or confused by application behavior (the "Principle of 1468 Least Astonishment")? 1470 o Does the profile introduce any new security concerns such as those 1471 described under Section 12 of this document (e.g., false accepts 1472 for authentication or authorization)? 1474 12. Security Considerations 1476 12.1. General Issues 1478 If input strings that appear "the same" to users are programmatically 1479 considered to be distinct in different systems, or if input strings 1480 that appear distinct to users are programmatically considered to be 1481 "the same" in different systems, then users can be confused. Such 1482 confusion can have security implications, such as the false accepts 1483 and false rejects discussed in [RFC6943] (the terms "false positives" 1484 and "false negatives" are used in that document). One starting goal 1485 of work on the PRECIS framework was to limit the number of times that 1486 users are confused (consistent with the "Principle of Least 1487 Astonishment"). Unfortunately, this goal has been difficult to 1488 achieve given the large number of application protocols already in 1489 existence. Despite these difficulties, profiles should not be 1490 multiplied beyond necessity (see Section 5.1). In particular, 1491 application protocol designers should think long and hard before 1492 defining a new profile instead of using one that has already been 1493 defined, and if they decide to define a new profile then they should 1494 clearly explain their reasons for doing so. 1496 The security of applications that use this framework can depend in 1497 part on the proper preparation, enforcement, and comparison of 1498 internationalized strings. For example, such strings can be used to 1499 make authentication and authorization decisions, and the security of 1500 an application could be compromised if an entity providing a given 1501 string is connected to the wrong account or online resource based on 1502 different interpretations of the string (again, see [RFC6943]). 1504 Specifications of application protocols that use this framework are 1505 strongly encouraged to describe how internationalized strings are 1506 used in the protocol, including the security implications of any 1507 false accepts and false rejects that might result from various 1508 enforcement and comparison operations. For some helpful guidelines, 1509 refer to [RFC6943], [RFC5890], [UTR36], and [UTS39]. 1511 12.2. Use of the IdentifierClass 1513 Strings that conform to the IdentifierClass and any profile thereof 1514 are intended to be relatively safe for use in a broad range of 1515 applications, primarily because they include only letters, digits, 1516 and "grandfathered" non-space code points from the ASCII range; thus, 1517 they exclude spaces, code points with compatibility equivalents, and 1518 almost all symbols and punctuation marks. However, because such 1519 strings can still include so-called confusable code points (see 1520 Section 12.5), protocol designers and implementers are encouraged to 1521 pay close attention to the security considerations described 1522 elsewhere in this document. 1524 12.3. Use of the FreeformClass 1526 Strings that conform to the FreeformClass and many profiles thereof 1527 can include virtually any Unicode code point. This makes the 1528 FreeformClass quite expressive, but also problematic from the 1529 perspective of possible user confusion. Protocol designers are 1530 hereby warned that the FreeformClass contains code points they might 1531 not understand, and are encouraged to profile the IdentifierClass 1532 wherever feasible; however, if an application protocol requires more 1533 code points than are allowed by the IdentifierClass, protocol 1534 designers are encouraged to define a profile of the FreeformClass 1535 that restricts the allowable code points as tightly as possible. 1536 (The PRECIS Working Group considered the option of allowing 1537 "superclasses" as well as profiles of PRECIS string classes, but 1538 decided against allowing superclasses to reduce the likelihood of 1539 security and interoperability problems.) 1541 12.4. Local Character Set Issues 1543 When systems use local character sets other than ASCII and Unicode, 1544 this specification leaves the problem of converting between the local 1545 character set and Unicode up to the application or local system. If 1546 different applications (or different versions of one application) 1547 implement different rules for conversions among coded character sets, 1548 they could interpret the same name differently and contact different 1549 application servers or other network entities. This problem is not 1550 solved by security protocols, such as Transport Layer Security (TLS) 1551 [RFC5246] and the Simple Authentication and Security Layer (SASL) 1552 [RFC4422], that do not take local character sets into account. 1554 12.5. Visually Similar Characters 1556 Some code points are visually similar and thus can cause confusion 1557 among humans. Such characters are often called "confusable 1558 characters" or "confusables". 1560 The problem of confusable characters is not necessarily caused by the 1561 use of Unicode code points outside the ASCII range. For example, in 1562 some presentations and to some individuals the string "ju1iet" 1563 (spelled with DIGIT ONE, U+0031, as the third character) might appear 1564 to be the same as "juliet" (spelled with LATIN SMALL LETTER L, 1565 U+006C), especially on casual visual inspection. This phenomenon is 1566 sometimes called "typejacking". 1568 However, the problem is made more serious by introducing the full 1569 range of Unicode code points into protocol strings. A well-known 1570 example is confusion between CYRILLIC SMALL LETTER A, U+0430, and 1571 LATIN SMALL LETTER A, U+0061. As another example, the code points 1572 U+13DA U+13A2 U+13B5 U+13AC U+13A2 U+13AC U+13D2 from the Cherokee 1573 block look similar to the ASCII code points representing "STPETER" as 1574 they might appear when presented using a "creative" font family. 1575 Confusion among such characters is perhaps not unexpected, given that 1576 the alphabetic writing systems involved all bear a family resemblance 1577 or historical lineage. Perhaps more surprising is confusion among 1578 characters from disparate writing systems, such as LATIN CAPITAL 1579 LETTER O, U+004F; DIGIT ZERO, U+0030; LAO DIGIT ZERO, U+0ED0; NEW TAI 1580 LUE DIGIT ZERO, U+19D0; ETHIOPIC SYLLABLE PHARYNGEAL A, U+12D0; and 1581 other graphemes that have the appearance of open circles. And the 1582 reader needs to be aware that the foregoing represent merely a small 1583 sample of characters that are confusable in Unicode. 1585 In some instances of confusable characters, it is unlikely that the 1586 average human could tell the difference between the real string and 1587 the fake string. (Indeed, there is no programmatic way to 1588 distinguish with full certainty which is the fake string and which is 1589 the real string; in some contexts, the string formed of Cherokee code 1590 points might be the real string and the string formed of ASCII code 1591 points might be the fake string.) Because PRECIS-compliant strings 1592 can contain almost any properly encoded Unicode code point, it can be 1593 relatively easy to fake or mimic some strings in systems that use the 1594 PRECIS framework. The fact that some strings are easily confused 1595 introduces security vulnerabilities of the kind that have also 1596 plagued the World Wide Web, specifically the phenomenon known as 1597 phishing. 1599 Despite the fact that some specific suggestions about identification 1600 and handling of confusable characters appear in the Unicode Security 1601 Considerations [UTR36] and the Unicode Security Mechanisms [UTS39], 1602 it is also true (as noted in [RFC5890]) that "there are no 1603 comprehensive technical solutions to the problems of confusable 1604 characters." Because it is impossible to map visually similar 1605 characters without a great deal of context (such as knowing the font 1606 families used), the PRECIS framework does nothing to map similar- 1607 looking characters together, nor does it prohibit some characters 1608 because they look like others. 1610 Nevertheless, specifications for application protocols that use this 1611 framework are strongly encouraged to describe how confusable 1612 characters can be abused to compromise the security of systems that 1613 use the protocol in question, along with any protocol-specific 1614 suggestions for overcoming those threats. In particular, software 1615 implementations and service deployments that use PRECIS-based 1616 technologies are strongly encouraged to define and implement 1617 consistent policies regarding the registration, storage, and 1618 presentation of visually similar characters. The following 1619 recommendations are appropriate: 1621 1. An application service SHOULD define a policy that specifies the 1622 scripts or blocks of code points that the service will allow to 1623 be registered (e.g., in an account name) or stored (e.g., in a 1624 filename). Such a policy SHOULD be informed by the languages and 1625 scripts that are used to write registered account names; in 1626 particular, to reduce confusion, the service SHOULD forbid 1627 registration or storage of strings that contain code points from 1628 more than one script and SHOULD restrict registrations to code 1629 points drawn from a very small number of scripts (e.g., scripts 1630 that are well understood by the administrators of the service, to 1631 improve manageability). 1633 2. User-oriented application software SHOULD define a policy that 1634 specifies how internationalized strings will be presented to a 1635 human user. Because every human user of such software has a 1636 preferred language or a small set of preferred languages, the 1637 software SHOULD gather that information either explicitly from 1638 the user or implicitly via the operating system of the user's 1639 device. 1641 The challenges inherent in supporting the full range of Unicode code 1642 points have in the past led some to hope for a way to 1643 programmatically negotiate more restrictive ranges based on locale, 1644 script, or other relevant factors; to tag the locale associated with 1645 a particular string; etc. As a general-purpose internationalization 1646 technology, the PRECIS framework does not include such mechanisms. 1648 12.6. Security of Passwords 1650 Two goals of passwords are to maximize the amount of entropy and to 1651 minimize the potential for false accepts. These goals can be 1652 achieved in part by allowing a wide range of code points and by 1653 ensuring that passwords are handled in such a way that code points 1654 are not compared aggressively. Therefore, it is NOT RECOMMENDED for 1655 application protocols to profile the FreeformClass for use in 1656 passwords in a way that removes entire categories (e.g., by 1657 disallowing symbols or punctuation). Furthermore, it is NOT 1658 RECOMMENDED for application protocols to map uppercase and titlecase 1659 code points to their lowercase equivalents in such strings; instead, 1660 it is RECOMMENDED to preserve the case of all code points contained 1661 in such strings and to compare them in a case-sensitive manner. 1663 That said, software implementers need to be aware that there exist 1664 tradeoffs between entropy and usability. For example, allowing a 1665 user to establish a password containing "uncommon" code points might 1666 make it difficult for the user to access a service when using an 1667 unfamiliar or constrained input device. 1669 Some application protocols use passwords directly, whereas others 1670 reuse technologies that themselves process passwords (one example of 1671 such a technology is the Simple Authentication and Security Layer 1672 [RFC4422]). Moreover, passwords are often carried by a sequence of 1673 protocols with backend authentication systems or data storage systems 1674 such as RADIUS [RFC2865] and the Lightweight Directory Access 1675 Protocol (LDAP) [RFC4510]. Developers of application protocols are 1676 encouraged to look into reusing these profiles instead of defining 1677 new ones, so that end-user expectations about passwords are 1678 consistent no matter which application protocol is used. 1680 In protocols that provide passwords as input to a cryptographic 1681 algorithm such as a hash function, the client will need to perform 1682 proper preparation of the password before applying the algorithm, 1683 because the password is not available to the server in plaintext 1684 form. 1686 Further discussion of password handling can be found in 1687 [I-D.ietf-precis-7613bis]. 1689 13. Interoperability Considerations 1691 13.1. Coded Character Sets 1693 It is known that some existing applications and systems do not 1694 support the full Unicode coded character set, or even any characters 1695 outside the ASCII repertoire [RFC20]. If two (or more) applications 1696 or systems need to interoperate when exchanging data (e.g., for the 1697 purpose of authenticating the combination of a username and 1698 password), naturally they will need to have in common at least one 1699 coded character set and the repertoire of characters being exchanged 1700 (see [RFC6365] for definitions of these terms). Establishing such a 1701 baseline is a matter for the application or system that uses PRECIS, 1702 not for the PRECIS framework. 1704 13.2. Dependency on Unicode 1706 The only coded character set supported by PRECIS is Unicode. If an 1707 application or system does not support Unicode or uses a different 1708 coded character set [RFC6365], then the PRECIS rules cannot be 1709 applied to that application or system. 1711 13.3. Encoding 1713 Although strings that are consumed in PRECIS-based application 1714 protocols are often encoded using UTF-8 [RFC3629], the exact encoding 1715 is a matter for the application protocol that uses PRECIS, not for 1716 the PRECIS framework or for specifications that define PRECIS string 1717 classes or profiles thereof. 1719 13.4. Unicode Versions 1721 It is extremely important for protocol designers and application 1722 developers to undersatnd that various changes can occur across 1723 versions of the Unicode Standard, and such changes can result in 1724 instability of PRECIS categories. The following are merely a few 1725 examples: 1727 o As described in [RFC6452], between Unicode 5.2 (current at the 1728 time IDNA2008 was originally published) and Unicode 6.0, three 1729 code points underwent changes in their GeneralCategory, resulting 1730 in modified handling depending on which version of Unicode is 1731 available on the underlying system. 1733 o The HasCompat() categorization of a given input string could 1734 change if, for example, the string includes a precomposed 1735 character that was added in a recent version of Unicode. 1737 o The East Asian width property, which is used in many PRECIS width- 1738 mapping rules, is not guaranteed to be stable across Unicode 1739 versions. 1741 Other such differences might arise between the version of Unicode 1742 current at the time of this writing (7.0) and future versions. 1744 13.5. Potential Changes to Handling of Certain Unicode Code Points 1746 As part of the review of Unicode 7.0 for IDNA, a question was raised 1747 about a newly added code point that led to a re-analysis of the 1748 normalization rules used by IDNA and inherited by this document 1749 (Section 5.2.4). Some of the general issues are described in 1750 [IAB-Statement] and pursued in more detail in [IDNA-Unicode]. 1752 At the time of writing, these issues have yet to be settled. 1753 However, implementers need to be aware that this specification is 1754 likely to be updated in the future to address these issues. The 1755 potential changes include but might not be limited to the following: 1757 o The range of code points in the LetterDigits category 1758 (Sections 4.2.1 and 9.1) might be narrowed. 1760 o Some code points with special properties that are now allowed 1761 might be excluded. 1763 o More "Additional Mapping Rules" (Section 5.2.2) might be defined. 1765 o Alternative normalization methods might be added. 1767 As described in Section 11.1, until these issues are settled, it is 1768 reasonable for the IANA to apply the same precautionary principle 1769 described in [IAB-Statement] to the PRECIS Derived Property Value 1770 Registry as is applied to the Internationalized Domain Names for 1771 Applications (IDNA) Parameters registry: that is, to not make further 1772 updates to the registry. 1774 Nevertheless, implementations and deployments are unlikely to 1775 encounter significant problems as a consequence of these issues or 1776 potential changes if they follow the advice given in this 1777 specification to use the more restrictive IdentifierClass whenever 1778 possible or, if using the FreeformClass, to allow only a restricted 1779 set of code points, particularly avoiding code points whose 1780 implications they do not understand. 1782 14. References 1784 14.1. Normative References 1786 [RFC20] Cerf, V., "ASCII format for network interchange", STD 80, 1787 RFC 20, DOI 10.17487/RFC0020, October 1969, 1788 . 1790 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1791 Requirement Levels", BCP 14, RFC 2119, 1792 DOI 10.17487/RFC2119, March 1997, 1793 . 1795 [RFC5198] Klensin, J. and M. Padlipsky, "Unicode Format for Network 1796 Interchange", RFC 5198, DOI 10.17487/RFC5198, March 2008, 1797 . 1799 [RFC6365] Hoffman, P. and J. Klensin, "Terminology Used in 1800 Internationalization in the IETF", BCP 166, RFC 6365, 1801 DOI 10.17487/RFC6365, September 2011, 1802 . 1804 [Unicode] The Unicode Consortium, "The Unicode Standard", 1805 . 1807 14.2. Informative References 1809 [DerivedCoreProperties] 1810 The Unicode Consortium, "DerivedCoreProperties-7.0.0.txt", 1811 Unicode Character Database, February 2014, 1812 . 1815 [Err4568] RFC Errata, "Erratum ID 4568", RFC 7564, 1816 . 1818 [I-D.ietf-precis-7613bis] 1819 Saint-Andre, P. and A. Melnikov, "Preparation, 1820 Enforcement, and Comparison of Internationalized Strings 1821 Representing Usernames and Passwords", draft-ietf-precis- 1822 7613bis-11 (work in progress), July 2017. 1824 [I-D.ietf-precis-7700bis] 1825 Saint-Andre, P., "Preparation, Enforcement, and Comparison 1826 of Internationalized Strings Representing Nicknames", 1827 draft-ietf-precis-7700bis-10 (work in progress), July 1828 2017. 1830 [IAB-Statement] 1831 Internet Architecture Board, "IAB Statement on Identifiers 1832 and Unicode 7.0.0", February 2015, 1833 . 1837 [IDNA-Unicode] 1838 Klensin, J. and P. Faltstrom, "IDNA Update for Unicode 1839 7.0.0", Work in Progress, draft-klensin-idna-5892upd- 1840 unicode70-04, March 2015. 1842 [PropertyAliases] 1843 The Unicode Consortium, "PropertyAliases-7.0.0.txt", 1844 Unicode Character Database, November 2013, 1845 . 1848 [RFC2865] Rigney, C., Willens, S., Rubens, A., and W. Simpson, 1849 "Remote Authentication Dial In User Service (RADIUS)", 1850 RFC 2865, DOI 10.17487/RFC2865, June 2000, 1851 . 1853 [RFC3454] Hoffman, P. and M. Blanchet, "Preparation of 1854 Internationalized Strings ("stringprep")", RFC 3454, 1855 DOI 10.17487/RFC3454, December 2002, 1856 . 1858 [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, 1859 "Internationalizing Domain Names in Applications (IDNA)", 1860 RFC 3490, DOI 10.17487/RFC3490, March 2003, 1861 . 1863 [RFC3491] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep 1864 Profile for Internationalized Domain Names (IDN)", 1865 RFC 3491, DOI 10.17487/RFC3491, March 2003, 1866 . 1868 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 1869 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 1870 2003, . 1872 [RFC4422] Melnikov, A., Ed. and K. Zeilenga, Ed., "Simple 1873 Authentication and Security Layer (SASL)", RFC 4422, 1874 DOI 10.17487/RFC4422, June 2006, 1875 . 1877 [RFC4510] Zeilenga, K., Ed., "Lightweight Directory Access Protocol 1878 (LDAP): Technical Specification Road Map", RFC 4510, 1879 DOI 10.17487/RFC4510, June 2006, 1880 . 1882 [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review and 1883 Recommendations for Internationalized Domain Names 1884 (IDNs)", RFC 4690, DOI 10.17487/RFC4690, September 2006, 1885 . 1887 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 1888 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 1889 DOI 10.17487/RFC5226, May 2008, 1890 . 1892 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 1893 Specifications: ABNF", STD 68, RFC 5234, 1894 DOI 10.17487/RFC5234, January 2008, 1895 . 1897 [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security 1898 (TLS) Protocol Version 1.2", RFC 5246, 1899 DOI 10.17487/RFC5246, August 2008, 1900 . 1902 [RFC5890] Klensin, J., "Internationalized Domain Names for 1903 Applications (IDNA): Definitions and Document Framework", 1904 RFC 5890, DOI 10.17487/RFC5890, August 2010, 1905 . 1907 [RFC5891] Klensin, J., "Internationalized Domain Names in 1908 Applications (IDNA): Protocol", RFC 5891, 1909 DOI 10.17487/RFC5891, August 2010, 1910 . 1912 [RFC5892] Faltstrom, P., Ed., "The Unicode Code Points and 1913 Internationalized Domain Names for Applications (IDNA)", 1914 RFC 5892, DOI 10.17487/RFC5892, August 2010, 1915 . 1917 [RFC5893] Alvestrand, H., Ed. and C. Karp, "Right-to-Left Scripts 1918 for Internationalized Domain Names for Applications 1919 (IDNA)", RFC 5893, DOI 10.17487/RFC5893, August 2010, 1920 . 1922 [RFC5894] Klensin, J., "Internationalized Domain Names for 1923 Applications (IDNA): Background, Explanation, and 1924 Rationale", RFC 5894, DOI 10.17487/RFC5894, August 2010, 1925 . 1927 [RFC5895] Resnick, P. and P. Hoffman, "Mapping Characters for 1928 Internationalized Domain Names in Applications (IDNA) 1929 2008", RFC 5895, DOI 10.17487/RFC5895, September 2010, 1930 . 1932 [RFC6452] Faltstrom, P., Ed. and P. Hoffman, Ed., "The Unicode Code 1933 Points and Internationalized Domain Names for Applications 1934 (IDNA) - Unicode 6.0", RFC 6452, DOI 10.17487/RFC6452, 1935 November 2011, . 1937 [RFC6885] Blanchet, M. and A. Sullivan, "Stringprep Revision and 1938 Problem Statement for the Preparation and Comparison of 1939 Internationalized Strings (PRECIS)", RFC 6885, 1940 DOI 10.17487/RFC6885, March 2013, 1941 . 1943 [RFC6943] Thaler, D., Ed., "Issues in Identifier Comparison for 1944 Security Purposes", RFC 6943, DOI 10.17487/RFC6943, May 1945 2013, . 1947 [RFC7564] Saint-Andre, P. and M. Blanchet, "PRECIS Framework: 1948 Preparation, Enforcement, and Comparison of 1949 Internationalized Strings in Application Protocols", 1950 RFC 7564, DOI 10.17487/RFC7564, May 2015, 1951 . 1953 [RFC7622] Saint-Andre, P., "Extensible Messaging and Presence 1954 Protocol (XMPP): Address Format", RFC 7622, 1955 DOI 10.17487/RFC7622, September 2015, 1956 . 1958 [RFC7790] Yoneya, Y. and T. Nemoto, "Mapping Characters for Classes 1959 of the Preparation, Enforcement, and Comparison of 1960 Internationalized Strings (PRECIS)", RFC 7790, 1961 DOI 10.17487/RFC7790, February 2016, 1962 . 1964 [UAX11] Unicode Standard Annex #11, "East Asian Width", edited by 1965 Ken Lunde. An integral part of The Unicode Standard, 1966 . 1968 [UAX15] Unicode Standard Annex #15, "Unicode Normalization Forms", 1969 edited by Mark Davis and Ken Whistler. An integral part of 1970 The Unicode Standard, . 1972 [UAX9] Unicode Standard Annex #9, "Unicode Bidirectional 1973 Algorithm", edited by Mark Davis, Aharon Lanin, and Andrew 1974 Glass. An integral part of The Unicode Standard, 1975 . 1977 [UTR36] Unicode Technical Report #36, "Unicode Security 1978 Considerations", by Mark Davis and Michel Suignard, 1979 . 1981 [UTS39] Unicode Technical Standard #39, "Unicode Security 1982 Mechanisms", edited by Mark Davis and Michel Suignard, 1983 . 1985 Appendix A. Changes from RFC 7564 1987 The following changes were made from [RFC7564]. 1989 o Recommended the Unicode toLowerCase() operation over the Unicode 1990 toCaseFold() operation in most PRECIS applications. 1992 o Clarified the meaning of "preparation" and described the 1993 motivation for including it in PRECIS. 1995 o Updated references. 1997 See [RFC7564] for a description of the differences from [RFC3454]. 1999 Appendix B. Acknowledgements 2001 Thanks to Martin Duerst, William Fisher, John Klensin, Christian 2002 Schudt, and Sam Whited for their feedback. Thanks to Sam Whited also 2003 for submitting [Err4568]. 2005 See [RFC7564] for acknowledgements related to the specification that 2006 this document supersedes. 2008 Some algorithms and textual descriptions have been borrowed from 2009 [RFC5892]. Some text regarding security has been borrowed from 2010 [RFC5890], [I-D.ietf-precis-7613bis], and [RFC7622]. 2012 Authors' Addresses 2014 Peter Saint-Andre 2015 Filament 2016 18335 E 103rd Ave, Suite 203 2017 Commerce City, CO 80022 2018 USA 2020 Phone: +1 720 256 6756 2021 EMail: peter@filament.com 2022 URI: https://filament.com/ 2024 Marc Blanchet 2025 Viagenie 2026 246 Aberdeen 2027 Quebec, QC G1R 2E1 2028 Canada 2030 EMail: Marc.Blanchet@viagenie.ca 2031 URI: http://www.viagenie.ca/