idnits 2.17.1 draft-ietf-ltru-4646bis-23.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** The document seems to lack a License Notice according IETF Trust Provisions of 28 Dec 2009, Section 6.b.i or Provisions of 12 Sep 2009 Section 6.b -- however, there's a paragraph with a matching beginning. Boilerplate error? (You're using the IETF Trust Provisions' Section 6.b License Notice from 12 Feb 2009 rather than one of the newer Notices. See https://trustee.ietf.org/license-info/.) Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- -- The draft header indicates that this document obsoletes RFC4646, but the abstract doesn't seem to mention this, which it should. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to use 'NOT RECOMMENDED' as an RFC 2119 keyword, but does not include the phrase in its RFC 2119 key words list. -- The document seems to contain a disclaimer for pre-RFC5378 work, and may have content which was first submitted before 10 November 2008. The disclaimer is necessary when there are original authors that you have been unable to contact, or if some do not wish to grant the BCP78 rights to the IETF Trust. If you are able to get all authors (current and original) to grant those rights, you can and should remove the disclaimer; otherwise, the disclaimer is needed and you can ignore this comment. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 11, 2009) is 5426 days in the past. Is this intentional? Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO15924' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO3166-1' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO639-1' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO639-2' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO639-3' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO639-5' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO646' ** Obsolete normative reference: RFC 2028 (Obsoleted by RFC 9281) ** Obsolete normative reference: RFC 5226 (Obsoleted by RFC 8126) -- Possible downref: Non-RFC (?) normative reference: ref. 'UAX14' -- Possible downref: Non-RFC (?) normative reference: ref. 'Unicode' -- Obsolete informational reference (is this intentional?): RFC 1766 (Obsoleted by RFC 3066, RFC 3282) -- Obsolete informational reference (is this intentional?): RFC 3066 (Obsoleted by RFC 4646, RFC 4647) -- Obsolete informational reference (is this intentional?): RFC 4646 (Obsoleted by RFC 5646) Summary: 3 errors (**), 0 flaws (~~), 2 warnings (==), 15 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Language Tag Registry Update A. Phillips, Ed. 3 Working Group Lab126 4 Internet-Draft M. Davis, Ed. 5 Obsoletes: 4646 (if approved) Google 6 Intended status: BCP June 11, 2009 7 Expires: December 13, 2009 9 Tags for Identifying Languages 10 draft-ietf-ltru-4646bis-23 12 Status of this Memo 14 This Internet-Draft is submitted to IETF in full conformance with the 15 provisions of BCP 78 and BCP 79. This document may contain material 16 from IETF Documents or IETF Contributions published or made publicly 17 available before November 10, 2008. The person(s) controlling the 18 copyright in some of this material may not have granted the IETF 19 Trust the right to allow modifications of such material outside the 20 IETF Standards Process. Without obtaining an adequate license from 21 the person(s) controlling the copyright in such materials, this 22 document may not be modified outside the IETF Standards Process, and 23 derivative works of it may not be created outside the IETF Standards 24 Process, except to format it for publication as an RFC or to 25 translate it into languages other than English. 27 Internet-Drafts are working documents of the Internet Engineering 28 Task Force (IETF), its areas, and its working groups. Note that 29 other groups may also distribute working documents as Internet- 30 Drafts. 32 Internet-Drafts are draft documents valid for a maximum of six months 33 and may be updated, replaced, or obsoleted by other documents at any 34 time. It is inappropriate to use Internet-Drafts as reference 35 material or to cite them other than as "work in progress." 37 The list of current Internet-Drafts can be accessed at 38 http://www.ietf.org/ietf/1id-abstracts.txt. 40 The list of Internet-Draft Shadow Directories can be accessed at 41 http://www.ietf.org/shadow.html. 43 This Internet-Draft will expire on December 13, 2009. 45 Copyright Notice 47 Copyright (c) 2009 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents in effect on the date of 52 publication of this document (http://trustee.ietf.org/license-info). 53 Please review these documents carefully, as they describe your rights 54 and restrictions with respect to this document. 56 Abstract 58 This document describes the structure, content, construction, and 59 semantics of language tags for use in cases where it is desirable to 60 indicate the language used in an information object. It also 61 describes how to register values for use in language tags and the 62 creation of user-defined extensions for private interchange. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 67 2. The Language Tag . . . . . . . . . . . . . . . . . . . . . . . 6 68 2.1. Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . 6 69 2.1.1. Formatting of Language Tags . . . . . . . . . . . . . 8 70 2.2. Language Subtag Sources and Interpretation . . . . . . . . 9 71 2.2.1. Primary Language Subtag . . . . . . . . . . . . . . . 11 72 2.2.2. Extended Language Subtags . . . . . . . . . . . . . . 13 73 2.2.3. Script Subtag . . . . . . . . . . . . . . . . . . . . 14 74 2.2.4. Region Subtag . . . . . . . . . . . . . . . . . . . . 15 75 2.2.5. Variant Subtags . . . . . . . . . . . . . . . . . . . 17 76 2.2.6. Extension Subtags . . . . . . . . . . . . . . . . . . 18 77 2.2.7. Private Use Subtags . . . . . . . . . . . . . . . . . 19 78 2.2.8. Grandfathered and Redundant Registrations . . . . . . 20 79 2.2.9. Classes of Conformance . . . . . . . . . . . . . . . . 21 80 3. Registry Format and Maintenance . . . . . . . . . . . . . . . 23 81 3.1. Format of the IANA Language Subtag Registry . . . . . . . 23 82 3.1.1. File Format . . . . . . . . . . . . . . . . . . . . . 23 83 3.1.2. Record and Field Definitions . . . . . . . . . . . . . 24 84 3.1.3. Type Field . . . . . . . . . . . . . . . . . . . . . . 27 85 3.1.4. Subtag and Tag Fields . . . . . . . . . . . . . . . . 28 86 3.1.5. Description Field . . . . . . . . . . . . . . . . . . 28 87 3.1.6. Deprecated Field . . . . . . . . . . . . . . . . . . . 30 88 3.1.7. Preferred-Value Field . . . . . . . . . . . . . . . . 30 89 3.1.8. Prefix Field . . . . . . . . . . . . . . . . . . . . . 32 90 3.1.9. Suppress-Script Field . . . . . . . . . . . . . . . . 34 91 3.1.10. Macrolanguage Field . . . . . . . . . . . . . . . . . 34 92 3.1.11. Scope Field . . . . . . . . . . . . . . . . . . . . . 35 93 3.1.12. Comments Field . . . . . . . . . . . . . . . . . . . . 36 94 3.2. Language Subtag Reviewer . . . . . . . . . . . . . . . . . 36 95 3.3. Maintenance of the Registry . . . . . . . . . . . . . . . 37 96 3.4. Stability of IANA Registry Entries . . . . . . . . . . . . 37 97 3.5. Registration Procedure for Subtags . . . . . . . . . . . . 43 98 3.6. Possibilities for Registration . . . . . . . . . . . . . . 48 99 3.7. Extensions and the Extensions Registry . . . . . . . . . . 51 100 3.8. Update of the Language Subtag Registry . . . . . . . . . . 53 101 4. Formation and Processing of Language Tags . . . . . . . . . . 55 102 4.1. Choice of Language Tag . . . . . . . . . . . . . . . . . . 55 103 4.1.1. Tagging Encompassed Languages . . . . . . . . . . . . 59 104 4.1.2. Using Extended Language Subtags . . . . . . . . . . . 60 105 4.2. Meaning of the Language Tag . . . . . . . . . . . . . . . 62 106 4.3. Lists of Languages . . . . . . . . . . . . . . . . . . . . 64 107 4.4. Length Considerations . . . . . . . . . . . . . . . . . . 65 108 4.4.1. Working with Limited Buffer Sizes . . . . . . . . . . 65 109 4.4.2. Truncation of Language Tags . . . . . . . . . . . . . 66 110 4.5. Canonicalization of Language Tags . . . . . . . . . . . . 67 111 4.6. Considerations for Private Use Subtags . . . . . . . . . . 69 112 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 71 113 5.1. Language Subtag Registry . . . . . . . . . . . . . . . . . 71 114 5.2. Extensions Registry . . . . . . . . . . . . . . . . . . . 72 115 6. Security Considerations . . . . . . . . . . . . . . . . . . . 74 116 7. Character Set Considerations . . . . . . . . . . . . . . . . . 75 117 8. Changes from RFC 4646 . . . . . . . . . . . . . . . . . . . . 76 118 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 80 119 9.1. Normative References . . . . . . . . . . . . . . . . . . . 80 120 9.2. Informative References . . . . . . . . . . . . . . . . . . 81 121 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 83 122 Appendix B. Examples of Language Tags (Informative) . . . . . . . 84 123 Appendix C. Examples of Registration Forms . . . . . . . . . . . 87 124 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 89 126 1. Introduction 128 Human beings on our planet have, past and present, used a number of 129 languages. There are many reasons why one would want to identify the 130 language used when presenting or requesting information. 132 The language of an information item or a user's language preferences 133 often need to be identified so that appropriate processing can be 134 applied. For example, the user's language preferences in a Web 135 browser can be used to select Web pages appropriately. Language 136 information can also be used to select among tools (such as 137 dictionaries) to assist in the processing or understanding of content 138 in different languages. Knowledge about the particular language used 139 by some piece of information content might be useful or even required 140 by some types of processing; for example, spell-checking, computer- 141 synthesized speech, Braille transcription, or high-quality print 142 renderings. 144 One means of indicating the language used is by labeling the 145 information content with an identifier or "tag". These tags can also 146 be used to specify the user's preferences when selecting information 147 content, or for labeling additional attributes of content and 148 associated resources. 150 Sometimes language tags are used to indicate additional language 151 attributes of content. For example, indicating specific information 152 about the dialect, writing system, or orthography used in a document 153 or resource may enable the user to obtain information in a form that 154 they can understand, or it can be important in processing or 155 rendering the given content into an appropriate form or style. 157 This document specifies a particular identifier mechanism (the 158 language tag) and a registration function for values to be used to 159 form tags. It also defines a mechanism for private use values and 160 future extension. 162 This document replaces [RFC4646]. This document, in combination with 163 [RFC4647], replaces [RFC3066] (and its predecessor [RFC1766]) as BCP 164 47. For a list of changes in this document, see Section 8. 166 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 167 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 168 document are to be interpreted as described in [RFC2119]. 170 2. The Language Tag 172 Language tags are used to help identify languages, whether spoken, 173 written, signed, or otherwise signaled, for the purpose of 174 communication. This includes constructed and artificial languages, 175 but excludes languages not intended primarily for human 176 communication, such as programming languages. 178 2.1. Syntax 180 A language tag is composed from a sequence of one or more "subtags", 181 each of which refines or narrows the range of language identified by 182 the overall tag. Subtags, in turn, are a sequence of alphanumeric 183 characters (letters and digits), distinguished and separated from 184 other subtags in a tag by a hyphen ("-", [Unicode] U+002D). 186 There are different types of subtag, each of which is distinguished 187 by length, position in the tag, and content: each subtag's type can 188 be recognized solely by these features. This makes it possible to 189 extract and assign some semantic information to the subtags, even if 190 the specific subtag values are not recognized. Thus, a language tag 191 processor need not have a list of valid tags or subtags (that is, a 192 copy of some version of the IANA Language Subtag Registry) in order 193 to perform common searching and matching operations. The only 194 exceptions to this ability to infer meaning from subtag structure are 195 the grandfathered tags listed in the productions 'regular' and 196 'irregular' below. These tags were registered under [RFC3066] and 197 are a fixed list that can never change. 199 The syntax of the language tag in ABNF [RFC5234] is: 201 Language-Tag = langtag ; normal language tags 202 / privateuse ; private use tag 203 / grandfathered ; grandfathered tags 205 langtag = language 206 ["-" script] 207 ["-" region] 208 *("-" variant) 209 *("-" extension) 210 ["-" privateuse] 212 language = 2*3ALPHA ; shortest ISO 639 code 213 ["-" extlang] ; sometimes followed by 214 ; extended language subtags 215 / 4ALPHA ; or reserved for future use 216 / 5*8ALPHA ; or registered language subtag 218 extlang = 3ALPHA ; selected ISO 639 codes 219 *2("-" 3ALPHA) ; permanently reserved 221 script = 4ALPHA ; ISO 15924 code 223 region = 2ALPHA ; ISO 3166-1 code 224 / 3DIGIT ; UN M.49 code 226 variant = 5*8alphanum ; registered variants 227 / (DIGIT 3alphanum) 229 extension = singleton 1*("-" (2*8alphanum)) 231 ; Single alphanumerics 232 ; "x" reserved for private use 233 singleton = DIGIT ; 0 - 9 234 / %x41-57 ; A - W 235 / %x59-5A ; Y - Z 236 / %x61-77 ; a - w 237 / %x79-7A ; y - z 239 privateuse = "x" 1*("-" (1*8alphanum)) 241 grandfathered = irregular ; non-redundant tags registered 242 / regular ; during the RFC 3066 era 244 irregular = "en-GB-oed" ; irregular tags do not match 245 / "i-ami" ; the 'langtag' production and 246 / "i-bnn" ; would not otherwise be 247 / "i-default" ; considered 'well-formed' 248 / "i-enochian" ; These tags are all valid, 249 / "i-hak" ; but most are deprecated 250 / "i-klingon" ; in favor of more modern 251 / "i-lux" ; subtags or subtag 252 / "i-mingo" ; combination 253 / "i-navajo" 254 / "i-pwn" 255 / "i-tao" 256 / "i-tay" 257 / "i-tsu" 258 / "sgn-BE-FR" 259 / "sgn-BE-NL" 260 / "sgn-CH-DE" 262 regular = "art-lojban" ; these tags match the 'langtag' 263 / "cel-gaulish" ; production, but their subtags 264 / "no-bok" ; are not extended language 265 / "no-nyn" ; or variant subtags: their meaning 266 / "zh-guoyu" ; is defined by their registration 267 / "zh-hakka" ; and all of these are deprecated 268 / "zh-min" ; in favor of a more modern 269 / "zh-min-nan" ; subtag or sequence of subtags 270 / "zh-xiang" 272 alphanum = (ALPHA / DIGIT) ; letters and numbers 274 Figure 1: Language Tag ABNF 276 For examples of language tags, see Appendix B. 278 All subtags have a maximum length of eight characters and whitespace 279 is not permitted in a language tag. There is a subtlety in the ABNF 280 production 'variant': a variant starting with a digit has a minimum 281 length of four characters, while those starting with a letter have a 282 minimum length of five characters. 284 Although [RFC5234] refers to octets, the language tags described in 285 this document are sequences of characters from the US-ASCII [ISO646] 286 repertoire. Language tags MAY be used in documents and applications 287 that use other encodings, so long as these encompass the relevant 288 part of the US-ASCII repertoire. An example of this would be an XML 289 document that uses the UTF-16LE [RFC2781] encoding of [Unicode]. 291 2.1.1. Formatting of Language Tags 293 At all times, language tags and their subtags, including private-use 294 and extensions, are to be treated as case insensitive: there exist 295 conventions for the capitalization of some of the subtags, but these 296 MUST NOT be taken to carry meaning. 298 Thus, the tag "mn-Cyrl-MN" is not distinct from "MN-cYRL-mn" or "mN- 299 cYrL-Mn" (or any other combination), and each of these variations 300 conveys the same meaning: Mongolian written in the Cyrillic script as 301 used in Mongolia. 303 The ABNF syntax also does not distinguish between upper and 304 lowercase: the uppercase US-ASCII letters in the range 'A' through 305 'Z' are always considered equivalent and mapped directly to their US- 306 ASCII lowercase equivalents in the range 'a' through 'z'. So the tag 307 "I-AMI" is considered equivalent to that value "i-ami" in the 308 'irregular' production. 310 Although case distinctions do not carry meaning in language tags, 311 consistent formatting and presentation of language tags will aid 312 users. The format of subtags in the registry is RECOMMENDED as the 313 form to use in language tags. This format generally corresponds to 314 the common conventions for the various ISO standards from which the 315 subtags are derived. 317 These conventions include: 319 o [ISO639-1] recommends that language codes be written in lowercase 320 ('mn' Mongolian). 322 o [ISO15924] recommends that script codes use lowercase with the 323 initial letter capitalized ('Cyrl' Cyrillic). 325 o [ISO3166-1] recommends that country codes be capitalized ('MN' 326 Mongolia). 328 An implementation can reproduce this format without accessing the 329 registry as follows: All subtags, including extension and private use 330 subtags, use lowercase letters, with two exceptions: two-letter and 331 four-letter subtags that neither appear at the start of the tag nor 332 occur after singletons. Such two-letter subtags are all uppercase 333 (as in the tags "en-CA-x-ca" or "sgn-BE-FR") and four-letter subtags 334 are titlecase (as in the tag "az-Latn-x-latn"). 336 Note: Case folding of ASCII letters in certain locales, unless 337 carefully handled, sometimes produces non-ASCII character values. 338 The Unicode Character Database file "SpecialCasing.txt" defines the 339 specific cases that are known to cause problems with this. In 340 particular, the letter 'i' (U+0069) in Turkish and Azerbaijani is 341 uppercased to U+0130 (LATIN CAPITAL LETTER I WITH DOT ABOVE). 342 Implementers SHOULD specify a locale-neutral casing operation to 343 ensure that case folding of subtags does not produce this value, 344 which is illegal in language tags. For example, if one were to 345 uppercase the region subtag 'in' using Turkish locale rules, the 346 sequence U+0130 U+004E would result instead of the expected 'IN'. 348 2.2. Language Subtag Sources and Interpretation 350 The namespace of language tags and their subtags is administered by 351 the Internet Assigned Numbers Authority (IANA) according to the rules 352 in Section 5 of this document. The Language Subtag Registry 353 maintained by IANA is the source for valid subtags: other standards 354 referenced in this section provide the source material for that 355 registry. 357 Terminology used in this document: 359 o "Tag" refers to a complete language tag, such as "sr-Latn-RS" or 360 "az-Arab-IR". Examples of tags in this document are enclosed in 361 double-quotes ("en-US"). 363 o "Subtag" refers to a specific section of a tag, delimited by 364 hyphen, such as the subtags 'zh', 'Hant', and 'CN' in the tag "zh- 365 Hant-CN". Examples of subtags in this document are enclosed in 366 single quotes ('Hant'). 368 o "Code" refers to values defined in external standards (and which 369 are used as subtags in this document). For example, 'Hant' is an 370 [ISO15924] script code that was used to define the 'Hant' script 371 subtag for use in a language tag. Examples of codes in this 372 document are enclosed in single quotes ('en', 'Hant'). 374 Language tags are designed so that each subtag type has unique length 375 and content restrictions. These make identification of the subtag's 376 type possible, even if the content of the subtag itself is 377 unrecognized. This allows tags to be parsed and processed without 378 reference to the latest version of the underlying standards or the 379 IANA registry and makes the associated exception handling when 380 parsing tags simpler. 382 Some of the subtags in the IANA registry do not come from an 383 underlying standard. These can only appear in specific positions in 384 a tag: they can only occur as primary language subtags or as variant 385 subtags. 387 Sequences of private use and extension subtags MUST occur at the end 388 of the sequence of subtags and MUST NOT be interspersed with subtags 389 defined elsewhere in this document. These sequences are introduced 390 by single-character subtags, which are reserved as follows: 392 o The single-letter subtag 'x' introduces a sequence of private use 393 subtags. The interpretation of any private use subtags is defined 394 solely by private agreement and is not defined by the rules in 395 this section or in any standard or registry defined in this 396 document. 398 o The single-letter subtag 'i' is used by some grandfathered tags, 399 such as "i-default", where it always appears in the first position 400 and cannot be confused with an extension. 402 o All other single-letter and single-digit subtags are reserved to 403 introduce standardized extension subtag sequences as described in 404 Section 3.7. 406 2.2.1. Primary Language Subtag 408 The primary language subtag is the first subtag in a language tag and 409 cannot be omitted, with two exceptions: 411 o The single-character subtag 'x' as the primary subtag indicates 412 that the language tag consists solely of subtags whose meaning is 413 defined by private agreement. For example, in the tag "x-fr-CH", 414 the subtags 'fr' and 'CH' do not represent the French language or 415 the country of Switzerland (or any other value in the IANA 416 registry) unless there is a private agreement in place to do so. 417 See Section 4.6. 419 o The single-character subtag 'i' is used by some grandfathered tags 420 (see Section 2.2.8) such as "i-klingon" and "i-bnn". (Other 421 grandfathered tags have a primary language subtag in their first 422 position.) 424 The following rules apply to the primary language subtag: 426 1. Two-character primary language subtags were defined in the IANA 427 registry according to the assignments found in the standard "ISO 428 639-1:2002, Codes for the representation of names of languages -- 429 Part 1: Alpha-2 code" [ISO639-1], or using assignments 430 subsequently made by the ISO 639-1 registration authority (RA) or 431 governing standardization bodies. 433 2. Three-character primary language subtags in the IANA registry 434 were defined according to the assignments found in one of these 435 additional ISO 639 parts or assignments subsequently made by the 436 relevant ISO 639 registration authorities or governing 437 standardization bodies: 439 A. "ISO 639-2:1998 - Codes for the representation of names of 440 languages -- Part 2: Alpha-3 code - edition 1" [ISO639-2] 442 B. "ISO 639-3:2007 - Codes for the representation of names of 443 languages -- Part 3: Alpha-3 code for comprehensive coverage 444 of languages" [ISO639-3] 446 C. "ISO 639-5:2008 - Codes for the representation of names of 447 languages -- Part 5: Alpha-3 code for language families and 448 groups" [ISO639-5] 450 3. The subtags in the range 'qaa' through 'qtz' are reserved for 451 private use in language tags. These subtags correspond to codes 452 reserved by ISO 639-2 for private use. These codes MAY be used 453 for non-registered primary language subtags (instead of using 454 private use subtags following 'x-'). Please refer to Section 4.6 455 for more information on private use subtags. 457 4. Four-character language subtags are reserved for possible future 458 standardization. 460 5. Any language subtags of 5 to 8 characters in length in the IANA 461 registry were defined via the registration process in Section 3.5 462 and MAY be used to form the primary language subtag. An example 463 of what such a registration might include: one of the 464 grandfathered IANA registrations is "i-enochian". The subtag 465 'enochian' could be registered in the IANA registry as a primary 466 language subtag (assuming that ISO 639 does not register this 467 language first), making tags such as "enochian-AQ" and "enochian- 468 Latn" valid. 470 At the time this document was created, there were no examples of 471 this kind of subtag. Future registrations of this type are 472 discouraged: an attempt to register any new proposed primary 473 language MUST be made to the ISO 639 registration authority. 474 Proposals rejected by the ISO 639 registration authority are 475 unlikely to meet the criteria for primary language subtags and 476 are thus unlikely to be registered. 478 6. Other values MUST NOT be assigned to the primary subtag except by 479 revision or update of this document. 481 When languages have both an ISO 639-1 two-character code and a three 482 character code (assigned by ISO 639-2, ISO 639-3, or ISO 639-5), only 483 the ISO 639-1 two-character code is defined in the IANA registry. 485 When languages that have no ISO 639-1 two-character code and for 486 which the ISO 639-2/T (Terminology) code and the ISO 639-2/B 487 (Bibliographic) codes differ, only the Terminology code is defined in 488 the IANA registry. At the time this document was created, all 489 languages that had both kinds of three-character code were also 490 assigned a two-character code; it is expected that future assignments 491 of this nature will not occur. 493 In order to avoid instability in the canonical form of tags, if a 494 two-character code is added to ISO 639-1 for a language for which a 495 three-character code was already included in either ISO 639-2 or ISO 496 639-3, the two-character code MUST NOT be registered. See 497 Section 3.4. 499 For example, if some content were tagged with 'haw' (Hawaiian), which 500 currently has no two-character code, the tag would not need to be 501 changed if ISO 639-1 were to assign a two-character code to the 502 Hawaiian language at a later date. 504 To avoid these problems with versioning and subtag choice (as 505 experienced during the transition between RFC 1766 and RFC 3066), as 506 well as to ensure the canonical nature of subtags defined by this 507 document, the ISO 639 Registration Authority Joint Advisory Committee 508 (ISO 639/RA-JAC) has included the following statement in 509 [iso639.prin]: 511 "A language code already in ISO 639-2 at the point of freezing ISO 512 639-1 shall not later be added to ISO 639-1. This is to ensure 513 consistency in usage over time, since users are directed in 514 Internet applications to employ the alpha-3 code when an alpha-2 515 code for that language is not available." 517 2.2.2. Extended Language Subtags 519 Extended language subtags are used to identify certain specially- 520 selected languages that, for various historical and compatibility 521 reasons, are closely identified with or tagged using an existing 522 primary language subtag. Extended language subtags are always used 523 with their enclosing primary language subtag (indicated with a 524 'Prefix' field in the registry) when used to form the language tag. 525 All languages that have an extended language subtag in the registry 526 also have an identical primary language subtag record in the 527 registry. This primary language subtag is RECOMMENDED for forming 528 the language tag. The following rules apply to the extended language 529 subtags: 531 1. Extended language subtags consist solely of three-letter subtags. 532 All extended language subtag records defined in the registry were 533 defined according to the assignments found in [ISO639-3]. 534 Language collections and groupings, such as defined in [ISO639-5] 535 are specifically excluded from being extended language subtags. 537 2. Extended language subtag records MUST include exactly one 538 'Prefix' field indicating an appropriate subtag or sequence of 539 subtags for that extended language subtag. 541 3. Extended language subtag records MUST include a 'Preferred- 542 Value'. The 'Preferred-Value' and 'Subtag' fields MUST be 543 identical. 545 4. Although the ABNF production 'extlang' permits up to three 546 extended language tags in the language tag, extended language 547 subtags MUST NOT include another extended language subtag in 548 their Prefix. That is, the second and third extended language 549 subtag positions in a language tag are permanently reserved and 550 tags that include those subtags in that position are, and will 551 always remain, invalid. 553 For example, the macrolanguage Chinese ('zh') encompasses a number of 554 languages. For compatibility reasons, each of these languages has 555 both a primary and extended language subtag in the registry. A few 556 selected examples of these include Gan Chinese ('gan'), Cantonese 557 Chinese ('yue') and Mandarin Chinese ('cmn'). Each is encompassed by 558 the macrolanguage 'zh' (Chinese). Therefore, they each have the 559 prefix "zh" in their registry records. Thus Gan Chinese is 560 represented with tags beginning "zh-gan" or "gan"; Cantonese with 561 tags beginning either "yue" or "zh-yue"; and Mandarin Chinese with 562 "zh-cmn" or "cmn". The language subtag 'zh' can still be used 563 without an extended language subtag to label a resource as some 564 unspecified variety of Chinese, while the primary language subtag 565 ('gan', 'yue', 'cmn') is preferred to using the extended language 566 form ("zh-gan", "zh-yue", "zh-cmn"). 568 2.2.3. Script Subtag 570 Script subtags are used to indicate the script or writing system 571 variations that distinguish the written forms of a language or its 572 dialects. The following rules apply to the script subtags: 574 1. Script subtags MUST follow any primary and extended language 575 subtags and MUST precede any other type of subtag. 577 2. Script subtags consist of four letters and were defined according 578 to the assignments found in [ISO15924] ("Information and 579 documentation -- Codes for the representation of names of 580 scripts"), or subsequently assigned by the ISO 15924 registration 581 authority or governing standardization bodies. Only codes 582 assigned by ISO 15924 will be considered for registration. 584 3. The script subtags 'Qaaa' through 'Qabx' are reserved for private 585 use in language tags. These subtags correspond to codes reserved 586 by ISO 15924 for private use. These codes MAY be used for non- 587 registered script values. Please refer to Section 4.6 for more 588 information on private use subtags. 590 4. There MUST be at most one script subtag in a language tag, and 591 the script subtag SHOULD be omitted when it adds no 592 distinguishing value to the tag or when the primary or extended 593 language subtag's record in the subtag registry includes a 594 'Suppress-Script' field listing the applicable script subtag. 596 For example: "sr-Latn" represents Serbian written using the Latin 597 script. 599 2.2.4. Region Subtag 601 Region subtags are used to indicate linguistic variations associated 602 with or appropriate to a specific country, territory, or region. 603 Typically, a region subtag is used to indicate variations such as 604 regional dialects or usage, or region-specific spelling conventions. 605 It can also be used to indicate that content is expressed in a way 606 that is appropriate for use throughout a region, for instance, 607 Spanish content tailored to be useful throughout Latin America. 609 The following rules apply to the region subtags: 611 1. Region subtags MUST follow any primary language, extended 612 language, or script subtags and MUST precede any other type of 613 subtag. 615 2. Two-letter region subtags were defined according to the 616 assignments found in [ISO3166-1] ("Codes for the representation 617 of names of countries and their subdivisions -- Part 1: Country 618 codes") using the list of alpha-2 country codes, or using 619 assignments subsequently made by the ISO 3166-1 maintenance 620 agency or governing standardization bodies. In addition, the 621 codes that are "exceptionally reserved" (as opposed to 622 "assigned") in ISO 3166-1 were also defined in the registry, with 623 the exception of 'UK', which is an exact synonym for the assigned 624 code 'GB'. 626 3. The region subtags 'AA', 'QM'-'QZ', 'XA'-'XZ', and 'ZZ' are 627 reserved for private use in language tags. These subtags 628 correspond to codes reserved by ISO 3166 for private use. These 629 codes MAY be used for private use region subtags (instead of 630 using a private use subtag sequence). Please refer to 631 Section 4.6 for more information on private use subtags. 633 4. Three-character region subtags consist solely of digit (number) 634 characters and were defined according to the assignments found in 635 UN Standard Country or Area Codes for Statistical Use [UN_M.49] 636 or assignments subsequently made by the governing standards body. 637 Not all of the UN M.49 codes are defined in the IANA registry. 638 The following rules define which codes are entered into the 639 registry as valid subtags: 641 A. UN numeric codes assigned to 'macro-geographical 642 (continental)' or sub-regions MUST be registered in the 643 registry. These codes are not associated with an assigned 644 ISO 3166-1 alpha-2 code and represent supra-national areas, 645 usually covering more than one nation, state, province, or 646 territory. 648 B. UN numeric codes for 'economic groupings' or 'other 649 groupings' MUST NOT be registered in the IANA registry and 650 MUST NOT be used to form language tags. 652 C. When ISO 3166-1 reassigns a code formerly used for one 653 country or area to another country or area and that code 654 already is present in the registry, the UN numeric code for 655 that country or area MUST be registered in the registry as 656 described in Section 3.4 and MUST be used to form language 657 tags that represent the country or region for which it is 658 defined (rather than the recycled ISO 3166-1 code). 660 D. UN numeric codes for countries or areas for which there is an 661 associated ISO 3166-1 alpha-2 code in the registry MUST NOT 662 be entered into the registry and MUST NOT be used to form 663 language tags. Note that the ISO 3166-based subtag in the 664 registry MUST actually be associated with the UN M.49 code in 665 question. 667 E. For historical reasons, the UN numeric code 830 (Channel 668 Islands), which was not registered at the time this document 669 was adopted and had, at that time, no corresponding ISO 670 3166-1 code, MAY be entered into the IANA registry via the 671 process described in Section 3.5, provided no ISO 3166-1 code 672 with that exact meaning has been previously registered. 674 F. All other UN numeric codes for countries or areas that do not 675 have an associated ISO 3166-1 alpha-2 code MUST NOT be 676 entered into the registry and MUST NOT be used to form 677 language tags. For more information about these codes, see 678 Section 3.4. 680 5. The alphanumeric codes in Appendix X of the UN document MUST NOT 681 be entered into the registry and MUST NOT be used to form 682 language tags. (At the time this document was created, these 683 values matched the ISO 3166-1 alpha-2 codes.) 685 6. There MUST be at most one region subtag in a language tag and the 686 region subtag MAY be omitted, as when it adds no distinguishing 687 value to the tag. 689 For example: 691 "de-AT" represents German ('de') as used in Austria ('AT'). 693 "sr-Latn-RS" represents Serbian ('sr') written using Latin script 694 ('Latn') as used in Serbia ('RS'). 696 "es-419" represents Spanish ('es') appropriate to the UN-defined 697 Latin America and Caribbean region ('419'). 699 2.2.5. Variant Subtags 701 Variant subtags are used to indicate additional, well-recognized 702 variations that define a language or its dialects that are not 703 covered by other available subtags. The following rules apply to the 704 variant subtags: 706 1. Variant subtags MUST follow any primary language, extended 707 language, script, or region subtags, and MUST precede any 708 extension or private use subtag sequences. 710 2. Variant subtags, as a collection, are not associated with any 711 particular external standard. The meaning of variant subtags in 712 the registry is defined in the course of the registration process 713 defined in Section 3.5. Note that any particular variant subtag 714 might be associated with some external standard. However, 715 association with a standard is not required for registration. 717 3. More than one variant MAY be used to form the language tag. 719 4. Variant subtags MUST be registered with IANA according to the 720 rules in Section 3.5 of this document before being used to form 721 language tags. In order to distinguish variants from other types 722 of subtags, registrations MUST meet the following length and 723 content restrictions: 725 1. Variant subtags that begin with a letter (a-z, A-Z) MUST be 726 at least five characters long. 728 2. Variant subtags that begin with a digit (0-9) MUST be at 729 least four characters long. 731 5. The same variant subtag MUST NOT be used more than once within a 732 language tag. 734 * For example, the tag "de-DE-1901-1901" is not valid. 736 Variant subtag records in the language subtag registry MAY include 737 one or more 'Prefix' (Section 3.1.8) fields. Each 'Prefix' indicates 738 a suitable sequence of subtags for forming (with other subtags, as 739 appropriate) a language tag when using the variant. 741 Most variants that share a prefix are mutually exclusive. For 742 example, the German orthographic variations '1996' and '1901' SHOULD 743 NOT be used in the same tag, as they represent the dates of different 744 spelling reforms. A variant that can meaningfully be used in 745 combination with another variant SHOULD include a 'Prefix' field in 746 its registry record that lists that other variant. For example, if 747 another German variant 'example' were created that made sense to use 748 with '1996', then 'example' should include two Prefix fields: "de" 749 and "de-1996". 751 For example: 753 "sl-nedis" represents the Natisone or Nadiza dialect of Slovenian. 755 "de-CH-1996" represents German as used in Switzerland and as 756 written using the spelling reform beginning in the year 1996 C.E. 758 2.2.6. Extension Subtags 760 Extensions provide a mechanism for extending language tags for use in 761 various applications. They are intended to identify information 762 which is commonly used in association with languages or language 763 tags, but which is not part of language identification. See 764 Section 3.7. The following rules apply to extensions: 766 1. An extension MUST follow at least a primary language subtag. 767 That is, a language tag cannot begin with an extension. 768 Extensions extend language tags, they do not override or replace 769 them. For example, "a-value" is not a well-formed language tag, 770 while "de-a-value" is. Note that extensions cannot be used in 771 tags that are entirely private use (that is, tags starting with 772 "x-"). 774 2. Extension subtags are separated from the other subtags defined in 775 this document by a single-character subtag (called a 776 "singleton"). The singleton MUST be one allocated to a 777 registration authority via the mechanism described in Section 3.7 778 and MUST NOT be the letter 'x', which is reserved for private use 779 subtag sequences. 781 3. Each singleton subtag MUST appear at most one time in each tag 782 (other than as a private use subtag). That is, singleton subtags 783 MUST NOT be repeated. For example, the tag "en-a-bbb-a-ccc" is 784 invalid because the subtag 'a' appears twice. Note that the tag 785 "en-a-bbb-x-a-ccc" is valid because the second appearance of the 786 singleton 'a' is in a private use sequence. 788 4. Extension subtags MUST meet whatever requirements are set by the 789 document that defines their singleton prefix and whatever 790 requirements are provided by the maintaining authority. Note 791 that there might not be a registry of these subtags and 792 validating processors are not required to validate extensions. 794 5. Each extension subtag MUST be from two to eight characters long 795 and consist solely of letters or digits, with each subtag 796 separated by a single '-'. Case distinctions are ignored in 797 extensions (as with any language subtag) and normalized subtags 798 of this type are expected to be in lowercase. 800 6. Each singleton MUST be followed by at least one extension subtag. 801 For example, the tag "tlh-a-b-foo" is invalid because the first 802 singleton 'a' is followed immediately by another singleton 'b'. 804 7. Extension subtags MUST follow all primary language, extended 805 language, script, region, and variant subtags in a tag and MUST 806 precede any private-use subtag sequences. 808 8. All subtags following the singleton and before another singleton 809 are part of the extension. Example: In the tag "fr-a-Latn", the 810 subtag 'Latn' does not represent the script subtag 'Latn' defined 811 in the IANA Language Subtag Registry. Its meaning is defined by 812 the extension 'a'. 814 9. In the event that more than one extension appears in a single 815 tag, the tag SHOULD be canonicalized as described in Section 4.5, 816 by ordering the various extension sequences into case-insensitive 817 ASCII order. 819 For example, if an extension were defined for the singleton 'r' and 820 it defined the subtags shown, then the following tag would be a valid 821 example: "en-Latn-GB-boont-r-extended-sequence-x-private" 823 2.2.7. Private Use Subtags 825 Private use subtags are used to indicate distinctions in language 826 important in a given context by private agreement. The following 827 rules apply to private use subtags: 829 1. Private use subtags are separated from the other subtags defined 830 in this document by the reserved single-character subtag 'x'. 832 2. Private use subtags MUST conform to the format and content 833 constraints defined in the ABNF for all subtags, that is, they 834 MUST consist solely of letters and digits and not exceed eight 835 characters in length. 837 3. Private use subtags MUST follow all primary language, extended 838 language, script, region, variant, and extension subtags in the 839 tag. Another way of saying this is that all subtags following 840 the singleton 'x' MUST be considered private use. Example: The 841 subtag 'US' in the tag "en-x-US" is a private use subtag. 843 4. A tag MAY consist entirely of private use subtags. 845 5. No source is defined for private use subtags. Use of private use 846 subtags is by private agreement only. 848 6. Private use subtags are NOT RECOMMENDED where alternatives exist 849 or for general interchange. See Section 4.6 for more information 850 on private use subtag choice. 852 For example, suppose a group of scholars are studying some texts in 853 medieval Greek. They might agree to use some collection of private- 854 use subtags to identify different styles of writing in the texts. 855 For example, they might use 'el-x-koine' for documents in the 856 "common" style while using 'el-x-attic' for other documents that 857 mimic the Attic style. These subtags would not be recognized by 858 outside processes or systems, but might be useful in categorizing 859 various texts for study by those in the group. 861 Do not confuse private-use subtags with what this document refers to 862 as "subtags reserved for private use". The latter are subtags such 863 as 'qaa', 'Qaaa', or 'AA' that are designated as codes for private 864 use by one of the underlying ISO standards. Although their purpose 865 is very similar, subtags reserved for private use are represented by 866 records of the appropriate type in the registry and, thus, are 867 syntactically "language", "script", or "region" subtags, rather than 868 "private-use" subtags. 870 2.2.8. Grandfathered and Redundant Registrations 872 Prior to RFC 4646, whole language tags were registered according to 873 the rules in RFC 1766 and/or RFC 3066. All of these registered tags 874 remain valid as language tags. 876 Many of these registered tags were made redundant by the advent of 877 either RFC 4646 or this document. A redundant tag is a grandfathered 878 registration whose individual subtags appear with the same semantic 879 meaning in the registry. For example, the tag "zh-Hant" (Traditional 880 Chinese) can now be composed from the subtags 'zh' (Chinese) and 881 'Hant' (Han script traditional variant). These redundant tags are 882 maintained in the registry as records of type "redundant", mostly as 883 a matter of historical curiosity. 885 The remainder of the previously registered tags are "grandfathered". 886 These tags are classified into two groups: 'regular' and 'irregular'. 888 Grandfathered tags that (appear to) match the 'langtag' production in 889 Figure 1 are considered 'regular' grandfathered tags. These tags 890 either contain subtags that do not individually appear in the 891 registry, or their subtags appear but with a different semantic 892 meaning: each tag, in its entirety, represents a language or 893 collection of languages. 895 Grandfathered tags that do not match the 'langtag' production in the 896 ABNF and would otherwise be invalid are considered 'irregular' 897 grandfathered tags. With the exception of "en-GB-oed", which is a 898 variant of "en-GB", each of them, in its entirety, represents a 899 language. 901 Many of the grandfathered tags have been superseded by the subsequent 902 addition of new subtags: each superseded record contains a Preferred- 903 Value field that ought to be used to form language tags representing 904 that value. For example, the tag "art-lojban" is superseded by the 905 primary language subtag 'jbo'. 907 2.2.9. Classes of Conformance 909 Implementations sometimes need to describe their capabilities with 910 regard to the rules and practices described in this document. Tags 911 can be checked or verified in a number of ways, but two particular 912 classes of tag conformance are formally defined here. 914 A tag is considered "well-formed" if it conforms to the ABNF 915 (Section 2.1). Language tags may be well-formed in terms of syntax 916 but not valid in terms of content. However, many operations 917 involving language tags work well without knowing anything about the 918 meaning or validity of the subtags. 920 A tag is considered "valid" if it satisfies these conditions: 922 o The tag is well-formed. 924 o The tag is either in the list of grandfathered tags or all of its 925 primary language, extended language, script, region, and variant 926 subtags appear in the IANA language subtag registry as of the 927 particular registry date. 929 o There are no duplicate variant subtags. 931 o There are no duplicate singleton (extension) subtags. 933 Note that a tag's validity depends on the date of the registry used 934 to validate the tag. A more recent copy of the registry might 935 contain a subtag that an older version does not. 937 A tag is considered "valid" for a given extension (Section 3.7) (as 938 of a particular version, revision, and date) if it meets the criteria 939 for "valid" above and also satisfies this condition: 941 Each subtag used in the extension part of the tag is valid 942 according to the extension. 944 Older specifications or language tag implementations sometimes 945 reference [RFC3066]. A wider array of tags was considered 'well- 946 formed' under that document. Any tags that were valid for use under 947 RFC 3066 are both 'well-formed' and 'valid' under this document's 948 syntax; only invalid or illegal tags were well-formed by the early 949 definition but no longer are. The language tag syntax under RFC 3066 950 was: 952 obs-language-tag = primary-subtag *( "-" subtag ) 953 primary-subtag = 1*8ALPHA 954 subtag = 1*8(ALPHA / DIGIT) 956 Figure 2: RFC 3066 Language Tag Syntax 958 Subtags designated for private use as well as private-use sequences 959 introduced by the 'x' subtag are available for cases in which no 960 assigned subtags are available and registration is not a suitable 961 option. For example, one might use a tag such as "no-QQ", where 'QQ' 962 is one of a range of private-use ISO 3166-1 codes to indicate an 963 otherwise-undefined region. Users MUST NOT assign language tags that 964 use subtags that do not appear in the registry other than in private- 965 use sequences (such the subtag 'personal' in the tag "en-x- 966 personal"). Besides not being 'valid', the user also risks collision 967 with a future possible assignment or registrations. 969 Note well: although the 'Language-Tag' production appearing in this 970 document is functionally equivalent to the one in [RFC4646], it has 971 been changed to prevent certain errors in well-formedness arising 972 from the old 'grandfathered' production. 974 3. Registry Format and Maintenance 976 The IANA Language Subtag Registry ("the registry") contains a 977 comprehensive list of all of the subtags valid in language tags. 978 This allows implementers a straightforward and reliable way to 979 validate language tags. The registry will be maintained so that, 980 except for extension subtags, it is possible to validate all of the 981 subtags that appear in a language tag under the provisions of this 982 document or its revisions or successors. In addition, the meaning of 983 the various subtags will be unambiguous and stable over time. (The 984 meaning of private use subtags, of course, is not defined by the 985 registry.) 987 This section defines the registry along with the maintenance and 988 update procedures associated with it, as well as a registry for 989 extensions to language tags (Section 3.7). 991 3.1. Format of the IANA Language Subtag Registry 993 The IANA Language Subtag Registry is a machine-readable file in the 994 format described in this section, plus copies of the registration 995 forms approved in accordance with the process described in 996 Section 3.5. 998 The existing registration forms for grandfathered and redundant tags 999 taken from RFC 3066 have been maintained as part of the obsolete RFC 1000 3066 registry. The subtags added to the registry by either [RFC4645] 1001 or [draft-4645bis] do not have separate registration forms (so no 1002 forms are archived for these additions). 1004 3.1.1. File Format 1006 The registry is a [Unicode] text file and consists of a series of 1007 records in a format based on "record-jar" (described in 1008 [record-jar]). Each record, in turn, consists of a series of fields 1009 that describe the various subtags and tags. The actual registry file 1010 is encoded using the UTF-8 [RFC3629] character encoding. 1012 Each field can be considered a single, logical line of characters. 1013 Each field contains a 'field-name' and a 'field-body'. These are 1014 separated by a 'field-separator'. The field-separator is a COLON 1015 character (U+003A) plus any surrounding whitespace. Each field is 1016 terminated by the newline sequence CRLF. The text in each field MUST 1017 be in Unicode Normalization Form C (NFC). 1019 A collection of fields forms a 'record'. Records are separated by 1020 lines containing only the sequence "%%" (U+0025 U+0025). 1022 Although fields are logically a single line of text, each line of 1023 text in the file format is limited to 72 bytes in length. To 1024 accommodate this, the field-body can be split into a multiple-line 1025 representation; this is called "folding". Folding is done according 1026 to customary conventions for line-wrapping. This is typically on 1027 whitespace boundaries, but can occur between other characters when 1028 the value does not include spaces, such as when a language does not 1029 use whitespace between words. In any event, there MUST NOT be breaks 1030 inside a multibyte UTF-8 sequence nor in the middle of a combining 1031 character sequence. For more information, see [UAX14]. 1033 Although the file format uses the Unicode character set and the file 1034 itself encoded using the UTF-8 encoding, fields are restricted to the 1035 printable characters from the US-ASCII [ISO646] repertoire unless 1036 otherwise indicated in the description of a specific field 1037 (Section 3.1.2). 1039 The format of the registry is described by the following ABNF 1040 [RFC5234]. Character numbers (code points) are taken from Unicode 1041 and terminals in the ABNF productions are in terms of characters 1042 rather than bytes. 1044 registry = record *("%%" CRLF record) 1045 record = 1*field 1046 field = ( field-name field-sep field-body CRLF ) 1047 field-name = (ALPHA / DIGIT) [*(ALPHA / DIGIT / "-") (ALPHA / DIGIT)] 1048 field-sep = *SP ":" *SP 1049 field-body = *([[*SP CRLF] 1*SP] 1*CHARS) 1050 CHARS = (%x21-10FFFF) ; Unicode code points 1052 Figure 3: Registry Format ABNF 1054 The sequence '..' (U+002E U+002E) in a field-body denotes a range of 1055 values. Such a range represents all subtags of the same length that 1056 are in alphabetic or numeric order within that range, including the 1057 values explicitly mentioned. For example 'a..c' denotes the values 1058 'a', 'b', and 'c' and '11..13' denotes the values '11', '12', and 1059 '13'. 1061 All fields whose field-body contains a date value use the "full-date" 1062 format specified in [RFC3339]. For example: "2004-06-28" represents 1063 June 28, 2004, in the Gregorian calendar. 1065 3.1.2. Record and Field Definitions 1067 There are three types of records in the registry: "File-Date", 1068 "Subtag", and "Tag". 1070 The first record in the registry is always the "File-Date" record. 1071 This record occurs only once in the file and contains a single field 1072 whose field-name is "File-Date". The field-body of this record 1073 contains a date (see Section 5.1) making it possible to easily 1074 recognize different versions of the registry. 1076 File-Date: 2004-06-28 1077 %% 1079 Figure 4: Example of the File-Date Record 1081 Subsequent records contain multiple fields and represent information 1082 about either subtags or tags. Both types of record have identical 1083 structure, except that "Subtag" records contain a field with a field- 1084 name of "Subtag", while, unsurprisingly, "Tag" records contain a 1085 field with a field-name of "Tag". Field-names MUST NOT occur more 1086 than once per record, with the exception of the 'Description', 1087 'Comments', and sometimes the 'Prefix' field. 1089 Each record MUST contain at least one of each of the following 1090 fields: 1092 o 'Type' 1094 * Type's field-body MUST consist of one of the following strings: 1095 "language", "extlang", "script", "region", "variant", 1096 "grandfathered", and "redundant", and denotes the type of tag 1097 or subtag. 1099 o Either 'Subtag' or 'Tag' 1101 * Subtag's field-body contains the subtag being defined. This 1102 field MUST appear in all records whose 'Type' has one of these 1103 values: "language", "extlang", "script", "region", or 1104 "variant". 1106 * Tag's field-body contains a complete language tag. This field 1107 MUST appear in all records whose 'Type' has one of these 1108 values: "grandfathered" or "redundant". If the 'Type' is 1109 "grandfathered", then the 'Tag' field-body will be one of the 1110 tags listed in either the 'regular' or 'irregular' production 1111 in found in Section 2.1. 1113 o Description 1115 * Description's field-body contains a non-normative description 1116 of the subtag or tag. 1118 o Added 1120 * Added's field-body contains the date the record was registered 1121 or, in the case of grandfathered or redundant tags, the date 1122 the corresponding tag was registered under the rules of 1123 [RFC1766] or [RFC3066]. 1125 Each record MAY also contain the following fields: 1127 o Deprecated 1129 * Deprecated's field-body contains the date the record was 1130 deprecated. In some cases this value is earlier than that of 1131 the 'Added' field in the same record. That is, the date of 1132 deprecation preceded the addition of the record to the 1133 registry. 1135 o Preferred-Value 1137 * Preferred-Value's field body contains a canonical mapping from 1138 this record's value to a modern equivalent that is preferred in 1139 its place. Depending on the value of the 'Type' field, this 1140 value can take different forms: 1142 + For fields of type 'language', 'Preferred-Value' contains 1143 the primary language subtag that is preferred when forming 1144 the language tag. 1146 + For fields of type 'script', 'region', or 'variant', 1147 'Preferred-Value' contains the subtag of the same type that 1148 is preferred for forming the language tag. 1150 + For fields of type 'extlang', 'grandfathered', or 1151 'redundant', 'Preferred-Value' contains an "extended 1152 language range" ([RFC4647]) that is preferred for forming 1153 the language tag. That is, the preferred language tag will 1154 contain, in order, each of the subtags that appears in the 1155 Preferred-Value; additional fields can be included in a 1156 language tag as described elsewhere in this document. For 1157 example, the replacement for the grandfathered tag "zh-min- 1158 nan" (Min Nan Chinese) is "nan", which can be used as the 1159 basis for tags such as "nan-Hant" or "nan-TW" (note that the 1160 extended language subtag form such as "zh-nan-Hant" or "zh- 1161 nan-TW" can also be used). 1163 o Prefix 1164 * Prefix's field-body contains a valid language tag that is 1165 RECOMMENDED as one possible prefix to this record's subtag. 1166 This field MAY appear in records whose 'Type' field-body is 1167 either 'extlang' or 'variant' (it MUST NOT appear in any other 1168 record type). 1170 o Suppress-Script 1172 * Suppress-Script's field-body contains a script subtag that 1173 SHOULD NOT be used to form language tags with the associated 1174 primary or extended language subtag. This field MUST appear 1175 only in records whose 'Type' field-body is "language" or 1176 "extlang". See Section 4.1. 1178 o Macrolanguage 1180 * Macrolanguage's field-body contains a primary language subtag 1181 defined by ISO 639 as the "macrolanguage" that encompasses this 1182 language subtag. This field MUST appear only in records whose 1183 'Type' field-body is either "language" or "extlang". 1185 o Scope 1187 * Scope's field-body contains information about a primary or 1188 extended language subtag indicating the type of language code 1189 according to ISO 639. The values permitted in this field are 1190 "macrolanguage", "collection", "special" and "private-use". 1191 This field only appears in records whose 'Type' field-body is 1192 either "language" or "extlang". When this field is omitted, 1193 the language is an individual language. 1195 o Comments 1197 * Comments's field-body contains additional information about the 1198 subtag, as deemed appropriate for understanding the registry 1199 and implementing language tags using the subtag or tag. 1201 Future versions of this document might add additional fields to the 1202 registry; implementations SHOULD ignore fields found in the registry 1203 that are not defined in this document. 1205 3.1.3. Type Field 1207 The field 'Type' contains the string identifying the record type it 1208 appears in. Values for the 'Type' field-body are: "language" 1209 (Section 2.2.1); "extlang" (Section 2.2.2); "script" (Section 2.2.3); 1210 "region" (Section 2.2.4); "variant" (Section 2.2.5); "grandfathered" 1211 or "redundant" (Section 2.2.8). 1213 3.1.4. Subtag and Tag Fields 1215 The field 'Subtag' contains the subtag defined in the record. The 1216 field 'Tag' appears in records whose 'Type' is either 'grandfathered' 1217 or 'redundant' and contains a tag registered under [RFC3066]. 1219 The 'Subtag' field-body MUST follow the casing conventions described 1220 in Section 2.1.1. All subtags use lowercase letters in the field- 1221 body, with two exceptions: 1223 Subtags whose 'Type' field is 'script' (in other words, subtags 1224 defined by ISO 15924) MUST use titlecase. 1226 Subtags whose 'Type' field is 'region' (in other words, the non- 1227 numeric region subtags defined by ISO 3166-1) MUST use all 1228 uppercase. 1230 The 'Tag' field-body MUST be formatted according to the rules 1231 described in Section 2.1.1. 1233 3.1.5. Description Field 1235 The field 'Description' contains a description of the tag or subtag 1236 in the record. The 'Description' field MAY appear more than once per 1237 record. The 'Description' field MAY include the full range of 1238 Unicode characters. At least one of the 'Description' fields MUST be 1239 written or transcribed into the Latin script; additional 1240 'Description' fields MAY be in any script or language. 1242 The 'Description' field is used for identification purposes. 1243 Descriptions SHOULD contain all and only that information necessary 1244 to distinguish one subtag from others that it might be confused with. 1245 They are not intended to provide general background information, nor 1246 to provide all possible alternate names or designations. 1247 'Description' fields don't necessarily represent the actual native 1248 name of the item in the record, nor are any of the descriptions 1249 guaranteed to be in any particular language (such as English or 1250 French, for example). 1252 Descriptions in the registry that correspond to ISO 639, ISO 15924, 1253 ISO 3166-1, or UN M.49 codes are intended only to indicate the 1254 meaning of that identifier as defined in the source standard at the 1255 time it was added to the registry or as subsequently modified, within 1256 the bounds of the stability rules (Section 3.4), via subsequent 1257 registration. The 'Description' does not replace the content of the 1258 source standard itself. 'Description' fields are not intended to be 1259 the localized English names for the subtags. Localization or 1260 translation of language tag and subtag descriptions is out of scope 1261 of this document. 1263 For subtags taken from a source standard (such as ISO 639 or ISO 1264 15924), the 'Description' fields in the record are also initially 1265 taken from that source standard. Multiple descriptions in the source 1266 standard are split into separate 'Description' fields. The source 1267 standard's descriptions MAY be edited or modified, either prior to 1268 insertion or via the registration process, and additional or 1269 extraneous descriptions omitted or removed. Each 'Description' field 1270 MUST be unique within the record in which it appears and formatting 1271 variations of the same description SHOULD NOT occur in that specific 1272 record. For example, while the ISO 639-1 code 'fy' has both the 1273 description "Western Frisian" and the description "Frisian, Western" 1274 in that standard, only one of these descriptions appears in the 1275 registry. 1277 To help ensure that users do not become confused about which subtag 1278 to use, 'Description' fields assigned to a record of any specific 1279 type ('language', 'extlang', 'script', and so on) MUST be unique 1280 within that given record type with the following exception: if a 1281 particular 'Description' field occurs in multiple records of a given 1282 type, then at most one of the records can omit the 'Deprecated' 1283 field; all deprecated records that share a 'Description' MUST have 1284 the same 'Preferred-Value'; and all non-deprecated records MUST be 1285 that 'Preferred-Value'. This means that two records of the same type 1286 that share a 'Description' are also semantically equivalent and no 1287 more than one record with a given 'Description' is preferred for that 1288 meaning. 1290 For example, consider the 'language' subtags 'zza' (Zaza) and 'diq' 1291 (Dimli). It so happens that 'zza' is a macrolanguage enclosing 'diq' 1292 and thus also has a description in ISO 639-3 of "Dimli". This 1293 description was edited to read "Dimli (macrolanguage)" in the 1294 registry record for 'zza' to prevent a collision. 1296 By contrast, the subtags 'he' and 'iw' share a 'Description' value of 1297 "Hebrew"; this is permitted because 'iw' is deprecated and its 1298 'Preferred-Value' is 'he'. 1300 For fields of type 'language', the first 'Description' field 1301 appearing in the Registry corresponds whenever possible to the 1302 Reference Name assigned by ISO 639-3. This helps facilitate cross- 1303 referencing between ISO 639 and the registry. 1305 When creating or updating a record due to the action of one of the 1306 source standards, the Language Subtag Reviewer MAY edit descriptions 1307 to correct irregularities in formatting (such as misspellings, 1308 inappropriate apostrophes or other punctuation, or excessive or 1309 missing spaces) prior to submitting the proposed record to the 1310 ietf-languages@iana.org list for consideration. 1312 3.1.6. Deprecated Field 1314 The field 'Deprecated' contains the date the record was deprecated 1315 and MAY be added, changed, or removed from any record via the 1316 maintenance process described in Section 3.3 or via the registration 1317 process described in Section 3.5. Usually, the addition of a 1318 'Deprecated' field is due to the action of one of the standards 1319 bodies, such as ISO 3166, withdrawing a code. Although valid in 1320 language tags, subtags and tags with a 'Deprecated' field are 1321 deprecated and validating processors SHOULD NOT generate these 1322 subtags. Note that a record that contains a 'Deprecated' field and 1323 no corresponding 'Preferred-Value' field has no replacement mapping. 1325 In some historical cases, it might not have been possible to 1326 reconstruct the original deprecation date. For these cases, an 1327 approximate date appears in the registry. Some subtags and some 1328 grandfathered or redundant tags were deprecated before the initial 1329 creation of the registry. The exact rules for this appear in Section 1330 2 of [RFC4645]. Note that these records have a 'Deprecated' field 1331 with an earlier date then the corresponding 'Added' field! 1333 3.1.7. Preferred-Value Field 1335 The field 'Preferred-Value' contains a mapping between the record in 1336 which it appears and another tag or subtag (depending on the record's 1337 'Type'). The value in this field is used for canonicalization (see 1338 Section 4.5). In cases where the subtag or tag also has a 1339 'Deprecated' field, then the 'Preferred-Value' is RECOMMENDED as the 1340 best choice to represent the value of this record when selecting a 1341 language tag. 1343 Records containing a Preferred-Value fall into one of these four 1344 groups: 1346 1. ISO 639 language codes that were later withdrawn in favor of 1347 other codes. These values are mostly a historical curiosity. 1348 The 'he'/'iw' pairing above is an example of this. 1350 2. Subtags (with types other than language or extlang) taken from 1351 codes or values that have been withdrawn in favor of a new code. 1352 In particular, this applies to region subtags taken from ISO 1353 3166-1, because sometimes a country will change its name or 1354 administration in such a way that warrants a new region code. In 1355 some cases, countries have reverted to an older name, which might 1356 already be encoded. For example, the subtag 'ZR' (Zaire) was 1357 replaced by the subtag 'CD' (Democratic Republic of the Congo) 1358 when that country's name was changed. 1360 3. Tags or subtags that have become obsolete because the values they 1361 represent were later encoded. Many of the grandfathered or 1362 redundant tags were later encoded by ISO 639, for example, and 1363 fall into this grouping. For example, "i-klingon" was deprecated 1364 when the subtag 'tlh' was added. The record for "i-klingon" has 1365 a 'Preferred-Value' of 'tlh'. 1367 4. Extended language subtags always have a mapping to their 1368 identical primary language subtag. For example, the extended 1369 language subtag 'yue' (Cantonese) can be used to form the tag 1370 "zh-yue". It has a Preferred-Value mapping to the primary 1371 language subtag 'yue', meaning that a tag such as 1372 "zh-yue-Hant-HK" can be canonicalized to "yue-Hant-HK". 1374 Records other than those of type 'extlang' that contain a 'Preferred- 1375 Value' field MUST also have a 'Deprecated' field. This field 1376 contains the date on which the tag or subtag was deprecated in favor 1377 of the preferred value. 1379 For records of type 'extlang', the 'Preferred-Value' field appears 1380 without a corresponding 'Deprecated' field. An implementation MAY 1381 ignore these preferred value mappings, although if it ignores the 1382 mapping, it SHOULD do so consistently. It SHOULD also treat the 1383 Preferred-Value as equivalent to the mapped item. For example, the 1384 tags "zh-yue-Hant-HK" and "yue-Hant-HK" are semantically equivalent 1385 and ought to be treated as if they were the same tag. 1387 Occasionally the deprecated code is preferred in certain contexts. 1388 For example, both "iw" and "he" can be used in the Java programming 1389 language, but "he" is converted on input to "iw", which is thus the 1390 canonical form in Java. 1392 'Preferred-Value' mappings in records of type 'region' sometimes do 1393 not represent exactly the same meaning as the original value. There 1394 are many reasons for a country code to be changed, and the effect 1395 this has on the formation of language tags will depend on the nature 1396 of the change in question. For example, the region subtag 'YD' 1397 (Democratic Yemen) was deprecated in favor of the subtag 'YE' (Yemen) 1398 when those two countries unified in 1990. 1400 A 'Preferred-Value' MAY be added to, changed, or removed from records 1401 according to the rules in Section 3.3. Addition, modification, or 1402 removal of a 'Preferred-Value' field in a record does not imply that 1403 content using the affected subtag needs to be retagged. 1405 The 'Preferred-Value' fields in records of type "grandfathered" and 1406 "redundant" each contain an "extended language range" ([RFC4647]) 1407 that is strongly RECOMMENDED for use in place of the record's value. 1408 In many cases, these mappings were created via deprecation of the 1409 tags during the period before [RFC4646] was adopted. For example, 1410 the tag "no-nyn" was deprecated in favor of the ISO 639-1-defined 1411 language code 'nn'. 1413 The 'Preferred-Value' field in subtag records of type "extlang" also 1414 contains an "extended language range". This allows the subtag to be 1415 deprecated in favor of either a single primary language subtag or a 1416 new language-extlang sequence. 1418 Usually the addition, removal, or change of a Preferred-Value field 1419 for a subtag is done to reflect changes in one of the source 1420 standards. For example, if an ISO 3166-1 region code is deprecated 1421 in favor of another code, that SHOULD result in the addition of a 1422 Preferred-Value field. 1424 Changes to one subtag can affect other subtags as well: when 1425 proposing changes to the registry, the Language Subtag Reviewer MUST 1426 review the registry for such effects and propose the necessary 1427 changes using the process in Section 3.5, although anyone MAY request 1428 such changes. For example: 1430 Suppose that subtag 'XX' has a Preferred-Value of 'YY'. If 'YY' 1431 later changes to have a Preferred-Value of 'ZZ', then the 1432 Preferred-Value for 'XX' MUST also change to be 'ZZ'. 1434 Suppose that a registered language subtag 'dialect' represents a 1435 language not yet available in any part of ISO 639. The later 1436 addition of a corresponding language code in ISO 639 SHOULD result 1437 in the addition of a Preferred-Value for 'dialect'. 1439 3.1.8. Prefix Field 1441 The field 'Prefix' contains a valid language tag that is RECOMMENDED 1442 as one possible prefix to this record's subtag, perhaps with other 1443 subtags. That is, when including an extended language or a variant 1444 subtag that has at least one 'Prefix' in a language tag, the 1445 resulting tag SHOULD match at least one of the subtag's 'Prefix' 1446 fields using the "Extended Filtering" algorithm (see [RFC4647]) and 1447 each of the subtags in that 'Prefix' SHOULD appear before the subtag 1448 itself. 1450 The 'Prefix' field MUST appear exactly once in a record of type 1451 'extlang'. The 'Prefix' field MAY appear multiple times (or not at 1452 all) in records of type 'variant'. Additional fields of this type 1453 MAY be added to a 'variant' record via the registration process, 1454 provided the 'variant' record already has at least one 'Prefix' 1455 field. 1457 Each 'Prefix' field indicates a particular sequence of subtags that 1458 form a meaningful tag with this subtag. For example, the extended 1459 language subtag 'cmn' (Mandarin Chinese) only makes sense with its 1460 prefix 'zh' (Chinese). Similarly, 'rozaj' (Resian, a dialect of 1461 Slovenian) would be appropriate when used with its prefix 'sl' 1462 (Slovenian), while tags such as "is-1994" are not appropriate (and 1463 probably not meaningful). Although the prefix for 'rozaj' is "sl", 1464 other subtags might appear between them. For example, the tag "sl- 1465 IT-rozaj" (Slovenian, Italy, Resian) matches the Prefix "sl". 1467 The 'Prefix' also indicates when variant subtags make sense when used 1468 together (many that otherwise share a 'Prefix' are mutually 1469 exclusive) and what the relative ordering of variants is supposed to 1470 be. For example, the variant '1994' (Standardized Resian 1471 orthography) has several 'Prefix' fields in the registry ("sl-rozaj", 1472 "sl-rozaj-biske", "sl-rozaj-njiva", "sl-rozaj-osojs", and "sl-rozaj- 1473 solba"). This not only indicates that '1994' is appropriate to use 1474 with each of these five Resian variant subtags ('rozaj', 'biske', 1475 'njiva', 'osojs', and 'solba') but also that it SHOULD appear 1476 following any of these variants in a tag. Thus, the language tag 1477 ought to take the form "sl-rozaj-biske-1994" rather than "sl-1994- 1478 rozaj-biske" or "sl-rozaj-1994-biske". 1480 If a record includes no 'Prefix' field, a 'Prefix' field MUST NOT be 1481 added to the record at a later date. Otherwise, changes (additions, 1482 deletions, or modifications) to the set of 'Prefix' fields MAY be 1483 registered, as long as they strictly widen the range of language tags 1484 that are recommended. For example, the Prefix "be-Latn" (Belarusian, 1485 Latin script) could be replaced by the Prefix "be" (Belarusian) but 1486 not by the Prefix "ru-Latn" (Russian, Latin script) or the Prefix 1487 "be-Latn-BY" (Belarusian, Latin script, Belarus), since these latter 1488 either change or narrow the range of suggested tags. 1490 The field-body of the 'Prefix' field MUST NOT conflict with any 1491 'Prefix' already registered for a given record. Such a conflict 1492 would occur when no valid tag could be constructed that would contain 1493 the prefix, such as when two subtags each have a 'Prefix' that 1494 contains the other subtag. For example, suppose that the subtag 1495 'avariant' has the prefix "es-bvariant". Then the subtag 'bvariant' 1496 cannot given the prefix 'avariant', for that would require a tag of 1497 the form "es-avariant-bvariant-avariant", which would not be valid. 1499 3.1.9. Suppress-Script Field 1501 The field 'Suppress-Script' contains a script subtag (whose record 1502 appears in the registry). The field 'Suppress-Script' MUST appear 1503 only in records whose 'Type' field-body is either 'language' or 1504 'extlang'. This field MUST NOT appear more than one time in a 1505 record. 1507 This field indicates a script used to write the overwhelming majority 1508 of documents for the given language. The subtag for such a script 1509 therefore adds no distinguishing information to a language tag and 1510 thus SHOULD NOT be used for most documents in that language. 1511 Omitting the script subtag indicated by this field helps ensure 1512 greater compatibility between the language tags generated according 1513 to the rules in this document and language tags and tag processors or 1514 consumers based on RFC 3066. For example, virtually all Icelandic 1515 documents are written in the Latin script, making the subtag 'Latn' 1516 redundant in the tag "is-Latn". 1518 Many language subtag records do not have a 'Suppress-Script' field. 1519 The lack of a 'Suppress-Script' might indicate that the language is 1520 customarily written in more than one script or that the language is 1521 not customarily written at all. It might also mean that sufficient 1522 information was not available when the record was created and thus 1523 remains a candidate for future registration. 1525 3.1.10. Macrolanguage Field 1527 The field 'Macrolanguage' contains a primary language subtag (whose 1528 record appears in the registry). This field indicates a language 1529 that encompasses this subtag's language according to assignments made 1530 by ISO 639-3. 1532 ISO 639-3 labels some languages in the registry as "macrolanguages". 1533 ISO 639-3 defines the term "Macrolanguage" to mean "clusters of 1534 closely-related language varieties that [...] can be considered 1535 distinct individual languages, yet in certain usage contexts a single 1536 language identity for all is needed". These correspond to codes 1537 registered in ISO 639-2 as individual languages that were found to 1538 correspond to more than one language in ISO 639-3. 1540 A language contained within a macrolanguage is called an "encompassed 1541 language". The record for each encompassed language contains a 1542 'Macrolanguage' field in the registry; the macrolanguages themselves 1543 are not specially marked. Note that some encompassed languages have 1544 ISO 639-1 or ISO 639-2 codes. 1546 The Macrolanguage field can only occur in records of type 'language' 1547 or 'extlang'. Only values assigned by ISO 639-3 will be considered 1548 for inclusion. Macrolanguage fields MAY be added or removed via the 1549 normal registration process whenever ISO 639-3 defines new values or 1550 withdraws old values. Macrolanguages are informational, and MAY be 1551 removed or changed if ISO 639-3 changes the values. For more 1552 information on the use of this field and choosing between 1553 macrolanguage and encompassed language subtags, see Section 4.1.1. 1555 For example, the language subtags 'nb' (Norwegian Bokmal) and 'nn' 1556 (Norwegian Nynorsk) each have a Macrolanguage entry of 'no' 1557 (Norwegian). For more information see Section 4.1. 1559 3.1.11. Scope Field 1561 The field 'Scope' contains classification information about a primary 1562 or extended language subtag derived from ISO 639. Most languages 1563 have a scope of 'individual', which means that the language is not a 1564 macrolanguage, collection, special code, or private use. That is, it 1565 is what one would normally consider to be 'a language'. Any primary 1566 or extended language subtag that has no 'Scope' field is an 1567 individual language. 1569 Scope information can sometimes be helpful in selecting language 1570 tags, since it indicates the purpose or "scope" of the code 1571 assignment within ISO 639. The available values are: 1573 o 'macrolanguage' - Indicates a macrolanguage as defined by ISO 1574 639-3 (see above (Section 3.1.10)). A macrolanguage is a cluster 1575 of closely-related languages that are sometimes considered to be a 1576 single language. 1578 o 'collection' - Indicates a subtag that represents a collection of 1579 languages, typically related by some type of historical, 1580 geographical, or linguistic association. Unlike a macrolanguage, 1581 a collection can contain languages that are only loosely related 1582 and a collection cannot be used interchangeably with languages 1583 that belong to it. 1585 o 'special' - Indicates a special language code. These are subtags 1586 used for identifying linguistic attributes not particularly 1587 associated with a concrete language. These include codes for when 1588 the language is undetermined or for non-linguistic content. 1590 o 'private-use' - Indicates a code reserved for private use in the 1591 underlying standard. Subtags with this scope can be used to 1592 indicate a primary language for which no ISO 639 or registered 1593 assignment exists. 1595 The Scope field MAY appear in records of type 'language' or 1596 'extlang'. Note that many of the prefixes for extended language 1597 subtags will have a Scope of 'macrolanguage' (although some will not) 1598 and that many languages that have a Scope of 'macrolanguage' will 1599 have extended language subtags associated with them. 1601 The Scope field MAY be added, modified, or removed via the 1602 registration process, provided the change mirrors changes by ISO 639 1603 to the assignment's classification. Such a change is expected to be 1604 rare. 1606 For example, the primary language subtag 'zh' (Chinese) has a Scope 1607 of 'macrolanguage', while its enclosed language 'nan' (Min Nan 1608 Chinese) has a Scope of 'individual'. The special value 'und' 1609 (Undetermined) has a Scope of 'special'. The ISO 639-5 collection 1610 'gem' (Germanic languages) has a Scope of 'collection'. 1612 3.1.12. Comments Field 1614 The field 'Comments' contains additional information about the record 1615 and MAY appear more than once per record. The field-body MAY include 1616 the full range of Unicode characters and is not restricted to any 1617 particular script. This field MAY be inserted or changed via the 1618 registration process and no guarantee of stability is provided. 1620 The content of this field is not restricted, except by the need to 1621 register the information, the suitability of the request, and by 1622 reasonable practical size limitations. The primary reason for the 1623 'Comments' field is subtag identification: to help distinguish the 1624 subtag from others with which it might be confused as an aid to 1625 usage. Large amounts of information about the use, history, or 1626 general background of a subtag are frowned upon, as these generally 1627 belong in a registration request rather than in the registry. 1629 3.2. Language Subtag Reviewer 1631 The Language Subtag Reviewer moderates the ietf-languages@iana.org 1632 mailing list, responds to requests for registration, and performs the 1633 other registry maintenance duties described in Section 3.3. Only the 1634 Language Subtag Reviewer is permitted to request IANA to change, 1635 update, or add records to the Language Subtag Registry. The Language 1636 Subtag Reviewer MAY delegate list moderation and other clerical 1637 duties as needed. 1639 The Language Subtag Reviewer is appointed by the IESG for an 1640 indefinite term, subject to removal or replacement at the IESG's 1641 discretion. The IESG will solicit nominees for the position (upon 1642 adoption of this document or upon a vacancy) and then solicit 1643 feedback on the nominees' qualifications. Qualified candidates 1644 should be familiar with BCP 47 and its requirements; be willing to 1645 fairly, responsively, and judiciously administer the registration 1646 process; and be suitably informed about the issues of language 1647 identification so that the reviewer can assess the claims and draw 1648 upon the contributions of language experts and subtag requesters. 1650 The subsequent performance or decisions of the Language Subtag 1651 Reviewer MAY be appealed to the IESG under the same rules as other 1652 IETF decisions (see [RFC2026]). The IESG can reverse or overturn the 1653 decisions of the Language Subtag Reviewer, provide guidance, or take 1654 other appropriate actions. 1656 3.3. Maintenance of the Registry 1658 Maintenance of the registry requires that, as codes are assigned or 1659 withdrawn by ISO 639, ISO 15924, ISO 3166, and UN M.49, the Language 1660 Subtag Reviewer MUST evaluate each change and determine the 1661 appropriate course of action according to the rules in this document. 1662 Such updates follow the registration process described in 1663 Section 3.5. Usually the Language Subtag Reviewer will start the 1664 process for the new or updated record by filling in the registration 1665 form and submitting it. If a change to one of these standards takes 1666 place and the Language Subtag Reviewer does not do this in a timely 1667 manner, then any interested party MAY submit the form. Thereafter 1668 the registration process continues normally. 1670 Note that some registrations affect other subtags--perhaps more than 1671 one--as when a region subtag is being deprecated in favor of a new 1672 value. The Language Subtag Reviewer is responsible for ensuring that 1673 any such changes are properly registered, with each change requiring 1674 its own registration form. 1676 The Language Subtag Reviewer MUST ensure that new subtags meet the 1677 requirements elsewhere in this document (and most especially in 1678 Section 3.4) or submit an appropriate registration form for an 1679 alternate subtag as described in that section. Each individual 1680 subtag affected by a change MUST be sent to the 1681 ietf-languages@iana.org list with its own registration form and in a 1682 separate message. 1684 3.4. Stability of IANA Registry Entries 1686 The stability of entries and their meaning in the registry is 1687 critical to the long-term stability of language tags. The rules in 1688 this section guarantee that a specific language tag's meaning is 1689 stable over time and will not change. 1691 These rules specifically deal with how changes to codes (including 1692 withdrawal and deprecation of codes) maintained by ISO 639, ISO 1693 15924, ISO 3166, and UN M.49 are reflected in the IANA Language 1694 Subtag Registry. Assignments to the IANA Language Subtag Registry 1695 MUST follow the following stability rules: 1697 1. Values in the fields 'Type', 'Subtag', 'Tag', and 'Added' MUST 1698 NOT be changed and are guaranteed to be stable over time. 1700 2. Values in the fields 'Preferred-Value' and 'Deprecated' MAY be 1701 added, altered, or removed via the registration process. These 1702 changes SHOULD be limited to changes necessary to mirror changes 1703 in one of the underlying standards (ISO 639, ISO 15924, ISO 1704 3166-1, or UN M.49) and typically alteration or removal of a 1705 Preferred-Value is limited specifically to region codes. 1707 3. Values in the 'Description' field MUST NOT be changed in a way 1708 that would invalidate any existing tags. The description MAY be 1709 broadened somewhat in scope, changed to add information, or 1710 adapted to the most common modern usage. For example, countries 1711 occasionally change their names; a historical example of this 1712 would be "Upper Volta" changing to "Burkina Faso". 1714 4. Values in the field 'Prefix' MAY be added to existing records of 1715 type 'variant' via the registration process, provided the 1716 'variant' already has at least one 'Prefix'. A 'Prefix' field 1717 SHALL NOT be registered for any 'variant' that has no existing 1718 'Prefix' field. If a prefix is added to a variant record, 1719 'Comment' fields MAY be used to explain different usages with 1720 the various prefixes. 1722 5. Values in the field 'Prefix' in records of type 'variant' MAY 1723 also be modified, so long as the modifications broaden the set 1724 of prefixes. That is, a prefix MAY be replaced by one of its 1725 own prefixes. For example, the prefix "en-US" could be replaced 1726 by "en", but not by the prefixes "en-Latn", "fr", or "en-US- 1727 boont". If one of those prefix values were needed, it would 1728 have to be separately registered. 1730 6. Values in the field 'Prefix' in records of type 'extlang' MUST 1731 NOT be added, modified, or removed. 1733 7. The field 'Prefix' MUST NOT be removed from any record in which 1734 it appears. This field SHOULD be included in the initial 1735 registration of any records of type 'variant' and MUST be 1736 included in any records of type 'extlang'. 1738 8. The field 'Comments' MAY be added, changed, modified, or removed 1739 via the registration process or any of the processes or 1740 considerations described in this section. 1742 9. The field 'Suppress-Script' MAY be added or removed via the 1743 registration process. 1745 10. The field 'Macrolanguage' MAY be added or removed via the 1746 registration process, but only in response to changes made by 1747 ISO 639. The Macrolanguage field appears whenever a language 1748 has a corresponding Macrolanguage in ISO 639. That is, the 1749 Macrolanguage fields in the registry exactly match those of ISO 1750 639. No other macrolanguage mappings will be considered for 1751 registration. 1753 11. The field 'Scope' MAY be added or removed from a primary or 1754 extended language subtag after initial registration, and it MAY 1755 be modified in order to match any changes made by ISO 639. 1756 Changes to the 'Scope' field MUST mirror changes made by ISO 1757 639. Note that primary or extended language subtags whose 1758 records do not contain a 'Scope' field (that is, most of them) 1759 are individual languages as described in Section 3.1.11. 1761 12. Primary and extended language subtags (other than independently 1762 registered values created using the registration process) are 1763 created according to the assignments of the various parts of ISO 1764 639, as follows: 1766 A. Codes assigned by ISO 639-1 that do not conflict with 1767 existing two-letter primary language subtags and which have 1768 no corresponding three-letter primary defined in the 1769 registry are entered into the IANA registry as new records 1770 of type 'language'. Note that languages given an ISO 639-1 1771 code cannot be given extended language subtags, even if 1772 encompassed by a macrolanguage. 1774 B. Codes assigned by ISO 639-3 or ISO 639-5 that do not 1775 conflict with existing three-letter primary language subtags 1776 and which do not have ISO 639-1 codes assigned (or expected 1777 to be assigned) are entered into the IANA registry as new 1778 records of type 'language'. Note that these two standards 1779 now comprise a superset of ISO 639-2 codes. Codes that have 1780 a defined "macrolanguage" mapping at the time of their 1781 registration MUST contain a "Macrolanguage" field. 1783 C. Codes assigned by ISO 639-3 MAY also be considered for an 1784 extended language subtag registration. Note that they MUST 1785 be assigned a primary language subtag record of type 1786 'language' even when an 'extlang' record is proposed. When 1787 considering extended language subtag assignment, these 1788 criteria apply: 1790 1. If a language has a macrolanguage mapping, and that 1791 macrolanguage has other encompassed languages that are 1792 assigned extended language subtags, then the new 1793 language SHOULD have an 'extlang' record assigned to it 1794 as well. For example, any language with a macrolanguage 1795 of 'zh' or 'ar' would be assigned an 'extlang' record. 1797 2. 'Extlang' records SHOULD NOT be created for languages if 1798 other languages encompassed by the macrolanguage do not 1799 also include 'extlang' records. For example, if a new 1800 Serbo-Croatian ('sh') language were registered, it would 1801 not get an extlang record because other languages 1802 encompassed such as Serbian ('sr') do not include one in 1803 the registry. 1805 3. Sign languages SHOULD have an 'extlang' record with a 1806 'Prefix' of 'sgn'. 1808 4. 'Extlang' records MUST NOT be created for items already 1809 in the registry. Extended language subtags will only be 1810 considered at the time of initial registration. 1812 5. Extended language subtag records MUST include the fields 1813 'Prefix' and 'Preferred-Value' with field-values 1814 assigned as described in Section 2.2.2. 1816 D. Any other codes assigned by ISO 639-2 that do not conflict 1817 with existing three-letter primary or extended language 1818 subtags and which do not have ISO 639-1 two-letter codes 1819 assigned are entered into the IANA registry as new records 1820 of type 'language'. This type of registration is not 1821 supposed to occur in the future. 1823 13. Codes assigned by ISO 15924 and ISO 3166-1 that do not conflict 1824 with existing subtags of the associated type and whose meaning 1825 is not the same as an existing subtag of the same type are 1826 entered into the IANA registry as new records. 1828 14. Codes assigned by ISO 639, ISO 15924, or ISO 3166-1 that are 1829 withdrawn by their respective maintenance or registration 1830 authority remain valid in language tags. A 'Deprecated' field 1831 containing the date of withdrawal MUST be added to the record. 1832 If a new record of the same type is added that represents a 1833 replacement value, then a 'Preferred-Value' field MAY also be 1834 added. The registration process MAY be used to add comments 1835 about the withdrawal of the code by the respective standard. 1837 For example: the region code 'TL' was assigned to the country 1838 'Timor-Leste', replacing the code 'TP' (which was assigned to 1839 'East Timor' when it was under administration by Portugal). 1840 The subtag 'TP' remains valid in language tags, but its 1841 record contains the 'Preferred-Value' of 'TL' and its field 1842 'Deprecated' contains the date the new code was assigned 1843 ('2004-07-06'). 1845 15. Codes assigned by ISO 639, ISO 15924, or ISO 3166-1 that 1846 conflict with existing subtags of the associated type, including 1847 subtags that are deprecated, MUST NOT be entered into the 1848 registry. The following additional considerations apply to 1849 subtag values that are reassigned: 1851 A. For ISO 639 codes, if the newly assigned code's meaning is 1852 not represented by a subtag in the IANA registry, the 1853 Language Subtag Reviewer, as described in Section 3.5, SHALL 1854 prepare a proposal for entering in the IANA registry as soon 1855 as practical a registered language subtag as an alternate 1856 value for the new code. The form of the registered language 1857 subtag will be at the discretion of the Language Subtag 1858 Reviewer and MUST conform to other restrictions on language 1859 subtags in this document. 1861 B. For all subtags whose meaning is derived from an external 1862 standard (that is, by ISO 639, ISO 15924, ISO 3166-1, or UN 1863 M.49), if a new meaning is assigned to an existing code and 1864 the new meaning broadens the meaning of that code, then the 1865 meaning for the associated subtag MAY be changed to match. 1866 The meaning of a subtag MUST NOT be narrowed, however, as 1867 this can result in an unknown proportion of the existing 1868 uses of a subtag becoming invalid. Note: ISO 639 1869 registration authority (RA) has adopted a similar stability 1870 policy. 1872 C. For ISO 15924 codes, if the newly assigned code's meaning is 1873 not represented by a subtag in the IANA registry, the 1874 Language Subtag Reviewer, as described in Section 3.5, SHALL 1875 prepare a proposal for entering in the IANA registry as soon 1876 as practical a registered variant subtag as an alternate 1877 value for the new code. The form of the registered variant 1878 subtag will be at the discretion of the Language Subtag 1879 Reviewer and MUST conform to other restrictions on variant 1880 subtags in this document. 1882 D. For ISO 3166-1 codes, if the newly assigned code's meaning 1883 is associated with the same UN M.49 code as another 'region' 1884 subtag, then the existing region subtag remains as the 1885 preferred value for that region and no new entry is created. 1886 A comment MAY be added to the existing region subtag 1887 indicating the relationship to the new ISO 3166-1 code. 1889 E. For ISO 3166-1 codes, if the newly assigned code's meaning 1890 is associated with a UN M.49 code that is not represented by 1891 an existing region subtag, then the Language Subtag 1892 Reviewer, as described in Section 3.5, SHALL prepare a 1893 proposal for entering the appropriate UN M.49 country code 1894 as an entry in the IANA registry. 1896 F. For ISO 3166-1 codes, if there is no associated UN numeric 1897 code, then the Language Subtag Reviewer SHALL petition the 1898 UN to create one. If there is no response from the UN 1899 within ninety days of the request being sent, the Language 1900 Subtag Reviewer SHALL prepare a proposal for entering in the 1901 IANA registry as soon as practical a registered variant 1902 subtag as an alternate value for the new code. The form of 1903 the registered variant subtag will be at the discretion of 1904 the Language Subtag Reviewer and MUST conform to other 1905 restrictions on variant subtags in this document. This 1906 situation is very unlikely to ever occur. 1908 16. UN M.49 has codes for both 'countries' and 'areas' (such as 1909 '276' for Germany) and 'geographical regions' and 'sub-regions' 1910 (such as '150' for Europe). UN M.49 'country' or 'area' codes 1911 for which there is no corresponding ISO 3166-1 code MUST NOT be 1912 registered, except as a surrogate for an ISO 3166-1 code that is 1913 blocked from registration by an existing subtag. 1915 If such a code becomes necessary, then the maintenance agency 1916 for ISO 3166-1 SHALL first be petitioned to assign a code to the 1917 region. If the petition for a code assignment by ISO 3166-1 is 1918 refused or not acted on in a timely manner, the registration 1919 process described in Section 3.5 can then be used to register 1920 the corresponding UN M.49 code. This way, UN M.49 codes remain 1921 available as the value of last resort in cases where ISO 3166-1 1922 reassigns a deprecated value in the registry. 1924 17. The redundant and grandfathered entries together form the 1925 complete list of tags registered under [RFC3066]. The redundant 1926 tags are those previously registered tags that can now be formed 1927 using the subtags defined in the registry. The grandfathered 1928 entries include those that can never be legal because they are 1929 'irregular' (that is, they do not match the 'langtag' production 1930 in Figure 1), are limited by rule (subtags such as 'nyn' and 1931 'min' look like the extlang production, but cannot be registered 1932 as extended language subtags), or their subtags are 1933 inappropriate for registration. All of the grandfathered tags 1934 are listed in either the 'regular' or the 'irregular' 1935 productions in the ABNF. Under [RFC4646] it was possible for 1936 grandfathered tags to become redundant. However, all of the 1937 tags for which this was possible became redundant before this 1938 document was produced. So the set of redundant and 1939 grandfathered tags is now permanent and immutable: new entries 1940 of either type MUST NOT be added and existing entries MUST NOT 1941 be removed. The decision-making process about which tags were 1942 initially grandfathered and which were made redundant is 1943 described in [RFC4645]. 1945 Many of the grandfathered tags are deprecated, indeed, they were 1946 deprecated even before [RFC4646]. For example, the tag "art- 1947 lojban" was deprecated in favor of the primary language subtag 1948 'jbo'. These tags could have been made 'redundant' by 1949 registering some of their subtags as 'variants'. The 'variant- 1950 like' subtags in the grandfathered registrations SHALL NOT be 1951 registered in the future, even with a similar or identical 1952 meaning. 1954 3.5. Registration Procedure for Subtags 1956 The procedure given here MUST be used by anyone who wants to use a 1957 subtag not currently in the IANA Language Subtag Registry or who 1958 wishes to add, modify, update, or remove information in existing 1959 records as permitted by this document. 1961 Only subtags of type 'language' and 'variant' will be considered for 1962 independent registration of new subtags. Subtags needed for 1963 stability and subtags necessary to keep the registry synchronized 1964 with ISO 639, ISO 15924, ISO 3166, and UN M.49 within the limits 1965 defined by this document also use this process, as described in 1966 Section 3.3 and subject to stability provisions as described in 1967 Section 3.4. 1969 Registration requests are accepted relating to information in the 1970 'Comments', 'Deprecated', 'Description', 'Prefix', 'Preferred-Value', 1971 or 'Suppress-Script' fields in a subtag's record as described in 1972 Section 3.4. Changes to all other fields in the IANA registry are 1973 NOT permitted. 1975 Registering a new subtag or requesting modifications to an existing 1976 tag or subtag starts with the requester filling out the registration 1977 form reproduced below. Note that each response is not limited in 1978 size so that the request can adequately describe the registration. 1979 The fields in the "Record Requested" section need to follow the 1980 requirements in Section 3.1 before the record will be approved. 1982 LANGUAGE SUBTAG REGISTRATION FORM 1983 1. Name of requester: 1984 2. E-mail address of requester: 1985 3. Record Requested: 1987 Type: 1988 Subtag: 1989 Description: 1990 Prefix: 1991 Preferred-Value: 1992 Deprecated: 1993 Suppress-Script: 1994 Macrolanguage: 1995 Comments: 1997 4. Intended meaning of the subtag: 1998 5. Reference to published description 1999 of the language (book or article): 2000 6. Any other relevant information: 2002 Figure 5: The Language Subtag Registration Form 2004 Examples of completed registration forms can be found in Appendix C. 2005 A complete list of approved registration forms is online at 2006 http://www.iana.org/assignments/lang-subtags-templates/. 2008 The subtag registration form MUST be sent to 2009 . Registration requests receive a two-week 2010 review period before being approved and submitted to IANA for 2011 inclusion in the registry. If modifications are made to the request 2012 during the course of the registration process (such as corrections to 2013 meet the requirements in Section 3.1 or to make the 'Description' 2014 fields unique for the given record type) the modified form MUST also 2015 be sent to at least one week prior to 2016 submission to IANA. 2018 The ietf-languages list is an open list and can be joined by sending 2019 a request to . The list can be 2020 hosted by IANA or by any third party at the request of IESG. 2022 Before forwarding any registration to IANA, the Language Subtag 2023 Reviewer MUST ensure that all requirements in this document are met. 2024 This includes ensuring that values in the 'Subtag' field match case 2025 according to the description in Section 3.1.4 and that 'Description' 2026 fields are unique for the given record type as described in 2027 Section 3.1.5. The Reviewer MUST also ensure that an appropriate 2028 File-Date record is included in the request, to assist IANA when 2029 updating the registry (see Section 5.1). 2031 Some fields in both the registration form as well as the registry 2032 record itself permit the use of non-ASCII characters. Registration 2033 requests SHOULD use the UTF-8 encoding for consistency and clarity. 2034 However, since some mail clients do not support this encoding, other 2035 encodings MAY be used for the registration request. The Language 2036 Subtag Reviewer is responsible for ensuring that the proper Unicode 2037 characters appear in both the archived request form and the registry 2038 record. In the case of a transcription or encoding error by IANA, 2039 the Language Subtag Reviewer will request that the registry be 2040 repaired, providing any necessary information to assist IANA. 2042 Extended language subtags (type 'extlang'), by definition, are always 2043 encompassed by another language. All records of type 'extlang' MUST, 2044 therefore, contain a 'Prefix' field at the time of registration. 2045 This Prefix field can never be altered or removed and requests to do 2046 so MUST be rejected. 2048 Variant subtags are usually registered for use with a particular 2049 range of language tags and variant subtags based on the terminology 2050 of the language to which they are apply are encouraged. For example, 2051 the subtag 'rozaj' (Resian) is intended for use with language tags 2052 that start with the primary language subtag "sl" (Slovenian), since 2053 Resian is a dialect of Slovenian. Thus, the subtag 'rozaj' would be 2054 appropriate in tags such as "sl-Latn-rozaj" or "sl-IT-rozaj". This 2055 information is stored in the 'Prefix' field in the registry. Variant 2056 registration requests SHOULD include at least one 'Prefix' field in 2057 the registration form. 2059 Requests to assign an additional record of a given type with an 2060 existing subtag value MUST be rejected. For example, the variant 2061 subtag 'rozaj' already exists in the registry, so adding a second 2062 record of type 'variant' with the subtag 'rozaj' is prohibited. 2064 The 'Prefix' field for a given registered variant subtag exists in 2065 the IANA registry as a guide to usage. Additional 'Prefix' fields 2066 MAY be added by filing an additional registration form. In that 2067 form, the "Any other relevant information:" field MUST indicate that 2068 it is the addition of a prefix. 2070 Requests to add a 'Prefix' field to a variant subtag that imply a 2071 different semantic meaning SHOULD be rejected. For example, a 2072 request to add the prefix "de" to the subtag '1994' so that the tag 2073 "de-1994" represented some German dialect or orthographic form would 2074 be rejected. The '1994' subtag represents a particular Slovenian 2075 orthography and the additional registration would change or blur the 2076 semantic meaning assigned to the subtag. A separate subtag SHOULD be 2077 proposed instead. 2079 Requests to add a 'Prefix' to a variant subtag that has no current 2080 'Prefix' field MUST be rejected. Variants are registered with no 2081 prefix because they are potentially useful with many or even all 2082 languages. Adding one or more 'Prefix' fields would be potentially 2083 harmful to the use of the variant, since it dramatically reduces the 2084 scope of the subtag (which is not allowed under the stability rules 2085 (Section 3.4), as opposed to broadening the scope of the subtag, 2086 which is what the addition of a 'Prefix' normally does. An example 2087 of such a "no-prefix" variant is the subtag 'fonipa', which 2088 represents the International Phonetic Alphabet, a scheme which can be 2089 used to transcribe many languages. 2091 The 'Description' fields provided in the request MUST contain at 2092 least one description written or transcribed into the Latin script; 2093 the request MAY also include additional 'Description' fields in any 2094 script or language. The 'Description' field is used for 2095 identification purposes and doesn't necessarily represent the actual 2096 native name of the language or variation. It also doesn't have to be 2097 in any particular language, but SHOULD be both suitable and 2098 sufficient to identify the item in the record. The Language Subtag 2099 Reviewer will check and edit any proposed 'Description' fields so as 2100 to ensure uniqueness and prevent collisions with 'Description' fields 2101 in other records of the same type. If this occurs in an independent 2102 registration request, the Language Subtag Reviewer MUST resubmit the 2103 record to ietf-languages@iana.org, treating it as a modification of a 2104 request due to discussion, as described in Section 3.5, unless the 2105 request's sole purpose is to introduce a duplicate 'Description' 2106 field, in which case the request SHALL be rejected. 2108 The 'Description' field is not guaranteed to be stable. Corrections 2109 or clarifications of intent are examples of possible changes. 2110 Attempts to provide translations or transcriptions of entries in the 2111 registry (which, by definition provide no new information) are 2112 unlikely to be approved. 2114 Soon after the two-week review period has passed, the Language Subtag 2115 Reviewer MUST take one of the following actions: 2117 o Explicitly accept the request and forward the form containing the 2118 record to be inserted or modified to iana@iana.org according to 2119 the procedure described in Section 3.3. 2121 o Explicitly reject the request because of significant objections 2122 raised on the list or due to problems with constraints in this 2123 document (which MUST be explicitly cited). 2125 o Extend the review period by granting an additional two-week 2126 increment to permit further discussion. After each two-week 2127 increment, the Language Subtag Reviewer MUST indicate on the list 2128 whether the registration has been accepted, rejected, or extended. 2130 Note that the Language Subtag Reviewer MAY raise objections on the 2131 list if he or she so desires. The important thing is that the 2132 objection MUST be made publicly. 2134 Sometimes the request needs to be modified as a result of discussion 2135 during the review period or due to requirements in this document. 2136 The applicant, Language Subtag Reviewer, or others MAY submit a 2137 modified version of the completed registration form, which will be 2138 considered in lieu of the original request with the explicit approval 2139 of the applicant. Such changes do not restart the two-week 2140 discussion period, although an application containing the final 2141 record submitted to IANA MUST appear on the list at least one week 2142 prior to the Language Subtag Reviewer forwarding the record to IANA. 2143 The applicant MAY modify a rejected application with more appropriate 2144 or additional information and submit it again; this starts a new two- 2145 week comment period. 2147 Registrations initiated due to the provisions of Section 3.3 or 2148 Section 3.4 SHALL NOT be rejected altogether (since they have to 2149 ultimately appear in the registry) and SHOULD be completed as quickly 2150 as possible. The review process allows list members to comment on 2151 the specific information in the form and the record it contains and 2152 thus help ensure that it is correct and consistent. The Language 2153 Subtag Reviewer MAY reject a specific version of the form, but MUST 2154 propose a suitable replacement, extending the review period as 2155 described above, until the form is in a format worthy of reviewer's 2156 approval and meets with rough consensus of the list. 2158 Decisions made by the Language Subtag Reviewer MAY be appealed to the 2159 IESG [RFC2028] under the same rules as other IETF decisions 2160 [RFC2026]. This includes a decision to extend the review period or 2161 the failure to announce a decision in a clear and timely manner. 2163 The approved records appear in the Language Subtag Registry. The 2164 approved registration forms are available online under 2165 http://www.iana.org/assignments/lang-subtags-templates/. 2167 Updates or changes to existing records follow the same procedure as 2168 new registrations. The Language Subtag Reviewer decides whether 2169 there is consensus to update the registration following the two week 2170 review period; normally, objections by the original registrant will 2171 carry extra weight in forming such a consensus. 2173 Registrations are permanent and stable. Once registered, subtags 2174 will not be removed from the registry and will remain a valid way in 2175 which to specify a specific language or variant. 2177 Note: The purpose of the "Reference to published description" section 2178 in the registration form is to aid in verifying whether a language is 2179 registered or what language or language variation a particular subtag 2180 refers to. In most cases, reference to an authoritative grammar or 2181 dictionary of that language will be useful; in cases where no such 2182 work exists, other well-known works describing that language or in 2183 that language MAY be appropriate. The Language Subtag Reviewer 2184 decides what constitutes "good enough" reference material. This 2185 requirement is not intended to exclude particular languages or 2186 dialects due to the size of the speaker population or lack of a 2187 standardized orthography. Minority languages will be considered 2188 equally on their own merits. 2190 3.6. Possibilities for Registration 2192 Possibilities for registration of subtags or information about 2193 subtags include: 2195 o Primary language subtags for languages not listed in ISO 639 that 2196 are not variants of any listed or registered language MAY be 2197 registered. At the time this document was created, there were no 2198 examples of this form of subtag. Before attempting to register a 2199 language subtag, there MUST be an attempt to register the language 2200 with ISO 639. Subtags MUST NOT be registered for languages 2201 defined by codes that exist in ISO 639-1, ISO 639-2, or ISO 639-3, 2202 or that are under consideration by the ISO 639 registration 2203 authorities, or that have never been attempted for registration 2204 with those authorities. If ISO 639 has previously rejected a 2205 language for registration, it is reasonable to assume that there 2206 must be additional, very compelling evidence of need before it 2207 will be registered as a primary language subtag in the IANA 2208 registry (to the extent that it is very unlikely that any subtags 2209 will be registered of this type). 2211 o Dialect or other divisions or variations within a language, its 2212 orthography, writing system, regional or historical usage, 2213 transliteration or other transformation, or distinguishing 2214 variation MAY be registered as variant subtags. An example is the 2215 'rozaj' subtag (the Resian dialect of Slovenian). 2217 o The addition or maintenance of fields (generally of an 2218 informational nature) in Tag or Subtag records as described in 2219 Section 3.1 and subject to the stability provisions in 2220 Section 3.4. This includes Description, Comments, Deprecated and 2221 Preferred-Value fields for obsolete or withdrawn codes, or the 2222 addition of Suppress-Script or Macrolanguage fields to primary 2223 language subtags, as well as other changes permitted by this 2224 document, such as the addition of an appropriate Prefix field to a 2225 variant subtag. 2227 o The addition of records and related field value changes necessary 2228 to reflect assignments made by ISO 639, ISO 15924, ISO 3166-1, and 2229 UN M.49 as described in Section 3.4. 2231 Subtags proposed for registration that would cause all or part of a 2232 grandfathered tag to become redundant but whose meaning conflicts 2233 with or alters the meaning of the grandfathered tag MUST be rejected. 2235 This document leaves the decision on what subtags or changes to 2236 subtags are appropriate (or not) to the registration process 2237 described in Section 3.5. 2239 Note: four-character primary language subtags are reserved to allow 2240 for the possibility of alpha4 codes in some future addition to the 2241 ISO 639 family of standards. 2243 ISO 639 defines a registration authority for additions to and changes 2244 in the list of languages in ISO 639. This agency is: 2246 International Information Centre for Terminology (Infoterm) 2247 Aichholzgasse 6/12, AT-1120 2248 Wien, Austria 2249 Phone: +43 1 26 75 35 Ext. 312 Fax: +43 1 216 32 72 2251 ISO 639-2 defines a registration authority for additions to and 2252 changes in the list of languages in ISO 639-2. This agency is: 2254 Library of Congress 2255 Network Development and MARC Standards Office 2256 Washington, D.C. 20540 USA 2257 Phone: +1 202 707 6237 Fax: +1 202 707 0115 2258 URL: http://www.loc.gov/standards/iso639-2 2260 ISO 639-3 defines a registration authority for additions to and 2261 changes in the list of languages in ISO 639-3. This agency is: 2263 SIL International 2264 ISO 639-3 Registrar 2265 7500 W. Camp Wisdom Rd. 2266 Dallas, TX 75236 USA 2267 Phone: +1 972 708 7400, ext. 2293 Fax: +1 972 708 7546 2268 Email: iso639-3@sil.org 2269 URL: http://www.sil.org/iso639-3 2271 ISO 639-5 defines a registration authority for additions to and 2272 changes in the list of languages in ISO 639-5. This agency is the 2273 same as for ISO 639-2 and is: 2275 Library of Congress 2276 Network Development and MARC Standards Office 2277 Washington, D.C. 20540 USA 2278 Phone: +1 202 707 6237 Fax: +1 202 707 0115 2279 URL: http://www.loc.gov/standards/iso639-5 2281 The maintenance agency for ISO 3166-1 (country codes) is: 2283 ISO 3166 Maintenance Agency 2284 c/o International Organization for Standardization 2285 Case postale 56 2286 CH-1211 Geneva 20 Switzerland 2287 Phone: +41 22 749 72 33 Fax: +41 22 749 73 49 2288 URL: http://www.iso.org/iso/en/prods-services/iso3166ma/index.html 2290 The registration authority for ISO 15924 (script codes) is: 2292 Unicode Consortium 2293 Box 391476 2294 Mountain View, CA 94039-1476, USA 2295 URL: http://www.unicode.org/iso15924 2297 The Statistics Division of the United Nations Secretariat maintains 2298 the Standard Country or Area Codes for Statistical Use and can be 2299 reached at: 2301 Statistical Services Branch 2302 Statistics Division 2303 United Nations, Room DC2-1620 2304 New York, NY 10017, USA 2306 Fax: +1-212-963-0623 2307 E-mail: statistics@un.org 2308 URL: http://unstats.un.org/unsd/methods/m49/m49alpha.htm 2310 3.7. Extensions and the Extensions Registry 2312 Extension subtags are those introduced by single-character subtags 2313 ("singletons") other than 'x'. They are reserved for the generation 2314 of identifiers that contain a language component and are compatible 2315 with applications that understand language tags. 2317 The structure and form of extensions are defined by this document so 2318 that implementations can be created that are forward compatible with 2319 applications that might be created using singletons in the future. 2320 In addition, defining a mechanism for maintaining singletons will 2321 lend stability to this document by reducing the likely need for 2322 future revisions or updates. 2324 Single-character subtags are assigned by IANA using the "IETF Review" 2325 policy defined by [RFC5226]. This policy requires the development of 2326 an RFC, which SHALL define the name, purpose, processes, and 2327 procedures for maintaining the subtags. The maintaining or 2328 registering authority, including name, contact email, discussion list 2329 email, and URL location of the registry, MUST be indicated clearly in 2330 the RFC. The RFC MUST specify or include each of the following: 2332 o The specification MUST reference the specific version or revision 2333 of this document that governs its creation and MUST reference this 2334 section of this document. 2336 o The specification and all subtags defined by the specification 2337 MUST follow the ABNF and other rules for the formation of tags and 2338 subtags as defined in this document. In particular, it MUST 2339 specify that case is not significant and that subtags MUST NOT 2340 exceed eight characters in length. 2342 o The specification MUST specify a canonical representation. 2344 o The specification of valid subtags MUST be available over the 2345 Internet and at no cost. 2347 o The specification MUST be in the public domain or available via a 2348 royalty-free license acceptable to the IETF and specified in the 2349 RFC. 2351 o The specification MUST be versioned, and each version of the 2352 specification MUST be numbered, dated, and stable. 2354 o The specification MUST be stable. That is, extension subtags, 2355 once defined by a specification, MUST NOT be retracted or change 2356 in meaning in any substantial way. 2358 o The specification MUST include in a separate section the 2359 registration form reproduced in this section (below) to be used in 2360 registering the extension upon publication as an RFC. 2362 o IANA MUST be informed of changes to the contact information and 2363 URL for the specification. 2365 IANA will maintain a registry of allocated single-character 2366 (singleton) subtags. This registry MUST use the record-jar format 2367 described by the ABNF in Section 3.1. Upon publication of an 2368 extension as an RFC, the maintaining authority defined in the RFC 2369 MUST forward this registration form to iesg@ietf.org, who MUST 2370 forward the request to iana@iana.org. The maintaining authority of 2371 the extension MUST maintain the accuracy of the record by sending an 2372 updated full copy of the record to iana@iana.org with the subject 2373 line "LANGUAGE TAG EXTENSION UPDATE" whenever content changes. Only 2374 the 'Comments', 'Contact_Email', 'Mailing_List', and 'URL' fields MAY 2375 be modified in these updates. 2377 Failure to maintain this record, maintain the corresponding registry, 2378 or meet other conditions imposed by this section of this document MAY 2379 be appealed to the IESG [RFC2028] under the same rules as other IETF 2380 decisions (see [RFC2026]) and MAY result in the authority to maintain 2381 the extension being withdrawn or reassigned by the IESG. 2382 %% 2383 Identifier: 2384 Description: 2385 Comments: 2386 Added: 2387 RFC: 2388 Authority: 2389 Contact_Email: 2390 Mailing_List: 2391 URL: 2392 %% 2394 Figure 6: Format of Records in the Language Tag Extensions Registry 2396 'Identifier' contains the single-character subtag (singleton) 2397 assigned to the extension. The Internet-Draft submitted to define 2398 the extension SHOULD specify which letter or digit to use, although 2399 the IESG MAY change the assignment when approving the RFC. 2401 'Description' contains the name and description of the extension. 2403 'Comments' is an OPTIONAL field and MAY contain a broader description 2404 of the extension. 2406 'Added' contains the date the extension's RFC was published in the 2407 "full-date" format specified in [RFC3339]. For example: 2004-06-28 2408 represents June 28, 2004, in the Gregorian calendar. 2410 'RFC' contains the RFC number assigned to the extension. 2412 'Authority' contains the name of the maintaining authority for the 2413 extension. 2415 'Contact_Email' contains the email address used to contact the 2416 maintaining authority. 2418 'Mailing_List' contains the URL or subscription email address of the 2419 mailing list used by the maintaining authority. 2421 'URL' contains the URL of the registry for this extension. 2423 The determination of whether an Internet-Draft meets the above 2424 conditions and the decision to grant or withhold such authority rests 2425 solely with the IESG and is subject to the normal review and appeals 2426 process associated with the RFC process. 2428 Extension authors are strongly cautioned that many (including most 2429 well-formed) processors will be unaware of any special relationships 2430 or meaning inherent in the order of extension subtags. Extension 2431 authors SHOULD avoid subtag relationships or canonicalization 2432 mechanisms that interfere with matching or with length restrictions 2433 that sometimes exist in common protocols where the extension is used. 2434 In particular, applications MAY truncate the subtags in doing 2435 matching or in fitting into limited lengths, so it is RECOMMENDED 2436 that the most significant information be in the most significant 2437 (left-most) subtags and that the specification gracefully handle 2438 truncated subtags. 2440 When a language tag is to be used in a specific, known, protocol, it 2441 is RECOMMENDED that the language tag not contain extensions not 2442 supported by that protocol. In addition, note that some protocols 2443 MAY impose upper limits on the length of the strings used to store or 2444 transport the language tag. 2446 3.8. Update of the Language Subtag Registry 2448 Upon adoption of this document the IANA Language Subtag Registry will 2449 need an update so that it contains the complete set of subtags valid 2450 in a language tag. This collection of subtags, along with a 2451 description of the process used to create it, is described by 2452 [draft-4645bis]. Once published by IANA, the maintenance procedures, 2453 rules, and registration processes described in this document will be 2454 available for new registrations or updates._RFC Editor: please modify 2455 this paragraph upon publication to reflect the fact that RFC 4645bis 2456 has also been published (past tense)._ 2458 Registrations that are in process under the rules defined in 2459 [RFC4646] when this document is adopted MUST be completed under the 2460 rules contained in this document. 2462 4. Formation and Processing of Language Tags 2464 This section addresses how to use the information in the registry 2465 with the tag syntax to choose, form, and process language tags. 2467 4.1. Choice of Language Tag 2469 The guiding principle in forming language tags is to "tag content 2470 wisely." Sometimes there is a choice between several possible tags 2471 for the same content. The choice of which tag to use depends on the 2472 content and application in question and some amount of judgment might 2473 be necessary when selecting a tag. 2475 Interoperability is best served when the same language tag is used 2476 consistently to represent the same language. If an application has 2477 requirements that make the rules here inapplicable, then that 2478 application risks damaging interoperability. It is strongly 2479 RECOMMENDED that users not define their own rules for language tag 2480 choice. 2482 Standards, protocols, and applications that reference this document 2483 normatively but apply different rules to the ones given in this 2484 section MUST specify how language tag selection varies from the 2485 guidelines given here. 2487 To ensure consistent backward compatibility, this document contains 2488 several provisions to account for potential instability in the 2489 standards used to define the subtags that make up language tags. 2490 These provisions mean that no valid language tag can become invalid, 2491 nor will a language tag have a narrower scope in the future (it may 2492 have a broader scope). The most appropriate language tag for a given 2493 application or content item might evolve over time, but once applied, 2494 the tag itself cannot become invalid or have its meaning wholly 2495 change. 2497 A subtag SHOULD only be used when it adds useful distinguishing 2498 information to the tag. Extraneous subtags interfere with the 2499 meaning, understanding, and processing of language tags. In 2500 particular, users and implementations SHOULD follow the 'Prefix' and 2501 'Suppress-Script' fields in the registry (defined in Section 3.1): 2502 these fields provide guidance on when specific additional subtags 2503 SHOULD be used or avoided in a language tag. 2505 The choice of subtags used to form a language tag SHOULD follow these 2506 guidelines: 2508 1. Use as precise a tag as possible, but no more specific than is 2509 justified. Avoid using subtags that are not important for 2510 distinguishing content in an application. 2512 * For example, 'de' might suffice for tagging an email written 2513 in German, while "de-CH-1996" is probably unnecessarily 2514 precise for such a task. 2516 * Note that some subtag sequences might not represent the 2517 language a casual user might expect. For example, the Swiss 2518 German (Schweizerdeutsch) language is represented by "gsw-CH" 2519 and not by "de-CH". This latter tag represents German ('de') 2520 as used in Switzerland ('CH'), also known as Swiss High German 2521 (Schweizer Hochdeutsch). Both are real languages and 2522 distinguishing between them could be important to an 2523 application. 2525 2. The script subtag SHOULD NOT be used to form language tags unless 2526 the script adds some distinguishing information to the tag. 2527 Script subtags were first formally defined in BCP 47 by 2528 [RFC4646]. Their use can affect matching and subtag 2529 identification for implementations of previous versions of BCP 47 2530 (i.e. [RFC1766] or [RFC3066]), as these subtags appear between 2531 the primary language and region subtags. Some applications can 2532 benefit from the use of script subtags in language tags, as long 2533 as the use is consistent for a given context. Script subtags are 2534 never appropriate for unwritten content (such as audio 2535 recordings). The field 'Suppress-Script' in the primary or 2536 extended language record in the registry indicates script subtags 2537 that do not add distinguishing information for most applications; 2538 this field defines when users SHOULD NOT include a script subtag 2539 with a particular primary language subtag. 2541 For example, if an implementation selects content using Basic 2542 Filtering [RFC4647] (originally described in Section 2.5 of 2543 [RFC3066]) and the user requested the language range "en-US", 2544 content labeled "en-Latn-US" will not match the request and thus 2545 not be selected. Therefore, it is important to know when script 2546 subtags will customarily be used and when they ought not be used. 2548 For example: 2550 * The subtag 'Latn' should not be used with the primary language 2551 'en' because nearly all English documents are written in the 2552 Latin script and it adds no distinguishing information. 2553 However, if a document were written in English mixing Latin 2554 script with another script such as Braille ('Brai'), then it 2555 might be appropriate to choose to indicate both scripts to aid 2556 in content selection, such as the application of a style 2557 sheet. 2559 * When labeling content that is unwritten (such as a recording 2560 of human speech), the script subtag should not be used, even 2561 if the language is customarily written in several scripts. 2562 Thus the subtitles to a movie might use the tag "uz-Arab" 2563 (Uzbek, Arabic script), but the audio track for the same 2564 language would be tagged simply "uz". (The tag "uz-Zxxx" 2565 could also be used where content is not written, as the subtag 2566 'Zxxx' represents the "Code for unwritten documents".) 2568 3. If a tag or subtag has a 'Preferred-Value' field in its registry 2569 entry, then the value of that field SHOULD be used to form the 2570 language tag in preference to the tag or subtag in which the 2571 preferred value appears. 2573 * For example, use 'jbo' for Lojban in preference to the 2574 grandfathered tag "art-lojban". 2576 4. Use subtags or sequences of subtags for individual languages in 2577 preference to subtags for language collections. A "language 2578 collection" is a group of languages that are descended from a 2579 common ancestor, are spoken in the same geographical area, or are 2580 otherwise related. Certain language collections are assigned 2581 codes by [ISO639-5] (and some of these [ISO639-5] codes are also 2582 defined as collections in [ISO639-2]). These codes are included 2583 as primary language subtags in the registry. Subtags for a 2584 language collection in the registry have a 'Scope' field with a 2585 value of 'collection'. A subtag for a language collection is 2586 always preferred to less-specific alternatives such as 'mul' and 2587 'und' (see below) and a subtag representing a language collection 2588 MAY be used when more specific language information is not 2589 available. However, most users and implementations do not know 2590 there is a relationship between the collection and its individual 2591 languages. In addition, the relationship between the individual 2592 languages in the collection is not well defined; in particular, 2593 the languages are usually not mutually intelligible. Since the 2594 subtags are different, a request for the collection will 2595 typically only produce items tagged with the collection's subtag, 2596 not items tagged with subtags for the individual languages 2597 contained in the collection. 2599 For example: 2601 1. Collections are interpreted inclusively, so the subtag 'gem' 2602 (Germanic languages) could, but SHOULD NOT, be used with 2603 content that would be better tagged with "en" (English), "de" 2604 (German), or "gsw" (Swiss German, Alemannic). While 'gem' 2605 collects all of these (and other) languages, most 2606 implementations will not match 'gem' to the individual 2607 languages; thus using the subtag will not produce the desired 2608 result. 2610 5. [ISO639-2] has defined several codes included in the subtag 2611 registry that require additional care when choosing language 2612 tags. In most of these cases, where omitting the language tag is 2613 permitted, such omission is preferable to using these codes. 2614 Language tags SHOULD NOT incorporate these subtags as a prefix, 2615 unless the additional information conveys some value to the 2616 application. 2618 * The 'mul' (Multiple) primary language subtag identifies 2619 content in multiple languages. This subtag SHOULD NOT be used 2620 when a list of languages or individual tags for each content 2621 element can be used instead. For example, the 'Content- 2622 Language' header ([RFC3282]) allows a list of languages to be 2623 used, not just a single language tag. 2625 * The 'und' (Undetermined) primary language subtag identifies 2626 linguistic content whose language is not determined. This 2627 subtag SHOULD NOT be used unless a language tag is required 2628 and language information is not available or cannot be 2629 determined. Omitting the language tag (where permitted) is 2630 preferred. The 'und' subtag might be useful for protocols 2631 that require a language tag to be provided or where a primary 2632 language subtag is required (such as in "und-Latn"). The 2633 'und' subtag MAY also be useful when matching language tags in 2634 certain situations. 2636 * The 'zxx' (Non-Linguistic, Not Applicable) primary language 2637 subtag identifies content for which a language classification 2638 is inappropriate or does not apply. Some examples might 2639 include instrumental or electronic music; sound recordings 2640 consisting of nonverbal sounds; audiovisual materials with no 2641 narration, dialog, printed titles, or subtitles; machine- 2642 readable data files consisting of machine languages or 2643 character codes; or programming source code. 2645 * The 'mis' (Uncoded) primary language subtag identifies content 2646 whose language is known but which does not currently have a 2647 corresponding subtag. This subtag SHOULD NOT be used. 2648 Because the addition of other codes in the future can render 2649 its application invalid, it is inherently unstable and hence 2650 incompatible with the stability goals of BCP 47. It is always 2651 preferable to use other subtags: either 'und' or (with prior 2652 agreement) private use subtags. 2654 6. Use variant subtags sparingly and in the correct order. Most 2655 variant subtags have one or more 'Prefix' fields in the registry 2656 that express the list of subtags that they are appropriate with. 2657 Variants SHOULD only be used with subtags that appear in one of 2658 these 'Prefix' fields. If a variant lists a second variant in 2659 one of its 'Prefix' fields, the first variant SHOULD appear 2660 directly after the second variant in any language tag where both 2661 occur. General purpose variants (those with no 'Prefix' fields 2662 at all) SHOULD appear after any other variant subtags. Order any 2663 remaining variants by placing the most significant subtag first. 2664 If none of the subtags is more significant or no relationship can 2665 be determined, alphabetize the subtags. Because variants are 2666 very specialized, using many of them together generally makes the 2667 tag so narrow as to override the additional precision gained. 2668 Putting the subtags into another order interferes with 2669 interoperability, as well as the overall interpretation of the 2670 tag. 2672 A. For example, the tag "en-scottish-fonipa" (English, Scottish 2673 dialect, IPA phonetic transcription) is correctly ordered 2674 because 'scottish' has a 'Prefix' of "en", while 'fonipa' has 2675 no 'Prefix' field. 2677 B. For example, the tag "sl-IT-rozaj-biske-1994" is correctly 2678 ordered: 'rozaj' lists "sl" as its sole 'Prefix'; 'biske' 2679 lists "sl-rozaj" as its sole Prefix. The subtag '1994' has 2680 several prefixes, including "sl-rozaj". However, it follows 2681 both 'rozaj' and 'biske' because one of its 'Prefix' fields 2682 is "sl-rozaj-biske". 2684 7. The grandfathered tag "i-default" (Default Language) was 2685 originally registered according to [RFC1766] to meet the needs of 2686 [RFC2277]. It is used to indicate not a specific language, but 2687 rather, it identifies the condition or content used where the 2688 language preferences of the user cannot be established. It 2689 SHOULD NOT be used except as a means of labeling the default 2690 content for applications or protocols that require default 2691 language content to be labeled with that specific tag. It MAY 2692 also be used by an application or protocol to identify when the 2693 default language content is being returned. 2695 4.1.1. Tagging Encompassed Languages 2697 Some primary language records in the registry have a "Macrolanguage" 2698 field (Section 3.1.10) that contains a mapping from each "encompassed 2699 language" to its macrolanguage. The Macrolanguage mapping doesn't 2700 define what the relationship between the encompassed language and its 2701 macrolanguage is, nor does it define how languages encompassed by the 2702 same macrolanguage are related to each other. Two different 2703 languages encompassed by the same macrolanguage may differ from one 2704 another more than say, French and Spanish do. 2706 A few specific macrolanguages, such as Chinese ('zh') and Arabic 2707 ('ar'), are handled differently. See Section 4.1.2. 2709 The more specific encompassed language subtag SHOULD be used to form 2710 the language tag, although either the macrolanguage's primary 2711 language subtag or the encompassed language's subtag MAY be used. 2712 This means, for example, tagging Plains Cree with 'crk' rather than 2713 'cre' (Cree); and so forth. 2715 Each macrolanguage subtag's scope, by definition, includes all of its 2716 encompassed languages. Since the relationship between encompassed 2717 languages varies, users cannot assume that the macrolanguage subtag 2718 means any particular encompassed language nor that any given pair of 2719 encompassed languages are mutually intelligible or otherwise 2720 interchangeable. 2722 Applications MAY use macrolanguage information to improve matching or 2723 language negotiation. For example, the information that 'sr' 2724 (Serbian) and 'hr' (Croatian) share a macrolanguage expresses a 2725 closer relation between those languages than between, say, 'sr' 2726 (Serbian) and 'ma' (Macedonian). However, this relationship is not 2727 guaranteed nor is it exclusive. For example, Romanian ('ro') and 2728 Moldavian ('mo') do not share a macrolanguage, but are far more 2729 closely related to each other than Cantonese ('yue') and Wu ('wuu') , 2730 which do share a macrolanguage. 2732 4.1.2. Using Extended Language Subtags 2734 To accommodate language tag forms used prior to the adoption of this 2735 document, language tags provide a special compatibility mechanism: 2736 the extended language subtag. Selected languages have been provided 2737 with both primary and extended language subtags. These include 2738 macrolanguages, such as Malay ('ms') and Uzbek ('uz'), that have a 2739 specific dominant variety that is generally synonymous with the 2740 macrolanguage. Other languages, such as the Chinese ('zh') and 2741 Arabic ('ar') macrolanguages and the various sign languages ('sgn'), 2742 have traditionally used their primary language subtag, possibly 2743 coupled with various region subtags or as part of a registered 2744 grandfathered tag, to indicate the language. 2746 With the adoption of this document, specific ISO 639-3 subtags became 2747 available to identify the languages contained within these diverse 2748 language families or groupings. This presents a choice of language 2749 tags where previously none existed: 2751 o Each encompassed language's subtag SHOULD be used as the primary 2752 language subtag. For example, a document in Mandarin Chinese 2753 would be tagged "cmn" (the subtag for Mandarin Chinese) in 2754 preference to "zh" (Chinese). 2756 o If compatibility is desired or needed, the encompassed subtag MAY 2757 be used as an extended language subtag. For example, a document 2758 in Mandarin Chinese could be tagged "zh-cmn" instead of either 2759 "cmn" or "zh". 2761 o The macrolanguage or prefixing subtag MAY still be used to form 2762 the tag instead of the more specific encompassed language subtag. 2763 That is, tags such as "zh-HK" or "sgn-RU" are still valid. 2765 Chinese ('zh') provides a useful illustration of this. In the past, 2766 various content has used tags beginning with the 'zh' subtag, with 2767 application specific meaning being associated with region codes, 2768 private-use sequences, or grandfathered registered values. This is 2769 because historically only the macrolanguage subtag 'zh' was available 2770 for forming language tags. However, the languages encompassed by the 2771 Chinese subtag 'zh' are, in the main, not mutually intelligible when 2772 spoken, and the written forms of these languages also show wide 2773 variation in form and usage. 2775 To provide compatibility, Chinese languages encompassed by the 'zh' 2776 subtag are in the registry as both primary language subtags and as 2777 extended language subtags. For example, the ISO 639-3 code for 2778 Cantonese is 'yue'. Content in Cantonese might historically have 2779 used a tag such as "zh-HK" (since Cantonese is commonly spoken in 2780 Hong Kong), although that tag actually means any type of Chinese as 2781 used in Hong Kong. With the availability of ISO 639-3 codes in the 2782 registry, content in Cantonese can be directly tagged using the 'yue' 2783 subtag. The content can use it as a primary language subtag, as in 2784 the tag "yue-HK" (Cantonese, Hong Kong). Or it can use an extended 2785 language subtag with 'zh', as in the tag "zh-yue-Hant" (Chinese, 2786 Cantonese, Traditional script). 2788 As noted above, applications can choose to use the macrolanguage 2789 subtag to form the tag instead of using the more specific encompassed 2790 language subtag. For example, an application with large quantities 2791 of data already using tags with the 'zh' (Chinese) subtag might 2792 continue to use this more general subtag even for new data, even 2793 though the content could be more precisely tagged with 'cmn' 2794 (Mandarin), 'yue' (Cantonese), 'wuu' (Wu), and so on. Similarly, an 2795 application already using tags that start with the 'ar' (Arabic) 2796 subtag might continue to use this more general subtag even for new 2797 data, which could be more precisely be tagged with 'arb' (Standard 2798 Arabic). 2800 In some cases, the encompassed languages had tags registered for them 2801 during the RFC 3066 era. Those grandfathered tags not already 2802 deprecated or rendered redundant were deprecated in the registry upon 2803 adoption of this document. As grandfathered values, they remain 2804 valid for use and some content or applications might use them. As 2805 with other grandfathered tags, since implementations might not be 2806 able to associate the grandfathered tags with the encompassed 2807 language subtag equivalents that are recommended by this document, 2808 implementations are encouraged to canonicalize tags for comparison 2809 purposes. Some examples of this include the tags "zh-hakka" (Hakka) 2810 and "zh-guoyu" (Mandarin or Standard Chinese). 2812 Sign languages share a mode of communication rather than a linguistic 2813 heritage. There are many sign languages which have developed 2814 independently and the subtag 'sgn' indicates only the presence of a 2815 sign language. A number of sign languages also had grandfathered 2816 tags registered for them during the RFC 3066 era. For example, the 2817 grandfathered tag "sgn-US" was registered to represent 'American Sign 2818 Language' specifically, without reference to the United States. This 2819 is still valid, but deprecated: a document in American Sign Language 2820 can be labeled either "ase" or "sgn-ase" (the 'ase' subtag is for the 2821 language called 'American Sign Language'). 2823 4.2. Meaning of the Language Tag 2825 The meaning of a language tag is related to the meaning of the 2826 subtags that it contains. Each subtag, in turn, implies a certain 2827 range of expectations one might have for related content, although it 2828 is not a guarantee. For example, the use of a script subtag such as 2829 'Arab' (Arabic script) does not mean that the content contains only 2830 Arabic characters. It does mean that the language involved is 2831 predominantly in the Arabic script. Thus a language tag and its 2832 subtags can encompass a very wide range of variation and yet remain 2833 appropriate in each particular instance. 2835 Validity of a tag is not the only factor determining its usefulness. 2836 While every valid tag has a meaning, it might not represent any real- 2837 world language usage. This is unavoidable in a system in which 2838 subtags can be combined freely. For example, tags such as 2839 "ar-Cyrl-CO" (Arabic, Cyrillic script, as used in Colombia) or "tlh- 2840 Kore-AQ-fonipa" (Klingon, Korean script, as used in Antarctica, IPA 2841 phonetic transcription) are both valid and unlikely to represent a 2842 useful combination of language attributes. 2844 The meaning of a given tag doesn't depend on the context in which it 2845 appears. The relationship between a tag's meaning and the 2846 information objects to which that tag is applied, however, can vary. 2848 o For a single information object, the associated language tags 2849 might be interpreted as the set of languages that is necessary for 2850 a complete comprehension of the complete object. Example: Plain 2851 text documents. 2853 o For an aggregation of information objects, the associated language 2854 tags could be taken as the set of languages used inside components 2855 of that aggregation. Examples: Document stores and libraries. 2857 o For information objects whose purpose is to provide alternatives, 2858 the associated language tags could be regarded as a hint that the 2859 content is provided in several languages and that one has to 2860 inspect each of the alternatives in order to find its language or 2861 languages. In this case, the presence of multiple tags might not 2862 mean that one needs to be multi-lingual to get complete 2863 understanding of the document. Example: MIME multipart/ 2864 alternative [RFC2046]. 2866 o For markup languages, such as HTML and XML, language information 2867 can be added to each part of the document identified by the markup 2868 structure (including the whole document itself). For example, one 2869 could write C'est la vie. inside a German 2870 document; the German-speaking user could then access a French- 2871 German dictionary to find out what the marked section meant. If 2872 the user were listening to that document through a speech 2873 synthesis interface, this formation could be used to signal the 2874 synthesizer to appropriately apply French text-to-speech 2875 pronunciation rules to that span of text, instead of applying the 2876 inappropriate German rules. 2878 o For markup languages and document formats that allow the audience 2879 to be identified, a language tag could indicate the audience(s) 2880 appropriate for that document. For example, the same HTML 2881 document described in the preceding bullet might have an HTTP 2882 header "Content-Language: de" to indicate that the intended 2883 audience for the file is German (even though three words appear 2884 and are identified as being in French within it). 2886 o For systems and APIs, language tags form the basis for most 2887 implementations of locale identifiers. For example, see Unicode's 2888 CLDR (Common Locale Data Repository) (see: UTS #35 [UTS35]) 2889 project. 2891 Language tags are related when they contain a similar sequence of 2892 subtags. For example, if a language tag B contains language tag A as 2893 a prefix, then B is typically "narrower" or "more specific" than A. 2894 Thus, "zh-Hant-TW" is more specific than "zh-Hant". 2896 This relationship is not guaranteed in all cases: specifically, 2897 languages that begin with the same sequence of subtags are NOT 2898 guaranteed to be mutually intelligible, although they might be. For 2899 example, the tag "az" shares a prefix with both "az-Latn" 2900 (Azerbaijani written using the Latin script) and "az-Cyrl" 2901 (Azerbaijani written using the Cyrillic script). A person fluent in 2902 one script might not be able to read the other, even though the 2903 linguistic content (e.g., what would be heard if both texts were read 2904 aloud) might be identical. Content tagged as "az" most probably is 2905 written in just one script and thus might not be intelligible to a 2906 reader familiar with the other script. 2908 Similarly, not all subtags specify an actual distinction in language. 2909 For example, the tags "en-US" and "en-CA" mean, roughly, English with 2910 features generally thought to be characteristic of the United States 2911 and Canada, respectively. They do not imply that a significant 2912 dialectical boundary exists between any arbitrarily selected point in 2913 the United States and any arbitrarily selected point in Canada. 2914 Neither does a particular region subtag imply that linguistic 2915 distinctions do not exist within that region. 2917 4.3. Lists of Languages 2919 In some applications, a single content item might best be associated 2920 with more than one language tag. Examples of such a usage include: 2922 o Content items that contain multiple, distinct varieties. Often 2923 this is used to indicate an appropriate audience for a given 2924 content item when multiple choices might be appropriate. Examples 2925 of this could include: 2927 * Metadata about the appropriate audience for a movie title. For 2928 example, a DVD might label its individual audio tracks 'de' 2929 (German), 'fr' (French), and 'es' (Spanish), but the overall 2930 title would list "de, fr, es" as its overall audience. 2932 * A French/English, English/French dictionary tagged as both "en" 2933 and "fr" to specify that it applies equally to French and 2934 English 2936 * A side-by-side or interlinear translation of a document, as is 2937 commonly done with classical works in Latin or Greek 2939 o Content items that contain a single language but which require 2940 multiple levels of specificity. For example, a library might wish 2941 to classify a particular work as both Norwegian ('no') and as 2942 Nynorsk ('nn') for audiences capable of appreciating the 2943 distinction or needing to select content more narrowly. 2945 4.4. Length Considerations 2947 There is no defined upper limit on the size of language tags. While 2948 historically most language tags have consisted of language and region 2949 subtags with a combined total length of up to six characters, larger 2950 tags have always been both possible and have actually appeared in 2951 use. 2953 Neither the language tag syntax nor other requirements in this 2954 document impose a fixed upper limit on the number of subtags in a 2955 language tag (and thus an upper bound on the size of a tag). The 2956 language tag syntax suggests that, depending on the specific 2957 language, more subtags (and thus a longer tag) are sometimes 2958 necessary to completely identify the language for certain 2959 applications; thus, it is possible to envision long or complex subtag 2960 sequences. 2962 4.4.1. Working with Limited Buffer Sizes 2964 Some applications and protocols are forced to allocate fixed buffer 2965 sizes or otherwise limit the length of a language tag. A conformant 2966 implementation or specification MAY refuse to support the storage of 2967 language tags that exceed a specified length. Any such limitation 2968 SHOULD be clearly documented, and such documentation SHOULD include 2969 what happens to longer tags (for example, whether an error value is 2970 generated or the language tag is truncated). A protocol that allows 2971 tags to be truncated at an arbitrary limit, without giving any 2972 indication of what that limit is, has the potential for causing harm 2973 by changing the meaning of tags in substantial ways. 2975 In practice, most language tags do not require more than a few 2976 subtags and will not approach reasonably sized buffer limitations; 2977 see Section 4.1. 2979 Some specifications or protocols have limits on tag length but do not 2980 have a fixed length limitation. For example, [RFC2231] has no 2981 explicit length limitation: the length available for the language tag 2982 is constrained by the length of other header components (such as the 2983 charset's name) coupled with the 76-character limit in [RFC2047]. 2984 Thus, the "limit" might be 50 or more characters, but it could 2985 potentially be quite small. 2987 The considerations for assigning a buffer limit are: 2989 Implementations SHOULD NOT truncate language tags unless the 2990 meaning of the tag is purposefully being changed, or unless the 2991 tag does not fit into a limited buffer size specified by a 2992 protocol for storage or transmission. 2994 Implementations SHOULD warn the user when a tag is truncated since 2995 truncation changes the semantic meaning of the tag. 2997 Implementations of protocols or specifications that are space 2998 constrained but do not have a fixed limit SHOULD use the longest 2999 possible tag in preference to truncation. 3001 Protocols or specifications that specify limited buffer sizes for 3002 language tags MUST allow for language tags of at least 35 3003 characters. Note that RFC 4646 [RFC4646] recommended a minimum 3004 field size of 42 characters because it included all three elements 3005 of the 'extlang' production. Two of these are now permanently 3006 reserved, so a registered primary language subtag of the maximum 3007 length of eight characters is now longer than the longest 3008 language-extlang combination. Protocols or specifications that 3009 commonly use extensions or private use subtags might wish to 3010 reserve or recommend a longer "minimum buffer" size. 3012 The following illustration shows how the 35-character recommendation 3013 was derived: 3015 language = 8 ; longest allowed registered value 3016 ; longer than primary+extlang 3017 ; which requires 7 characters 3018 script = 5 ; if not suppressed: see Section 4.1 3019 region = 4 ; UN M.49 numeric region code 3020 ; ISO 3166-1 codes require 3 3021 variant1 = 9 ; needs 'language' as a prefix 3022 variant2 = 9 ; very rare, as it needs 3023 ; 'language-variant1' as a prefix 3025 total = 35 characters 3027 Figure 7: Derivation of the Limit on Tag Length 3029 4.4.2. Truncation of Language Tags 3031 Truncation of a language tag alters the meaning of the tag, and thus 3032 SHOULD be avoided. However, truncation of language tags is sometimes 3033 necessary due to limited buffer sizes. Such truncation MUST NOT 3034 permit a subtag to be chopped off in the middle or the formation of 3035 invalid tags (for example, one ending with the "-" character). 3037 This means that applications or protocols that truncate tags MUST do 3038 so by progressively removing subtags along with their preceding "-" 3039 from the right side of the language tag until the tag is short enough 3040 for the given buffer. If the resulting tag ends with a single- 3041 character subtag, that subtag and its preceding "-" MUST also be 3042 removed. For example: 3044 Tag to truncate: zh-Latn-CN-variant1-a-extend1-x-wadegile-private1 3045 1. zh-Latn-CN-variant1-a-extend1-x-wadegile 3046 2. zh-Latn-CN-variant1-a-extend1 3047 3. zh-Latn-CN-variant1 3048 4. zh-Latn-CN 3049 5. zh-Latn 3050 6. zh 3052 Figure 8: Example of Tag Truncation 3054 4.5. Canonicalization of Language Tags 3056 Since a particular language tag can be used by many processes, 3057 language tags SHOULD always be created or generated in canonical 3058 form. 3060 A language tag is in 'canonical form' when the tag is well-formed 3061 according to the rules in Section 2.1 and Section 2.2 and it has been 3062 canonicalized by applying each of the following steps in order, using 3063 data from the IANA registry (see Section 3.1): 3065 1. Extension sequences are ordered into case-insensitive ASCII order 3066 by singleton subtag. 3068 * For example, the subtag sequence '-a-babble' comes before 3069 '-b-warble'. 3071 2. Redundant or grandfathered tags are replaced by their Preferred- 3072 Value, if there is one. 3074 * The field-body of the Preferred-Value for grandfathered and 3075 redundant tags is an "extended language range" ([RFC4647]) and 3076 might consist of more than one subtag. 3078 * Preferred-Value fields in the registry provide mappings from 3079 deprecated tags to modern equivalents. Many of these were 3080 created before the adoption of this document (such as the 3081 mapping of "no-nyn" to "nn" or "i-klingon" to "tlh"). Others 3082 are the result of later registrations or additions to the 3083 registry as permitted or required by this document (for 3084 example, "zh-hakka" was deprecated in favor of the ISO 639-3 3085 code 'hak' when this document was adopted). 3087 3. Subtags are replaced by their Preferred-Value, if there is one. 3088 For extlangs, the original primary language subtag is also 3089 replaced if there is a primary language subtag in the Preferred- 3090 Value. 3092 * The field-body of the Preferred-Value for extlangs is an 3093 "extended language range" and typically maps to a primary 3094 language subtag. For example, the subtag sequence "zh-hak" 3095 (Chinese, Hakka) is replaced with the subtag 'hak' (Hakka). 3097 * Most of the non-extlang subtags are either Region subtags 3098 where the country name or designation has changed or clerical 3099 corrections to ISO 639-1. 3101 The canonical form contains no 'extlang' subtags. There is an 3102 alternate 'extlang form' that maintains or reinstates extlang 3103 subtags. This form can be useful in environments where the presence 3104 of the Prefix subtag is considered beneficial in matching or 3105 selection (see Section 4.1.2). 3107 A language tag is in 'extlang form', when the tag is well-formed 3108 according to the rules in Section 2.1 and Section 2.2 and it has been 3109 processed by applying each of the following two steps in order, using 3110 data from the IANA registry: 3112 1. The language tag is first transformed into canonical form, as 3113 described above. 3115 2. If the language tag starts with a primary language subtag that is 3116 also an extlang subtag, then the language tag is prepended with 3117 the extlang's Prefix. 3119 * For example, "hak-CN" (Hakka, China) has the primary language 3120 subtag 'hak', which in turn has an 'extlang' record with a 3121 Prefix 'zh' (Chinese). The extlang form is "zh-hak-CN" 3122 (Chinese, Hakka, China). 3124 * Note that Step 2 (prepending a prefix) can restore a subtag 3125 that was removed by Step 1 (canonicalizing). 3127 Example: The language tag "en-a-aaa-b-ccc-bbb-x-xyz" is in canonical 3128 form, while "en-b-ccc-bbb-a-aaa-X-xyz" is well-formed and potentially 3129 valid (extensions 'a' and 'b' are not defined as of the publication 3130 of this document) but not in a canonical form (the extensions are not 3131 in alphabetical order). 3133 Example: Although the tag "en-BU" (English as used in Burma) 3134 maintains its validity, the language tag "en-BU" is not in canonical 3135 form because the 'BU' subtag has a canonical mapping to 'MM' 3136 (Myanmar). 3138 Canonicalization of language tags does not imply anything about the 3139 use of upper or lowercase letters when processing or comparing 3140 subtags (and as described in Section 2.1). All comparisons MUST be 3141 performed in a case-insensitive manner. 3143 When performing canonicalization of language tags, processors MAY 3144 regularize the case of the subtags (that is, this process is 3145 OPTIONAL), following the case used in the registry (see 3146 Section 2.1.1). 3148 If more than one variant appears within a tag, processors MAY reorder 3149 the variants to obtain better matching behavior or more consistent 3150 presentation. Reordering of the variants SHOULD follow the 3151 recommendations for variant ordering in Section 4.1. 3153 If the field 'Deprecated' appears in a registry record without an 3154 accompanying 'Preferred-Value' field, then that tag or subtag is 3155 deprecated without a replacement. These values are canonical when 3156 they appear in a language tag. However, tags that include these 3157 values SHOULD NOT be selected by users or generated by 3158 implementations. 3160 An extension MUST define any relationships that exist between the 3161 various subtags in the extension and thus MAY define an alternate 3162 canonicalization scheme for the extension's subtags. Extensions MAY 3163 define how the order of the extension's subtags are interpreted. For 3164 example, an extension could define that its subtags are in canonical 3165 order when the subtags are placed into ASCII order: that is, "en-a- 3166 aaa-bbb-ccc" instead of "en-a-ccc-bbb-aaa". Another extension might 3167 define that the order of the subtags influences their semantic 3168 meaning (so that "en-b-ccc-bbb-aaa" has a different value from "en-b- 3169 aaa-bbb-ccc"). However, extension specifications SHOULD be designed 3170 so that they are tolerant of the typical processes described in 3171 Section 3.7. 3173 4.6. Considerations for Private Use Subtags 3175 Private use subtags, like all other subtags, MUST conform to the 3176 format and content constraints in the ABNF. Private use subtags have 3177 no meaning outside the private agreement between the parties that 3178 intend to use or exchange language tags that employ them. The same 3179 subtags MAY be used with a different meaning under a separate private 3180 agreement. They SHOULD NOT be used where alternatives exist and 3181 SHOULD NOT be used in content or protocols intended for general use. 3183 Private use subtags are simply useless for information exchange 3184 without prior arrangement. The value and semantic meaning of private 3185 use tags and of the subtags used within such a language tag are not 3186 defined by this document. 3188 Private use sequences introduced by the 'x' singleton are completely 3189 opaque to users or implementations outside of the private use 3190 agreement. So, in addition to private use subtag sequences 3191 introduced by the singleton subtag 'x', the Language Subtag Registry 3192 provides private use language, script, and region subtags derived 3193 from the private use codes assigned by the underlying standards. 3194 These subtags are valid for use in forming language tags; they are 3195 RECOMMENDED over the 'x' singleton private use subtag sequences 3196 because they convey more information via their linkage to the 3197 language tag's inherent structure. 3199 For example, the region subtags 'AA', 'ZZ', and in the ranges 3200 'QM'-'QZ' and 'XA'-'XZ' (derived from the ISO 3166-1 private use 3201 codes) can be used to form a language tag. A tag such as 3202 "zh-Hans-XQ" conveys a great deal of public, interchangeable 3203 information about the language material (that it is Chinese in the 3204 simplified Chinese script and is suitable for some geographic region 3205 'XQ'). While the precise geographic region is not known outside of 3206 private agreement, the tag conveys far more information than an 3207 opaque tag such as "x-somelang" or even "zh-Hans-x-xq" (where the 3208 'xq' subtag's meaning is entirely opaque). 3210 However, in some cases content tagged with private use subtags can 3211 interact with other systems in a different and possibly unsuitable 3212 manner compared to tags that use opaque, privately defined subtags, 3213 so the choice of the best approach sometimes depends on the 3214 particular domain in question. 3216 5. IANA Considerations 3218 This section deals with the processes and requirements necessary for 3219 IANA to undertake to maintain the subtag and extension registries as 3220 defined by this document and in accordance with the requirements of 3221 [RFC5226]. 3223 The impact on the IANA maintainers of the two registries defined by 3224 this document will be a small increase in the frequency of new 3225 entries or updates. IANA also is required to create a new mailing 3226 list (described below in Section 5.1) to announce registry changes 3227 and updates. 3229 5.1. Language Subtag Registry 3231 Upon adoption of this document, IANA will update the registry using 3232 instructions and content provided in a companion document: 3233 [draft-4645bis]. The criteria and process for selecting the updated 3234 set of records are described in that document. The updated set of 3235 records represents no impact on IANA, since the work to create it 3236 will be performed externally. 3238 Future work on the Language Subtag Registry includes the following 3239 activities: 3241 o Inserting or replacing whole records. These records are 3242 preformatted for IANA by the Language Subtag Reviewer, as 3243 described in Section 3.3. 3245 o Archiving and making publicly available the registration forms. 3247 o Announcing each updated version of the registry on the 3248 "ietf-languages-announcements@iana.org" mailing list. 3250 Each registration form sent to IANA contains a single record for 3251 incorporation into the registry. The form will be sent to 3252 "iana@iana.org" by the Language Subtag Reviewer. It will have a 3253 subject line indicating whether the enclosed form represents an 3254 insertion of a new record (indicated by the word "INSERT" in the 3255 subject line) or a replacement of an existing record (indicated by 3256 the word "MODIFY" in the subject line). At no time can a record be 3257 deleted from the registry. 3259 IANA will extract the record from the form and place the inserted or 3260 modified record into the appropriate section of the language subtag 3261 registry, grouping the records by their 'Type' field. Inserted 3262 records can be placed anywhere within the appropriate section; there 3263 is no guarantee that the registry's records will be placed in any 3264 particular order except that they will always be grouped by 'Type.' 3265 Modified records overwrite the record they replace. 3267 Whenever an entry is created or modified in the registry, the 'File- 3268 Date' record at the start of the registry is updated to reflect the 3269 most recent modification date. The date format SHALL be the "full- 3270 date" format of [RFC3339]. The date SHALL be the date on which that 3271 version of the registry was first published by IANA. There SHALL be 3272 at most one version of the registry published in a day. A 'File- 3273 Date' record is also included in each request to IANA to insert or 3274 modify records, indicating the acceptance date of the records in the 3275 request. 3277 The updated registry file MUST use the UTF-8 character encoding and 3278 IANA MUST check the registry file for proper encoding. Non-ASCII 3279 characters can be sent to IANA by attaching the registration form to 3280 the email message or by using various encodings in the mail message 3281 body (UTF-8 is recommended). IANA will verify any unclear or 3282 corrupted characters with the Language Subtag Reviewer prior to 3283 posting the updated registry. 3285 IANA will also archive and make publicly available from 3286 "http://www.iana.org/assignments/lang-subtags-templates/" each 3287 registration form. Note that multiple registrations can pertain to 3288 the same record in the registry. 3290 Developers who are dependent upon the language subtag registry 3291 sometimes would like to be informed of changes in the registry so 3292 that they can update their implementations. When any change is made 3293 to the language subtag registry, IANA will send an announcement 3294 message to "ietf-languages-announcements@iana.org" (a self- 3295 subscribing list that only IANA can post to). 3297 5.2. Extensions Registry 3299 The Language Tag Extensions Registry can contain at most 35 records 3300 and thus changes to this registry are expected to be very infrequent. 3302 Future work by IANA on the Language Tag Extensions Registry is 3303 limited to two cases. First, the IESG MAY request that new records 3304 be inserted into this registry from time to time. These requests 3305 MUST include the record to insert in the exact format described in 3306 Section 3.7. In addition, there MAY be occasional requests from the 3307 maintaining authority for a specific extension to update the contact 3308 information or URLs in the record. These requests MUST include the 3309 complete, updated record. IANA is not responsible for validating the 3310 information provided, only that it is properly formatted. IANA 3311 SHOULD take reasonable steps to ascertain that the request comes from 3312 the maintaining authority named in the record present in the 3313 registry. 3315 6. Security Considerations 3317 Language tags used in content negotiation, like any other information 3318 exchanged on the Internet, might be a source of concern because they 3319 might be used to infer the nationality of the sender, and thus 3320 identify potential targets for surveillance. 3322 This is a special case of the general problem that anything sent is 3323 visible to the receiving party and possibly to third parties as well. 3324 It is useful to be aware that such concerns can exist in some cases. 3326 The evaluation of the exact magnitude of the threat, and any possible 3327 countermeasures, is left to each application protocol (see BCP 72 3328 [RFC3552] for best current practice guidance on security threats and 3329 defenses). 3331 The language tag associated with a particular information item is of 3332 no consequence whatsoever in determining whether that content might 3333 contain possible homographs. The fact that a text is tagged as being 3334 in one language or using a particular script subtag provides no 3335 assurance whatsoever that it does not contain characters from scripts 3336 other than the one(s) associated with or specified by that language 3337 tag. 3339 Since there is no limit to the number of variant, private use, and 3340 extension subtags, and consequently no limit on the possible length 3341 of a tag, implementations need to guard against buffer overflow 3342 attacks. See Section 4.4 for details on language tag truncation, 3343 which can occur as a consequence of defenses against buffer overflow. 3345 To prevent denial-of-service attacks, applications SHOULD NOT depend 3346 on either the Language Subtag Registry or the Language Tag Extensions 3347 Registry being always accessible. Additionally, although the 3348 specification of valid subtags for an extension (see Section 3.7) 3349 MUST be available over the Internet, implementations SHOULD NOT 3350 mechanically depend on those sources being always accessible. 3352 The registries specified in this document are not suitable for 3353 frequent or real-time access to, or retrieval, of the full registry 3354 contents. Most applications do not need registry data at all. For 3355 others, being able to validate or canonicalize language tags as of a 3356 particular registry date will be sufficient, as the registry contents 3357 change only occasionally. Changes are announced to 3358 . Changes, or the absence 3359 thereof, can also easily be detected by looking at the 'File-Date' 3360 record at the start of the registry, or by using features of the 3361 protocol used for downloading, without having to download the full 3362 registry. 3364 7. Character Set Considerations 3366 The syntax in this document requires that language tags use only the 3367 characters A-Z, a-z, 0-9, and HYPHEN-MINUS, which are present in most 3368 character sets, so the composition of language tags shouldn't have 3369 any character set issues. 3371 The rendering of text based on the language tag is not addressed 3372 here. Historically, some processes have relied on the use of 3373 character set/encoding information (or other external information) in 3374 order to infer how a specific string of characters should be 3375 rendered. Notably this applies to language- and culture-specific 3376 variations of Han ideographs as used in Japanese, Chinese, and 3377 Korean, where use of, for example, a Japanese character encoding such 3378 as EUC-JP implies that the text itself is in Japanese. When language 3379 tags are applied to spans of text, rendering engines might be able to 3380 use that information to better select fonts or make other rendering 3381 choices, particularly where languages with distinct writing 3382 traditions use the same characters. 3384 8. Changes from RFC 4646 3386 The main goal for this revision of this document was to incorporate 3387 two new parts of ISO 639 (ISO 639-3 and ISO 639-5) and their 3388 attendant sets of language codes into the IANA Language Subtag 3389 Registry. This permits the identification of many more languages and 3390 language collections than previously supported. 3392 The specific changes in this document to meet these goals are: 3394 o Defines the incorporation of ISO 639-3 and ISO 639-5 codes for use 3395 as primary and extended language subtags. It also permanently 3396 reserves and disallows the use of additional 'extlang' subtags. 3397 The changes necessary to achieve this were: 3399 * Modified the ABNF comments. 3401 * Updated various registration and stability requirements 3402 sections to reference ISO 639-3 and ISO 639-5 in addition to 3403 ISO 639-1 and ISO 639-2. 3405 * Edited the text to eliminate references to extended language 3406 subtags where they are no longer used. 3408 * Explained the change in the section on extended language 3409 subtags. 3411 o Changed the ABNF related to grandfathered tags. The irregular 3412 tags are now listed. Well-formed grandfathered tags are now 3413 described by the 'langtag' production and the 'grandfathered' 3414 production was removed as a result. Also: added description of 3415 both types of grandfathered tags to Section 2.2.8. 3417 o Added the paragraph on "collections" to Section 4.1. 3419 o Changed the capitalization rules for 'Tag' fields in Section 3.1. 3421 o Split section 3.1 up into subsections. 3423 o Modified section 3.5 to allow Suppress-Script fields to be added, 3424 modified, or removed via the registration process. This was an 3425 erratum from RFC 4646. 3427 o Modified examples that used region code 'CS' (formerly Serbia and 3428 Montenegro) to use 'RS' (Serbia) instead. 3430 o Modified the rules for creating and maintaining record 3431 'Description' fields to prevent duplicates, including inverted 3432 duplicates. 3434 o Removed the lengthy description of why RFC 4646 was created from 3435 this section, which also caused the removal of the reference to 3436 XML Schema. 3438 o Modified the text in section 2.1 to place more emphasis on the 3439 fact that language tags are not case sensitive. 3441 o Replaced the example "fr-Latn-CA" in Section 2.1 with "sr-Latn-RS" 3442 and "az-Arab-IR" because "fr-Latn-CA" doesn't respect the 3443 Suppress-Script on 'Latn' with 'fr'. 3445 o Changed the requirements for well-formedness to make singleton 3446 repetition checking optional (it is required for validity 3447 checking) in Section 2.2.9. 3449 o Changed the text in Section 2.2.9 referring to grandfathered 3450 checking to note that the list is now included in the ABNF. 3452 o Modified and added text to Section 3.2. The job description was 3453 placed first. A note was added making clear that the Language 3454 Subtag Reviewer may delegate various non-critical duties, 3455 including list moderation. Finally, additional text was added to 3456 make the appointment process clear and to clarify that decisions 3457 and performance of the reviewer are appealable. 3459 o Added text to Section 3.5 clarifying that the 3460 ietf-languages@iana.org list is operated by whomever the IESG 3461 appoints. 3463 o Added text to Section 3.1.5 clarifying that the first Description 3464 in a 'language' record matches the corresponding Reference Name 3465 for the language in ISO 639-3. 3467 o Modified Section 2.2.9 to define classes of conformance related to 3468 specific tags (formerly 'well-formed' and 'valid' referred to 3469 implementations). Notes were added about the removal of 'extlang' 3470 from the ABNF provided in RFC 4646, allowing for well-formedness 3471 using this older definition. Reference to RFC 3066 well- 3472 formedness was also added. 3474 o Added text to the end of Section 3.1.2 noting that future versions 3475 of this document might add new field types to the Registry format 3476 and recommending that implementations ignore any unrecognized 3477 fields. 3479 o Added text about what the lack of a Suppress-Script field means in 3480 a record to Section 3.1.9. 3482 o Added text allowing the correction of misspellings and typographic 3483 errors to Section 3.1.5. 3485 o Added text to Section 3.1.8 disallowing Prefix field conflicts 3486 (such as circular prefix references). 3488 o Modified text in Section 3.5 to require the subtag reviewer to 3489 announce his/her decision (or extension) following the two-week 3490 period. Also clarified that any decision or failure to decide can 3491 be appealed. 3493 o Modified text in Section 4.1 to include the (heretofore anecdotal) 3494 guiding principle of tag choice, and clarifying the non-use of 3495 script subtags in non-written applications. Also updated examples 3496 in this section to use Chamic languages as an example of language 3497 collections. 3499 o Prohibited multiple use of the same variant in a tag (i.e. "de- 3500 1901-1901"). Previously this was only a recommendation 3501 ("SHOULD"). 3503 o Removed inappropriate [RFC2119] language from the illustration in 3504 Section 4.4.1. 3506 o Replaced the example of deprecating "zh-guoyu" with "zh- 3507 hakka"->"hak" in Section 4.5, noting that it was this document 3508 that caused the change. 3510 o Replaced the section in Section 4.1 dealing with "mul"/"und" to 3511 include the subtags 'zxx' and 'mis', as well as the tag 3512 "i-default". A normative reference to RFC 2277 was added, along 3513 with an informative reference to MARC21. 3515 o Added text to Section 3.5 clarifying that any modifications of a 3516 registration request must be sent to the ietf-languages@iana.org 3517 list before submission to IANA. 3519 o Changed the ABNF for the record-jar format from using the LWSP 3520 production to use a folding whitespace production similar to obs- 3521 FWS in [RFC5234]. This effectively prevents unintentional blank 3522 lines inside a field. 3524 o Clarified and revised text in Section 3.3, Section 3.5, and 3525 Section 5.1 to clarify that the Language Subtag Reviewer sends the 3526 complete registration forms to IANA, that IANA extracts the record 3527 from the form, and that the forms must also be archived separately 3528 from the registry. 3530 o Added text to Section 5 requiring IANA to send an announcement to 3531 an ietf-languages-announcements list whenever the registry is 3532 updated. 3534 o Modification of the registry to use UTF-8 as its character 3535 encoding. This also entails additional instructions to IANA and 3536 the Language Subtag Reviewer in the registration process. 3538 o Modified the rules in Section 2.2.4 so that "exceptionally 3539 reserved" ISO 3166-1 codes other than 'UK' were included into the 3540 registry. In particular, this allows the code 'EU' (European 3541 Union) to be used to form language tags or (more commonly) for 3542 applications that use the registry for region codes to reference 3543 this subtag. 3545 o Modified the IANA considerations section (Section 5) to remove 3546 unnecessary normative [RFC2119] language. 3548 9. References 3550 9.1. Normative References 3552 [ISO15924] 3553 International Organization for Standardization, "ISO 3554 15924:2004. Information and documentation -- Codes for 3555 the representation of names of scripts", January 2004. 3557 [ISO3166-1] 3558 International Organization for Standardization, "ISO 3166- 3559 1:2006. Codes for the representation of names of 3560 countries and their subdivisions -- Part 1: Country 3561 codes", November 2006. 3563 [ISO639-1] 3564 International Organization for Standardization, "ISO 639- 3565 1:2002. Codes for the representation of names of 3566 languages -- Part 1: Alpha-2 code", July 2002. 3568 [ISO639-2] 3569 International Organization for Standardization, "ISO 639- 3570 2:1998. Codes for the representation of names of 3571 languages -- Part 2: Alpha-3 code", October 1998. 3573 [ISO639-3] 3574 International Organization for Standardization, "ISO 639- 3575 3:2007. Codes for the representation of names of 3576 languages - Part 3: Alpha-3 code for comprehensive 3577 coverage of languages", February 2007. 3579 [ISO639-5] 3580 International Organization for Standardization, "ISO 639- 3581 5:2008. Codes for the representation of names of languages 3582 -- Part 5: Alpha-3 code for language families and groups", 3583 May 2008. 3585 [ISO646] International Organization for Standardization, "ISO/IEC 3586 646:1991, Information technology -- ISO 7-bit coded 3587 character set for information interchange.", 1991. 3589 [RFC2026] Bradner, S., "The Internet Standards Process -- Revision 3590 3", BCP 9, RFC 2026, October 1996. 3592 [RFC2028] Hovey, R. and S. Bradner, "The Organizations Involved in 3593 the IETF Standards Process", BCP 11, RFC 2028, 3594 October 1996. 3596 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 3597 Requirement Levels", BCP 14, RFC 2119, March 1997. 3599 [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and 3600 Languages", BCP 18, RFC 2277, January 1998. 3602 [RFC3339] Klyne, G., Ed. and C. Newman, "Date and Time on the 3603 Internet: Timestamps", RFC 3339, July 2002. 3605 [RFC4647] Phillips, A. and M. Davis, "Matching of Language Tags", 3606 BCP 47, RFC 4647, September 2006. 3608 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an 3609 IANA Considerations Section in RFCs", BCP 26, RFC 5226, 3610 May 2008. 3612 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 3613 Specifications: ABNF", STD 68, RFC 5234, January 2008. 3615 [UAX14] Freitag, A., "Unicode Standard Annex #14: Line Breaking 3616 Properties", August 2006, 3617 . 3619 [UN_M.49] Statistics Division, United Nations, "Standard Country or 3620 Area Codes for Statistical Use", Revision 4 (United 3621 Nations publication, Sales No. 98.XVII.9, June 1999. 3623 [Unicode] Unicode Consortium, "The Unicode Consortium. The Unicode 3624 Standard, Version 5.0, (Boston, MA, Addison-Wesley, 2003. 3625 ISBN 0-321-49081-0)", January 2007. 3627 9.2. Informative References 3629 [RFC1766] Alvestrand, H., "Tags for the Identification of 3630 Languages", RFC 1766, March 1995. 3632 [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 3633 Extensions (MIME) Part Two: Media Types", RFC 2046, 3634 November 1996. 3636 [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail Extensions) 3637 Part Three: Message Header Extensions for Non-ASCII Text", 3638 RFC 2047, November 1996. 3640 [RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and Encoded 3641 Word Extensions: 3642 Character Sets, Languages, and Continuations", RFC 2231, 3643 November 1997. 3645 [RFC2781] Hoffman, P. and F. Yergeau, "UTF-16, an encoding of ISO 3646 10646", RFC 2781, February 2000. 3648 [RFC3066] Alvestrand, H., "Tags for the Identification of 3649 Languages", RFC 3066, January 2001. 3651 [RFC3282] Alvestrand, H., "Content Language Headers", RFC 3282, 3652 May 2002. 3654 [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC 3655 Text on Security Considerations", BCP 72, RFC 3552, 3656 July 2003. 3658 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 3659 10646", STD 63, RFC 3629, November 2003. 3661 [RFC4645] Ewell, D., "Initial Language Subtag Registry", RFC 4645, 3662 September 2006. 3664 [RFC4646] Phillips, A. and M. Davis, "Tags for Identifying 3665 Languages", BCP 47, RFC 4646, September 2006. 3667 [UTS35] Davis, M., "Unicode Technical Standard #35: Locale Data 3668 Markup Language (LDML)", December 2007, 3669 . 3671 [draft-4645bis] 3672 Ewell, D., Ed., "Update to the Language Subtag Registry", 3673 May 2009, . 3676 [iso639.prin] 3677 ISO 639 Joint Advisory Committee, "ISO 639 Joint Advisory 3678 Committee: Working principles for ISO 639 maintenance", 3679 March 2000, 3680 . 3683 [record-jar] 3684 Raymond, E., "The Art of Unix Programming", 2003, 3685 . 3687 Appendix A. Acknowledgements 3689 Any list of contributors is bound to be incomplete; please regard the 3690 following as only a selection from the group of people who have 3691 contributed to make this document what it is today. 3693 The contributors to RFC 4646, RFC 4647, RFC 3066, and RFC 1766, the 3694 precursors of this document, made enormous contributions directly or 3695 indirectly to this document and are generally responsible for the 3696 success of language tags. 3698 The following people contributed to this document: 3700 Stephane Bortzmeyer, Karen Broome, Peter Constable, John Cowan, 3701 Martin Duerst, Frank Ellerman, Doug Ewell, Deborah Garside, Marion 3702 Gunn, Alfred Hoenes, Kent Karlsson, Chris Newman, Randy Presuhn, 3703 Stephen Silver, Shawn Steele, and many, many others. 3705 Very special thanks must go to Harald Tveit Alvestrand, who 3706 originated RFCs 1766 and 3066, and without whom this document would 3707 not have been possible. 3709 Special thanks go to Michael Everson, who served as the Language Tag 3710 Reviewer for almost the entire RFC 1766/RFC 3066 period, as well as 3711 the Language Subtag Reviewer since the adoption of RFC 4646. 3713 Special thanks also to Doug Ewell, for his production of the first 3714 complete subtag registry, his work to support and maintain new 3715 registrations, and his careful editorship of both RFC 4645 and 3716 [draft-4645bis]. 3718 Appendix B. Examples of Language Tags (Informative) 3720 Simple language subtag: 3722 de (German) 3724 fr (French) 3726 ja (Japanese) 3728 i-enochian (example of a grandfathered tag) 3730 Language subtag plus Script subtag: 3732 zh-Hant (Chinese written using the Traditional Chinese script) 3734 zh-Hans (Chinese written using the Simplified Chinese script) 3736 sr-Cyrl (Serbian written using the Cyrillic script) 3738 sr-Latn (Serbian written using the Latin script) 3740 Extended language subtags and their primary language subtag 3741 counterparts: 3743 zh-cmn-Hans-CN (Chinese, Mandarin, Simplified script, as used in 3744 China) 3746 cmn-Hans-CN (Mandarin Chinese, Simplified script, as used in 3747 China) 3749 zh-yue-HK (Chinese, Cantonese, as used in Hong Kong SAR) 3751 yue-HK (Cantonese Chinese, as used in Hong Kong SAR) 3753 Language-Script-Region: 3755 zh-Hans-CN (Chinese written using the Simplified script as used in 3756 mainland China) 3758 sr-Latn-RS (Serbian written using the Latin script as used in 3759 Serbia) 3761 Language-Variant: 3763 sl-rozaj (Resian dialect of Slovenian) 3764 sl-rozaj-biske (San Giorgio dialect of Resian dialect of 3765 Slovenian) 3767 sl-nedis (Nadiza dialect of Slovenian) 3769 Language-Region-Variant: 3771 de-CH-1901 (German as used in Switzerland using the 1901 variant 3772 [orthography]) 3774 sl-IT-nedis (Slovenian as used in Italy, Nadiza dialect) 3776 Language-Script-Region-Variant: 3778 hy-Latn-IT-arevela (Eastern Armenian written in Latin script, as 3779 used in Italy) 3781 Language-Region: 3783 de-DE (German for Germany) 3785 en-US (English as used in the United States) 3787 es-419 (Spanish appropriate for the Latin America and Caribbean 3788 region using the UN region code) 3790 Private use subtags: 3792 de-CH-x-phonebk 3794 az-Arab-x-AZE-derbend 3796 Private use registry values: 3798 x-whatever (private use using the singleton 'x') 3800 qaa-Qaaa-QM-x-southern (all private tags) 3802 de-Qaaa (German, with a private script) 3804 sr-Latn-QM (Serbian, Latin-script, private region) 3806 sr-Qaaa-RS (Serbian, private script, for Serbia) 3808 Tags that use extensions (examples ONLY: extensions MUST be defined 3809 by revision or update to this document or by RFC): 3811 en-US-u-islamcal 3813 zh-CN-a-myext-x-private 3815 en-a-myext-b-another 3817 Some Invalid Tags: 3819 de-419-DE (two region tags) 3821 a-DE (use of a single-character subtag in primary position; note 3822 that there are a few grandfathered tags that start with "i-" that 3823 are valid) 3825 ar-a-aaa-b-bbb-a-ccc (two extensions with same single-letter 3826 prefix) 3828 Appendix C. Examples of Registration Forms 3829 LANGUAGE SUBTAG REGISTRATION FORM 3830 1. Name of requester: Han Steenwijk 3831 2. E-mail address of requester: han.steenwijk @ unipd.it 3832 3. Record Requested: 3834 Type: variant 3835 Subtag: biske 3836 Description: The San Giorgio dialect of Resian 3837 Description: The Bila dialect of Resian 3838 Prefix: sl-rozaj 3839 Comments: The dialect of San Giorgio/Bila is one of the 3840 four major local dialects of Resian 3842 4. Intended meaning of the subtag: The local variety of Resian as 3843 spoken in San Giorgio/Bila 3845 5. Reference to published description of the language (book or 3846 article): 3847 -- Jan I.N. Baudouin de Courtenay - Opyt fonetiki rez'janskich 3848 govorov, Varsava - Peterburg: Vende - Kozancikov, 1875. 3850 LANGUAGE SUBTAG REGISTRATION FORM 3851 1. Name of requester: Jaska Zedlik 3852 2. E-mail address of requester: jz53 @ zedlik.com 3853 3. Record Requested: 3855 Type: variant 3856 Subtag: tarask 3857 Description: Belarusian in Taraskievica orthography 3858 Prefix: be 3859 Comments: The subtag represents Branislau Taraskievic's Belarusian 3860 orthography as published in "Bielaruski klasycny pravapis" by Juras 3861 Buslakou, Vincuk Viacorka, Zmicier Sanko, and Zmicier Sauka 3862 (Vilnia-Miensk 2005). 3864 4. Intended meaning of the subtag: 3866 The subtag is intended to represent the Belarusian orthography as 3867 published in "Bielaruski klasycny pravapis" by Juras Buslakou, Vincuk 3868 Viacorka, Zmicier Sanko, and Zmicier Sauka (Vilnia-Miensk 2005). 3870 5. Reference to published description of the language (book or article): 3872 Taraskievic, Branislau. Bielaruskaja gramatyka dla skol. Vilnia: Vyd. 3873 "Bielaruskaha kamitetu", 1929, 5th edition. 3875 Buslakou, Juras; Viacorka, Vincuk; Sanko, Zmicier; Sauka, Zmicier. 3876 Bielaruski klasycny pravapis. Vilnia-Miensk, 2005. 3878 6. Any other relevant information: 3880 Belarusian in Taraskievica orthography became widely used, especially in 3881 Belarusian-speaking Internet segment, but besides this some books and 3882 newspapers are also printed using this orthography of Belarusian. 3884 Authors' Addresses 3886 Addison Phillips (editor) 3887 Lab126 3889 Email: addison@inter-locale.com 3890 URI: http://www.inter-locale.com 3892 Mark Davis (editor) 3893 Google 3895 Email: mark.davis@google.com