Network Working Group A. Phillips, Ed. Internet-Draft webMethods, Inc. Expires: August 11, 2003 M. Davis IBM February 10, 2003 Tags for Identifying Languages draft-phillips-langtags-01 Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http:// www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on August 11, 2003. Copyright Notice Copyright (C) The Internet Society (2003). All Rights Reserved. Abstract This document describes a language tag for use in cases where it is desired to indicate the language used in an information object, how to register values for use in this language tag, and a construct for matching such language tags, including user defined extensions for private interchange. Phillips & Davis Expires August 11, 2003 [Page 1] Internet-Draft langtags February 2003 Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. The Language Tag . . . . . . . . . . . . . . . . . . . . . . 4 2.1 Syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Language Tag Sources . . . . . . . . . . . . . . . . . . . . 4 2.2.1 Pre-Existing RFC3066 Registrations . . . . . . . . . . . . . 8 2.2.2 Possibilities for Registration . . . . . . . . . . . . . . . 9 2.3 Choice of Language Tag . . . . . . . . . . . . . . . . . . . 10 2.4 Meaning of the Language Tag . . . . . . . . . . . . . . . . 13 2.4.1 Language Range . . . . . . . . . . . . . . . . . . . . . . . 14 2.4.2 Matching Language Tags . . . . . . . . . . . . . . . . . . . 15 3. IANA Considerations . . . . . . . . . . . . . . . . . . . . 16 3.1 Registration Procedure for Subtags . . . . . . . . . . . . . 16 4. Security Considerations . . . . . . . . . . . . . . . . . . 19 5. Character Set Considerations . . . . . . . . . . . . . . . . 20 6. Changes from RFC3066 . . . . . . . . . . . . . . . . . . . . 21 References . . . . . . . . . . . . . . . . . . . . . . . . . 22 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 23 A. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 24 B. Examples of Language Tags (Informative) . . . . . . . . . . 25 Intellectual Property and Copyright Statements . . . . . . . 27 Phillips & Davis Expires August 11, 2003 [Page 2] Internet-Draft langtags February 2003 1. Introduction Human beings on our planet have, past and present, used a number of languages. There are many reasons why one would want to identify the language used when presenting information. Information about a user's language preferences commonly needs to be identified so that appropriate processing can be applied. For example, the user's language preferences in a brower can be used to select web pages appropriately. A choice of language preference can also be used to select among tools (such as dictionaries) to assist in the processing or understanding of content in different langauges. In addition, knowledge about the particular language of information content may be useful or even required by some types of information processing; for example spell-checking, computer-synthesized speech, Braille transcription, or high-quality print renderings. One means of indicating the language used is by labeling the information content with a language identifier. These identifiers can also be used to specify user preferences when selecting information content, or for labeling additional attributes of content and associated resources. These identifiers can also be used to indicate additional attributes of content that are closely related to the language. In particular, it is often necessary to indicate specific information about the dialect, writing system, or orthography used in a document or resource, as these attributes may be important for the user to obtain information in a form that they can understand, or important in selecting appropriate processing resources for the given content. This document specifies an identifier mechanism, a registration function for values to be used with that identifier mechanism, and a construct for matching against those values. It also defines a mechanism for private use extension and how private use, registered values, and matching interact. The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119] [10]. Phillips & Davis Expires August 11, 2003 [Page 3] Internet-Draft langtags February 2003 2. The Language Tag 2.1 Syntax The language tag is composed of one or more parts: A primary language subtag and a (possibly empty) series of subsequent subtags. The sequence of subtags has a specific structure that depends on the length of the subtag to distinguish each tag type. The syntax of this tag in ABNF [RFC 2234] [11] is: = lang *["-s-" extlang] ["-" script] ["-" region] *["-" variant] ["-x" extensions] =/ "x" extensions ; private use tag =/ grandfathered-registrations lang = 2*3 ALPHA ; shortest ISO 639 tag =/ registered-lang extlang = 2*15 ALPHA ; additional language subtag script = 4 ALPHA ; ISO 15924 tag region = 2 ALPHA ; ISO 3166 tag =/ 3 DIGIT ; UN country number variant = 5*15 alphanum extensions = 1* ("-" value) ; private use extensions value = 1*32 alphanum registered-lang = 5*15 alphanum grandfathered-registrations = ALPHA * (alphanumdash) alphanum = (ALPHA / DIGIT) alphanumdash = (alphanum / "-") Figure 1: Language Tag ABNF The character "-" is HYPHEN-MINUS (ABNF: %x2D). All tags and subtags, including private use extensions, are to be treated as case insensitive: there exist conventions for the capitalization of some of them, but these should not be taken to carry meaning. For instance, [ISO 3166] [4] recommends that country codes be capitalized (MN Mongolia), while [ISO 639] [3] recommends that language codes be written in lower case (mn Mongolian). For examples, see Appendix B at the end of this document. 2.2 Language Tag Sources The namespace of language tags is administered by the Internet Assigned Numbers Authority (IANA) [15] according to the rules in Section 3.1. Terminology in this section: Phillips & Davis Expires August 11, 2003 [Page 4] Internet-Draft langtags February 2003 o Tag or tags refers to a complete language tag, such as 'fr-Latn-CA' o Subtag refers to a specific section of a tag, separated by hyphen, such as 'Latn' in 'fr-Latn-CA' o Code or codes refers to the tags defined in external standards. For example, 'Latn' is an ISO 15924 [2] script code (which can be used as a script subtag in a language tag) The rules in this section apply to the various subtags within the language tags defined in this document, excepting those "grandfathered" tags defined in Section 2.2.1. Those tags should be considered as exceptions to the rules presented here. Single letter and digit subtags are reserved for current or future use with revisions of this document. These include the following current uses: o The single letter subtag "i" is reserved for use with grandfathered IANA registrations that begin with that tag, such as "i-hakka". o The single letter subtag "x" is reserved to introduce private use tags (as the primary subtag of a tag) or to introduce private use extensions (in any other position in the tag). o The single letter subtag "s" is reserved to introduce a language extension subtag. Language extension subtags are currently reserved for future standardization. The following rules apply to the primary (language) subtag: o All 2-character language subtags are interpreted according to assignments found in ISO standard 639, "Code for the representation of names of languages" [ISO639-1] [3], or assignments subsequently made by the ISO 639 Part 1 maintenance agency or governing standardization bodies. o All 3-character language subtags are interpreted according to assignments found in ISO 639 part 2, "Codes for the representation of names of languages -- Part 2: Alpha-3 code [ISO 639-2] [1]", or assignments subsequently made by the ISO 639 part 2 maintenance agency or governing standardization bodies, or assignments of 3-character disambiguation registrations according to Rule 7a. Ambiguity in Section 2.3 of this document. o ISO639-2 reserves for private use codes the range 'qaa' through Phillips & Davis Expires August 11, 2003 [Page 5] Internet-Draft langtags February 2003 'qtz'. These codes should be used for non-registered language subtags. o Subtags of 5 to 15 characters may be registered with IANA, according to the rules in Section 3.1 of this document. (Note that previously, in rfc3066 [16], the IANA registry contained whole tag registrations such as de-CH-1994, whereas this document refers to the registration of subtags such as 'klingon') o The single character subtag "x" as the primary subtag indicates that the whole language tag is a private use tag. The value and semantic meaning of such a tag as a whole and of the subtags used within such as tag are undefined by this document. o Other values shall not be assigned except by revision of this document. The following rules apply to the extended language subtags: o Each extended language subtag must be prefixed with the single letter subtag "s". o Extended language subtags are currently reserved for future standardization. The following rules apply to the script subtags: o All 4-character subtags are interpreted as ISO 15924 alpha-4 script codes from [2], or subsequently assigned by the ISO 15924 maintenance agency or governing standardization bodies, denoting the script or writing system used in conjunction with this language. These alpha4 tags may only occur as the second subtag in a tag. For example: 'de-Latn' represents German written using Latin script. o ISO 15924 reserves the codes Qaaa-Qacz for private use values. These codes should be used for non-registered script values. o Script subtags can NOT be registered using the process in Section 3.1 of this document. Variant subtags may be considered for registration for that purpose. Note that registered subtags can only appear in the first (primary) position (for registered language subtags) or as a sequence on the end of a language tag. This makes the associated exception handling when parsing tags simpler. The following rules apply to the region subtags: Phillips & Davis Expires August 11, 2003 [Page 6] Internet-Draft langtags February 2003 o All 2-character subtags following the primary subtag denote the area to which this language variant relates, and are interpreted according to assignments found in ISO 3166 alpha-2 country codes from [4], assignments subsequently made by the ISO 3166 maintenance agency or governing standardization bodies. o Region subtags must occur after any script subtags and before any variant subtags or extensions. o The ISO3166 code must be used to form the subtag, except in cases where the tag is ambiguous according to Rule 7a in Section 2.3.Example: 'de-Latn-CH' represents German written using Latin script for Switzerland. o Three digit numeric codes from the UN Standard Country or Area Codes for Statistical Use [5] must be used for countries with ambiguous ISO 3166 alpha-2 codes as defined in Rule 7a in Section 2.3. These codes may also be used for countries or regions for which no corresponding ISO 3166 code has been assigned, including supra-national and sub-national regions. Note: the alphanumeric codes in Appendix X of the UN document must not be used. (At the time this document was created these values match the ISO 3166 alpha-2 codes.) o ISO 3166 reserves the country codes AA, QM-QZ, XA-XZ and ZZ as user-assigned codes. These codes should be used for private use region subtags. o No region subtags can be registered using the process in Section 3.1 of this document. Variant subtags may be considered for registration for this purpose. Note that registered subtags can only appear in the first position or as a sequence on the end of a language tag. This makes the associated exception handling when parsing tags simpler. The following rules apply to the variant subtags: o Alphanumeric subtags of 5 to 15 characters may be registered with IANA, according to the rules in Section 3.1 of this document. Registered subtags must not begin with the character 'x', which is reserved for private use subtags. (Note that previously, in rfc3066, the IANA registry contained whole tag registrations such as 'en-boont', whereas this document refers to the registration of subtags such as 'boont') o Alphanumeric subtags of 5 to 15 characters starting with 'x' are reserved for private use. The semantics of these subtags must be defined by the end users of such subtags and the semantic meaning Phillips & Davis Expires August 11, 2003 [Page 7] Internet-Draft langtags February 2003 should be considered external to this document. The following rules apply to private use extensions: o Private Use Extensions subtags are separated from the other subtags defined in this document by the reserved single-character subtag "x". o Private Use Extensions must follow all language, extended language, script, region, and variant subtags. o The reserved subtag "x" must be followed by at least one subtag. o Private Use Extension subtags must consist of one to thirty-two alphanumeric characters, with each subtag separated by a single "-". o No source is defined for private use extensions. Use or standardization of the private use extension subtags is by private agreement and should not be considered part of this document. For example: Users who wished to utilize SIL Ethonologue for identification might agree to exchange tags such as 'az-Arab-x-AZE-derbend'. This example contains two extension subtags. The first is "AZE" and the second is "derbend". 2.2.1 Pre-Existing RFC3066 Registrations Existing IANA registered language tags from RFC1766/RFC3066 that are not defined by additions to this document maintain their validity. IANA will maintain these tags, adding a notation that they are "grandfathered from RFC 3066". The rules governing existing RFC 1766 and RFC 3066 registered tags are: o If the formerly registered tag would now be defined by this document, then the existing tag is marked as superseded by this document and no subtag will be registered as a result. For example, zh-Hans would now be defined by the addition of ISO 15924 script codes. o If the registered tag contained one or more subtags that follow the guidelines for registered language or variant subtags, and all of the subtags are either now defined by this document or would be valid to register, then each subtag not already covered by this document will be registered automatically by IANA without further review and the existing tag marked as superseded by this document. Phillips & Davis Expires August 11, 2003 [Page 8] Internet-Draft langtags February 2003 For example: the tag 'en-boont' fits the pattern for a registered variant. The variant subtag "boont" will be registered automatically and 'en-boont' marked as superseded. o If the registered tag contains any subtags that are not otherwise valid for registration according to the rules in this document, then the tag as a whole is maintained as an exceptional case (that is, it is "grandfathered"). This includes special cases of Sign Language tags. For example, the tag 'i-klingon' is not covered by any addition and is grandfathered, as is sgn-BE-fr (Belgian French Sign Language). Users of tags that are grandfathered should consider registering the appropriate subtags using the new format (but are not required to). 2.2.2 Possibilities for Registration Possibilities for registration of subtags include: o Languages not listed in ISO 639 that are not variants of any listed language, can be registered, such as tsolyani. Before attempting to register a language subtag, there should be a good faith attempt to register the language with ISO 639. No language subtags will be registered for codes that exist in ISO 639-1 or ISO 639-2. o Dialect or other divisions or variations within a language, its orthography, writing system, regional variation, or historical usage, such as the "scouse" subtag (the Scouse dialect of English). This document leaves the decision on what subtags are appropriate or not to the registration process described in Section 3.1. ISO 639 defines a maintenance agency for additions to and changes in the list of languages in ISO 639. This agency is: International Information Centre for Terminology (Infoterm) Aichholzgasse 6/12, AT-1120 Wien, Austria Phone: +43 1 26 75 35 Ext. 312 Fax: +43 1 216 32 72 ISO 639-2 defines a maintenance agency for additions to and changes in the list of languages in ISO 639-2. This agency is: Library of Congress Network Development and MARC Standards Office Washington, D.C. 20540 USA Phillips & Davis Expires August 11, 2003 [Page 9] Internet-Draft langtags February 2003 Phone: +1 202 707 6237 Fax: +1 202 707 0115 URL: http://www.loc.gov/standards/iso639 The maintenance agency for ISO 3166 (country codes) is: ISO 3166 Maintenance Agency c/o International Organization for Standardization Case postale 56 CH-1211 Geneva 20 Switzerland Phone: +41 22 749 72 33 Fax: +41 22 749 73 49 URL: http://www.iso.org/iso/en/prods-services/iso3166ma/index.html The registration authority for ISO 15924 (script codes) is: Unicode Consortium Box 391476 Mountain View, CA 94039-1476, USA URL: http://www.unicode.org/iso15924 The Statistics Division of the United Nations Secretariat maintains the Standard Country or Area Codes for Statistical Use and can be reached at: Statistical Services Branch Statistics Division United Nations, Room DC2-1620 New York, NY 10017, USA Fax: +1-212-963-0623 E-mail: statistics@un.org 2.3 Choice of Language Tag One may occasionally be faced with several possible tags for the same body of text. Interoperability is best served if all users send the same tag, and use the same tag for the same language for all documents. If an application has requirements that make the rules here inapplicable, the application protocol specification MUST specify how the procedure varies from the one given here. The text below is based on the set of tags known to the tagging entity. 1. Use as precise a tag as possible, but no more specific than is justified. For example, 'de' might suffice for tagging an email written in German, while 'de-CH-1996' is probably unnecessarily precise for such a task. Phillips & Davis Expires August 11, 2003 [Page 10] Internet-Draft langtags February 2003 2. Avoid using subtags that add no distinguishing information about the content. For example, the script subtag in 'en-Latn-US' is generally unnecessary, since nearly all English texts are written in the Latin script. 3. When a language has both an ISO 639-1 2-character code and an ISO 639-2 3-character code, you MUST use the ISO 639-1 2-character code. 4. When a language has no ISO 639-1 2-character code, and the ISO 639- 2/T (Terminology) code and the ISO 639-2/B (Bibliographic) codes differ, you MUST use the Terminology code. NOTE: At present all languages that have both kinds of 3-character code also are assigned a 2-character code, and the displeasure of developers about the existence of two different code sets has been adequately communicated to ISO. So this situation will hopefully not arise. 5. You SHOULD NOT use the UND (Undetermined) code unless the protocol in use forces you to give a value for the language tag, even if the language is unknown. Omitting the tag is preferred. 6. You SHOULD NOT use the MUL (Multiple) tag if the protocol allows you to use multiple languages, as is the case for the Content-Language header in HTTP. NOTE: In order to avoid versioning difficulties in applications such as that experienced in RFC 1766 [8], the ISO 639 Registration Authority Joint Advisory Committee (RA-JAC) has agreed on the following policy statement: "After the publication of ISO/DIS 639-1 as an International Standard, no new 2-letter code shall be added to ISO 639-1 unless a 3-letter code is also added at the same time to ISO 639-2. In addition, no language with a 3-letter code available at the time of publication of ISO 639-1 which at that time had no 2-letter code shall be subsequently given a 2-letter code." This will ensure that, for example, a user who implements "haw" (Hawaiian), which currently has no 2-character code, will not find his or her data invalidated by eventual addition of a 2-character code for that language." 7. To maintain backwards compatibility, there are two provisions to account for potential instability in ISO 639, 3166, and 15924 codes. Phillips & Davis Expires August 11, 2003 [Page 11] Internet-Draft langtags February 2003 a) Ambiguity. In the event that one of these standards assigns a code a new meaning or reassigns a deprecated code, the new use of the code will not be permitted in language tags defined by this document. In the event that either ISO 639 or ISO 15924 assigns a new meaning to an existing code, the language subtag reviewer, as described in Section 3, shall prepare a proposal for entering in the IANA registry as soon as practical a variant or registered language subtag as a surrogate value for the new code. The form of the registered language subtag or variant subtag will be at the discretion of the language subtag reviewer and must conform to other restrictions on language or variant subtags in this document. In the event ISO 3166 assigns a new meaning to an existing code, then the language subtag reviewer, as described in Section 3, shall prepare a proposal for entering the appropriate numeric UN country code as an informative entry in the IANA registry. The normal registration process described in Section 3.1 of this document applies to the review and registration of the registered subtags described above. Note that these subtags should never be used in combination with the subtag type for which they are a surrogate. For example, a "region" variant subtag should not be used with a region subtag. For example: cs-CS (Czech for Czechoslovakia) sr-891 (Serbian for Serbia and Montenegro, using the UN country code) qx-Latn (hypothetical reassigned value 'qx') qx2003-Latn (hypothetical registered language subtag) b) Stability. All other ISO codes are valid, even if they have been deprecated. Some examples, current at the time this document was drafted, are listed below. Where a new equivalent code has been defined (given below on the right side after a tilde), implementations should treat these tags as identical. For example, some deprecated ISO 639 codes: Phillips & Davis Expires August 11, 2003 [Page 12] Internet-Draft langtags February 2003 iw ~ he in ~ id ji ~ yi sh For example, some deprecated ISO 3166 codes: FX TP ~ TL YU 2.4 Meaning of the Language Tag The language tag always defines a language as spoken (or written, signed or otherwise signaled) by human beings for communication of information to other human beings. Computer languages such as programming languages are explicitly excluded. If a language tag B contains language tag A as a prefix, then B is typically "narrower" or "more specific" than A. For example, 'zh-Hant-TW' is more specific than 'zh-Hant'. This relationship is not guaranteed in all cases: specifically, languages that begin with the same sequence of subtags are NOT guaranteed to be mutually intelligible, although they may be. For example, the tag 'az' shares a prefix with both 'az-Latn' (Azerbaijani written using the Latin script) and 'az-Cyrl' (Azerbaijani written using the Cyrillic script). A person fluent in one script may not be able to read the other, even though the text might be identical. Content tagged as 'az' most probably is written in just one script and thus might not be intelligible to a reader familiar with the other script. The relationship between the tag and the information it relates to is defined by the standard describing the context in which it appears. Accordingly, this section can only give possible examples of its usage. o For a single information object, it could be taken as the set of languages that is required for a complete comprehension of the complete object. Example: Plain text documents. Phillips & Davis Expires August 11, 2003 [Page 13] Internet-Draft langtags February 2003 o For an aggregation of information objects, it should be taken as the set of languages used inside components of that aggregation. Examples: Document stores and libraries. o For information objects whose purpose is to provide alternatives, the set of tags associated with it should be regarded as a hint that the content is provided in several languages, and that one has to inspect each of the alternatives in order to find its language or languages. In this case, a tag with multiple languages does not mean that one needs to be multi-lingual to get complete understanding of the document. Example: MIME multipart/ alternative. o In markup languages, such as HTML and XML, language information can be added to each part of the document identified by the markup structure (including the whole document itself). For example, one could write C'est la vie. inside a Norwegian document; the Norwegian-speaking user could then access a French-Norwegian dictionary to find out what the marked section meant. If the user were listening to that document through a speech synthesis interface, this formation could be used to signal the synthesizer to appropriately apply French text-to-speech pronunciation rules to that span of text, instead of misapplying the Norwegian rules. 2.4.1 Language Range A Language Range is a set of languages whose tags all begin with the same sequence of subtags. The following definition of language-range is derived from HTTP/1.1 [14]. language-range = language-tag / "*" That is, a language-range has the same syntax as a language-tag, or is the single character "*" and implicitly assumes that there is a semantic relationship between tags that share the same subtag prefixes. A language-range matches a language-tag if it exactly equals the tag, or if it exactly equals a prefix of the tag such that the first character following the prefix is "-". The special range "*" matches any tag. A protocol which uses language ranges may specify additional rules about the semantics of "*"; for instance, HTTP/1.1 specifies that the range "*" matches only languages not matched by any other range within an "Accept-Language:" header. Phillips & Davis Expires August 11, 2003 [Page 14] Internet-Draft langtags February 2003 As noted above, not all languages or content denoted by a specific language-range may be mutually intelligible and this use of a prefix matching rule does not imply that language tags are assigned to languages in such a way that it is always true that if a user understands a language with a certain tag, then this user will also understand all languages with tags for which this tag is a prefix. The prefix rule simply allows the use of prefix tags if this is the case. 2.4.2 Matching Language Tags Implementations that are searching for content or otherwise matching language tags to a language-range [Section 2.4.1] may choose to assume that there is a semantic relationship between two tags that share common prefixes. This is called 'language tag fallback'. The most common implementations follow this pattern: 1. When searching for content using language tag fallback, the language tag is progressively truncated from the end until a match is located. For example, starting with the tag 'en-US-boont', searchs or matches would first be performed with the whole tag, then with 'en-US', and finally with 'en'. This allows some flexibility in finding content in accordance with Rules 1 and 2 in Section 2.3; allows better maintenance; and usually provides better results when data is not available at a specific level of tag granularity or is sparsely populated (than if the default language for the system or content were used). Any implementation that uses this technique should ensure that appropriate data is available on each level. Tag to match: en-US-boont 1. en-US-boont 2. en-US 3. en Figure 2: Default Fallback Pattern Example 2. Private Use Extensions are orthogonal to language tag fallback. By default, implementations should ignore private use extensions and follow the default fallback pattern (above). Thus matching the tag "en-US-boont-x-traditional" would be exactly the same as the example above. 3. Implementations that choose to interpret one or more private use extension subtags can choose a different fallback pattern or use the private use extensions to interpret content in a different fashion. Phillips & Davis Expires August 11, 2003 [Page 15] Internet-Draft langtags February 2003 3. IANA Considerations This section deals with the registration of subtags for use in language tags defined by this document, in accordance with the requirements of RFC2434 [13]. 3.1 Registration Procedure for Subtags The procedure given here MUST be used by anyone who wants to use a subtag not given an interpretation in Section 2.2 of this document or previously registered with IANA. This procedure MAY also be used to register information with the IANA about a tag or subtag defined by this document, for instance if one wishes to make publicly available a reference to the definition for a language such as sgn-US (American Sign Language), or additional information about a registration previously made via this procedure. Variant subtags MUST NOT be registered using the pattern 2 ALPHA * DIGIT to accommodate the provisions in Section 2.3, rule 7a of this document. That is, the subtag yx1234 can NOT be registered except under the aforementioned provisions. Extended language subtags that are registered MUST be at least 4 characters long. Subtags MUST NOT be registered that start with the letter 'x', since this prefix is reserved for Private Use subtags. The process starts by filling out the registration form reproduced below. LANGUAGE SUBTAG REGISTRATION FORM Name of requester: E-mail address of requester: Subtag to be registered: Type of Subtag: [ ] language [ ] extended language [ ] variant [ ] region (informative; for use by language subtag reviewer only) Full English name of subtag: Intended meaning of the subtag: If variant subtag, the intended prefix(es) of subtag: If extended language subtag, the intended prefix(es) of subtag: Native name of language (transcribed into ASCII): Reference to published description of the language (book or article): Any other relevant information: Phillips & Davis Expires August 11, 2003 [Page 16] Internet-Draft langtags February 2003 Figure 3 The subtag registration form MUST be sent to for a two week review period before it can be submitted to IANA. (This is an open list. Requests to be added should be sent to .) Variant subtags are generally registered for use with a particular prefix or set of prefixes. For example, the subtag 'boont' is intended for use with the prefix 'en-', since Boontling is a dialect of English. This information MUST be provided in the registration form. Any registered subtag MAY be incorporated into a variety of language tags, according to the rules of Section 2.1. This makes validation simpler and thus more uniform across implementations, and does not require new registrations for different intended prefixes. However, the intended prefixes for a given registered subtag will be maintained in the IANA registry as a guide to usage. If it is necessary to add an additional intended prefix to that list for an existing language tag, that can be done by filing an additional registration form. In that form, the "Any other relevant information: " field should indicate that it is the addition of an additional intended prefix. When the two week period has passed, the subtag reviewer, who is appointed by the IETF Applications Area Director, either forwards the request to IANA@IANA.ORG, or rejects it because of significant objections raised on the list. Note that the reviewer can raise objections on the list himself, if he or she so desires. The important thing is that the objection must be made publicly. The applicant is free to modify a rejected application with additional information and submit it again; this restarts the two week comment period. Decisions made by the reviewer may be appealed to the IESG [RFC 2028] [9] under the same rules as other IETF decisions [RFC 2026] [18]. All registered forms are available online in the directory http:// www.iana.org/numbers.html under "languages". Updates of registrations follow the same procedure as registrations. The subtag reviewer decides whether to allow a new registrant to update a registration made by someone else; normally objections by the original registrant would carry extra weight in such a decision. Registrations are permanent and stable. When some registered subtag Phillips & Davis Expires August 11, 2003 [Page 17] Internet-Draft langtags February 2003 should not be used any more, for instance because a corresponding ISO 639 code has been created, the registration should be amended by adding a remark like "DEPRECATED: use instead" to the "other relevant information" section. Note: The purpose of the "published description" is intended as an aid to people trying to verify whether a language is registered, or what language a particular subtag refers to. In most cases, reference to an authoritative grammar or dictionary of that language will be useful; in cases where no such work exists, other well known works describing that language or in that language may be appropriate. The subtag reviewer decides what constitutes "good enough" reference material. Phillips & Davis Expires August 11, 2003 [Page 18] Internet-Draft langtags February 2003 4. Security Considerations The only security issue that has been raised with language tags since the publication of RFC 1766, which stated that "Security issues are believed to be irrelevant to this memo", is a concern with language ranges used in content negotiation - that they may be used to infer the nationality of the sender, and thus identify potential targets for surveillance. This is a special case of the general problem that anything you send is visible to the receiving party. It is useful to be aware that such concerns can exist in some cases. The evaluation of the exact magnitude of the threat, and any possible countermeasures, is left to each application protocol. Phillips & Davis Expires August 11, 2003 [Page 19] Internet-Draft langtags February 2003 5. Character Set Considerations Language tags may always be presented using the characters A-Z, a-z, 0-9, and HYPHEN-MINUS, which are present in most character sets, so presentation of language tags should not have any character set issues. The issue of deciding upon the rendering of a character set based on the language tag is not addressed in this memo; however, it is thought impossible to make such a decision correctly for all cases unless means of switching language in the middle of a text are defined (for example, a rendering engine that decides font based on Japanese or Chinese language may produce sub-optimal output when a mixed Japanese- Chinese text is encountered) Phillips & Davis Expires August 11, 2003 [Page 20] Internet-Draft langtags February 2003 6. Changes from RFC3066 The main goals were to maintain backward compatibility (so that all previous codes would remain valid); reduce the need for large numbers of registrations; to provide a more formal structure to allow parsing into subtags even where software does not have the latest registrations; to provide stability in the face of potential instability in ISO 639, 3166, and 15924 codes (*demonstrated instability* in the case of ISO 3166); and to allow for external extension mechanisms. o Allows ISO15924 script code subtags and allows them to be used generatively. o Adds the concept of a variant subtag and allows variants to be used generatively. o Adds an extension mechanism which does not require registration to use. o Defines the private use tags in ISO639, ISO15924, and ISO3166 as the mechanism for creating private use language, script, and region subtags respectively o Defines a syntax for private use variant subtags which can be used without registration. o Defines a process for handling reuse of values by ISO639, ISO15924, and ISO3166 in the event that they register a previously used value for a new purpose. o Changes the IANA language tag registry to a language subtag registry Substantive changes between draft-01 and this version are: Removed the key-value structure from extensions Added the concept of extended language subtags, reserved for future use with ISO639-4 or some other mechanism. Reserved single language subtags explicitly Phillips & Davis Expires August 11, 2003 [Page 21] Internet-Draft langtags February 2003 References [1] International Organization for Standardization, "ISO 639-2:1998 - Codes for the representation of names of languages -- Part 2: Alpha-3 code - edition 1", August 1988. [2] ISO TC46/WG3 and M. Everson, Ed., "ISO 15924:2003 (E/F) - Codes for the representation of names of scripts", March 2003. [3] International Organization for Standardization, "Code for the representation of names of languages, 1st edition", ISO Standard 639, 1988. [4] International Organization for Standardization, "Codes for the representation of names of countries, 3rd edition", ISO Standard 3166, August 1988. [5] Statistical Division, United Nations, "Standard Country or Area Codes for Statistical Use", UN Standard Country or Area Codes for Statistical Use, Revision 4 (United Nations publication, Sales No. 98.XVII.9, June 1999. [6] Hardcastle-Kille, S., "Mapping between X.400(1988) / ISO 10021 and RFC 822", RFC 1327, May 1992. [7] Borenstein, N. and N. Freed, "MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies", RFC 1521, September 1993. [8] Alvestrand, H., "Tags for the Identification of Languages", RFC 1766, March 1995. [9] Hovey, R. and S. Bradner, "The Organizations Involved in the IETF Standards Process", BCP 11, RFC 2028, October 1996. [10] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [11] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 2234, November 1997. [12] Berners-Lee, T., Fielding, R. and L. Masinter, "Uniform Resource Identifiers (URI): Generic Syntax", RFC 2396, August 1998. [13] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 2434, October Phillips & Davis Expires August 11, 2003 [Page 22] Internet-Draft langtags February 2003 1998. [14] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [15] Carpenter, B., Baker, F. and M. Roberts, "Memorandum of Understanding Concerning the Technical Work of the Internet Assigned Numbers Authority", RFC 2860, June 2000. [16] Alvestrand, H., "Tags for the Identification of Languages", BCP 47, RFC 3066, January 2001. [17] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. [18] Authors' Addresses Addison Phillips (editor) webMethods, Inc. 432 Lakeside Drive Sunnyvale, CA 94088 US EMail: aphillips@webmethods.com Mark Davis IBM EMail: mark.davis@us.ibm.com Phillips & Davis Expires August 11, 2003 [Page 23] Internet-Draft langtags February 2003 Appendix A. Acknowledgements Any list of contributors is bound to be incomplete; please regard the following as only a selection from the group of people who have contributed to make this document what it is today. The contributors to RFC 3066 and RFC 1766, the precursors of this document, made enormous contributions directly or indirectly to this document and are generally responsible for the success of language tags. The following people (in alphabetical order) contributed to this document or to RFCs 1766 and 3066: Glenn Adams, Harald Tveit Alvestrand, Tim Berners-Lee, Marc Blanchet, Nathaniel Borenstein, Eric Brunner, Sean M. Burke, John Clews, Jim Conklin, Peter Constable, John Cowan, Mark Crispin, Dave Crocker, Martin Duerst, Michael Everson, Ned Freed, Tim Goodwin, Dirk-Willem van Gulik, Marion Gunn, Paul Hoffman, Richard Ishida, Olle Jarnefors, Kent Karlsson, John Klensin, Alain LaBonte, Eric Mader, Keith Moore, Chris Newman, Masataka Ohta, George Rhoten, Markus Scherer, Keld Jorn Simonsen, Thierry Sourbier, Otto Stolz, Tex Texin, Rhys Weatherley, Misha Wolf, Francois Yergeau and many, many others. Very special thanks must go to Harald Tveit Alvestrand, who originated RFCs 1766 and 3066, and without whom this document would not have been possible. Special thanks must go to Michael Everson, who has served as language tag reviewer for almost the complete period since the publication of RFC 1766. Phillips & Davis Expires August 11, 2003 [Page 24] Internet-Draft langtags February 2003 Appendix B. Examples of Language Tags (Informative) Simple language code: de (German) fr (French) ja (Japanese) Language code plus Script code : zh-Hant (Traditional Chinese) en-Latn (English written in Latin script) sr-Cyrl (Serbian written with Cyrillic script) Language-Script-Region: zh-Hans-CN (Simplified Chinese for the PRC) sr-Latn-891 (Serbian, Latin script, Serbia and Montenegro) Language-Script-Region-Variant: en-Latn-US-boont (Boontling dialect of English) Language-Region: de-DE (German for Germany) zh-SG (Chinese for Singapore) cs-CS (Czech for Czechoslovakia) sr-891 (Serbian for Serbia and Montenegro, UN country code, see 7a in Section 2.3 Other Mixtures: zh-CN (Chinese for the PRC) en-boont (Boontling dialect of English) Extension mechanism: Phillips & Davis Expires August 11, 2003 [Page 25] Internet-Draft langtags February 2003 de-CH-x-phonebook az-Arab-x-AZE-derbend Extended language subtags: zh-s-min zh-s-min-s-nan-Hant-CN Private Use tags: qaa-Qaaa-QM-xsouthern (all private tags) de-Qaaa (German, with a private script) de-Latn-QM (German, Latin-script, private region) de-Qaaa-DE (German, private script, for Germany) Some Invalid Tags: de-891-DE (two region tags) a-DE (use of a single character tag) zh-xsouthern-DE (private use variant followed by another tag) Phillips & Davis Expires August 11, 2003 [Page 26] Internet-Draft langtags February 2003 Intellectual Property Statement The IETF takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Information on the IETF's procedures with respect to rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementors or users of this specification can be obtained from the IETF Secretariat. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director. Full Copyright Statement Copyright (C) The Internet Society (2003). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assignees. This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION Phillips & Davis Expires August 11, 2003 [Page 27] Internet-Draft langtags February 2003 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgment Funding for the RFC Editor function is currently provided by the Internet Society. Phillips & Davis Expires August 11, 2003 [Page 28]