idnits 2.17.1 draft-ietf-ipvbi-nabts-05.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1061 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (Sept 1999) is 9257 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: 'Romkey 1988' on line 351 -- Looks like a reference, but probably isn't: 'RFC 2434' on line 524 -- Looks like a reference, but probably isn't: 'A' on line 713 -- Looks like a reference, but probably isn't: 'B' on line 713 == Missing Reference: '0' is mentioned on line 761, but not defined -- Looks like a reference, but probably isn't: 'Byte' on line 918 == Unused Reference: '4' is defined on line 572, but no explicit reference was found in the text == Unused Reference: '5' is defined on line 575, but no explicit reference was found in the text == Unused Reference: '6' is defined on line 578, but no explicit reference was found in the text == Unused Reference: '7' is defined on line 581, but no explicit reference was found in the text == Unused Reference: '8' is defined on line 584, but no explicit reference was found in the text == Unused Reference: '9' is defined on line 588, but no explicit reference was found in the text == Unused Reference: '10' is defined on line 591, but no explicit reference was found in the text == Unused Reference: '14' is defined on line 603, but no explicit reference was found in the text == Unused Reference: '15' is defined on line 606, but no explicit reference was found in the text == Unused Reference: '16' is defined on line 609, but no explicit reference was found in the text == Unused Reference: '17' is defined on line 612, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. '2' -- Possible downref: Non-RFC (?) normative reference: ref. '3' -- Possible downref: Non-RFC (?) normative reference: ref. '4' -- Possible downref: Non-RFC (?) normative reference: ref. '5' -- Possible downref: Non-RFC (?) normative reference: ref. '6' -- Possible downref: Non-RFC (?) normative reference: ref. '8' ** Obsolete normative reference: RFC 2434 (ref. '9') (Obsoleted by RFC 5226) -- Possible downref: Non-RFC (?) normative reference: ref. '10' -- Possible downref: Non-RFC (?) normative reference: ref. '11' -- Possible downref: Non-RFC (?) normative reference: ref. '12' -- Possible downref: Non-RFC (?) normative reference: ref. '13' -- Possible downref: Non-RFC (?) normative reference: ref. '14' -- Possible downref: Non-RFC (?) normative reference: ref. '16' -- Possible downref: Non-RFC (?) normative reference: ref. '17' Summary: 7 errors (**), 0 flaws (~~), 14 warnings (==), 20 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 IPVBI R. Panabaker, Microsoft 2 Internet Draft S. Wegerif, Philips Semiconductors 3 D. Zigmond, WebTV Networks 5 Document: Sept 1999 7 The Transmission of IP Over the Vertical Blanking Interval of a 8 Television Signal 10 Status of this Memo 12 This document is an Internet-Draft and is in full conformance with 13 all provisions of Section 10 of RFC2026. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six months 21 and may be updated, replaced, or obsoleted by other documents at any 22 time. It is inappropriate to use Internet-Drafts as reference 23 material or to cite them other than as "work in progress."100 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 To view the entire list of current Internet-Drafts, please check the 32 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 33 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern 34 Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific 35 Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 37 A revised version of this draft document will be submitted to the RFC 38 editor as a Proposed Standard for the Internet Community. Discussion 39 and suggestions for improvement are requested. This document will 40 expire before August 1999. Distribution of this draft is unlimited. 42 1. Abstract 44 This document describes a method for broadcasting IP data in a 45 unidirectional manner using the vertical blanking interval of 46 television signals. It includes a description for compressing IP 47 headers on unidirectional networks, a framing protocol identical to 48 SLIP, a forward error correction scheme, and the NABTS byte 49 structures. 51 2. Introduction 53 This RFC proposes several protocols to be used in the transmission of 54 IP datagrams using the Vertical Blanking Interval (VBI) of a 55 television signal. The VBI is a non-viewable portion of the 56 television signal that can be used to provide point-to-multipoint IP 57 data services which will relieve congestion and traffic in the 58 traditional Internet access networks. Wherever possible these 60 Panabaker 1 62 IPVBI Sept, 1999 64 protocols make use of existing RFC standards and non-standards. 66 Traditionally, point-to-point connections (TCP/IP) have been used 67 even for the transmission of broadcast type data. Distribution of 68 the same content--news feeds, stock quotes, newsgroups, weather 69 reports, and the like--are typically sent repeatedly to individual 70 clients rather than being broadcast to the large number of users who 71 want to receive such data. 73 Today, IP is quickly becoming the preferred method of distributing 74 one-to-many data on intranets and the Internet. The coming 75 availability of low cost PC hardware for receiving television signals 76 accompanied by broadcast data streams makes a defined standard for 77 the transmission of data over traditional broadcast networks 78 imperative. A lack of standards in this area as well as the expense 79 of hardware has prevented traditional broadcast networks from 80 becoming effective deliverers of data to the home and office. 82 This document describes the transmission of IP using the North 83 American Basic Teletext Standard (NABTS), a recognized and industry- 84 supported method of transporting data on the VBI. NABTS is 85 traditionally used on 525-line television systems such as NTSC. 86 Another byte structure, WST, is traditionally used on 625-line 87 systems such as PAL and SECAM. These generalizations have 88 exceptions, and countries should be treated on an individual basis. 89 These existing television system standards will enable the television 90 and Internet communities to provide inexpensive broadcast data 91 services. A set of existing protocols will be layered above the 92 specific FEC for NABTS including a serial stream framing protocol 93 similar to SLIP (RFC 1055 [Romkey 1988]) and a compression technique 94 for unidirectional UDP/IP headers. 96 The protocols described in this document are intended for the 97 unidirectional delivery of IP datagrams using the VBI. That is, no 98 return channel is described, or for that matter possible, in the VBI. 100 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 101 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 102 document are to be interpreted as described in RFC 2119. 104 3. Proposed protocol stack 106 The following protocol stack demonstrates the layers used in the 107 transmission of VBI data. Each layer has no knowledge of the data it 108 encapsulates, and is therefore abstracted from the other layers. At 109 the link layer, the NABTS protocol defines the modulation scheme used 110 to transport data on the VBI. At the network layer, IP handles the 111 movement of data to the appropriate clients. In the transport layer, 112 UDP determines the flow of data to the appropriate processes and 113 applications. 115 Panabaker 2 117 IPVBI Sept, 1999 119 +-------------------+ 120 | | 121 | Application | 122 | | 123 +-------------------+ 124 | | ) 125 | UDP | ) 126 | | ) 127 +-------------------+ (-- IP 128 | | ) 129 | IP | ) 130 | | ) 131 +-------------------+ 132 | SLIP-style | 133 | encapsulation | 134 | | 135 +-------------------+ 136 | FEC | 137 |-------------------| 138 | NABTS | 139 | | 140 +---------+---------+ 141 | | 142 | NTSC/other | 143 | | 144 +-------------------+ 145 | 146 | 147 | cable, off-air, etc. 148 +--------<----<----<-------- 150 These protocols can be described in a bottom up component model using 151 the example of NABTS carried over NTSC modulation as follows: 153 Video signal --> NABTS --> FEC --> serial data stream --> Framing 154 protocol --> compressed UDP/IP headers --> application data 156 This diagram can be read as follows: television signals have NABTS 157 packets, which contain a Forward Error Correction (FEC) protocol, 158 modulated onto them. The data contained in these sequential, ordered 159 packets form a serial data stream on which a framing protocol 160 indicates the location of IP packets, with compressed headers, 161 containing application data. 163 The structure of these components and protocols are described in 164 following subsections. 166 3.1. VBI 168 The characteristics and definition of the VBI is dependent on the 169 television system in use, be it NTSC, PAL, SECAM or some other. For 170 more information on Television standards worldwide, see ref [12]. 172 Panabaker 3 174 IPVBI Sept, 1999 176 3.1.1. 525 line systems 178 A 525-line television frame is comprised of two fields of 262.5 179 horizontal scan lines each. The first 21 lines of each field are not 180 part of the visible picture and are collectively called the Vertical 181 Blanking Interval (VBI). 183 Of these 21 lines, the first 9 are used while repositioning the 184 cathode ray to the top of the screen, but the remaining lines are 185 available for data transport. 187 There are 12 possible VBI lines being broadcast 60 times a second 188 (each field 30 times a second). In some countries Line 21 is 189 reserved for the transport of closed captioning data (Ref.[11]). In 190 that case, there are 11 possible VBI lines, some or all of which 191 could be used for IP transport. It should be noted that some of 192 these lines are sometimes used for existing, proprietary, data and 193 testing services. IP delivery therefore becomes one more data service 194 using a subset or all of these lines. 196 3.1.2. 625 Line Systems 198 A 625-line television frame is comprised of two fields of 312.5 199 horizontal scan lines each. The first few lines of each field are 200 used while repositioning the cathode ray to the top of the screen. 201 The lines available for data insertion are 6-22 in the first field 202 and 319-335 in the second field. 204 There are, therefore, 17 possible VBI lines being broadcast 50 times 205 a second (each field 25 times a second), some or all of which could 206 be used for IP transport. It should be noted that some of these 207 lines are sometimes used for existing, proprietary, data and testing 208 services. IP, therefore, becomes one more data service using a subset 209 or all of these lines. 211 3.2. NABTS 213 The North American Basic Teletext Standard is defined in the 214 Electronic Industry Association's EIA-516, Ref. [2], and ITU.R 215 BT.653-2, system C, Ref. [13]. It provides an industry-accepted 216 method of modulating data onto the VBI, usually of an NTSC signal. 217 This section describes the NABTS packet format as it is used for the 218 transport of IP. 220 It should be noted that only a subset of the NABTS standard is used, 221 as is common practice in NABTS implementations. Further information 222 concerning the NABTS standard and its implementation can be found in 223 EIA-516. 225 The NABTS packet is a 36-byte structure encoded onto one horizontal 227 Panabaker 4 229 IPVBI Sept, 1999 231 scan line of a television signal having the following structure: 233 0 1 2 3 234 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 235 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 236 | clock sync | byte sync | packet addr. | 237 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 238 | packet address (cont.) | cont. index |PcktStructFlags| 239 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 240 | user data (26 bytes) | 241 : : 242 : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 243 | | FEC | 244 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 246 The two-byte Clock Synchronization and one-byte Byte Synchronization 247 are located at the beginning of every scan line containing a NABTS 248 packet and are used to synchronize the decoding sampling rate and 249 byte timing. 251 The three-byte Packet Address field is Hamming encoded (as specified 252 in EIA-516), provides 4 data bits per byte, and thus provides 4096 253 possible packet addresses. These addresses are used to distinguish 254 related services originating from the same source. This is necessary 255 for the receiver to determine which packets are related, and part of 256 the same service. NABTS packet addresses therefore distinguish 257 different data services, possibly inserted at different points of the 258 transmission system, and most likely totally unrelated. Section 4 of 259 this document discusses Packet Addresses in detail. 261 The one-byte Continuity Index field is a Hamming encoded byte, which 262 is incremented by one for each subsequent packet of a given Packet 263 Address. The value or number of the Continuity Index sequences from 264 0 to 15. It increments by one each time a data packet is transmitted. 265 This allows the decoder to determine if packets were lost during 266 transmission. 268 The Packet Structure field is also a Hamming encoded byte, which 269 contains information about the structure of the remaining portions of 270 the packet. The least significant bit is always "0" in this 271 implementation. The second least significant bit specifies if the 272 Data Block is full--"0" indicates the data block is full of useful 273 data, and "1" indicates some or all of the data is filler data. The 274 two most significant bits are used to indicate the length of the 275 suffix of the Data Block--in this implementation, either 2 or 28 276 bytes (10 for 2-byte FEC suffix, 11 for 28-byte FEC suffix). This 277 suffix is used for the forward error correction described in the next 278 section. The following table shows the possible values of the Packet 279 Structure field: 281 Type of Packet PS field value 283 Panabaker 5 285 IPVBI Sept, 1999 287 Data Packet, no filler D0 288 Data Packet, with filler 8C 289 FEC Packet A1 291 The Data Block field is 26 bytes, zero to 26 of which is useful data 292 (part of a IP packet or SLIP frame), the remainder is filler data. 293 Data is byte-ordered least significant bit first. Filler data is 294 indicated by an Ox15 followed by as many OxEA as are needed to fill 295 the Data Block field. Sequential data blocks minus the filler data 296 form an asynchronous serial stream of data. 298 These NABTS packets are modulated onto the television signal 299 sequentially and on any combination of lines. 301 3.3. FEC 303 Due to the unidirectional nature of VBI data transport, Forward Error 304 Correction (FEC) is needed to ensure the integrity of data at the 305 receiver. The type of FEC described here and in the appendix of this 306 document for NABTS has been in use for a number of years, and has 307 proven popular with the broadcast industry. It is capable of 308 correcting single-byte errors and single- and double-byte erasures in 309 the data block and suffix of a NABTS packet. 310 In a system using NABTS, the FEC algorithm splits a serial stream of 311 data into 364-byte "bundles". The data is arranged as 14 packets of 312 26 bytes each. A function is applied to the 26 bytes of each packet 313 to determine the two suffix bytes, which (with the addition of a 314 header) complete the NABTS packet (8+26+2). 316 For every 14 packets in the bundle, two additional packets are 317 appended which contain only FEC data, each of which contain 28 bytes 318 of FEC data. The first packet in the bundle has a Continuity Index 319 value of 0, and the two FEC only packets at the end have Continuity 320 Index values of 14 and 15 respectively. This data is obtained by 321 first writing the packets into a table of 16 rows and 28 columns. 322 The same expression as above can be used on the columns of the table 323 with the suffix now represented by the bytes at the base of the 324 columns in rows 15 and 16. With NABTS headers on each of these rows, 325 we now have a bundle of 16 NABTS packets ready for sequential 326 modulation onto lines of the television signal. 328 At the receiver, these formulae can be used to verify the validity of 329 the data with very high accuracy. If single bit errors, double bit 330 errors, single-byte errors or single- and double-byte erasures are 331 found in any row or column (including an entire packet lost) they can 332 be corrected using the algorithms found in the appendix. The success 333 at correcting errors will depend on the particular implementation 334 used on the receiver. 336 3.4. Framing 338 Panabaker 6 340 IPVBI Sept, 1999 342 A framing protocol identical to SLIP is proposed for encapsulating 343 the packets described in the following section, thus abstracting this 344 data from the lower protocol layers. This protocol uses two special 345 characters: END (0xc0) and ESC (0xdb). To send a packet, the host 346 will send the packet followed by the END character. If a data byte 347 in the packet is the same code as END character, a two- byte sequence 348 of ESC (0xdb) and 0xdc is sent instead. If a data byte is the same 349 code as ESC character, a two-byte sequence of ESC (0xdb) and 0xdd is 350 sent instead. SLIP implementations are widely available; see RFC 351 1055 [Romkey 1988] for more detail. 353 +--------------+--+------------+--+--+---------+--+ 354 | packet |c0| packet |db|dd| |c0| 355 +--------------+--+------------+--+--+---------+--+ 356 END ESC END 358 The packet framed in this manner shall be formatted according to its 359 schema type identified by the schema field, which shall start every 360 packet: 362 +-----------+---------------------------------------------+ 363 | schema | packet (formatted according to schema) | 364 | 1 byte | ?? bytes (schema dependant length) | 365 +-----------+---------------------------------------------+ 367 In the case where the most significant bit in this field is set to 368 "1", the length of the field extends to two bytes, allowing for 32768 369 additional schemas: 371 +-----------+---------------------------------------------+ 372 | extended | packet (formatted according to schema) | 373 | schema | ?? bytes (schema dependant length) | 374 | 2 bytes | | 375 +-----------+---------------------------------------------+ 377 In the section 3.5, one such schema for sending IP is described. 378 This is the only schema specified by this document. Additional 379 schemas may be proposed for other packet types or other compression 380 schemes as described in section 7. 382 3.4.1 Maximum Transmission Unit Size 384 The maximum length of an uncompressed IP packet, or Maximum- 385 Transmission Unit (MTU) size is 1500 octets. Packets larger than 386 1500 octets MUST be fragmented before transmission, and the client 387 VBI interface MUST be able to receive full 1500 octet packet 388 transmissions. 390 3.5. IP Header Compression Scheme 392 The one-byte scheme defines the format for encoding the IP packet 394 Panabaker 7 396 IPVBI Sept, 1999 398 itself. Due to the value of bandwidth, it may be desirable to 399 introduce as much efficiency as possible in this encoding. One such 400 efficiency is the optional compression of the UDP/IP header using a 401 method related to the TCP/IP header compression as described by Van 402 Jacobson (RFC 1144). UDP/IP header compression is not identical due 403 to the limitation of unidirectional transmission. One such scheme is 404 proposed in this document for the compression of UDP/IP version 4. 405 It is assigned a value of 0x00. All future encapsulation schemes 406 should use a unique value in this field. 408 Only schema 0x00 is defined in this document; this schema must be 409 supported by all receivers. In schema 0x00, the format of the IP 410 packet itself takes one of two forms. Packets can be sent with full, 411 uncompressed headers as follows: 413 0 1 2 3 414 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 415 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 416 |0| group | uncompressed IP header (20 bytes) | 417 +-+-+-+-+-+-+-+-+ + 418 | | 419 : .... : 420 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 | | uncompressed UDP header (8 bytes) | 422 +-+-+-+-+-+-+-+-+ + 423 | | 424 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 425 | | payload (<1472 bytes) | 426 +-+-+-+-+-+-+-+-+ + 427 | | 428 : .... : 429 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 430 | CRC | 431 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 433 The first byte in the 0x00 scheme is the Compression Key. It is a 434 one-byte value: the first bit indicates if the header has been 435 compressed, and the remaining seven bits indicate the compression 436 group to which it belongs. 438 If the high bit of the Compression Key is set to zero, no compression 439 is performed and the full header is sent, as shown above. The client 440 VBI interface should store the most recent uncompressed header for a 441 given group value for future potential use in rebuilding subsequent 442 compressed headers. Packets with identical group bits are assumed to 443 have identical UDP/IP headers for all UDP and IP fields, with the 444 exception of the "IP identification" and "UDP checksum" fields. 446 Group values may be recycled following 60 seconds of nonuse by the 447 preceding UDP/IP session (no uncompressed packets sent), or by 448 sending packets with uncompressed headers for the 60-second duration 449 following the last packet in the preceding UDP/IP session. 450 Furthermore, the first packet sent following 60 seconds of nonuse, 452 Panabaker 8 454 IPVBI Sept, 1999 456 or compressed header packets only use, must use an uncompressed 457 header. Client VBI interfaces should disregard compressed packets 458 received 60 or more seconds after the last uncompressed packet using 459 a given group address. This avoids any incorrectly decompressed 460 packets due to group number reuse, and limits the outage due to a 461 lost uncompressed packet to 60 seconds. 462 If the high bit of the Compression Key is set to one, a compressed 463 version of the UDP/IP header is sent. The client VBI interface must 464 then combine the compressed header with the stored uncompressed 465 header of the same group and recreate a full packet. For compressed 466 packets, the only portions of the UDP/IP header sent are the "IP 467 identification" and "UDP checksum" fields: 469 0 1 2 3 470 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 471 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 472 |1| group | IP identification | UDP checksum| 473 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 474 |UDP cksm (cont)| payload (<1472 bytes) | 475 +-+-+-+-+-+-+-+-+ + 476 : .... : 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 | CRC | 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 481 All datagrams belonging to a multi fragment IP packet shall be sent 482 with full headers, in the uncompressed header format. Therefore, 483 only packets that have not been fragmented can be sent with the most 484 significant bit of the compression key set to "1". 486 A 32-bit CRC has also been added to the end of every packet in this 487 scheme to ensure data integrity. It is performed over the entire 488 packet including schema byte, compression key, and either compressed 489 or uncompressed headers. It uses the same algorithm as the MPEG-2 490 transport stream (ISO/IEC 13818-1). The generator polynomial is: 492 1 + D + D2 + D4 + D5 + D7 + D8 + D10 + D11 + D12 + D16 + D22 + D23 + 493 D26 + D32 495 As in the ISO/IEC 13818-1 specification, the initial state of the sum 496 is 0xFFFFFFFF. This is not the same algorithm used by Ethernet. This 497 CRC provides a final check for damaged datagrams that span FEC 498 bundles or were not properly corrected by FEC. 500 4. Addressing Considerations 502 The addressing of IP packets should adhere to existing standards in 503 this area. The inclusion of an appropriate source address is needed 504 to ensure the receiving client can distinguish between sources and 505 thus services if multiple hosts are sharing an insertion point and 506 NABTS packet address. 507 NABTS packet addressing is not regulated or standardized and requires 509 Panabaker 9 511 IPVBI Sept, 1999 513 care to ensure that unrelated services on the same channel are not 514 broadcasting with the same packet address. This could occur due to 515 multiple possible NABTS insertion sites, including show production, 516 network redistribution, regional operator, and local operator. 517 Traditionally, the marketplace has recognized this concern and made 518 amicable arrangements for the distribution of these addresses for 519 each channel. 521 5. IANA Considerations 523 IANA will register new schemas on a "First Come First Served" basis 524 [RFC 2434]. Anyone can register a scheme, so long as they provide a 525 point of contact and a brief description. The scheme number will be 526 assigned by IANA. Registrants are encouraged to publish complete 527 specifications for new schemas (preferably as standards-track RFCs), 528 but this is not required. 530 6. Security considerations 532 As with any broadcast network, there are security issues due to the 533 accessibility of data. It is assumed that the responsibility for 534 securing data lies in other protocol layers, including the IP 535 Security (IPSEC) protocol suite, Transport Layer Security (TLS) 536 protocols, as well as application layer protocols appropriate for a 537 broadcast-only network. 539 7. Conclusions 541 This document provides a method for broadcasting Internet data over a 542 television signal for reception by client devices. With an 543 appropriate broadcast content model, this will become an attractive 544 method of providing data services to end users. By using existing 545 standards and a layered protocol approach, this document has also 546 provided a model for data transmission on unidirectional and 547 broadcast networks. 549 8. Acknowledgements 551 The description of the FEC in Appendix A is taken from a document 552 prepared by Trevor Dee of Norpak Corporation. Dean Blackketter of 553 WebTV Networks, Inc., edited the final draft of this document. 555 9. References 557 [1] Bradner, S., 1997. "Key words for use in RFCs to Indicate 558 Requirement Levels," RFC 2119, 3 pages (March). 560 [2] Deering, S. E., 1989. "Host Extensions for IP Multicasting," RFC 562 Panabaker 10 564 IPVBI Sept, 1999 566 1112, 17 pages (Aug.). 568 [3] EIA-516, "Joint EIA/CVCC Recommended Practice for Teletext: North 569 American Basic Teletext Specification (NABTS)" Washington: Electronic 570 Industries Association, c1988 572 [4] International Telecommunications Union Recommendation. ITU-R 573 BT.470-5 (02/98) "Conventional TV Systems" 575 [5] International Telecommunications Union Recommendation. ITU.R 576 BT.653-2, system C 578 [6] Jack, Keith. "Video Demystified: A Handbook for the Digital 579 Engineer, Second Edition," San Diego: HighText Pub. c1996. 581 [7] Jacobson, V., 1990a. "Compressing TCP/IP Headers for Low-Speed 582 Serial Links," RFC 1144, 43 pages (Feb.). 584 [8] Mortimer, Brian. "An Error-correction system for the Teletext 585 Transmission in the Case of Transparent Data" c1989 Department of 586 Mathematics and Statistics, Carleton University, Ottawa Canada 588 [9] Narten, T. Alvestrand, H., 1998. "Guidelines for Writing an IANA 589 Considerations Section in RFCs," RFC 2434, (Oct.). 591 [10] Norpak Corporation "TTX71x Programming Reference Manual", c1996, 592 Kanata, Ontario, Canada 594 [11] Norpak Corporation, "TES3 EIA-516 NABTS Data Broadcast Encoder 595 Software User's Manual." c1996, Kanata, Ontario, Canada 597 [12] Norpak Corporation, "TES3/GES3 Hardware Manual" c1996, Kanata, 598 Ontario, Canada 600 [13] Pretzel, Oliver. "Correcting Codes and Finite Fields: Student 601 Edition" OUP, c1996 603 [14] Rorabaugh, C. Britton. "Error Coding Cookbook" McGraw Hill, 604 c1996 606 [15] Romkey, J. L., 1988. "A Nonstandard for Transmission of IP 607 Datagrams Over Serial Lines: SLIP," RFC 1055, 6 pages (June). 609 [16] Recommended Practice for Line 21 Data Service (ANSI/EIA-608-94) 610 (Sept., 1994) 612 [17] Stevens, W. Richard. "TCP/IP Illustrated, Volume 1,: The 613 Protocols" Reading: Addison-Wesley Publishing Company, c1994. 615 10. Acronyms 617 FEC - Forward Error Correction 619 Panabaker 11 621 IPVBI Sept, 1999 623 IP - Internet Protocol 624 NABTS - North American Basic Teletext Standard 625 NTSC - National Television Standards Committee 626 NTSC-J - NTSC-Japanese 627 PAL - Phase Alternation Line 628 RFC - Request for Comments 629 SECAM - Sequentiel Couleur Avec Memoire 630 (sequential color with memory) 631 SLIP - Serial Line Internet Protocol 632 TCP - Transmission Control Protocol 633 UDP - User Datagram Protocol 634 VBI - Vertical Blanking Interval 635 WST - World System Teletext 637 11. Editors' address and contacts 639 Ruston Panabaker, co-editor 640 Microsoft 641 One Microsoft Way Redmond, WA 98052 642 rustonp@microsoft.com 644 Simon Wegerif, co-editor 645 Philips Semiconductors 646 811 E. Arques Avenue 647 M/S 52, P.O. Box 3409 Sunnyvale, CA 94088-3409 648 Simon.Wegerif@sv.sc.philips.com 650 Dan Zigmond, WG Chair 651 WebTV Networks 652 One Microsoft Way Redmond, WA 98052 653 djz@corp.webtv.net 655 12. Appendix A: Forward Error Correction Specification 657 This FEC is optimized for data carried in the VBI of a television 658 signal. Teletext has been in use for many years and data 659 transmission errors have been categorized into three main types: 1) 660 Randomly distributed single bit errors 2) Loss of lines of NABTS data 661 3) Burst Errors 663 The quantity and distribution of these errors is highly variable and 664 dependent on many factors. The FEC is designed to fix all these 665 types of errors. 667 12.1. Mathematics used in the FEC 669 Galois fields form the basis for the FEC algorithm presented here. 670 Rather then explain these fields in general, a specific explanation 671 is given of the Galois field used in the FEC algorithm. 673 Panabaker 12 675 IPVBI Sept, 1999 677 The Galois field used is GF(2^8) with a primitive element alpha of 678 00011101. This is a set of 256 elements, along with the operations 679 of "addition", "subtraction", "division", and "multiplication" on 680 these elements. An 8-bit binary number represents each element. 682 The operations of "addition" and "subtraction" are the same for this 683 Galois field. Both operations are the XOR of two elements. Thus, 684 the "sum" of 00011011 and 00000111 is 00011100. 686 Division of two elements is done using long division with subtraction 687 operations replaced by XOR. For multiplication, standard long 688 multiplication is used but with the final addition stage replaced 689 with XOR. 691 All arithmetic in the following FEC is done modulo 100011101; for 692 instance, after you multiply two numbers, you replace the result with 693 its remainder when divided by 100011101. There are 256 values in 694 this field (256 possible remainders), the 8-bit numbers. It is very 695 important to remember that when we write A*B = C, we more accurately 696 imply modulo(A*B) = C. 698 It is obvious from the above description that multiplication and 699 division is time consuming to perform. Elements of the Galois Field 700 share two important properties with real numbers. 702 A*B = POWERalpha(LOGalpha(A) + LOGalpha(B)) 703 A/B = POWERalpha(LOGalpha(A) - LOGalpha(B)) 705 The Galois Field is limited to 256 entries so the power and log 706 tables are limited to 256 entries. The addition and subtraction 707 shown is standard so the result must be modulo alpha. Written as a 708 "C" expression: 710 A*B = apower[alog[A] + alog[B]] 711 A/B = apower[255 + alog[A] - alog[B]] 713 You may note that alog[A] + alog[B] can be greater than 255 and 714 therefore a modulo operation should be performed. This is not 715 necessary if the power table is extended to 512 entries by repeating 716 the table. The same logic is true for division as shown. The power 717 and log tables are calculated once using the long multiplication 718 shown above. 720 12.2. Calculating FEC bytes 722 The FEC algorithm splits a serial stream of data into "bundles". 723 These are arranged as 16 packets of 28 bytes when the correction 724 bytes are included. The bundle therefore has 16 horizontal codewords 725 interleaved with 28 vertical codewords. Two sums are calculated for 726 a codeword, S0 and S1. S0 is the sum of all bytes of the codeword 727 each multiplied by alpha to the power of its index in the codeword. 728 S1 is the sum of all bytes of the codeword each multiplied by alpha 730 Panabaker 13 732 IPVBI Sept, 1999 734 to the power of three times its index in the codeword. In "C" the 735 sum calculations would look like: 737 Sum0 = 0; 738 Sum1 = 0; 739 For (i = 0;i < m;i++) 740 { 741 if (codeword[i] != 0) 742 { 743 Sum0 = sum0 ^ power[i + alog[codeword[i]]]; 744 Sum1 = sum1 ^ power[3*i + alog[codeword[i]]]; 745 } 746 } 748 Note that the codeword order is different from the packet order. 749 Codeword positions 0 and 1 are the suffix bytes at the end of a 750 packet horizontally or at the end of a column vertically. 752 When calculating the two FEC bytes, the summation above must produce 753 two sums of zero. All codewords except 0 and 1 are know so the sums 754 for the known codewords can be calculated. Let's call these values 755 tot0 and tot1. 757 Sum0 = tot0^power[0+alog[codeword[0]]]^power[1+alog[codeword[1]]] 758 Sum1 = tot1^power[0+alog[codeword[0]]]^power[3+alog[codeword[1]]] 759 This gives us two equations with the two unknowns that we can solve: 760 codeword[1] = power[255+alog[tot0^tot1]-alog[power[1]^power[3]]] 761 codeword[0] = tot0^power[alog[codeword[1]]+alog[power[1]]] 763 12.3. Correcting Errors 765 This section describes the procedure for detecting and correcting 766 errors using the FEC data calculated above. Upon reception, we begin 767 by rebuilding the bundle. This is perhaps the most important part of 768 the procedure because if the bundle is not built correctly it cannot 769 possibly correct any errors. The continuity index is used to 770 determine the order of the packets and if any packets are missing 771 (not captured by the hardware). The recommendation, when building 772 the bundle, is to convert the bundle from packet order to codeword 773 order. This conversion will simplify the codeword calculations. This 774 is done by taking the last byte of a packet and making it the second 775 byte of the codeword and taking the second last byte of a packet and 776 making it the first byte of a codeword. Also the packet with 777 continuity index 15 becomes codeword position one and the packet with 778 continuity index 14 becomes codeword position zero. 779 The same FEC is used regardless of the number of bytes in the 780 codeword. So let's think of the codewords as an array 781 codeword[vert][hor] where vert is 16 packets and hor is 28. Each 782 byte in the array is protected by both a horizontal and a vertical 783 codeword. For each of the codewords, the sums must be calculated. If 784 both the sums for a codeword are zero then no errors have been 785 detected for that codeword. Otherwise, an error has been detected 787 Panabaker 14 789 IPVBI Sept, 1999 791 and further steps need to be taken to see if the error can be 792 corrected. In "C" the horizontal summation would look like: 794 for (i = 0; i < 16; i++) 795 { 796 sum0 = 0; 797 sum1 = 0; 798 for (j = 0;j < hor;j++) 799 { 800 if (codeword[i][j] != 0) 801 { 802 sum0 = sum0 ^ power[j + alog[codeword[i][j]]; 803 sum1 = sum1 ^ power[3*j + alog[codeword[i][j]]; 804 } 805 } 806 if ((sum0 != 0) || (sum1 != 0)) 807 { 808 Try Correcting Packet 809 } 810 } 812 Similarly, vertical looks like: 813 for (j = 0;i < hor;i++) 814 { 815 sum0 = 0; 816 sum1 = 0; 817 for (i = 0;i < 16;i++) 818 { 819 if (codeword[i][j] != 0) 820 { 821 sum0 = sum0 ^ power[i + alog[codeword[i][j]]; 822 sum1 = sum1 ^ power[3*i + alog[codeword[i][j]]; 823 } 824 } 825 if ((sum0 != 0) || (sum1 != 0)) 826 { 827 Try Correcting Column 828 } 829 } 831 12.4. Correction Schemes 833 This FEC provides four possible corrections: 834 1) Single bit correction in codeword. All single bit errors. 835 2) Double bit correction in a codeword. Most two-bit errors. 836 3) Single byte correction in a codeword. All single-byte errors. 837 4) Packet replacement. One or two missing packets from a bundle. 839 12.4.1. Single Bit Correction 841 When correcting a single-bit in a codeword, the byte and bit position 843 Panabaker 15 845 IPVBI Sept, 1999 847 must be calculated. The equations are: 849 Byte = 1/2LOGalpha(S1/S0) 850 Bit = 8LOGalpha(S0/POWERalpha(Byte)) 852 In "C" this is written: 854 Byte = alog[power[255 + alog[sum1] - alog[sum0]]]; 855 if (Byte & 1) 856 Byte = Byte + 255; 857 Byte = Byte >> 1; 858 Bit = alog[power[255 + alog[sum0] - Byte]] << 3; 859 while (Bit > 255) 860 Bit = Bit - 255; 862 The error is correctable if Byte is less than the number of bytes in 863 the codeword and Bit is less than eight. For this math to be valid 864 both sum0 and sum1 must be non-zero. The codeword is corrected by: 865 codeword[Byte] = codeword[Byte] ^ (1 << Bit); 867 12.4.2. Double Bit Correction 869 Double bit correction is much more complex than single bit correction 870 for two reasons. First, not all double bit errors are deterministic. 871 That is two different bit patterns can generate the same sums. 872 Second, the solution is iterative. To find two bit errors you assume 873 one bit in error and then solve for the second error as a single bit 874 error. 876 The procedure is to iteratively move through the bits of the codeword 877 changing each bit's state. The new sums are calculated for the 878 modified codeword. Then the single bit calculation above determines 879 if this is the correct solution. If not, the bit is restored and the 880 next bit is tried. 882 For a long codeword, this can involve many calculations. However, 883 tricks can speed the process. For example, the vertical sums give a 884 strong indication of which bytes are in error horizontally. Bits in 885 other bytes need not be tried. 887 12.4.3. Single Byte Correction 889 For single byte correction, the byte position and bits to correct 890 must be calculated. The equations are: 891 Byte = 1/2*LOGalpha(S1/S0) 892 Mask = S0/POWERalpha[Byte] 894 Notice that the byte position is the same calculation as for single 895 bit correction. The mask will allow more than one bit in the byte to 896 be corrected. In "C" the mask calculation looks like: 897 Mask = power[255 + alog[sum0] - Byte] 899 Panabaker 16 901 IPVBI Sept, 1999 903 Both sum0 and sum1 must be non-zero for the calculations to be valid. 904 The Byte value must be less than the codeword length but Mask can be 905 any value. This corrects the byte in the codeword by: 907 Codeword[Byte] = Codeword[Byte] ^ Mask 909 12.4.4. Packet Replacement 911 If a packet is missing, as determined by the continuity index, then 912 its byte position is known and does not need to be calculated. The 913 formula for single packet replacement is therefore the same as for 914 the Mask calculation of single byte correction. Instead of XORing an 915 existing byte with the Mask, the Mask replaces the missing codeword 916 position: 918 Codeword[Byte] = Mask 920 When two packets are missing, both the codeword positions are known 921 by the continuity index. This again gives two equations with two 922 unknowns, which is solved to give the following equations. 923 Mask2 = POWERalpha(2*Byte1)*S0+S1 925 POWERalpha(2*Byte1+Byte2) +POWERalpha(3*BYTE2) 926 Mask1 = S0 + Mask2*POWERalpha(Byte2)/POWERalpha(BYTE1) 928 In "C" these equations are written: 929 if (sum0 == 0) 930 { 931 if (sum1 == 0) 932 mask2 = 0; 933 else 934 mask2=power[255+alog[sum1]-alog[power[byte2+2*byte1]^ 935 power[3*Byte2]]]; 936 } else { 937 if ((a=sum1^power[alog[sum0]+2*byte1]) == 0) 938 mask2 = 0; 939 else 940 mask2 = power[255+alog[a]-alog[power[byte2+2*byte1]^ 941 power[3*byte2]]]; 942 } 944 if (mask2 = 0) 945 { 946 if (sum0 == 0) 947 mask1 = 0; 948 else 949 mask1 = power[255+alog[sum0]-byte1]; 950 } else { 951 if ((a=sum0^power[alog[mask2] + byte2]) == 0) 952 mask1 = 0; 953 else 954 mask1 = power[255+alog[a]-byte1]; 956 Panabaker 17 958 IPVBI Sept, 1999 960 } 962 Notice that, in the code above, care is taken to check for zero 963 values. The missing codeword position can be fixed by: 965 codeword[byte1] = mask1; 966 codeword[byte2] = mask2; 968 12.5. FEC Performance Considerations 970 The section above shows how to correct the different types of errors. 971 It does not suggest how these corrections may be used in an algorithm 972 to correct a bundle. There are many possible algorithms and the one 973 chosen depends on many variables. These include: 975 . The amount of processing power available 976 . The number of packets per VBI to process 977 . The type of hardware capturing the data 978 . The delivery path of the VBI 979 . How the code is implemented 981 As a minimum, it is recommended that the algorithm use single bit or 982 single byte correction for one pass in each direction followed by 983 packet replacement if appropriate. It is possible to do more than 984 one pass of error correction in each direction. The theory is that 985 errors not corrected in the first pass may be corrected in the second 986 pass because error correction in the other direction has removed some 987 errors. 989 In making choices, it is important to remember that the code has 990 several possible states: 992 1) Shows codeword as correct and it is. 993 2) Shows codeword as correct and it is not (detection failure). 994 3) Shows codeword as incorrect but cannot correct (detection). 995 4) Shows codeword as incorrect and corrects it correctly 996 (correction). 997 5) Shows codeword as incorrect but corrects wrong bits (false 998 correction). 1000 There is actually overlap among the different types of errors. For 1001 example, a pair of sums may indicate both a double bit error and a 1002 byte error. It is not possible to know at the code level which is 1003 correct and which is a false correction. In fact, neither might be 1004 correct if both are false corrections. 1005 If you know something about the types of errors in the delivery 1006 channel, you can greatly improve efficiency. If you know that errors 1007 are randomly distributed (as in a weak terrestrial broadcast) then 1008 single and double bit correction are more powerful than single byte. 1010 13. Appendix B: Architecture 1012 Panabaker 18 1014 IPVBI Sept, 1999 1016 The architecture that this document is addressing can be broken down 1017 into three areas: insertion, distribution network, and receiving 1018 client. 1020 The insertion of IP data onto the television signal can occur at any 1021 part of the delivery system. A VBI encoder typically accepts a video 1022 signal and an asynchronous serial stream of bytes forming framed IP 1023 packets as inputs and subsequently packetizes the data onto a 1024 selected set of lines using NABTS and an FEC. This composite signal 1025 is then modulated with other channels before being broadcast onto the 1026 distribution network. Operators further down the distribution chain 1027 could then add their own data, to other unused lines, as well. 1028 The distribution networks include coax plant, off-air, and analog 1029 satellite systems and are primarily unidirectional broadcast 1030 networks. They must provide a signal to noise ratio, which is 1031 sufficient for FEC to recover any lost data for the broadcast of data 1032 to be effective. 1034 The receiving client must be capable of tuning, NABTS waveform 1035 sampling as appropriate, filtering on NABTS addresses as appropriate, 1036 forward error correction, unframing, verification of the CRC and 1037 decompressing the UDP/IP header if they are compressed. All of the 1038 above functions can be carried out in PC software and inexpensive 1039 off-the-shelf hardware. 1041 14. Appendix C: Scope of proposed protocols 1043 The protocols described in this document are for transmitting IP 1044 packets. However, their scope may be extensible to other 1045 applications outside this area. Many of the protocols in this 1046 document could be implemented on any unidirectional network. 1047 s 1048 The unidirectional framing protocol provides encapsulation of IP 1049 datagrams on the serial stream, and the compression of the UDP/IP 1050 headers reduces the overhead on transmission, thus conserving 1051 bandwidth. These two protocols could be widely used on different 1052 unidirectional broadcast networks or modulation schemes to 1053 efficiently transport any type of packet data. In particular, new 1054 versions of Internet protocols can be supported to provide a 1055 standardized method of data transport. 1057 - end - 1059 Panabaker 19