idnits 2.17.1 draft-ietf-ipvbi-nabts-03.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1059 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 382: '... 1500 octets MUST be fragmented befo...' RFC 2119 keyword, line 383: '... VBI interface MUST be able to recei...' Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (June 1999) is 9082 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: 'Romkey 1988' on line 343 -- Looks like a reference, but probably isn't: 'RFC 2434' on line 509 -- Looks like a reference, but probably isn't: 'A' on line 677 -- Looks like a reference, but probably isn't: 'B' on line 677 == Missing Reference: '0' is mentioned on line 727, but not defined -- Looks like a reference, but probably isn't: 'Byte' on line 886 == Unused Reference: '7' is defined on line 565, but no explicit reference was found in the text == Unused Reference: '8' is defined on line 568, but no explicit reference was found in the text == Unused Reference: '9' is defined on line 571, but no explicit reference was found in the text == Unused Reference: '10' is defined on line 574, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. '2' -- Possible downref: Non-RFC (?) normative reference: ref. '3' ** Obsolete normative reference: RFC 2434 (ref. '5') (Obsoleted by RFC 5226) -- Possible downref: Non-RFC (?) normative reference: ref. '6' -- Possible downref: Non-RFC (?) normative reference: ref. '7' -- Possible downref: Non-RFC (?) normative reference: ref. '8' -- Possible downref: Non-RFC (?) normative reference: ref. '10' -- Possible downref: Non-RFC (?) normative reference: ref. '11' -- Possible downref: Non-RFC (?) normative reference: ref. '12' -- Possible downref: Non-RFC (?) normative reference: ref. '13' Summary: 7 errors (**), 0 flaws (~~), 7 warnings (==), 16 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 IPVBI R. Panabaker, Microsoft 2 Internet Draft S. Wegerif, Philips Semiconductors 3 D. Zigmond, WebTV Networks 5 Document: June 1999 7 The Transmission of IP Over the 8 Vertical Blanking Interval of a Television Signal 10 Status of this Memo 12 This document is an Internet-Draft and is in full conformance with 13 all provisions of Section 10 of RFC2026. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. 20 Internet-Drafts are draft documents valid for a maximum of six 21 months and may be updated, replaced, or obsoleted by other documents 22 at any time. It is inappropriate to use Internet-Drafts as 23 reference material or to cite them other than as "work in progress." 25 The list of current Internet-Drafts can be accessed at 26 http://www.ietf.org/ietf/1id-abstracts.txt 28 The list of Internet-Draft Shadow Directories can be accessed at 29 http://www.ietf.org/shadow.html. 31 To view the entire list of current Internet-Drafts, please check the 32 "1id-abstracts.txt" listing contained in the Internet-Drafts Shadow 33 Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern 34 Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific 35 Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast). 37 A revised version of this draft document will be submitted to the 38 RFC editor as a Proposed Standard for the Internet Community. 39 Discussion and suggestions for improvement are requested. This 40 document will expire before August 1999. Distribution of this draft 41 is unlimited. 43 1. Abstract 45 This document describes a method for broadcasting IP data using the 46 vertical blanking interval of television signals. It includes a 47 description for compressing IP headers on unidirectional networks, a 48 framing protocol identical to SLIP, a forward error correction 49 scheme, and the NABTS byte structures. 51 2. Introduction 53 Panabaker 1 55 IPVBI June, 1999 57 This RFC proposes several protocols to be used in the transmission 58 of IP datagrams using the Vertical Blanking Interval (VBI) of a 59 television signal. The VBI is a non-viewable portion of the 60 television signal that can be used to provide point-to-multipoint IP 61 data services which will relieve congestion and traffic in the 62 traditional Internet access networks. Wherever possible these 63 protocols make use of existing RFC standards and non-standards. 65 Traditionally, point-to-point connections (TCP/IP) have been used 66 even for the transmission of broadcast type data. Distribution of 67 the same content--news feeds, stock quotes, newsgroups, weather 68 reports, and the like--are typically sent repeatedly to individual 69 clients rather than being broadcast to the large number of users who 70 want to receive such data. 72 Today, IP is quickly becoming the preferred method of distributing 73 one-to-many data on intranets and the Internet. The coming 74 availability of low cost PC hardware for receiving television 75 signals accompanied by broadcast data streams makes a defined 76 standard for the transmission of data over traditional broadcast 77 networks imperative. A lack of standards in this area as well as 78 the expense of hardware has prevented traditional broadcast networks 79 from becoming effective deliverers of data to the home and office. 81 This document describes the transmission of IP using the North 82 American Basic Teletext Standard (NABTS), a recognized and industry- 83 supported method of transporting data on the VBI. NABTS is 84 traditionally used on 525-line television systems such as NTSC. 85 Another byte structure, WST, is traditionally used on 625-line 86 systems such as PAL and SECAM. These generalizations have 87 exceptions, and countries should be treated on an individual basis. 88 These existing television system standards will enable the 89 television and Internet communities to provide inexpensive broadcast 90 data services. A set of existing protocols will be layered above 91 the specific FEC for NABTS including a serial stream framing 92 protocol similar to SLIP (RFC 1055 [Romkey 1988]) and a compression 93 technique for unidirectional UDP/IP headers. 95 3. Proposed protocol stack 97 The following protocol stack demonstrates the layers used in the 98 transmission of VBI data. Each layer has no knowledge of the data 99 it encapsulates, and is therefore abstracted from the other layers. 100 At the link layer, the NABTS protocol defines the modulation scheme 101 used to transport data on the VBI. At the network layer, IP handles 102 the movement of data to the appropriate clients. In the transport 103 layer, UDP determines the flow of data to the appropriate processes 104 and applications. 106 Panabaker 2 108 IPVBI June, 1999 110 +-------------------+ 111 | | 112 | Application | 113 | | 114 +-------------------+ 115 | | ) 116 | UDP | ) 117 | | ) 118 +-------------------+ (-- IP 119 | | ) 120 | IP | ) 121 | | ) 122 +-------------------+ 123 | SLIP-style | 124 | encapsulation | 125 | | 126 +-------------------+ 127 | FEC | 128 |-------------------| 129 | NABTS | 130 | | 131 +---------+---------+ 132 | | 133 | NTSC/other | 134 | | 135 +-------------------+ 136 | 137 | 138 | cable, off-air, etc. 139 +--------<----<----<-------- 141 These protocols can be described in a bottom up component model 142 using the example of NABTS carried over NTSC modulation as follows: 144 Video signal --> NABTS --> FEC --> serial data stream --> Framing 145 protocol --> compressed UDP/IP headers --> application data 147 This diagram can be read as follows: television signals have NABTS 148 packets, which contain a Forward Error Correction (FEC) protocol, 149 modulated onto them. The data contained in these sequential, 150 ordered packets form a serial data stream on which a framing 151 protocol indicates the location of IP packets, with compressed 152 headers, containing application data. 154 The structure of these components and protocols are described in 155 following subsections. 157 3.1. VBI 159 Panabaker 3 161 IPVBI June, 1999 163 The characteristics and definition of the VBI is dependent on the 164 television system in use, be it NTSC, PAL, SECAM or some other. For 165 more information on Television standards worldwide, see ref [12]. 167 3.1.1. 525 line systems 169 A 525-line television frame is comprised of two fields of 262.5 170 horizontal scan lines each. The first 21 lines of each field are 171 not part of the visible picture and are collectively called the 172 Vertical Blanking Interval (VBI). 174 Of these 21 lines, the first 9 are used while repositioning the 175 cathode ray to the top of the screen, but the remaining lines are 176 available for data transport. 178 There are 12 possible VBI lines being broadcast 60 times a second 179 (each field 30 times a second). In some countries Line 21 is 180 reserved for the transport of closed captioning data (Ref.[11]). In 181 that case, there are 11 possible VBI lines, some or all of which 182 could be used for IP transport. It should be noted that some of 183 these lines are sometimes used for existing, proprietary, data and 184 testing services. IP delivery therefore becomes one more data 185 service using a subset or all of these lines. 187 3.1.2. 625 Line Systems 189 A 625-line television frame is comprised of two fields of 312.5 190 horizontal scan lines each. The first few lines of each field are 191 used while repositioning the cathode ray to the top of the screen. 192 The lines available for data insertion are 6-22 in the first field 193 and 319-335 in the second field. 195 There are, therefore, 17 possible VBI lines being broadcast 50 times 196 a second (each field 25 times a second), some or all of which could 197 be used for IP transport. It should be noted that some of these 198 lines are sometimes used for existing, proprietary, data and testing 199 services. IP, therefore, becomes one more data service using a 200 subset or all of these lines. 202 3.2. NABTS 204 The North American Basic Teletext Standard is defined in the 205 Electronic Industry Association's EIA-516, Ref. [2], and ITU.R 206 BT.653-2, system C, Ref. [13]. It provides an industry-accepted 207 method of modulating data onto the VBI, usually of an NTSC signal. 208 This section describes the NABTS packet format as it is used for the 209 transport of IP. 211 Panabaker 4 213 IPVBI June, 1999 215 It should be noted that only a subset of the NABTS standard is used, 216 as is common practice in NABTS implementations. Further information 217 concerning the NABTS standard and its implementation can be found in 218 EIA-516. 220 The NABTS packet is a 36-byte structure encoded onto one horizontal 221 scan line of a television signal having the following structure: 223 0 1 2 3 224 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 225 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 226 | clock sync | byte sync | packet addr. | 227 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 228 | packet address (cont.) | cont. index |PcktStructFlags| 229 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 230 | user data (26 bytes) | 231 : : 232 : +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 233 | | FEC | 234 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 236 The two-byte Clock Synchronization and one-byte Byte Synchronization 237 are located at the beginning of every scan line containing a NABTS 238 packet and are used to synchronize the decoding sampling rate and 239 byte timing. 241 The three-byte Packet Address field is Hamming encoded (as specified 242 in EIA-516), provides 4 data bits per byte, and thus provides 4096 243 possible packet addresses. These addresses are used to distinguish 244 related services originating from the same source. This is 245 necessary for the receiver to determine which packets are related, 246 and part of the same service. NABTS packet addresses therefore 247 distinguish different data services, possibly inserted at different 248 points of the transmission system, and most likely totally 249 unrelated. Section 4 of this document discusses Packet Addresses in 250 detail. 252 The one-byte Continuity Index field is a Hamming encoded byte, which 253 is incremented by one for each subsequent packet of a given Packet 254 Address. The value or number of the Continuity Index sequences from 255 0 to 15. It increments by one each time a data packet is 256 transmitted. This allows the decoder to determine if packets were 257 lost during transmission. 259 The Packet Structure field is also a Hamming encoded byte, which 260 contains information about the structure of the remaining portions 261 of the packet. The least significant bit is always "0" in this 262 implementation. The second least significant bit specifies if the 263 Data Block is full--"0" indicates the data block is full of useful 264 data, and "1" indicates some or all of the data is filler data. The 266 Panabaker 5 268 IPVBI June, 1999 270 two most significant bits are used to indicate the length of the 271 suffix of the Data Block--in this implementation, either 2 or 28 272 bytes (10 for 2-bit FEC suffix, 11 for 28-byte FEC suffix). This 273 suffix is used for the forward error correction described in the 274 next section. The following table shows the possible values of the 275 Packet Structure field: 277 Type of Packet PS field value 278 Data Packet, no filler D0 279 Data Packet, with filler 8C 280 FEC Packet A1 282 The Data Block field is 26 bytes, zero to 26 of which is useful data 283 (part of a IP packet or SLIP frame), the remainder is filler data. 284 Data is byte-ordered least significant bit first. Filler data is 285 indicated by an Ox15 followed by as many OxEA as are needed to fill 286 the Data Block field. Sequential data blocks minus the filler data 287 form an asynchronous serial stream of data. 289 These NABTS packets are modulated onto the television signal 290 sequentially and on any combination of lines. 292 3.3. FEC 294 Due to the unidirectional nature of VBI data transport, Forward 295 Error Correction (FEC) is needed to ensure the integrity of data at 296 the receiver. The type of FEC described here and in the appendix of 297 this document for NABTS has been in use for a number of years, and 298 has proven popular with the broadcast industry. It is capable of 299 correcting single-byte errors and single- and double-byte erasures 300 in the data block and suffix of a NABTS packet. 301 In a system using NABTS, the FEC algorithm splits a serial stream of 302 data into 364-byte "bundles". The data is arranged as 14 packets of 303 26 bytes each. A function is applied to the 26 bytes of each packet 304 to determine the two suffix bytes, which (with the addition of a 305 header) complete the NABTS packet (8+26+2). 307 For every 14 packets in the bundle, two additional packets are 308 appended which contain only FEC data, each of which contain 28 bytes 309 of FEC data. The first packet in the bundle has a Continuity Index 310 value of 0, and the two FEC only packets at the end have Continuity 311 Index values of 14 and 15 respectively. This data is obtained by 312 first writing the packets into a table of 16 rows and 28 columns. 313 The same expression as above can be used on the columns of the table 314 with the suffix now represented by the bytes at the base of the 315 columns in rows 15 and 16. With NABTS headers on each of these 316 rows, we now have a bundle of 16 NABTS packets ready for sequential 317 modulation onto lines of the television signal. 319 At the receiver, these formulae can be used to verify the validity 320 of the data with very high accuracy. If single bit errors, double 322 Panabaker 6 324 IPVBI June, 1999 326 bit errors, single-byte errors or single- and double-byte erasures 327 are found in any row or column (including an entire packet lost) 328 they can be corrected using the algorithms found in the appendix. 329 The success at correcting errors will depend on the particular 330 implementation used on the receiver. 332 3.4. Framing 334 A framing protocol identical to SLIP is proposed for encapsulating 335 the packets described in the following section, thus abstracting 336 this data from the lower protocol layers. This protocol uses two 337 special characters: END (0xc0) and ESC (0xdb). To send a packet, 338 the host will send the packet followed by the END character. If a 339 data byte in the packet is the same code as END character, a two- 340 byte sequence of ESC (0xdb) and 0xdc is sent instead. If a data 341 byte is the same code as ESC character, a two-byte sequence of ESC 342 (0xdb) and 0xdd is sent instead. SLIP implementations are widely 343 available; see RFC 1055 [Romkey 1988] for more detail. 345 +--------------+--+------------+--+--+---------+--+ 346 | packet |c0| packet |db|dd| |c0| 347 +--------------+--+------------+--+--+---------+--+ 348 END ESC END 350 The packet framed in this manner shall be formatted according to its 351 schema type identified by the schema field, which shall start every 352 packet: 354 +-----------+---------------------------------------------+ 355 | schema | packet (formatted according to schema) | 356 | 1 byte | ?? bytes (schema dependant length) | 357 +-----------+---------------------------------------------+ 359 In the case where the most significant bit in this field is set to 360 "1", the length of the field extends to two bytes, allowing for 361 32768 additional schemas: 363 +-----------+---------------------------------------------+ 364 | extended | packet (formatted according to schema) | 365 | schema | ?? bytes (schema dependant length) | 366 | 2 bytes | | 367 +-----------+---------------------------------------------+ 369 In the section 3.5, one such schema for sending IP is described. 370 This is the only schema specified by this document. Additional 371 schemas may be proposed for other packet types or other compression 372 schemes as described in section 7. 374 3.4.1 Maximum Transmission Unit Size 376 Panabaker 7 378 IPVBI June, 1999 380 The maximum length of an uncompressed IP packet, or Maximum- 381 Transmission Unit (MTU) size is 1500 octets. Packets larger than 382 1500 octets MUST be fragmented before transmission, and the client 383 VBI interface MUST be able to receive full 1500 octet packet 384 transmissions. 386 3.5. IP Header Compression Scheme 388 The one-byte scheme defines the format for encoding the IP packet 389 itself. Due to the value of bandwidth, it may be desirable to 390 introduce as much efficiency as possible in this encoding. One such 391 efficiency is the optional compression of the UDP/IP header using a 392 method related to the TCP/IP header compression as described by Van 393 Jacobson (RFC 1144). UDP/IP header compression is not identical due 394 to the limitation of unidirectional transmission. One such scheme 395 is proposed in this document for the compression of UDP/IP version 396 4. It is assigned a value of 0x00. All future encapsulation 397 schemes should use a unique value in this field. 399 Only schema 0x00 is defined in this document; this schema must be 400 supported by all receivers. In schema 0x00, the format of the IP 401 packet itself takes one of two forms. Packets can be sent with 402 full, uncompressed headers as follows: 404 0 1 2 3 405 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 406 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 407 |0| group | uncompressed IP header (20 bytes) | 408 +-+-+-+-+-+-+-+-+ + 409 | | 410 : .... : 411 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 412 | | uncompressed UDP header (8 bytes) | 413 +-+-+-+-+-+-+-+-+ + 414 | | 415 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 416 | | payload (<1472 bytes) | 417 +-+-+-+-+-+-+-+-+ + 418 | | 419 : .... : 420 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 421 | CRC | 422 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 424 The first byte in the 0x00 scheme is the Compression Key. It is a 425 one-byte value: the first bit indicates if the header has been 426 compressed, and the remaining seven bits indicate the compression 427 group to which it belongs. 429 If the high bit of the Compression Key is set to zero, no 430 compression is performed and the full header is sent, as shown 432 Panabaker 8 434 IPVBI June, 1999 436 above. The client VBI interface should store the most recent 437 uncompressed header for a given group value for future potential use 438 in rebuilding subsequent compressed headers. Packets with identical 439 group bits are assumed to have identical UDP/IP headers for all UDP 440 and IP fields, with the exception of the "IP identification" and 441 "UDP checksum" fields. Group values may be recycled as long as an 442 uncompressed header is transmitted for a given group value before 443 any compressed headers. In the event that a sender recycles a group 444 value but the receiver somehow misses the uncompressed header, the 445 CRC check will fail and the receiver may wait for an uncompressed 446 header with this group value before trying again. 448 If the high bit of the Compression Key is set to one, a compressed 449 version of the UDP/IP header is sent. The client VBI interface must 450 then combine the compressed header with the stored uncompressed 451 header of the same group and recreate a full packet. For compressed 452 packets, the only portions of the UDP/IP header sent are the "IP 453 identification" and "UDP checksum" fields: 455 0 1 2 3 456 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 457 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 458 |1| group | IP identification | UDP checksum| 459 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 460 |UDP cksm (cont)| payload (<1472 bytes) | 461 +-+-+-+-+-+-+-+-+ + 462 : .... : 463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 464 | CRC | 465 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 467 All datagrams belonging to a multi fragment IP packet shall be sent 468 with full headers, in the uncompressed header format. Therefore, 469 only packets that have not been fragmented can be sent with the most 470 significant bit of the compression key set to "1". 472 A 32-bit CRC has also been added to the end of every packet in this 473 scheme to ensure data integrity. It is performed over the entire 474 packet including schema byte, compression key, and either compressed 475 or uncompressed headers. It uses the same algorithm as the MPEG-2 476 transport stream (ISO/IEC 13818-1). The generator polynomial is: 478 1 + D + D2 + D4 + D5 + D7 + D8 + D10 + D11 + D12 + D16 + D22 + D23 + 479 D26 + D32 481 As in the ISO/IEC 13818-1 specification, the initial state of the 482 sum is 0xFFFFFFFF. This is not the same algorithm used by Ethernet. 483 This CRC provides a final check for damaged datagrams that span FEC 484 bundles or were not properly corrected by FEC. 486 4. Addressing Considerations 488 Panabaker 9 490 IPVBI June, 1999 492 The addressing of IP packets should adhere to existing standards in 493 this area. The inclusion of an appropriate source address is needed 494 to ensure the receiving client can distinguish between sources and 495 thus services if multiple hosts are sharing an insertion point and 496 NABTS packet address. 497 NABTS packet addressing is not regulated or standardized and 498 requires care to ensure that unrelated services on the same channel 499 are not broadcasting with the same packet address. This could occur 500 due to multiple possible NABTS insertion sites, including show 501 production, network redistribution, regional operator, and local 502 operator. Traditionally, the marketplace has recognized this 503 concern and made amicable arrangements for the distribution of these 504 addresses for each channel. 506 5. IANA Considerations 508 IANA will register new schemas on a "First Come First Served" basis 509 [RFC 2434]. Anyone can register a scheme, so long as they provide a 510 point of contact and a brief description. The scheme number will be 511 assigned by IANA. Registrants are encouraged to publish complete 512 specifications for new schemas (preferably as standards-track RFCs), 513 but this is not required. 515 6. Security considerations 517 As with any broadcast network, there are security issues due to the 518 accessibility of data. It is assumed that the responsibility for 519 securing data lies in other protocol layers, including the IP 520 Security (IPSEC) protocol suite, Transport Layer Security (TLS) 521 protocols, as well as application layer protocols appropriate for a 522 broadcast-only network. 524 7. Conclusions 526 This document provides a method for broadcasting Internet data over 527 a television signal for reception by client devices. With an 528 appropriate broadcast content model, this will become an attractive 529 method of providing data services to end users. By using existing 530 standards and a layered protocol approach, this document has also 531 provided a model for data transmission on unidirectional and 532 broadcast networks. 534 8. Acknowledgements 536 The description of the FEC in Appendix A is taken from a document 537 prepared by Trevor Dee of Norpak Corporation. Dean Blackketter of 538 WebTV Networks, Inc., edited the final draft of this document. 540 Panabaker 10 542 IPVBI June, 1999 544 9. References 546 [1] Deering, S. E., 1989. "Host Extensions for IP Multicasting," 547 RFC 1112, 17 pages (Aug.). 549 [2] EIA-516, "Joint EIA/CVCC Recommended Practice for Teletext: 550 North American Basic Teletext Specification (NABTS)" Washington: 551 Electronic Industries Association, c1988 553 [3] Jack, Keith. "Video Demystified: A Handbook for the Digital 554 Engineer, Second Edition," San Diego: HighText Pub. c1996. 556 [4] Jacobson, V., 1990a. "Compressing TCP/IP Headers for Low-Speed 557 Serial Links," RFC 1144, 43 pages (Feb.). 559 [5] Narten, T. Alvestrand, H., 1998. "Guidelines for Writing an 560 IANA Considerations Section in RFCs," RFC 2434, (Oct.). 562 [6] Norpak Corporation "TTX71x Programming Reference Manual", c1996, 563 Kanata, Ontario, Canada 565 [7] Norpak Corporation, "TES3 EIA-516 NABTS Data Broadcast Encoder 566 Software User's Manual." c1996, Kanata, Ontario, Canada 568 [8] Norpak Corporation, "TES3/GES3 Hardware Manual" c1996, Kanata, 569 Ontario, Canada 571 [9] Romkey, J. L., 1988. "A Nonstandard for Transmission of IP 572 Datagrams Over Serial Lines: SLIP," RFC 1055, 6 pages (June). 574 [10] Stevens, W. Richard. "TCP/IP Illustrated, Volume 1,: The 575 Protocols" Reading: Addison-Wesley Publishing Company, c1994. 577 [11] Recommended Practice for Line 21 Data Service (ANSI/EIA-608-94) 578 (Sept., 1994) 580 [12] International Telecommunications Union Recommendation. ITU-R 581 BT.470-5 (02/98) "Conventional TV Systems" 583 [13] International Telecommunications Union Recommendation. ITU.R 584 BT.653-2, system C 586 10. Acronyms 588 FEC - Forward Error Correction 589 IP - Internet Protocol 590 NABTS - North American Basic Teletext Standard 591 NTSC - National Television Standards Committee 592 NTSC-J - NTSC-Japanese 593 PAL - Phase Alternation Line 595 Panabaker 11 597 IPVBI June, 1999 599 RFC - Request for Comments 600 SECAM - Sequentiel Couleur Avec Memoire 601 (sequential color with memory) 602 SLIP - Serial Line Internet Protocol 603 TCP - Transmission Control Protocol 604 UDP - User Datagram Protocol 605 VBI - Vertical Blanking Interval 606 WST - World System Teletext 608 11. Editors' address and contacts 610 Ruston Panabaker, co-editor Microsoft 611 One Microsoft Way Redmond, WA 98052 rustonp@microsoft.com 612 Simon Wegerif, co-editor Philips Semiconductors 811 E. Arques Avenue 613 M/S 52, P.O. Box 3409 Sunnyvale, CA 94088-3409 614 Simon.Wegerif@sv.sc.philips.com 615 Dan Zigmond, WG Chair WebTV Networks One Microsoft Way Redmond, WA 616 98052 djz@corp.webtv.net 618 12. Appendix A: Forward Error Correction Specification 620 This FEC is optimized for data carried in the VBI of a television 621 signal. Teletext has been in use for many years and data 622 transmission errors have been categorized in to three main types: 1) 623 Randomly distributed single bit errors 2) Loss of lines of NABTS 624 data 3) Burst Errors 626 The quantity and distribution of these errors is highly variable and 627 dependent on many factors. The FEC is designed to fix all these 628 types of errors. 630 12.1. Mathematics used in the FEC 632 Galois fields form the basis for the FEC algorithm presented here. 633 Rather then explain these fields in general, a specific explanation 634 is given of the Galois field used in the FEC algorithm. 636 The Galois field used is GF(2^8) with a primitive element alpha of 637 00011101. This is a set of 256 elements, along with the operations 638 of "addition", "subtraction", "division", and "multiplication" on 639 these elements. An 8-bit binary number represents each element. 641 The operations of "addition" and "subtraction" are the same for this 642 Galois field. Both operations are the XOR of two elements. Thus, 643 the "sum" of 00011011 and 00000111 is 00011100. 645 Division of two elements is done using long division with 646 subtraction operations replaced by XOR. For multiplication, 648 Panabaker 12 650 IPVBI June, 1999 652 standard long multiplication is used but with the final addition 653 stage replaced with XOR. 655 All arithmetic in the following FEC is done modulo 100011101; for 656 instance, after you multiply two numbers, you replace the result 657 with its remainder when divided by 100011101. There are 256 values 658 in this field (256 possible remainders), the 8-bit numbers. It is 659 very important to remember that when we write A*B = C, we more 660 accurately imply modulo(A*B) = C. 662 It is obvious from the above description that multiplication and 663 division is time consuming to perform. Elements of the Galois Field 664 share two important properties with real numbers. 666 A*B = POWERalpha(LOGalpha(A) + LOGalpha(B)) 667 A/B = POWERalpha(LOGalpha(A) - LOGalpha(B)) 669 The Galois Field is limited to 256 entries so the power and log 670 tables are limited to 256 entries. The addition and subtraction 671 shown is standard so the result must be modulo alpha. Written as a 672 "C" expression: 674 A*B = apower[alog[A] + alog[B]] 675 A/B = apower[255 + alog[A] - alog[B]] 677 You may note that alog[A] + alog[B] can be greater than 255 and 678 therefore a modulo operation should be performed. This is not 679 necessary if the power table is extended to 512 entries by repeating 680 the table. The same logic is true for division as shown. The power 681 and log tables are calculated once using the long multiplication 682 shown above. 684 12.2. Calculating FEC bytes 686 The FEC algorithm splits a serial stream of data into "bundles". 687 These are arranged as 16 packets of 28 bytes when the correction 688 bytes are included. The bundle therefore has 16 horizontal 689 codewords interleaved with 28 vertical codewords. Two sums are 690 calculated for a codeword, S0 and S1. S0 is the sum of all bytes of 691 the codeword each multiplied by alpha to the power of its index in 692 the codeword. S1 is the sum of all bytes of the codeword each 693 multiplied by alpha to the power of three times its index in the 694 codeword. In "C" the sum calculations would look like: 696 Sum0 = 0; 697 Sum1 = 0; 698 for (i = 0;i < m;i++) 699 { 700 if (codeword[i] != 0) 701 { 702 Sum0 = sum0 ^ power[i + alog[codeword[i]]]; 704 Panabaker 13 706 IPVBI June, 1999 708 Sum1 = sum1 ^ power[3*i + alog[codeword[i]]]; 709 } 710 } 712 Note that the codeword order is different from the packet order. 713 Codeword positions 0 and 1 are the suffix bytes at the end of a 714 packet horizontally or at the end of a column vertically. 716 When calculating the two FEC bytes, the summation above must produce 717 two sums of zero. All codewords except 0 and 1 are know so the sums 718 for the known codewords can be calculated. Let's call these values 719 tot0 and tot1. 721 Sum0 = tot0^power[0+alog[codeword[0]]]^power[1+alog[codeword[1]]] 722 Sum1 = tot1^power[0+alog[codeword[0]]]^power[3+alog[codeword[1]]] 724 This gives us two equations with the two unknowns that we can solve: 726 codeword[1] = power[255+alog[tot0^tot1]-alog[power[1]^power[3]]] 727 codeword[0] = tot0^power[alog[codeword[1]]+alog[power[1]]] 729 12.3. Correcting Errors 731 This section describes the procedure for detecting and correcting 732 errors using the FEC data calculated above. Upon reception, we 733 begin by rebuilding the bundle. This is perhaps the most important 734 part of the procedure because if the bundle is not built correctly 735 it cannot possibly correct any errors. The continuity index is used 736 to determine the order of the packets and if any packets are missing 737 (not captured by the hardware). The recommendation, when building 738 the bundle, is to convert the bundle from packet order to codeword 739 order. This conversion will simplify the codeword calculations. 740 This is done by taking the last byte of a packet and making it the 741 second byte of the codeword and taking the second last byte of a 742 packet and making it the first byte of a codeword. Also the packet 743 with continuity index 15 becomes codeword position one and the 744 packet with continuity index 14 becomes codeword position zero. 745 The same FEC is used regardless of the number of bytes in the 746 codeword. So let's think of the codewords as an array 747 codeword[vert][hor] where vert is 16 packets and hor is 28. Each 748 byte in the array is protected by both a horizontal and a vertical 749 codeword. For each of the codewords, the sums must be calculated. 750 If both the sums for a codeword are zero then no errors have been 751 detected for that codeword. Otherwise, an error has been detected 752 and further steps need to be taken to see if the error can be 753 corrected. In "C" the horizontal summation would look like: 755 for (i = 0; i < 16; i++) 756 { 757 sum0 = 0; 758 sum1 = 0; 760 Panabaker 14 762 IPVBI June, 1999 764 for (j = 0;j < hor;j++) 765 { 766 if (codeword[i][j] != 0) 767 { 768 sum0 = sum0 ^ power[j + alog[codeword[i][j]]; 769 sum1 = sum1 ^ power[3*j + alog[codeword[i][j]]; 770 } 771 } 772 if ((sum0 != 0) || (sum1 != 0)) 773 { 774 Try Correcting Packet 775 } 776 } 778 Similarly, vertical looks like: 780 for (j = 0;i < hor;i++) 781 { 782 sum0 = 0; 783 sum1 = 0; 784 for (i = 0;i < 16;i++) 785 { 786 if (codeword[i][j] != 0) 787 { 788 sum0 = sum0 ^ power[i + alog[codeword[i][j]]; 789 sum1 = sum1 ^ power[3*i + alog[codeword[i][j]]; 790 } 791 } 792 if ((sum0 != 0) || (sum1 != 0)) 793 { 794 Try Correcting Column 795 } 796 } 798 12.4. Correction Schemes 800 This FEC provides four possible corrections: 802 1) Single bit correction in codeword. All single bit errors. 803 2) Double bit correction in a codeword. Most two-bit errors. 804 3) Single byte correction in a codeword. All single-byte errors. 805 4) Packet replacement. One or two missing packets from a bundle. 807 12.4.1. Single Bit Correction 809 When correcting a single-bit in a codeword, the byte and bit 810 position must be calculated. The equations are: 812 Byte = 1/2LOGalpha(S1/S0) 813 Bit = 8LOGalpha(S0/POWERalpha(Byte)) 815 Panabaker 15 817 IPVBI June, 1999 819 In "C" this is written: 821 Byte = alog[power[255 + alog[sum1] - alog[sum0]]]; 822 if (Byte & 1) 823 Byte = Byte + 255; 824 Byte = Byte >> 1; 825 Bit = alog[power[255 + alog[sum0] - Byte]] << 3; 826 while (Bit > 255) 827 Bit = Bit - 255; 829 The error is correctable if Byte is less than the number of bytes in 830 the codeword and Bit is less than eight. For this math to be valid 831 both sum0 and sum1 must be non-zero. The codeword is corrected by: 832 codeword[Byte] = codeword[Byte] ^ (1 << Bit); 834 12.4.2. Double Bit Correction 836 Double bit correction is much more complex than single bit 837 correction for two reasons. First, not all double bit errors are 838 deterministic. That is two different bit patterns can generate the 839 same sums. Second, the solution is iterative. To find two bit 840 errors you assume one bit in error and then solve for the second 841 error as a single bit error. 843 The procedure is to iteratively move through the bits of the 844 codeword changing each bit's state. The new sums are calculated for 845 the modified codeword. Then the single bit calculation above 846 determines if this is the correct solution. If not, the bit is 847 restored and the next bit is tried. 849 For a long codeword, this can involve many calculations. However, 850 tricks can speed the process. For example, the vertical sums give a 851 strong indication of which bytes are in error horizontally. Bits in 852 other bytes need not be tried. 854 12.4.3. Single Byte Correction 856 For single byte correction, the byte position and bits to correct 857 must be calculated. The equations are: 859 Byte = 1/2*LOGalpha(S1/S0) 860 Mask = S0/POWERalpha[Byte] 862 Notice that the byte position is the same calculation as for single 863 bit correction. The mask will allow more than one bit in the byte 864 to be corrected. In "C" the mask calculation looks like: 865 Mask = power[255 + alog[sum0] - Byte] 867 Panabaker 16 869 IPVBI June, 1999 871 Both sum0 and sum1 must be non-zero for the calculations to be 872 valid. The Byte value must be less than the codeword length but 873 Mask can be any value. This corrects the byte in the codeword by: 875 Codeword[Byte] = Codeword[Byte] ^ Mask 877 12.4.4. Packet Replacement 879 If a packet is missing, as determined by the continuity index, then 880 its byte position is known and does not need to be calculated. The 881 formula for single packet replacement is therefore the same as for 882 the Mask calculation of single byte correction. Instead of XORing 883 an existing byte with the Mask, the Mask replaces the missing 884 codeword position: 886 Codeword[Byte] = Mask 888 When two packets are missing, both the codeword positions are known 889 by the continuity index. This again gives two equations with two 890 unknowns, which is solved to give the following equations. 892 Mask2 = POWERalpha(2*Byte1)*S0+S1 894 POWERalpha(2*Byte1+Byte2) +POWERalpha(3*BYTE2) 896 Mask1 = S0 + Mask2*POWERalpha(Byte2)/POWERalpha(BYTE1) 898 In "C" these equations are written: 900 if (sum0 == 0) 901 { 902 if (sum1 == 0) 903 mask2 = 0; 904 else 905 mask2=power[255+alog[sum1] - 906 alog[power[byte2+2*byte1]^power[3*Byte2]]]; 907 } else { 908 if ((a=sum1^power[alog[sum0]+2*byte1]) == 0) 909 mask2 = 0; 910 else 911 mask2 = power[255+alog[a] - 912 alog[power[byte2+2*byte1]^power[3*byte2]]]; 913 } 915 if (mask2 = 0) 916 { 917 if (sum0 == 0) 918 mask1 = 0; 919 else 920 mask1 = power[255+alog[sum0]-byte1]; 921 } else { 923 Panabaker 17 925 IPVBI June, 1999 927 if ((a=sum0^power[alog[mask2] + byte2]) == 0) 928 mask1 = 0; 929 else 930 mask1 = power[255+alog[a]-byte1]; 931 } 933 Notice that, in the code above, care is taken to check for zero 934 values. The missing codeword position can be fixed by: 936 codeword[byte1] = mask1; 937 codeword[byte2] = mask2; 939 12.5. FEC Performance Considerations 941 The section above shows how to correct the different types of 942 errors. It does not suggest how these corrections may be used in an 943 algorithm to correct a bundle. There are many possible algorithms 944 and the one chosen depends on many variables. These include: 946 . The amount of processing power available 947 . The number of packets per VBI to process 948 . The type of hardware capturing the data 949 . The delivery path of the VBI 950 . How the code is implemented 952 As a minimum, it is recommended that the algorithm use single bit or 953 single byte correction for one pass in each direction followed by 954 packet replacement if appropriate. It is possible to do more than 955 one pass of error correction in each direction. The theory is that 956 errors not corrected in the first pass may be corrected in the 957 second pass because error correction in the other direction has 958 removed some errors. 960 In making choices, it is important to remember that the code has 961 several possible states: 963 1) Shows codeword as correct and it is. 964 2) Shows codeword as correct and it is not (detection failure). 965 3) Shows codeword as incorrect but cannot correct (detection). 966 4) Shows codeword as incorrect and corrects it correctly 967 (correction). 968 5) Shows codeword as incorrect but corrects wrong bits 969 (false correction). 971 There is actually overlap among the different types of errors. For 972 example, a pair of sums may indicate both a double bit error and a 973 byte error. It is not possible to know at the code level which is 974 correct and which is a false correction. In fact, neither might be 975 correct if both are false corrections. 976 If you know something about the types of errors in the delivery 977 channel, you can greatly improve efficiency. If you know that 979 Panabaker 18 981 IPVBI June, 1999 983 errors are randomly distributed (as in a weak terrestrial broadcast) 984 then single and double bit correction are more powerful than single 985 byte. 987 12.6. Appendix References 989 [1] Norpak Corporation "TTX71x Programming Reference Manual", c1996, 990 Kanata, Ontario, Canada 992 [2] Norpak Corporation, "TES3 EIA-516 NABTS Data Broadcast Encoder 993 Software User's Manual." c1996, Kanata, Ontario, Canada 995 [3] Norpak Corporation, "TES3/GES3 Hardware Manual" c1996, Kanata, 996 Ontario, Canada 998 [4] Pretzel, Oliver. "Correcting Codes and Finite Fields: Student 999 Edition" OUP, c1996 1001 [5] Rorabaugh, C. Britton. "Error Coding Cookbook" McGraw Hill, 1002 c1996 1004 [6] Mortimer, Brian. "An Error-correction system for the Teletext 1005 Transmission in the Case of Transparent Data" c1989 Department of 1006 Mathematics and Statistics, Carleton University, Ottawa Canada 1008 13. Appendix B: Architecture 1010 The architecture that this document is addressing can be broken down 1011 into three areas: insertion, distribution network, and receiving 1012 client. 1014 The insertion of IP data onto the television signal can occur at any 1015 part of the delivery system. A VBI encoder typically accepts a 1016 video signal and an asynchronous serial stream of bytes forming 1017 framed IP packets as inputs and subsequently packetizes the data 1018 onto a selected set of lines using NABTS and an FEC. This composite 1019 signal is then modulated with other channels before being broadcast 1020 onto the distribution network. Operators further down the 1021 distribution chain could then add their own data, to other unused 1022 lines, as well. 1023 The distribution networks include coax plant, off-air, and analog 1024 satellite systems and are primarily unidirectional broadcast 1025 networks. They must provide a signal to noise ratio, which is 1026 sufficient for FEC to recover any lost data for the broadcast of 1027 data to be effective. 1029 The receiving client must be capable of tuning, NABTS waveform 1030 sampling as appropriate, filtering on NABTS addresses as 1031 appropriate, forward error correction, unframing, verification of 1032 the CRC and decompressing the UDP/IP header if they are compressed. 1034 Panabaker 19 1036 IPVBI June, 1999 1038 All of the above functions can be carried out in PC software and 1039 inexpensive off-the-shelf hardware. 1041 14. Appendix C: Scope of proposed protocols 1043 The protocols described in this document are for transmitting IP 1044 packets. However, their scope may be extensible to other 1045 applications outside this area. Many of the protocols in this 1046 document could be implemented on any unidirectional network. 1048 The unidirectional framing protocol provides encapsulation of IP 1049 datagrams on the serial stream, and the compression of the UDP/IP 1050 headers reduces the overhead on transmission, thus conserving 1051 bandwidth. These two protocols could be widely used on different 1052 unidirectional broadcast networks or modulation schemes to 1053 efficiently transport any type of packet data. In particular, new 1054 versions of Internet protocols can be supported to provide a 1055 standardized method of data transport. 1057 Panabaker 20