idnits 2.17.1 draft-lugan-rtp-jpegxs-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (May 25, 2018) is 2162 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 831 == Unused Reference: 'ISO18477-3' is defined on line 725, but no explicit reference was found in the text -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO15444-1' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO18477-3' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO21122-1' -- Possible downref: Non-RFC (?) normative reference: ref. 'ISO21122-3' -- Possible downref: Non-RFC (?) normative reference: ref. 'SMPTE-ST2110-10' -- Possible downref: Non-RFC (?) normative reference: ref. 'SMPTE-ST2110-21' Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Payload Working Group S. Lugan 3 Internet-Draft G. Rouvroy 4 Intended status: Standards Track A. Descampe 5 Expires: November 26, 2018 intoPIX 6 T. Richter 7 IIS 8 A. Willeme 9 UCL/ICTEAM 10 May 25, 2018 12 RTP Payload Format for ISO/IEC 21122 (JPEG XS) 13 draft-lugan-rtp-jpegxs-00 15 Abstract 17 This document specifies a Real-Time Transport Protocol (RTP) payload 18 format to be used for transporting ISO/IEC 21122 (JPEG XS) encoded 19 video. ISO/IEC 21122 (JPEG XS) is a low-latency, lightweight image 20 coding system allowing for an increased resolution and frame rate, 21 while offering visually lossless quality with reduced amount of 22 ressources such as power and bandwidth. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on November 26, 2018. 41 Copyright Notice 43 Copyright (c) 2018 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3 60 2.1. Application Dependent Unit . . . . . . . . . . . . . . . 3 61 2.2. JPEG XS codestream . . . . . . . . . . . . . . . . . . . 3 62 2.3. JPEG XS frame . . . . . . . . . . . . . . . . . . . . . . 3 63 2.4. Marker . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 2.5. Marker Sequence . . . . . . . . . . . . . . . . . . . . . 3 65 2.6. JPEG XS Header . . . . . . . . . . . . . . . . . . . . . 3 66 2.7. Video Essence Box . . . . . . . . . . . . . . . . . . . . 4 67 2.8. JPEG XS Header Segment . . . . . . . . . . . . . . . . . 4 68 2.9. Slice . . . . . . . . . . . . . . . . . . . . . . . . . . 4 69 2.10. Slice group . . . . . . . . . . . . . . . . . . . . . . . 4 70 2.11. Fragment . . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Media Format Description . . . . . . . . . . . . . . . . . . 4 72 3.1. Wavelet decomposition . . . . . . . . . . . . . . . . . . 4 73 3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5 74 3.3. Video Essence Box . . . . . . . . . . . . . . . . . . . . 6 75 3.4. JPEG XS Stream . . . . . . . . . . . . . . . . . . . . . 6 76 3.5. Fragments . . . . . . . . . . . . . . . . . . . . . . . . 6 77 4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 7 78 4.1. RTP Header Usage . . . . . . . . . . . . . . . . . . . . 7 79 4.2. Payload Header . . . . . . . . . . . . . . . . . . . . . 8 80 4.3. Payload Data . . . . . . . . . . . . . . . . . . . . . . 11 81 4.4. Traffic Shaping and Delivery Timing . . . . . . . . . . . 12 82 5. Congestion Control Considerations . . . . . . . . . . . . . . 12 83 6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 13 84 6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 13 85 6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 13 86 6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 13 87 6.2.2. Media type and subtype . . . . . . . . . . . . . . . 13 88 6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 13 89 6.2.4. Other parameters . . . . . . . . . . . . . . . . . . 14 90 6.2.5. Offer/Answer Considerations . . . . . . . . . . . . . 14 91 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 92 8. Security Considerations . . . . . . . . . . . . . . . . . . . 14 93 9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 15 94 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 95 10.1. Normative References . . . . . . . . . . . . . . . . . . 16 96 10.2. Informative References . . . . . . . . . . . . . . . . . 17 97 10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 100 1. Introduction 102 This document specifies a payload format for packetization of ISO/IEC 103 21122 (JPEG XS) [ISO21122-1] encoded video signals into the Real-time 104 Transport Protocol (RTP) [RFC3550]. 106 2. Conventions, Definitions, and Abbreviations 108 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 109 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 110 document are to be interpreted as described in RFC 2119 [RFC2119]. 112 2.1. Application Dependent Unit 114 See Real-time Transport Protocol (RTP) [RFC3550], though for the 115 purpose of this document identical to a JPEG XS frame. 117 2.2. JPEG XS codestream 119 A sequence of bytes representing compressed images formatted 120 according to ISO/IEC 21122-1. 122 2.3. JPEG XS frame 124 Concatenation of the Video Essence box and a JPEG XS codestream 126 2.4. Marker 128 A two-byte functional sequence that is part of a JPEG XS codestream 129 starting with a 0xff byte and a subsequent byte defining its 130 function. 132 2.5. Marker Sequence 134 A marker along with a 16-bit marker size and payload data following 135 the size. 137 2.6. JPEG XS Header 139 A sequence of bytes at the beginning of each JPEG XS codestream 140 encoded in multiple markers and marker sequences that does not carry 141 entropy coded data, but metadata such as the frame dimension and 142 component precision. 144 2.7. Video Essence Box 146 A ISO super box in the sense of ISO/IEC 15444-1 defined in ISO/IEC 147 21122-3 that includes metadata required to play back a JPEG XS video 148 stream, such as its color space, its buffer model and its frame rate. 150 2.8. JPEG XS Header Segment 152 The concatenation of the Video Essence Box and the JPEG XS Header. 154 2.9. Slice 156 The smallest independently decodable unit of a JPEG XS stream. 158 2.10. Slice group 160 A contiguous sequence of slices belonging to a fragment. 162 2.11. Fragment 164 A slice group along with the metadata immediately preceeding and/or 165 following it sized such that the first byte of the fragment and the 166 byte following the last byte of the fragment are in two distinct 167 packets, except for the last fragment of a ADU. 169 3. Media Format Description 171 3.1. Wavelet decomposition 173 JPEG XS is a low-latency lightweight image coding system for coding 174 continuous-tone grayscale or continuous-tone color digital images. 176 This coding system provides an efficient representation of image 177 signals through the mathematical tool of wavelet analysis. The 178 wavelet filter process separates each component into multiple bands, 179 where each band consists of multiple coefficients describing the 180 image signal of a given component within a frequency domain specific 181 to the wavelet filter type, i.e. the particular filter corresponding 182 to the band. 184 Wavelet coefficients are grouped into precincts, where each precinct 185 includes all coefficients over all bands that contribute to a spatial 186 region of the image. 188 One or multiple Precincts are furthermore combined into slices 189 consisting of an integral number of precincts. Precincts do not 190 cross slice boundaries, and wavelet coefficients in precincts that 191 are part of different slices can be decoded independently from each 192 other. Note, however, that the wavelet transformation runs across 193 site boundaries. A slice always extends over the full width of the 194 image, but may only cover parts of its height. 196 A slice the the smallest indepedently decodable unit of a JPEG XS 197 codestream, bearing in mind that it decodes to wavelet coefficients 198 which still require inverse wavelet filtering to give an image. 200 3.2. Codestream 202 The codestream is a linear stream of bits from the first bit to the 203 last bit representing the sample values of a single frame, bare any 204 interpretation relative to a colorspace. It can be divided into 205 (8-bit) bytes, starting with the first bit of the codestream. Bits 206 within bytes are enumerated from the least significant bit (LSB) to 207 the most significant bit (MSB), with the least significant bit having 208 the index zero. Bits within bytes are transmitted in decreasing 209 magnitude order, with the MSB of a byte transmitted first and the LSB 210 transmitted last. This implies, in particular, that fields that are 211 longer than 8 bits are transmitted with the most significant byte 212 first. This is also denoted as "big endian" format. 214 The codestream consists of multiple syntax elements: markers, marker 215 segments and entropy coded data. 217 Markers inidicate syntactical elements of the codestreams. They 218 consist of an 0xff-byte and a second byte defining the nature of the 219 marker. The SOC marker (hex 0xff10) indicates the start of the 220 codestream, the EOC marker (hex 0xff11) its end. 222 Marker segments are markers along with a length field and payload 223 data following the length field. Marker segments define control 224 information necessary to steer the decoding process. The JPEG XS 225 specification ISO/IEC 21122-1 [ISO21122-1] defines additional markers 226 beyond SOC and EOC. 228 The sequence of bytes made up by all markers that precede the entropy 229 coded data is also denoted as JPEG XS Header in the following. 231 Entropy coded data represents the image data itself. The data is 232 organized in slices, where each slice consists of a slice header that 233 starts with the SLC marker (hex 0xff20) and payload data, consisting 234 of encoded wavelet coefficients. 236 The overall codestream format, including the definition of all 237 markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. 239 3.3. Video Essence Box 241 While the information defined in the codestream is sufficient to 242 reconstruct the sample values of one video frame, the interpretation 243 of the samples remains undefined by the codestream itself. This 244 interpretation, including the color space, frame rate and other 245 information significant to play a JPEG XS stream are contained the 246 Video Essence box, which preceeds each JPEG XS codestream. The 247 syntax of the Video Essence box follows ISO/IEC 15444-1 [ISO15444-1]; 248 it consists of multiple subboxes, each with a particular meaning. 249 Its contents, in particular its subboxes are defined in ISO/IEC 250 21122-3 [ISO21122-3]. 252 3.4. JPEG XS Stream 254 A JPEG XS stream is a sequence of frames, where each frame is coded 255 independently of each other. For the purpose of RTP transport, each 256 frame forms an Application Dependent Unit (ADU). 258 A JPEG XS frame consists the concatenation of a Video Essence box (as 259 defined in ISO/IEC 21122-3 [ISO21122-3]) and a JPEG XS codestream (as 260 defined in ISO/IEC 21122-1 [ISO21122-1]). As defined above, the 261 codestream consists of a JPEG XS header, one or multiple slice 262 groups, and an EOC marker. 264 3.5. Fragments 266 For the purpose of transport, JPEG XS frames are separated into one 267 or multiple fragments such that the start of the fragment and the 268 byte following the last byte of a fragment are in two distinct 269 packets used for RTP transport, except for the last fragment of a 270 JPEG XS frame which may be contained in only a single packet. 272 A fragment consists of all metadata preceeding its first slice, one 273 or multiple slices, and potentially the EOC marker following the last 274 slice. 276 The collection of slices in a fragment is also denoted as slice 277 group, and slice groups within a frame are enumerated from top to 278 bottom by the slice group counter. That is, the first slice group of 279 a frame is slice group #0, and the slice group counter increments by 280 1 from top to bottom for each slice group, and by that for each 281 fragment. 283 NOTE: By this definition, the first fragment consists of at least the 284 Video Essence Box, the JPEG XS header, and the first slice group. 285 The last fragment consists of at least the last slice group and the 286 EOC marker. In case the frame consists of only a single fragment, 287 this fragment contains both the JPEG XS header segment and the EOC. 289 4. Payload Format 291 This section specifies the payload format for JPEG XS video streams 292 over the Real-time Transport Protocol (RTP) [RFC3550]. 294 In order to be transported over RTP, each JPEG XS stream is 295 transported in a distinct RTP stream, identified by a distinct SSRC. 297 Each of those RTP streams is divided into Application Data Units 298 (ADUs). Each ADU shall correspond to a single JPEG XS frame. 300 Each ADU is split into packets, depending e.g. on the Maximum 301 Transmission unit (MTU) of the network. Every packet shall have same 302 size, except the last packet of every ADU which could be shorter. 303 Packet boundaries shall coincide with ADU boundaries, i.e. the first 304 byte of an ADU shall be the first byte of payload data within a JPEG 305 XS segment. 307 A JPEG XS frame, and by that each ADU, shall consist of a Video 308 Essence box defining the meta information required for playback, 309 concatenated to the JPEG XS codestream, defining the sample values of 310 the picture. 312 The JPEG XS stream, as defined in ISO/IEC 21122-1 [ISO21122-1] itself 313 consists of a JPEG XS header that defines picture parameters, and one 314 or multiple slices that contain the entropy coded picture data and an 315 EOC marker. A slice is the smallest independently decodable unit of 316 a JPEG XS codestream. 318 JPEG XS frames are separated into fragments such that the first byte 319 of a fragment and the byte following the last byte of a fragment are 320 in two disinct packets, except for the last fragment of the frame. 321 Fragments are enumerated by the slice group index of the slice group 322 contained within. 324 4.1. RTP Header Usage 326 The SSRC RTP field is used to discriminate each separate JPEG XS 327 video stream from others. Within a specific JPEG XS video stream, 328 identified by its SSRC, the picture counter field is used to identify 329 to which picture a packet corresponds to. 331 4.2. Payload Header 333 The following figure illustrates the RTP payload header used in order 334 to transport each JPEG XS video stream (identified by a distinct 335 SSRC). 337 0 1 2 3 338 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 339 +---+-+-+-------+-+-------------+-------------------------------+ 340 | V |P|X| CC |M| PT | Sequence number | 341 +---+-+-+-------+-+-------------+-------------------------------+ 342 | Timestamp | 343 +---------------------------------------------------------------+ 344 | Synchronization source (SSRC) identifier | 345 +-----+-+-+-------+-----------------------+-+-------------------+ 346 | Ver |f|l| SlcGp | SliceGrpStart |L| Picture Counter | 347 +-----+-+-+-------+-----------------------+-+-------------------+ 348 | Data | 349 +---------------------------------------------------------------+ 351 Figure 1: RTP and payload headers 353 The version (V), padding (P), extension (X), CSRC count (CC), 354 sequence number and synchronization source (SSRC) fields follow their 355 respective definitions in RFC 3550 [RFC3550]. 357 The timestamp should be based on a globally synchronized 90 kHz clock 358 reference, and should correspond to the number of cycles since the 359 SMPTE Epoch (as per defined in SMPTE ST 2059-1:2015 [SMPTE-ST2059]) 360 modulo 2^32: 362 timestamp = floor((now - epoch)*90000) % 2^32 364 where now and epoch are real numbers expressed in seconds, now being 365 the current timestamp and epoch the reference timestamp and floor 366 indicates rounding to the next lower integer. 368 As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the 369 RTP timestamp designates the sampling instant of the first octet of 370 the picture to which the RTP packet belongs. Packets shall not 371 include data from multiple frames, and all packets belonging to the 372 same frame shall have the same timestamp. Several successive RTP 373 packets will consequently have equal timestamps if they belong to the 374 same picture (that is until the marker bit is set to 1, marking the 375 last packet of the frame), and the timestamp is only increased when a 376 new frame begins. 378 If the sampling instant does not correspond to an integer value of 379 the clock, the value shall be truncated to the next lowest integer, 380 with no ambiguity. 382 The remaining fields are defined as follows: 384 +-----------------+----------+--------------------------------------+ 385 | Field | Width | Description | 386 +-----------------+----------+--------------------------------------+ 387 | Marker (M) | 1 bit | The marker bit is used to indicate | 388 | | | the last packet of a frame. This | 389 | | | enables a decoder to finish decoding | 390 | | | the picture, where it otherwise may | 391 | | | need to wait for the next packet to | 392 | | | explicitly know that the frame is | 393 | | | finished. | 394 | Payload Type | 7 bits | A dynamically allocated payload type | 395 | (PT) | | field that designates the payload as | 396 | | | JPEG XS video. | 397 | Vers | 3 bits | This field indicates the version | 398 | | | number of the payload header. The | 399 | | | value of this field shall be 0 for | 400 | | | the purpose of this edition of the | 401 | | | RFC. | 402 | f | 1 bit | The f field shall be set if a new | 403 | | | fragment is started within this | 404 | | | packet, i.e. if this packet contains | 405 | | | the first byte of a fragment. NOTE: | 406 | | | The first slice group of a frame and | 407 | | | the JPEG XS header segment form a | 408 | | | fragment. For that reason, the f-bit | 409 | | | remains unset in the packet that | 410 | | | contains the first byte of slice | 411 | | | group 0 but does not also contains | 412 | | | the first byte of the Video Essence | 413 | | | box. All other slice groups form | 414 | | | framgents of their own. The f bit | 415 | | | allows a quick identificaiton of | 416 | | | packets that start a fragment. The | 417 | | | SliceGrpStart field (see below) can | 418 | | | be used to identify the start of a | 419 | | | slice group. | 420 | l | 1 bit | The l field is a one-bit field that | 421 | | | is cleared if the fragment to which | 422 | | | the first byte of the packet belongs | 423 | | | extends througout a subsequent | 424 | | | packet. It is set if the fragment to | 425 | | | which the first byte of the packet | 426 | | | belongs ends in this packet. | 427 | SlcGp | 5 bits | The SlcGp (Slice Group) field | 428 | | | contains the slice group index | 429 | | | modulo 64 that is contained in the | 430 | | | fragment that is started in this | 431 | | | packet. If no fragment starts in | 432 | | | this packet, it contains the slice | 433 | | | group index modulo 64 of the slice | 434 | | | group that is contained in the | 435 | | | fragment to which the first byte of | 436 | | | the payload data of this packet | 437 | | | belongs. | 438 | SliceGrpStart | 11 bits | This field indicates the byte offset | 439 | | | of the slice header marker (SLH, hex | 440 | | | 0xff20, see ISO/IEC 21122-1 | 441 | | | [ISO21122-1]) of the slice group | 442 | | | that starts in this packet, relative | 443 | | | to the start of the packet. If no | 444 | | | slice group starts in this packet, | 445 | | | this field shall be 0. NOTE: Since | 446 | | | the payload data has a non-zero | 447 | | | offset within a packet, this field | 448 | | | can also be used to identify whether | 449 | | | a slice group starts in a packet. If | 450 | | | 0, no slice group starts in this | 451 | | | packet. Consequently, for slice | 452 | | | groups with a non-zero slice group | 453 | | | index, this field will be non-zero | 454 | | | if and only if the f-field is set. | 455 | | | For the first slice gorup of a | 456 | | | frame, however, the f bit indiecate | 457 | | | sthe start of the fragment. whereas | 458 | | | this field indicates the start of | 459 | | | the slice group. Due to the non-zero | 460 | | | size of the JPEG XS header segment, | 461 | | | this need not to happen in the same | 462 | | | packet. | 463 | F | 1 bit | The F flag in the JPEG XS payload | 464 | | | header shall be set if the packet | 465 | | | contains the first byte of the JPEG | 466 | | | XS Header Segment, and hence | 467 | | | includes the first bytes of the | 468 | | | Video Essence box. Readers may use | 469 | | | this flag to extract information | 470 | | | easily from the video essence box. | 471 | Picture number | 10 bits | Counter indicating the current | 472 | | | picture number modulo 2^11. The | 473 | | | picture number is incremented by one | 474 | | | at the beginning of each frame, and | 475 | | | stays constant throuout all packets | 476 | | | that contribute to to the same | 477 | | | frame. | 478 +-----------------+----------+--------------------------------------+ 480 Table 1: Payload header fields description 482 4.3. Payload Data 484 The payload data of a JPEG XS transport stream consists of a 485 concatenation of multiple JPEG XS Frames. 487 Each JPEG XS frame is the concatenation of multiple fragments where 488 each fragment contains one and only one slice group. The first 489 fragment of a frame also contains the Video Essence box and the JPEG 490 XS header, the last fragment also contains the EOC marker. 491 Figure Figure 2 depicts this layout. 493 Fragments may extend over multiple RTP packets. In particular, slice 494 groups and by that fragments have to be sized such that the first 495 byte of a fragment and the byte following the last byte of a fragment 496 are in two distinct packets. 498 The start of a fragment can be identified by the "f" bit in the 499 Payload header, the start of a slice group within a packet and its 500 location in the packet by the SliceGrpStart field in the same Payload 501 header. 503 ^ +-------------------------------------------+ ^ 504 | | Video Essence Box | | 505 | | +-------------------------------------+ | | 506 | | | Sub boxes of the Video Essence Box | | | 507 Frag- | +-------------------------------------+ | JPEG 508 ment | : additional sub-boxes of the VE-Box : | XS 509 #0 | +-------------------------------------+ | Header 510 | | | Seg- 511 | +-------------------------------------------+ ment 512 | | JPEG XS Header | | 513 | | +-------------------------------------+ | | 514 | | | SOC Marker | | | 515 | | +-------------------------------------+ | | 516 | | : Additional Marker Segments : | | 517 | | +-------------------------------------+ | | 518 | | | | 519 | +-------------------------------------------+ v 520 | | Slice Group #0 | 521 | | +-------------------------------------+ | 522 | | | Slice #0 of Slice Group #0 | | 523 | | | +-------------------------------+ | | 524 | | | | SLH Marker | | | 525 | | | +-------------------------------+ | | 526 | | | : Entropy Coded Data : | | 527 | | | +-------------------------------+ | | 528 | | +-------------------------------------+ | 529 | | | Slice #1 of Slice Group #0 | | 530 | | : : | 531 | | +-------------------------------------+ | 532 | | | Slice #n-1 of Slice Group #0 | | 533 | | : : | 534 v | +-------------------------------------+ | 535 ^ +-------------------------------------------+ 536 | | Slice Group #1 | 537 Frag- : : 538 ment : : 539 #1 : : 540 | : : 541 v +-------------------------------------------+ 542 : : 543 ^ +-------------------------------------------+ 544 | | Slice Group #n-1 | 545 Frag- : : 546 ment : : 547 #n-1 +-------------------------------------------+ 548 | | EOC Marker | 549 v +-------------------------------------------+ 551 Figure 2: JPEG XS Payload Data 553 4.4. Traffic Shaping and Delivery Timing 555 The traffic shaping and delivery timing shall be in accordance with 556 the Network Compatibility Model compliance definitions specified in 557 SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders 558 (Type NL) or Wide Senders (Type W). 560 Note: The Virtual Receiver Buffer Model compliance definitions of ST 561 2110-21 do not apply. 563 5. Congestion Control Considerations 565 Congestion control for RTP SHALL be used in accordance with RFC 3550 566 [RFC3550], and with any applicable RTP profile: e.g., RFC 3551 567 [RFC3551]. An additional requirement if best-effort service is being 568 used is users of this payload format MUST monitor packet loss to 569 ensure that the packet loss rate is within acceptable parameters. 571 Circuit Breakers [RFC8083] is an update to RTP [RFC3550] that defines 572 criteria for when one is required to stop sending RTP Packet Streams. 573 The circuit breakers is to be implemented and followed. 575 6. Payload Format Parameters 577 6.1. Media Type Definition 579 Type name: video 581 Subtype name: jpeg-xs 583 Encoding considerations: 585 This media type is framed and binary; see Section 4.8 in 586 RFC 6838 [RFC6838]. 588 Security considerations: 590 Please see the Security Considerations section in RFC XXXX 592 6.2. Mapping to SDP 594 6.2.1. General 596 A Session Description Protocol (SDP) object shall be created for each 597 RTP stream and it shall be in accordance with the provisions of SMPTE 598 ST 2110-10 [SMPTE-ST2110-10]. 600 The information carried in the media type specification has a 601 specific mapping to fields in the Session Description Protocol (SDP), 602 which is commonly used to describe RTP sessions. When SDP is used to 603 specify sessions employing the DV encoding, the mapping is as 604 follows: 606 6.2.2. Media type and subtype 608 The media type ("video") goes in SDP "m=" as the media name. 610 The media subtype ("jpeg-xs") goes in SDP "a=rtpmap" as the encoding 611 name. The RTP clock rate in "a=rtpmap" MUST be 90000, which for the 612 payload format defined in this document is a 90 kHz clock. 614 6.2.3. Traffic shaping 616 The SDP object shall include the TP parameter and may include the 617 CMAX parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21]. 619 6.2.4. Other parameters 621 The SDP object shall include the following payload-format-specific 622 parameter in the a=fmtp line: 624 SSN SMPTE Standard Number in the format: ST-: 625 e.g. ST2110-20:2017 626 The number shall be that of the JPEG XS standard 628 Any remaining parameters go in the SDP "a=fmtp" attribute by copying 629 them directly from the media type string as a semicolon-separated 630 list of parameter=value pairs. 632 6.2.5. Offer/Answer Considerations 634 The following considerations apply when using SDP offer/answer 635 procedures [RFC3264] to negotiate the use of the JPEG XS payload in 636 RTP: 638 o The "encode" parameter can be used for sendrecv, sendonly, and 639 recvonly streams. Each encode type MUST use a separate payload 640 type number. 642 o Any unknown parameter in an offer MUST be ignored by the receiver 643 and MUST NOT be included in the answer. 645 7. IANA Considerations 647 This memo requests that IANA registers video/jpeg-xs as specified in 648 Section 6.1. The media type is also requested to be added to the 649 IANA registry for "RTP Payload Format MIME types" [1]. 651 8. Security Considerations 653 [ FIXME: imported from RFC 7587 ] 655 RTP packets using the payload format defined in this specification 656 are subject to the security considerations discussed in the RTP 657 specification [RFC3550] and in any applicable RTP profile such as 658 RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/ 659 SAVPF [RFC5124]. This implies that confidentiality of the media 660 streams is achieved by encryption. 662 However, as "Securing the RTP Framework: Why RTP Does Not Mandate a 663 Single Media Security Solution" [RFC7202] discusses, it is not an RTP 664 payload format's responsibility to discuss or mandate what solutions 665 are used to meet the basic security goals like confidentiality, 666 integrity, and source authenticity for RTP in general. This 667 responsibility lies on anyone using RTP in an application. They can 668 find guidance on available security mechanisms and important 669 considerations in "Options for Securing RTP Sessions" [RFC7201]. 670 Applications SHOULD use one or more appropriate strong security 671 mechanisms. 673 This payload format and the JPEG XS encoding do not exhibit any 674 substantial non-uniformity, either in output or in complexity to 675 perform the decoding operation and thus are unlikely to pose a 676 denial-of-service threat due to the receipt of pathological 677 datagrams. 679 [ FIXME: imported from RFC 4175 ] 681 It is important to note that HD or UHDTV JPEG XS-encoded video can 682 have significant bandwidth requirements (typically more than 1 Gbps 683 for ultra high-definition video, especially if using high framerate). 684 This is sufficient to cause potential for denial-of-service if 685 transmitted onto most currently available Internet paths. 687 Accordingly, if best-effort service is being used, users of this 688 payload format MUST monitor packet loss to ensure that the packet 689 loss rate is within acceptable parameters. Packet loss is considered 690 acceptable if a TCP flow across the same network path, and 691 experiencing the same network conditions, would achieve an average 692 throughput, measured on a reasonable timescale, that is not less than 693 the RTP flow is achieving. This condition can be satisfied by 694 implementing congestion control mechanisms to adapt the transmission 695 rate (or the number of layers subscribed for a layered multicast 696 session), or by arranging for a receiver to leave the session if the 697 loss rate is unacceptably high. 699 This payload format may also be used in networks that provide 700 quality-of-service guarantees. If enhanced service is being used, 701 receivers SHOULD monitor packet loss to ensure that the service that 702 was requested is actually being delivered. If it is not, then they 703 SHOULD assume that they are receiving best-effort service and behave 704 accordingly. 706 9. RFC Editor Considerations 708 Note to RFC Editor: This section may be removed after carrying out 709 all the instructions of this section. 711 RFC XXXX is to be replaced by the RFC number this specification 712 receives when published. 714 10. References 716 10.1. Normative References 718 [ISO15444-1] 719 International Organization for Standardization (ISO) - 720 International Electrotechnical Commission (IEC), 721 "Information technology - JPEG 2000 image coding system: 722 Core coding system", ISO/IEC IS 15444-1, 2016, 723 . 725 [ISO18477-3] 726 International Organization for Standardization (ISO) - 727 International Electrotechnical Commission (IEC), 728 "Information technology - Scalable compression and coding 729 of continuous-tone still images - Part 3: Box file 730 format", ISO/IEC 18477-3:2015, 2015, 731 . 733 [ISO21122-1] 734 International Organization for Standardization (ISO) - 735 International Electrotechnical Commission (IEC), 736 "Information technology - Low-latency lightweight image 737 coding system - Part 1: Core coding system", ISO/IEC DIS 738 21122-1, under development, 739 . 741 [ISO21122-3] 742 International Organization for Standardization (ISO) - 743 International Electrotechnical Commission (IEC), 744 "Information technology - Low-latency lightweight image 745 coding system - Part 3: Transport and container formats", 746 ISO/IEC NP 21122-3, under development, 747 . 749 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 750 Requirement Levels", BCP 14, RFC 2119, 751 DOI 10.17487/RFC2119, March 1997, 752 . 754 [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model 755 with Session Description Protocol (SDP)", RFC 3264, 756 DOI 10.17487/RFC3264, June 2002, 757 . 759 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 760 Jacobson, "RTP: A Transport Protocol for Real-Time 761 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 762 July 2003, . 764 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 765 Video Conferences with Minimal Control", STD 65, RFC 3551, 766 DOI 10.17487/RFC3551, July 2003, 767 . 769 [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. 770 Norrman, "The Secure Real-time Transport Protocol (SRTP)", 771 RFC 3711, DOI 10.17487/RFC3711, March 2004, 772 . 774 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 775 Specifications and Registration Procedures", BCP 13, 776 RFC 6838, DOI 10.17487/RFC6838, January 2013, 777 . 779 [RFC8083] Perkins, C. and V. Singh, "Multimedia Congestion Control: 780 Circuit Breakers for Unicast RTP Sessions", RFC 8083, 781 DOI 10.17487/RFC8083, March 2017, 782 . 784 [SMPTE-ST2110-10] 785 Society of Motion Picture and Television Engineers, "SMPTE 786 Standard - Professional Media Over Managed IP Networks: 787 System Timing and Definitions", SMPTE ST 2110-10:2017, 788 2017, . 790 [SMPTE-ST2110-21] 791 Society of Motion Picture and Television Engineers, "SMPTE 792 Standard - Professional Media Over Managed IP Networks: 793 Traffic Shaping and Delivery Timing for Video", SMPTE ST 794 2110-21:2017, 2017, 795 . 797 10.2. Informative References 799 [RFC4175] Gharai, L. and C. Perkins, "RTP Payload Format for 800 Uncompressed Video", RFC 4175, DOI 10.17487/RFC4175, 801 September 2005, . 803 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 804 "Extended RTP Profile for Real-time Transport Control 805 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 806 DOI 10.17487/RFC4585, July 2006, 807 . 809 [RFC5124] Ott, J. and E. Carrara, "Extended Secure RTP Profile for 810 Real-time Transport Control Protocol (RTCP)-Based Feedback 811 (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February 812 2008, . 814 [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP 815 Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, 816 . 818 [RFC7202] Perkins, C. and M. Westerlund, "Securing the RTP 819 Framework: Why RTP Does Not Mandate a Single Media 820 Security Solution", RFC 7202, DOI 10.17487/RFC7202, April 821 2014, . 823 [SMPTE-ST2059] 824 Society of Motion Picture and Television Engineers, "SMPTE 825 Standard - Generation and Alignment of Interface Signals 826 to the SMPTE Epoch", SMPTE ST 2059-1:2015, 2015, 827 . 829 10.3. URIs 831 [1] http://www.iana.org/assignments/rtp-parameters 833 Authors' Addresses 835 Sebastien Lugan 836 intoPIX S.A. 837 Rue Emile Francqui, 9 838 1435 Mont-Saint-Guibert 839 Belgium 841 Phone: +32 10 23 84 70 842 Email: s.lugan@intopix.com 843 URI: http://www.intopix.com 844 Gael Rouvroy 845 intoPIX S.A. 846 Rue Emile Francqui, 9 847 1435 Mont-Saint-Guibert 848 Belgium 850 Phone: +32 10 23 84 70 851 Email: g.rouvroy@intopix.com 852 URI: http://www.intopix.com 854 Antonin Descampe 855 intoPIX S.A. 856 Rue Emile Francqui, 9 857 1435 Mont-Saint-Guibert 858 Belgium 860 Phone: +32 10 23 84 70 861 Email: a.descampe@intopix.com 862 URI: http://www.intopix.com 864 Thomas Richter 865 Fraunhofer IIS 866 Am Wolfsmantel 33 867 91048 Erlangen 868 Germany 870 Phone: +49 9131 776 5126 871 Email: thomas.richter@iis.fraunhofer.de 872 URI: https://www.iis.fraunhofer.de/ 874 Alexandre Willeme 875 Universite catholique de Louvain 876 Place du Levant, 2 - bte L5.04.04 877 1348 Louvain-la-Neuve 878 Belgium 880 Phone: +32 10 47 80 82 881 Email: alexandre.willeme@uclouvain.be 882 URI: https://uclouvain.be/en/icteam