idnits 2.17.1 draft-roca-nwcrg-generic-fec-api-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 46 instances of too long lines in the document, the longest one being 5 characters in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Line 407 has weird spacing: '...ol_size sour...' == Line 561 has weird spacing: '...ol_size sour...' == Line 588 has weird spacing: '...ram dec cont...' == Line 699 has weird spacing: '...ram dec cont...' == Line 731 has weird spacing: '...ram dec cont...' -- The document date (November 17, 2019) is 1614 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 NWCRG V. Roca (Ed.) 3 Internet-Draft INRIA 4 Intended status: Informational J. Detchart 5 Expires: May 20, 2020 ISAE - Supaero 6 C. Adjih 7 INRIA 8 M. Pedersen 9 Steinwurf ApS 10 November 17, 2019 12 Generic Application Programming Interface (API) for Sliding Window FEC 13 Codes 14 draft-roca-nwcrg-generic-fec-api-07 16 Abstract 18 This document introduces a generic Application Programming Interface 19 (API) for sliding window FEC codes. This API is meant to be 20 compatible with any sliding window FEC code. It defines the core 21 procedures and functions meant to control the codec (i.e., 22 implementation of the FEC code). However, it leaves out all upper 23 layer aspects that are the responsibility of the application or 24 protocol making use of the codec. As a consequence, this is not an 25 API for a FEC Scheme since certain mechanisms that must be defined by 26 any FEC Scheme (e.g., signalling and FEC Payload IDs) are the 27 responsibility of the caller instead of being addressed by the codec. 28 A first goal of this document is to pave the way for a future open- 29 source implementation of such codes, another goal is to simplify the 30 development of content delivery protocols that rely on sliding window 31 FEC codes for robust transmissions. 33 Status of This Memo 35 This Internet-Draft is submitted in full conformance with the 36 provisions of BCP 78 and BCP 79. 38 Internet-Drafts are working documents of the Internet Engineering 39 Task Force (IETF). Note that other groups may also distribute 40 working documents as Internet-Drafts. The list of current Internet- 41 Drafts is at https://datatracker.ietf.org/drafts/current/. 43 Internet-Drafts are draft documents valid for a maximum of six months 44 and may be updated, replaced, or obsoleted by other documents at any 45 time. It is inappropriate to use Internet-Drafts as reference 46 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on May 20, 2020. 50 Copyright Notice 52 Copyright (c) 2019 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents 57 (https://trustee.ietf.org/license-info) in effect on the date of 58 publication of this document. Please review these documents 59 carefully, as they describe your rights and restrictions with respect 60 to this document. Code Components extracted from this document must 61 include Simplified BSD License text as described in Section 4.e of 62 the Trust Legal Provisions and are provided without warranty as 63 described in the Simplified BSD License. 65 Table of Contents 67 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 68 2. Definitions and Abbreviations . . . . . . . . . . . . . . . . 3 69 3. AL-FEC Codes and Mechanisms Considered by the Generic API . . 3 70 3.1. Mechanisms Considered or Ignored by the API . . . . . . . 5 71 4. Generic API for Sliding Window FEC Codes . . . . . . . . . . 6 72 4.1. General Definitions Common to the Encoder and Decoder . . 6 73 4.2. Encoder . . . . . . . . . . . . . . . . . . . . . . . . . 9 74 4.3. Decoder . . . . . . . . . . . . . . . . . . . . . . . . . 13 75 4.4. Coding Window Functions at an Encoder and Decoder . . . . 17 76 4.5. Coding Coefficients Functions at an Encoder and Decoder . 19 77 5. Security Considerations . . . . . . . . . . . . . . . . . . . 22 78 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22 79 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 80 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 23 81 8.1. Normative References . . . . . . . . . . . . . . . . . . 23 82 8.2. Informative References . . . . . . . . . . . . . . . . . 23 83 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23 85 1. Introduction 87 Forward Erasure Correction (FEC) codes are a key element of 88 communication systems, used to efficiently recover from packet losses 89 during content delivery sessions. Among the FEC codes working at the 90 network and higher layers, one can broadly distinguish block codes 91 and sliding window codes. Block FEC codes require the data flow 92 coming from the application to be segmented into blocks of a 93 predefined maximum size, before generating a certain number of repair 94 packets. With the second type of FEC codes, an encoding window 95 continuously slides over the set of source data and repair packets 96 are generated at any time by computing for instance a linear 97 combination of data present in the encoding window. This fundamental 98 difference seriously impacts the way they can be used by a content 99 delivery protocol or application. 101 This document introduces a generic Application Programming Interface 102 (API) for sliding window FEC codes. This API is meant to be usable 103 by any sliding window FEC code and FEC Scheme independently of the 104 protocol that may rely on it. This API defines the core procedures 105 and functions meant to control the codec (i.e., implementation of the 106 FEC code), but leaves out all upper layer aspects that are the 107 responsibility of the application making use of the codec. 109 This API is meant to be usable by any sliding window FEC code. 110 independently of the FEC Scheme or network coding protocol that may 111 rely on it This API defines the core procedures and functions meant 112 to control the codec (i.e., implementation of the FEC code), but 113 leaves out all upper layer aspects that are the responsibility of the 114 application making use of the codec. For instance, those restricted 115 to end-to-end use-cases as well as those compatible with in-network 116 re-encoding use-cases. Additionally, this API is not impacted by the 117 intra-flow versus inter-flow nature of the use-case, nor is it 118 impacted by the single-path versus multi-paths nature of the use- 119 case, since those are usage considerations under the responsibility 120 of the caller. 122 A goal of this document is to pave the way for a future open-source 123 implementation of such codes. 125 2. Definitions and Abbreviations 127 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 128 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 129 document are to be interpreted as described in [RFC2119]. 131 This document uses the following definitions and abbreviations: 133 XXX 135 3. AL-FEC Codes and Mechanisms Considered by the Generic API 137 This generic FEC API is meant to be used with: 139 o sliding window codes, that manage an encoding window (of fixed or 140 variable size) that slides over the set of source symbols at the 141 sender. On the opposite, block codes (e.g., Reed-Solomon, LDPC, 142 Raptor(Q)) are out of scope; 143 o codes that are restricted to use-cases that involve a single 144 encoding point and a single decoding point (i.e., FEC operations 145 are carried out either within the end-hosts or middle-boxes), as 146 well as codes that can be used with use-cases that involve in- 147 network re-coding operations; 148 o use-cases that are limited to an intra-flow coding (simple case), 149 as well as use-cases that involve inter-flow coding. This second 150 case is more complex to address (e.g., with questions such as how 151 to identify a packet of a flow?) however this is the 152 responsibility of the application or protocol using this codec and 153 not the codec itself. This aspect is therefore transparent to the 154 API; 155 o use-cases that are limited to single-path communications and use- 156 cases that consider multi-path communications. Here also this is 157 a usage consideration that is transparent to the API; 158 o use-cases that involve a dynamic adaptation of the codec 159 parameters (e.g., its code rate because the communication path 160 losses is known thanks to feedbacks and an appropriate strategy 161 can be defined); 162 o fixed code rate or not FEC codes, including rateless codes where 163 the number of repair symbols that can be generated is huge (in 164 theory unlimited); 165 o ideal (MDS) or non-ideal (non-MDS) codes. However most of the 166 time, sliding window codes are non-ideal codes, meaning that 167 slightly more than l repair symbols may be required to recover all 168 the l lost source symbols; 170 A key question is to determine what mechanisms are included in the 171 codec and what mechanisms are left to the responsibility of the 172 caller (i.e., an application or a protocol making use of this codec) 173 (Figure 1). More precisely, an FEC Scheme (such as the RLC FEC 174 Scheme [RLC] in case of FECFRAME [fecframe-ext]) defines all the 175 internal code details in order to enable interoperable 176 implementations, but also signaling considerations that are essential 177 to use them in a specific context. 179 +----------------------------------------------------------+ 180 | Application / Protocol Mechanisms | 181 | (out of scope) | 182 | | 183 | + - - - - - - - - - - - - - - - - - - - - - + | 184 | FEC Scheme | 185 | +--|-------------------------------------------|-+ 186 | | 187 | | | <----- FEC Codec API (this doc.) -----> | 188 | | +---------------------------------------+ 189 | | | | Low Level FEC Codec | | 190 | | +---------------------------------------+ 191 +---------+ + - - - - - - - - - - - - - - - - - - - - - + 193 Figure 1: Position of the FEC Codec API with respect to the low level 194 FEC Codec, the FEC Scheme, the protocol and other caller services. 196 3.1. Mechanisms Considered or Ignored by the API 198 Applying FEC coding, through an FEC Scheme, in a given protocol to 199 improve transmission robustness involves many mechanisms. However, 200 these mechanisms are not all the responsibility of the codec and can 201 be implemented within the application or within the protocol that 202 uses this FEC codec. For instance, the following mechanisms are 203 considered **out of scope of the API**, being implemented by the 204 caller, without any impact on the codec: 206 o memory management; 207 o packet transmission and reception; 208 o signaling header creation / parsing; 209 o ADU to source symbol mapping; 210 o code rate adjustment, for instance thanks to the knowledge of 211 losses at a receiver via feedbacks; 212 o selective ACK creation and parsing; 213 o congestion control. 215 The following mechanisms are **within scope of the API**: 217 o session management (sender and receiver); 218 o encoding window management (sender and receiver); 219 o set/get/generate coding coefficients (sender and receiver); 220 o build coded symbol (sender only); 221 o decode with newly received source or repair symbol (receiver 222 only); 224 4. Generic API for Sliding Window FEC Codes 226 The following sections describe the generic API, following a 227 C-language formalism. This API tries to adhere to C99 version of C, 228 although it may not strictly be guaranteed. Everything is prefixed 229 by "swif" (sliding window FEC). 231 4.1. General Definitions Common to the Encoder and Decoder 233 This section gathers general definitions that are used by both an 234 encoder and decoder. 236 About FEC Codepoints: 238 An application first needs to negotiate with its remote side the 239 right FEC Scheme to use. This negotiation usually relies on the 240 FEC Encoding ID associated to this FEC Scheme for this 241 application. A difficulty is that the FEC Encoding ID space, 242 associated to an IANA registry, is protocol specific and the same 243 value are usually associated to different FEC Schemes depending on 244 the protocol. For instance, the FEC Encoding ID value 2 may be 245 used for two totally different FEC Schemes in protocol A and 246 protocol B. Therefore, the FEC Encoding ID, from the Generic FEC 247 API point of view, cannot be used to uniquely identify the target 248 codec. 249 The use of a codepoint to identify locally the right FEC codec 250 requires that the application knows a mapping between the FEC 251 Encoding ID it uses for a given protocol, and the local FEC 252 Codepoints corresponding to available codecs. This will be done 253 at development time, the FEC API header file giving access to the 254 swif_codepoint_t enumeration with the list of all codecs available 255 locally. 257 258 /** 259 * Return value of any function. 260 * 261 * SWIF_STATUS_OK = 0 Success 262 * SWIF_STATUS_FAILURE Failure. The function called did not succeed to 263 * perform its task, however this is not an error 264 * (e.g., it happens when decoding fails). 265 * SWIF_STATUS_ERROR Generic error type. The detailed error type is 266 * stored in the errno variable of swif_encoder_t and 267 * swif_decoder_t structures. 268 */ 269 typedef enum { 270 SWIF_STATUS_OK = 0, 271 SWIF_STATUS_FAILURE, 272 SWIF_STATUS_ERROR 273 } swif_status_t; 275 /** 276 * Potential errors. 277 */ 278 typedef enum { 279 SWIF_ERRNO_NULL = 0, /* everything is fine */ 280 SWIF_ERRNO_UNSUPPORTED_CODEPOINT, 281 /* and many more... */ 282 } swif_errno_t; 284 /** 285 * FEC Codepoints. 286 * These identifiers are opaque identifiers that fully identify an FEC 287 * code locally, including certain parameters like its Galois Field. 288 * These codepoints are codec specific and only have a local meaning. 289 * They should not be transmitted as different implementations may use 290 * them inconsistently. 291 * Note that the same FEC code may be used by several FEC Encoding IDs 292 * and therefore share the same codepoint. On the opposite multiple 293 * implementations of a given FEC code may exist locally, for instance 294 * with different optimizations, and then several codepoints, one per 295 * codec, will exist for the same FEC code. The following names are 296 * therefore only provided as examples. 297 */ 298 typedef enum { 299 SWIF_CODEPOINT_NULL = 0, /* codepoint 0 is reserved */ 301 /* codepoint for sliding window codec AAA. */ 302 SWIF_CODEPOINT_AAA_CODEC, 304 /* codepoint for sliding window codec BBB. */ 305 SWIF_CODEPOINT_BBB_CODEC, 307 /* list here other identifiers for any codec of interest... */ 308 } swif_codepoint_t; 310 /** 311 * Encoding Symbol Identifier (ESI) generic type. 312 * With Sliding Window FEC codes, an ESI is in fact a source symbol 313 * identifier, unlike block FEC codes. 314 */ 315 typedef uint32_t esi_t; 316 /** 317 * Throughout the API, a pointer to this structure is used as an 318 * identifier of the encoder instance (or "enc"). 319 * 320 * This generic structure is meant to be extended by each codec with 321 * new pieces of information that are specific to each codec. 322 */ 323 typedef struct swif_encoder { 324 swif_codepoint_t codepoint; 326 /* when a function returns with SWIF_STATUS_ERROR, the errno 327 * variable contains a more detailed error type. This variable 328 * is set by the codec and accessible to the application in 329 * READ ONLY mode. Otherwise its value is undefined. */ 330 swif_errno_t swif_errno; 332 /* pointers to codec specific versions of API functions. */ 333 swif_status_t (*set_callback_functions) ( 334 struct swif_encoder*, void (*) (void*, esi_t), void*); 335 swif_status_t (*set_parameters) ( 336 struct swif_encoder*, uint32_t, uint32_t, void*); 337 swif_status_t (*get_parameters) ( 338 struct swif_encoder*, uint32_t, uint32_t, void*); 339 swif_status_t (*build_repair_symbol) ( 340 struct swif_encoder*, void*); 341 swif_status_t (*reset_coding_window) (struct swif_encoder*); 342 swif_status_t (*add_source_symbol_to_coding_window) ( 343 struct swif_encoder*, void*, esi_t); 344 swif_status_t (*remove_source_symbol_from_coding_window) ( 345 struct swif_encoder*, esi_t); 346 swif_status_t (*get_coding_window_information) ( 347 struct swif_encoder*, esi_t*, esi_t*, uint32_t*); 348 swif_status_t (*set_coding_coefs_tab) ( 349 struct swif_encoder*, void*, uint32_t); 350 swif_status_t (*generate_coding_coefs) ( 351 struct swif_encoder*, uint32_t, uint32_t); 352 swif_status_t (*get_coding_coefs_tab) ( 353 struct swif_encoder*, void**, uint32_t*); 354 } swif_encoder_t; 356 /** 357 * Decoder structure that contains whatever is needed for decoding. 358 * The exact content of this structure is FEC code dependent, the 359 * structure below being a non normative example. 360 */ 361 typedef struct swif_decoder { 362 swif_codepoint_t codepoint; 363 /* when a function returns with SWIF_STATUS_ERROR, the errno 364 * variable contains a more detailed error type. This variable 365 * is set by the codec and accessible to the application in 366 * READ ONLY mode. Otherwise its value is undefined. */ 367 swif_errno_t swif_errno; 369 /* pointers to codec specific versions of API functions. */ 370 swif_status_t (*set_callback_functions) ( 371 struct swif_decoder*, void (*) (void*, esi_t), 372 void* (*) (void*, esi_t), 373 void* (*) (void*, void*, esi_t), void*); 374 swif_status_t (*set_parameters) ( 375 struct swif_decoder*, uint32_t, uint32_t, void*); 376 swif_status_t (*get_parameters) ( 377 struct swif_decoder*, uint32_t, uint32_t, void*); 378 swif_status_t (*decode_with_new_source_symbol) ( 379 struct swif_decoder*, void* const, esi_t); 380 swif_status_t (*decode_with_new_repair_symbol) ( 381 struct swif_decoder*, void* const); 382 swif_status_t (*reset_coding_window) (swif_encoder_t*); 383 swif_status_t (*add_source_symbol_to_coding_window) ( 384 struct swif_decoder*, esi_t); 385 swif_status_t (*remove_source_symbol_from_coding_window) ( 386 struct swif_decoder*, esi_t); 387 swif_status_t (*set_coding_coefs_tab) ( 388 struct swif_decoder*, void*, uint32_t); 389 swif_status_t (*generate_coding_coefs) ( 390 struct swif_decoder*, uint32_t, uint32_t); 391 } swif_decoder_t; 392 394 General definitions. 396 4.2. Encoder 398 399 /** 400 * Create and initialize an encoder, providing only key parameters. 401 * 402 * @param codepoint opaque identifier that fully identifies the FEC 403 * code to use. 404 * @param verbosity print information on the codec processing. 405 * 0 is the minimum verbosity, the maximum verbosity 406 * level being implementation specific. 407 * @param symbol_size source and repair symbol size in bytes. Cannot 408 * change during the codec instance lifetime. 409 * @param max_encoding_window_size 410 * @return pointer to a swif_encoder_t structure if okay, or 411 * NULL in case of error. 412 **/ 413 swif_encoder_t* swif_encoder_create ( 414 swif_codepoint_t codepoint, 415 uint32_t verbosity, 416 uint32_t symbol_size, 417 uint32_t max_coding_window_size); 419 /** 420 * Release an encoder and its associated ressources. 421 **/ 422 swif_status_t swif_encoder_release (swif_encoder_t* enc); 424 /** 425 * Set the various callback functions for this encoder. 426 * All the callback functions require an opaque context parameter, that 427 * must be initialized accordingly by the application, since it is 428 * application specific. 429 * 430 * @param enc 431 * @param source_symbol_removed_from_coding_window_callback 432 * (IN) Pointer to the function, within the application, 433 * that needs to be called each time a source symbol is 434 * removed from the left side of the coding window. 435 * This callback is called each time the encoding window 436 * slides to the right and an old source symbol needs to 437 * be removed on the left. The application therefore knows 438 * this source symbol will no longer be used by the codec 439 * and can free the associated buffer if need be. This 440 * function does not return anything. 441 * @param context_4_callback 442 * (IN) Pointer to the application-specific context that 443 * will be passed to the callback function (if any). This 444 * context is not interpreted by this function. 445 * @return 446 */ 447 swif_status_t swif_encoder_set_callback_functions ( 448 swif_encoder_t* enc, 449 void (*source_symbol_removed_from_coding_window_callback) ( 450 void* context, 451 esi_t old_symbol_esi), 452 void* context_4_callback); 454 /** 455 * This function sets one or more FEC codec specific parameters, 456 * using a type/length/value approach for maximum flexibility. 458 * 459 * @param enc 460 * @param type (IN) Type of parameter. 461 * @param length (IN) length of the pointed value. 462 * @param value (IN) Pointer to the value. The exact type of 463 * the object pointed is FEC codec specific. 464 * @return 465 */ 466 swif_status_t swif_encoder_set_parameters ( 467 swif_encoder_t* enc, 468 uint32_t type, 469 uint32_t length, 470 void* value); 472 /** 473 * This function gets one or more FEC codec specific parameters, 474 * using a type/length/value approach for maximum flexibility. 475 * 476 * @param enc 477 * @param type (IN) Type of parameter. 478 * @param length (IN) length of the pointed value. 479 * @param value (IN/OUT) Pointer to the value. The exact type of 480 * the object pointed is FEC codec specific. 481 * This function updates the value object 482 * accordingly. The caller, who knows the FEC codec, 483 * is responsible to allocate the appropriate 484 * object buffer. 485 * @return 486 */ 487 swif_status_t swif_encoder_get_parameters ( 488 swif_encoder_t* enc, 489 uint32_t type, 490 uint32_t length, 491 void* value); 493 /** 494 * List here the FEC codec specific control parameters. 495 */ 496 enum { 497 swif_ENCODER_GET_PARAM_ENCODER_STATISTICS = 1, 498 swif_ENCODER_SET_PARAM_RLC_DENSITY_THRESHOLD 499 }; 501 /** 502 * Create a single repair symbol (i.e. perform an encoding). 503 * Upon return of this function, the application has full control of the 504 * buffer and is in charge of freeing it when appropriate. 506 * 507 * @param new_buf (IN) The pointer to the buffer for the repair 508 * symbol to build can either point to a buffer 509 * allocated by the application and initialized to 510 * zero, or let to NULL meaning that this function 511 * will allocate memory. 512 * @return 513 */ 514 swif_status_t swif_build_repair_symbol ( 515 swif_encoder_t* enc, 516 void* new_buf); 517 /* FIX ME: must be void** to enable returning a pointer to buffer! */ 518 520 Encoder API proposal 522 523 /** 524 * Encoder structure that contains whatever is needed for encoding. 525 * The exact content of this structure is FEC code dependent, the 526 * structure below being a non normative example. 527 * However it MUST be aligned with swif_encoder_t (same first items) in 528 * order to be able to cast a pointer to one of the two structures, 529 * depending on the context. 530 */ 531 typedef struct swif_encoder_internal { 532 /* generic part of any control block. MUST be first in structure */ 533 swif_encoder_t gen; 535 /* desired verbosity: 0 is the minimum verbosity, the maximum 536 * level being implementation specific. */ 537 uint32_t verbosity; 539 /* maximum number of source symbols used for any repair symbol */ 540 uint32_t max_coding_window_size; 542 /* exact size (in bytes) of any source or repair symbol */ 543 uint32_t symbol_size; 545 /* add whatever may be needed hereafter... */ 546 } swif_encoder_internal_t; 548 Non normative example of internal structure used by an encoder. 550 4.3. Decoder 552 553 /** 554 * Create and initialize a decoder, providing only key parameters. 555 * 556 * @param codepoint opaque identifier that fully identifies the FEC 557 * code to use. 558 * @param verbosity print information on the codec processing. 559 * 0 is the minimum verbosity, the maximum verbosity 560 * level being implementation specific. 561 * @param symbol_size source and repair symbol size in bytes. Cannot 562 * change during the codec instance lifetime. 563 * @param max_coding_window_size 564 * @param max_linear_system_size 565 * @return pointer to a swif_decoder_t structure if okay, or 566 * NULL in case of error. 567 **/ 568 swif_decoder_t* swif_decoder_create ( 569 swif_codepoint_t codepoint, 570 uint32_t verbosity, 571 uint32_t symbol_size, 572 uint32_t max_coding_window_size, 573 uint32_t max_linear_system_size); 575 /** 576 * Release a decoder and its associated ressources. 577 * 578 * @param dec context (i.e., pointer to decoder structure). 579 **/ 580 swif_status_t swif_decoder_release (swif_decoder_t* dec); 582 /** 583 * Set the various callback functions for this decoder. 584 * All the callback functions require an opaque context parameter, that 585 * must be initialized accordingly by the application, since it is 586 * application specific. 587 * 588 * @param dec context (i.e., pointer to decoder structure). 589 * @param source_symbol_removed_from_linear_system_callback 590 * (IN) Pointer to the function, within the application, that 591 * needs to be called each time a source symbol is removed from 592 * the left side of the linear system. 593 * This callback is called each time the linear system slides 594 * to the right and an old source symbol needs to be removed 595 * on the left. This function does not return anything. 597 * @param decodable_source_symbol_callback 598 * (IN) Pointer to the function, within the application, that 599 * needs to be called each time a source symbol is decodable. 600 * What it does is application-dependent, but it MUST return 601 * either a pointer to a data buffer, left uninitialized, of 602 * the appropriate size, or NULL if the application prefers to 603 * let the codec allocate the buffer. 604 * In any case the codec is responsible for storing the actual 605 * symbol value within the data buffer. Also, no matter 606 * whether the data buffer is allocated by the application or 607 * the codec, it is the responsibility of the application to 608 * free this buffer when needed, once decoding is over (but 609 * not before since the codec does not keep any internal copy). 610 * @param decoded_source_symbol_callback 611 * (IN) Pointer to the function, within the application, that 612 * needs to be called each time a source symbol is decodable and 613 * all computations performed (i.e., the buffer does contain the 614 * symbol value). 615 * This callback is called in a second time, when the newly 616 * decodable source symbol is actually decoded and ready, 617 * i.e., when all the computations (like XOR and GF(2**8) 618 * operations) have been performed. In any case, it is the 619 * responsibility of the application to free this buffer when 620 * needed, once decoding is over (but not before since the 621 * codec does not keep any internal copy). This function does 622 * not return anything. 623 * @param context_4_callback 624 * (IN) Pointer to the application-specific context that will be 625 * passed to the callback function (if any). This context is not 626 * interpreted by this function. 627 * @return 628 */ 629 swif_status_t swif_decoder_set_callback_functions ( 630 swif_decoder_t* dec, 631 void (*source_symbol_removed_from_linear_system_callback) ( 632 void* context, 633 esi_t old_symbol_esi), 634 void* (*decodable_source_symbol_callback) ( 635 void *context, 636 esi_t esi), 637 void (*decoded_source_symbol_callback) ( 638 void *context, 639 void *new_symbol_buf, 640 esi_t esi), 641 void* context_4_callback); 643 /** 644 * This function sets one or more FEC codec specific parameters, 645 * using a type/length/value approach for maximum flexibility. 646 * 647 * @param dec context (i.e., pointer to decoder structure). 648 * @param type (IN) Type of parameter. 649 * @param length (IN) length of the pointed value. 650 * @param value (IN) Pointer to the value. The exact type of 651 * the object pointed is FEC codec specific. 652 * @return 653 */ 654 swif_status_t swif_decoder_set_parameters ( 655 swif_decoder_t* dec, 656 uint32_t type, 657 uint32_t length, 658 void* value); 660 /** 661 * This function gets one or more FEC codec specific parameters, 662 * using a type/length/value approach for maximum flexibility. 663 * 664 * @param dec context (i.e., pointer to decoder structure). 665 * @param type (IN) Type of parameter. 666 * @param length (IN) length of the pointed value. 667 * @param value (IN/OUT) Pointer to the value. The exact type of 668 * the object pointed is FEC codec specific. 669 * This function updates the value object 670 * accordingly. The caller, who knows the FEC codec, 671 * is responsible to allocate the appropriate 672 * object buffer. 673 * @return 674 */ 675 swif_status_t swif_decoder_get_parameters ( 676 swif_decoder_t* dec, 677 uint32_t type, 678 uint32_t length, 679 void* value); 681 /** 682 * List here the FEC codec specific control parameters. 683 */ 684 enum { 685 swif_DECODER_GET_PARAM_DECODER_STATISTICS = 1, 686 swif_DECODER_SET_PARAM_RLC_DENSITY_THRESHOLD 687 }; 689 /** 690 * Submit a received source symbol and try to progress in the decoding. 692 * For each decoded source symbol (if any), the application is informed 693 * through the dedicated callback functions. 694 * 695 * This function usually returns SWIF_STATUS_OK, regardless of whether 696 * this new symbol enabled the decoding of one or several source symbols, 697 * or SWIF_STATUS_ERROR. It cannot return SWIF_STATUS_FAILURE. 698 * 699 * @param dec context (i.e., pointer to decoder structure). 700 * @param new_symbol_buf 701 * (IN) Pointer to the new source symbol now available (i.e. 702 * a new symbol received by the application, or a decoded 703 * symbol in case of a recursive call if it makes sense). 704 * @param new_symbol_esi 705 * (IN) encoding symbol ID of the new source symbol. 706 * @return 707 */ 708 swif_status_t swif_decoder_decode_with_new_source_symbol ( 709 swif_decoder_t* dec, 710 void* const new_symbol_buf, 711 esi_t new_symbol_esi); 713 /** 714 * Submit a received repair symbol and try to progress in the decoding. 715 * For each decoded source symbol (if any), the application is informed 716 * through the dedicated callback functions. 717 * 718 * This function requires that the application has previously initialized 719 * the coding window and coding coefficients appropriately. The application 720 * keeps a full control of the repair symbol buffer, i.e., the application 721 * is in charge of freeing this buffer as soon as it believes appropriate 722 * (a copy is kept by the codec). This is motivated by the fact that a 723 * repair symbol may be part of a larger buffer (e.g., if there are 724 * several repair symbols per packet, or because of a packet header): only 725 * the application knows when the buffer can be safely freed. 726 * 727 * This function usually returns SWIF_STATUS_OK, regardless of whether 728 * this new symbol enabled the decoding of one or several source symbols, 729 * or SWIF_STATUS_ERROR. It cannot return SWIF_STATUS_FAILURE. 730 * 731 * @param dec context (i.e., pointer to decoder structure). 732 * @param new_symbol_buf 733 * (IN) Pointer to the new repair symbol now available (i.e. 734 * a new symbol received by the application or a decoded 735 * symbol in case of a recursive call if it makes sense). 736 * @return 737 */ 738 swif_status_t swif_decoder_decode_with_new_repair_symbol ( 739 swif_decoder_t* dec, 740 void* const new_symbol_buf); 741 743 Decoder API proposal 745 746 /** 747 * Decoder structure that contains whatever is needed for decoding. 748 * The exact content of this structure is FEC code dependent, the 749 * structure below being a non normative example. 750 * However it MUST be aligned with swif_decoder_t (same first items) in 751 * order to be able to cast a pointer to one of the two structures, 752 * depending on the context. 753 */ 754 typedef struct swif_decoder_internal { 755 /* generic part of any control block. MUST be first in structure */ 756 swif_decoder_t gen; 758 /* desired verbosity: 0 is the minimum verbosity, the maximum 759 * level being implementation specific. */ 760 uint32_t verbosity; 762 /* maximum number of source symbols used for any repair symbol */ 763 uint32_t max_coding_window_size; 765 /* max. number of source symbols kepts in current linear system. 766 * If the linear system grows above this limit, old source 767 * symbols in excess are removed and the application callback 768 * called. This value should be larger than the 769 * max_coding_window_size. */ 770 uint32_t max_linear_system_size; 772 /* exact size (in bytes) of any source or repair symbol */ 773 uint32_t symbol_size; 775 /* add whatever may be needed hereafter... */ 776 } swif_decoder_internal_t; 778 Non normative example (RLC) of internal structure used by a decoder. 780 4.4. Coding Window Functions at an Encoder and Decoder 782 This section gathers functions used to manage the coding window, both 783 at an encoder and at a decoder. At an encoder a sliding (of fixed or 784 elastic size) encoding window is managed. Whenever a repair symbol 785 needs to be created, a linear combination (that is code specific) of 786 source symbols currently in the encoding window is performed. This 787 encoding window is managed with the functions below plus, 788 potentially, internal mechanisms that are code specific. 790 At a decoder, before submitting a new repair symbol to the codec, the 791 application must specify the associated encoding window used at the 792 source. This is done by the reset/add a single or set of symbols/ 793 remove a symbol functions. Once this coding window is ready, as well 794 as the coding coefficient list if applicable, the application calls 795 the decode_with_new_repair_symbol() function. A coding window may be 796 reused for several repair symbols as long as they are all built from 797 the same set of source symbols. In that case resetting the coding 798 window and setting it from scratch would be a waste of time. The 799 coding window must be viewed as a temporary list used solely by the 800 decode_with_new_repair_symbol() function and kept independent from 801 the linear system managed by the codec. 803 804 /** 805 * This function resets the current coding window. We assume here that 806 * this window is maintained by the FEC codec instance. 807 * Encoder: reset the encoding window for the encoding of future 808 * repair symbols. 809 * Decoder: reset the coding window under preparation associated to 810 * a repair symbol just received. 811 * 812 * @return 813 */ 814 swif_status_t swif_encoder_reset_coding_window (swif_encoder_t* enc); 816 swif_status_t swif_decoder_reset_coding_window (swif_decoder_t* dec); 818 /** 819 * Add this source symbol to the coding window. 820 * Encoder: add a source symbol to the coding window. 821 * Decoder: add a source symbol to the coding window under preparation. 822 * 823 * @param new_src_symbol_buf (encoder only) pointer to a buffer 824 * containing the source symbol. The application MUST NOT 825 * free nor modify this buffer as long as the source symbol 826 * is in the coding window. 827 * @param new_src_symbol_esi ESI of the source symbol to add. 828 * @return 829 */ 830 swif_status_t swif_encoder_add_source_symbol_to_coding_window ( 831 swif_encoder_t* enc, 832 void* new_src_symbol_buf, 833 esi_t new_src_symbol_esi); 835 swif_status_t swif_decoder_add_source_symbol_to_coding_window ( 836 swif_decoder_t* dec, 837 esi_t new_src_symbol_esi); 839 /** 840 * Remove this source symbol from the coding window. 841 * 842 * Encoder: remove a source symbol from the encoding window, e.g. 843 * because the application knows that a source symbol has 844 * been acknowledged by the peer (if applicable). Note that 845 * the left side of the sliding window is automatically 846 * managed by the codec and no action is needed from the 847 * application. If needed a callback is available to inform 848 * the application that a source symbol has been removed). 849 * Decoder: remove a source symbol from the coding window under 850 * preparation. 851 * 852 * @param old_src_symbol_esi ESI of the source symbol to remove from 853 * the coding window. 854 * @return 855 */ 856 swif_status_t swif_encoder_remove_source_symbol_from_coding_window ( 857 swif_encoder_t* enc, 858 esi_t old_src_symbol_esi); 860 swif_status_t swif_decoder_remove_source_symbol_from_coding_window ( 861 swif_decoder_t* dec, 862 esi_t old_src_symbol_esi); 863 865 Coding Window Functions at an Encoder and Decoder. 867 4.5. Coding Coefficients Functions at an Encoder and Decoder 869 This section gathers functions used to manage the coding 870 coefficients, both at an encoder and at a decoder. Since different 871 FEC codecs will have different requirements, it is important to keep 872 these functions separate from the build_repair_symbol() and 873 decode_with_new_repair_symbol() functions. Several situations exist: 875 o the application provides the list of coding coefficients to use 876 for the next build_repair_symbol(); 877 o the application provides a key (typically a PRNG seed) that the 878 codec uses to produce the coding coefficients to use for the next 879 build_repair_symbol(); 880 o the choice of the coding coefficients is totally performed by the 881 codec, in an autonomous manner (e.g., the codec includes an 882 algorithm that produces an appropriate seed based on various 883 criteria, or the codec selects a set of coding coefficients based 884 on various criteria). In that case the application needs to 885 retrieve the list of coding coefficients or the key selected by 886 the codec; 888 889 /** 890 * The following functions enable an encoder (resp. decoder) to 891 * initialize the set of coefficients to be used for encoding 892 * or associated to a received repair symbol. 893 * 894 * Encoder: calling one of them MUST be done before calling 895 * build_repair_symbol(). 896 * Decoder: calling one of them MUST be done before calling 897 * decode_with_new_repair_symbol(). 898 */ 900 /** 901 * Encoder: this function specifies the coding coefficients chosen by 902 * the application if this is the way the codec works. 903 * Decoder: communicate with this function the coding coefficients 904 * associated to a repair symbol and carried in the packet 905 * header. 906 * 907 * @param coding_coefs_tab 908 * (IN) table of coding coefficients associated to each of 909 * the source symbols currently in the encoding window. 910 * The size (number of bits) of each coefficient depends on 911 * the FEC Scheme. The allocation and release of this table 912 * is under the responsibility of the application. 913 * @param nb_coefs_in_tab 914 * (IN) number of entries (i.e., coefficients) in the table. 915 * @return 916 */ 917 swif_status_t swif_encoder_set_coding_coefs_tab ( 918 swif_encoder_t* enc, 919 void* coding_coefs_tab, 920 uint32_t nb_coefs_in_tab); 922 swif_status_t swif_decoder_set_coding_coefs_tab ( 923 swif_decoder_t* dec, 924 void* coding_coefs_tab, 925 uint32_t nb_coefs_in_tab); 927 /** 928 * The coding coefficients may be generated in a deterministic manner, 929 * for instance by a PRNG known by the codec and a seed (perhaps with 930 * other parameters) provided by the application. 931 * The codec may also choose in an autonomous manner these coefficients. 932 * This function is used to trigger this process. 933 * When the choice is made in an autonomous manner, the actual coding 934 * coefficient or key used by the codec can be retrieved with 935 * swif_encoder_get_coding_coefs_tab(). 936 * 937 * @param key (IN) Value that can be used as a seed in case of a PRNG 938 * for instance, or by a specific coding coefficients 939 * function. Set to 0 if not required by a codec. 940 * @param add_param 941 * (IN) an opaque 32-bit integer that contains a codec 942 * specific parameter if needed. Set to 0 if not used. 943 * @return 944 */ 945 swif_status_t swif_encoder_generate_coding_coefs ( 946 swif_encoder_t* enc, 947 uint32_t key, 948 uint32_t add_param); 950 swif_status_t swif_decoder_generate_coding_coefs ( 951 swif_decoder_t* dec, 952 uint32_t key, 953 uint32_t add_param); 955 /** 956 * This function enables the application to retrieve the set of coding 957 * coefficients generated and used by build_repair_symbol(). This is 958 * useful when the choice of coefficients is performed by the codec in 959 * an autonomous manner but needs to be sent in the repair packet header. 960 * This function is only used by an encoder. 961 * 962 * @param coding_coefs_tab 963 * (OUT) pointer to a table of coding coefficients. 964 * The size (number of bits) of each coefficient depends on 965 * the FEC scheme. Upon return of this function, this table 966 * is allocated and filled with coefficient values. The 967 * release of this table is under the responsibility of the 968 * application. 969 * @param nb_coefs_in_tab 970 * (IN/OUT) pointer to the number of entries (i.e., 971 * coefficients) in the table. 972 * Upon calling this function, this number must be zero. 973 * Upon return of this function this variable is initialized 974 * with the actual number of entries in the coeffs_tab[]. 975 * @return 976 */ 977 swif_status_t swif_encoder_get_coding_coefs_tab ( 978 swif_encoder_t* enc, 979 void** coding_coefs_tab, 980 uint32_t* nb_coefs_in_tab); 982 /** 983 * Get information on the current coding window at the encoder. 984 * This function stores the ESI of the first source symbol and 985 * last source symbol in the coding window, as well as the number 986 * of symbols. In theory the application should be able to recover 987 * the information (it knows when new symbols are added and old 988 * symbols removed), but it's easier to let the SWiF Codec care 989 * about it. The number of source symbols is also returned. 990 * In situations where there's no gap (i.e., when 991 * swif_encoder_remove_source_symbol_from_coding_window() has not 992 * been used), nss can also be calculated with first/last. However 993 * it is more convenient to use nss directly (in particular in case 994 * of wraping to zero of either first or last). 995 * 996 * @param enc 997 * @param first (in/out) pointer to ESI of the first source 998 * symbol in the coding window (inclusive) 999 * @param last (in/out) pointer to ESI of the last source 1000 * symbol in the coding window (inclusive) 1001 * @param nss (in/out) pointer to number of source symbols 1002 * in the coding window 1003 * @return 1004 */ 1005 swif_status_t swif_encoder_get_coding_window_information ( 1006 swif_encoder_t* enc, 1007 esi_t* first, 1008 esi_t* last, 1009 uint32_t* nss); 1010 1012 Coding Coefficients Functions at an Encoder and Decoder. 1014 5. Security Considerations 1016 TBD 1018 6. IANA Considerations 1020 This document has no IANA requirement. 1022 7. Acknowledgments 1024 The authors would like to thank Marie-Jose Montpetit, Francois 1025 Michel, and Oumaima Attia for their valuable contributions to the 1026 IETF Hackathon SWiF-Codec project and their inputs to this document. 1028 8. References 1030 8.1. Normative References 1032 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1033 Requirement Levels", BCP 14, RFC 2119, 1034 DOI 10.17487/RFC2119, March 1997, 1035 . 1037 8.2. Informative References 1039 [fecframe-ext] 1040 Roca, V. and A. Begen, "Forward Error Correction (FEC) 1041 Framework Extension to Sliding Window Codes", Transport 1042 Area Working Group (TSVWG) draft-ietf-tsvwg-fecframe-ext 1043 (Work in Progress), June 2018, 1044 . 1047 [RLC] Roca, V. and B. Teibi, "Sliding Window Random Linear Code 1048 (RLC) Forward Erasure Correction (FEC) Scheme for 1049 FECFRAME", Transport Area Working Group (TSVWG) draft- 1050 ietf-tsvwg-rlc-fec-scheme (Work in Progress), June 2018, 1051 . 1054 Authors' Addresses 1056 Vincent Roca 1057 INRIA 1058 Univ. Grenoble Alpes 1059 France 1061 EMail: vincent.roca@inria.fr 1063 Jonathan Detchart 1064 ISAE - Supaero 1065 France 1067 EMail: jonathan.detchart@isae-supaero.fr 1068 Cedric Adjih 1069 INRIA 1070 France 1072 EMail: cedric.adjih@inria.fr 1074 Morten V. Pedersen 1075 Steinwurf ApS 1076 Denmark 1078 EMail: morten@steinwurf.com