WebRTC Forward Error Correction
Requirements
Google
747 6th St S
Kirkland
WA
98033
USA
justin@uberti.name
RAI
This document provides information and requirements for how Forward
Error Correction (FEC) should be used by WebRTC implementations.
In situations where packet loss is high, or perfect media quality is
essential, Forward Error Correction (FEC) can be used to proactively
recover from packet losses. This specification provides guidance on which
FEC mechanisms to use, and how to use them, for WebRTC
implementations.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in
.
FEC describes the sending of redundant information in an outgoing
packet stream so that information can still be recovered even in the face
of packet loss. There are multiple ways in which this can be
accomplished; this section enumerates the various mechanisms and
describes their tradeoffs.
This approach, as described in
, Section 4.3, sends FEC packets as an
independent SSRC-multiplexed stream, with its own SSRC and payload
type. While this approach can protect multiple packets of the primary
encoding with a single FEC packet, each FEC packet will have its own
IP+UDP+RTP+FEC header, and this overhead can be excessive in some
cases, e.g., when protecting each primary packet with a FEC packet.
This approach allows for recovery of entire RTP packets, including
the full RTP header.
This approach, as descibed in
, allows for redundant data to be piggybacked
on an existing primary encoding, all in a single packet. This redundant
data may be an exact copy of a previous packet, or for codecs that
support variable-bitrate encodings, possibly a smaller, lower-quality
representation. In certain cases, the redundant data could include
multiple prior packets.
Since there is only a single set of packet headers, this approach
allows for a very efficient representation of primary + redundant data.
However, this savings is only realized when the data all fits into a
single packet (i.e. the size is less than a MTU). As a result, this
approach is generally not useful for video content.
As described in
, Section 4, this approach cannot recover
certain parts of the RTP header, including the marker bit, CSRC
information, and header extensions.
Some audio codecs, notably Opus
and AMR
support their own in-band FEC mechanism, where
redundant data is included in the codec payload.
For Opus, packets deemed as important are re-encoded at a lower
bitrate and added to the subsequent packet, allowing partial recovery
of a lost packet. This scheme is fairly efficient; experiments
performed indicate that when Opus FEC is used, the overhead imposed is
about 20-30%, depending on the amount of protection needed. Note that
this mechanism can only carry redundancy information for the
immediately preceding packet; as such the decoder cannot fully recover
multiple consecutive lost packets, which can be a problem on wireless
networks. See
, Section 2.1.7 for complete details.
For AMR/AMR-WB, packets can contain copies or lower-quality
encodings of multiple prior audio frames. This mechanism is similar to
the
mechanism described above, but as it adds no
additional framing, it can be slightly more efficient. See
, Section 3.7.1 for details on this
mechanism.
In-band FEC mechanisms cannot recover any of the RTP header.
The following section provides guidance on how to best use FEC for
transmitting audio data. As indicated in
below, FEC should only be activated if
network conditions warrant it, or upon explicit application request.
When using variable-bitrate codecs without an internal FEC,
redundant encoding with lower-fidelity
version(s) of the previous packet(s) is RECOMMENDED. This provides
reasonable protection of the payload with only moderate bitrate
increase, as the redundant encodings can be significantly smaller than
the primary encoding.
When using the Opus codec, use of the built-in Opus FEC mechanism is
RECOMMENDED. This provides reasonable protection of the audio stream
against individual losses, with minimal overhead. Note that as
indicated above the built-in Opus FEC only provides single-frame
redundancy; if multi-packet protection is needed, the aforementioned
redundancy with reduced-bitrate Opus encodings
SHOULD be used instead.
When using the AMR/AMR-WB codecs, use of their built-in FEC
mechanism is RECOMMENDED. This provides slightly more efficient
protection of the audio stream than
.
When using constant-bitrate codecs, e.g. PCMU, use of
redundant encoding MAY be used, but note that
this will result in a potentially significant bitrate increase, and
that suddenly increasing bitrate to deal with losses from congestion
may actually make things worse.
Because of the lower packet rate of audio encodings, usually a
single packet per frame, use of a separate FEC stream comes with a
higher overhead than other mechanisms, and therefore is NOT
RECOMMENDED.
As mentioned above, the recommended mechanisms do not allow recovery
of parts of the RTP header that may be important in certain audio
applications, e.g., CSRCs and RTP header extensions like those
specified in
and
. Implementations SHOULD account for this and
attempt to approximate this information, using an approach similar to
those described in
, Section 4, and
, Section 5.
Support for redundant encoding of a given RTP stream SHOULD be
indicated by including audio/red
as an additional supported media type for the
associated m= section in the SDP offer
. Answerers can reject the use of redundant
encoding by not including the audio/red media type in the corresponding
m= section in the SDP answer.
Support for codec-specific FEC mechanisms are typically indicated
via "a=fmtp" parameters.
For Opus, a receiver MUST indicate that it is prepared to use
incoming FEC data with the "useinbandfec=1" parameter, as specified in
. This parameter is declarative and can be
negotiated separately for either media direction.
For AMR/AMR-WB, support for redundant encoding, and the maximum
supported depth, are controlled by the 'max-red' parameter, as
specified in
, Section 8.1. Receivers MUST include this
parameter, and set it to an appropriate value, as specified in
, Table 6.3.
The following section provides guidance on how to best use FEC for
transmitting video data. As indicated in
below, FEC should only be activated if
network conditions warrant it, or upon explicit application request.
Video frames, due to their size, often require multiple RTP packets.
As discussed above, a separate FEC stream can protect multiple packets
with a single FEC packet. In addition, the "flexfec" FEC mechanism
described in
is also capable
of protecting multiple RTP streams via a single FEC stream, including
all the streams that are part of a BUNDLE
group. As a
result, for video content, use of a separate FEC stream with the
flexfec RTP payload format is RECOMMENDED.
To process the incoming FEC stream, the receiver can demultiplex it
by SSRC, and then correlate it with the appropriate primary stream(s)
via the CSRC(s) present in the RTP header of flexfec repair packets, or
the SSRC field present in the FEC header of flexfec retransmission
packets.
Support for a SSRC-multiplexed flexfec stream to protect a given RTP
stream SHOULD be indicated by including one of the formats described in
, Section 5.1, as
an additional supported media type for the associated m= section in the
SDP offer
. As mentioned above, when BUNDLE is used,
only a single flexfec repair stream will be created for each BUNDLE
group, even if flexfec is negotiated for each primary stream.
Answerers can reject the use of SSRC-multiplexed FEC, by not
including the offered FEC formats in the corresponding m= section in
the SDP answer.
Use of FEC-only m-lines, and grouping using the SDP group mechanism
as described in
, Section 4.1 is not currently defined for
WebRTC, and SHOULD NOT be offered.
Answerers SHOULD reject any FEC-only m-lines, unless they
specifically know how to handle such a thing in a WebRTC context
(perhaps defined by a future version of the WebRTC specifications).
WebRTC also supports the ability to send generic application data, and
provides transport-level retransmission mechanisms to support full and
partial (e.g. timed) reliability. See
for details.
Because the application can control exactly what data to send, it has
the ability to monitor packet statistics and perform its own
application-level FEC, if necessary.
As a result, this document makes no recommendations regarding FEC for
the underlying data transport.
To support the functionality recommended above, implementations MUST
be able to receive and make use of the relevant FEC formats for their
supported audio codecs, and MUST indicate this support, as described in
. Use of these formats when sending, as
mentioned above, is RECOMMENDED.
The general FEC mechanism described in
SHOULD also be
supported, as mentioned in
.
Implementations MAY support additional FEC mechanisms if desired, e.g.
.
Because use of FEC always causes redundant data to be transmitted, and
the total amount of data must remain within any bandwidth limits indicated
by congestion control and the receiver, this will lead to less bandwidth
available for the primary encoding, even when the redundant data is not
being used. This is in contrast to methods like RTX
or flexfec
retransmissions,
which only transmit redundant data when necessary, at the cost of an
extra roundtrip.
Given this, WebRTC implementations SHOULD consider using RTX or
flexfec retransmissions instead of FEC when RTT is low, and SHOULD only
transmit the amount of FEC needed to protect against the observed packet
loss (which can be determined, e.g., by monitoring transmit packet loss
data from RTCP Receiver Reports
), unless the application indicates it is
willing to pay a quality penalty to proactively avoid losses.
Note that when probing bandwidth, i.e., speculatively sending extra
data to determine if additional link capacity exists, FEC SHOULD be used
in all cases. Given that extra data is going to be sent regardless, it
makes sense to have that data protect the primary payload; in addition,
FEC can be applied in a way that increases bandwidth only modestly, which
is necessary when probing.
When using FEC with layered codecs, e.g.,
, where only base layer frames are critical to
the decoding of future frames, implementations SHOULD only apply FEC to
these base layer frames.
This document makes recommendations regarding the use of FEC.
Generally, it should be noted that although applying redundancy is often
useful in protecting a stream against packet loss, if the loss is caused
by network congestion, the additional bandwidth used by the redundant
data may actually make the situation worse, and can lead to significant
degradation of the network.
As described in
, Section 10, the default processing when using
FEC with SRTP is to perform FEC followed by SRTP at the sender, and SRTP
followed by FEC at the receiver. This ordering is used for all the SRTP
Protection Profiles used in DTLS-SRTP
, as described in
, Section 4.1.2.
Additional security considerations for each individual FEC mechanism
are enumerated in their respective documents.
This document requires no actions from IANA.
Several people provided significant input into this document,
including Bernard Aboba, Jonathan Lennox, Giri Mandyam, Varun Singh, Tim
Terriberry, Magnus Westerlund, and Mo Zanaty.
IP Multimedia Subsystem (IMS); Multimedia telephony; Media
handling and interaction
3GPP
Changes in draft -07:
Clarify how bandwidth management interacts with FEC.
Make 3GPP reference normative.
Changes in draft -06:
Discuss how multiple streams can be protected by a single FlexFEC
stream.
Discuss FEC for bandwidth probing.
Add note about recovery of RTP headers and header extensions.
Add note about FEC/SRTP ordering.
Clarify flexfec demux text, and mention retransmits.
Clarify text regarding offers/answers.
Make RFC2198 support SHOULD strength.
Clean up references.
Changes in draft -05:
No changes.
Changes in draft -04:
Discussion of layered codecs.
Discussion of RTX.
Clarified implementation requirements.
FlexFEC MUST -> SHOULD.
Clarified AMR max-red handling.
Updated references.
Changes in draft -03:
Added overhead stats for Opus.
Expanded discussion of multi-packet FEC for Opus.
Added discussion of AMR/AMR-WB.
Removed discussion of ssrc-group.
Referenced the data channel doc.
Referenced the RTP/RTCP RFC.
Several small edits based on feedback from Magnus.
Changes in draft -02:
Expanded discussion of FEC-only m-lines, and how they should be
handled in offers and answers.
Changes in draft -01:
Tweaked abstract/intro text that was ambiguously normative.
Removed text on FEC for Opus in CELT mode.
Changed RFC 2198 recommendation for PCMU to be MAY instead of NOT
RECOMMENDED, based on list feedback.
Explicitly called out application data as something not addressed in
this document.
Updated flexible-fec reference.
Changes in draft -00:
Initial version, from sidebar conversation at IETF 90.