<?xml version="1.0" encoding="us-ascii"?>
  <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>

<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!ENTITY RFC2119 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml">
<!ENTITY RFC6184 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6184.xml">
<!ENTITY RFC6236 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6236.xml">
<!ENTITY RFC6386 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6386.xml">
<!ENTITY I-D.ietf-payload-vp8 SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-payload-vp8.xml">
<!ENTITY I-D.ietf-rtcweb-overview SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-rtcweb-overview.xml">
<!ENTITY I-D.ietf-rtcweb-rtp-usage SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-rtcweb-rtp-usage.xml">
<!ENTITY I-D.ietf-rtcweb-security SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-rtcweb-security.xml">
<!ENTITY I-D.ietf-rtcweb-security-arch SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-rtcweb-security-arch.xml">
]>

<rfc ipr="trust200902" docName="draft-ietf-rtcweb-video-06" category="std">

<?rfc toc="yes"?>
<?rfc sortrefs="yes"?>
<?rfc symrefs="yes"?>

  <front>
    <title abbrev="WebRTC Video">WebRTC Video Processing and Codec Requirements</title>

    <author initials="A.B." surname="Roach" fullname="Adam Roach">
      <organization>Mozilla</organization>
      <address>
        <postal>
          <street>\</street>
          <city>Dallas</city>
          <country>US</country>
        </postal>
        <phone>+1 650 903 0800 x863</phone>
        <email>adam@nostrum.com</email>
      </address>
    </author>

    <date year="2015" month="June" day="12"/>

    
    
    

    <abstract>


<t>This specification provides the requirements and considerations for WebRTC
applications to send and receive video across a network. It specifies the
video processing that is required, as well as video codecs and their parameters.</t>



    </abstract>


  </front>

  <middle>


<section anchor="introduction" title="Introduction">

<t>One of the major functions of WebRTC endpoints is the ability to send and
receive interactive video. The video might come from a camera, a screen
recording, a stored file, or some other source.  This specification provides
the requirements and considerations for WebRTC applications to send and
receive video across a network. It specifies the video processing that is
required, as well as video codecs and their parameters.</t>

<t>Note that this document only discusses those issues dealing with video
codec handling. Issues that are related to transport of media streams
across the network are specified in <xref target="I-D.ietf-rtcweb-rtp-usage"/>.</t>

</section>
<section anchor="terminology" title="Terminology">

<t>The key words &ldquo;MUST&rdquo;, &ldquo;MUST NOT&rdquo;, &ldquo;REQUIRED&rdquo;, &ldquo;SHALL&rdquo;, &ldquo;SHALL NOT&rdquo;, &ldquo;SHOULD&rdquo;,
&ldquo;SHOULD NOT&rdquo;, &ldquo;RECOMMENDED&rdquo;, &ldquo;MAY&rdquo;, and &ldquo;OPTIONAL&rdquo; in this document are to be
interpreted as described in <xref target="RFC2119"/>.</t>

</section>
<section anchor="pre-and-post-processing" title="Pre and Post Processing">

<t>This section provides guidance on pre- and post-processing of video streams.</t>

<t>Unless specified otherwise by the SDP or codec, the color space SHOULD be sRGB
<xref target="SRGB"/>.  For clarity, this is the color space indicated by codepoint 1 from
&ldquo;ColourPrimaries&rdquo; as defined in <xref target="IEC23001-8"/>.</t>

<t>Unless specified otherwise by the SDP or codec, the video scan pattern for
video codecs is Y&rsquo;CbCr 4:2:0.</t>

<section anchor="camera-source-video" title="Camera Source Video">

<t>This document imposes no normative requirements on camera capture; however,
implementors are encouraged to take advantage of the following features,
if feasible for their platform:</t>

<t><list style='symbols'>
  <t>Automatic focus, if applicable for the camera in use</t>
  <t>Automatic white balance</t>
  <t>Automatic light level control</t>
  <t>Dynamic frame rate for video capture based on actual encoding in use
(e.g., if encoding at 15 fps due to bandwidth constraints, low light
conditions, or application settings, the camera will ideally capture at 15
fps rather than a higher rate).</t>
</list></t>

</section>
<section anchor="screen-source-video" title="Screen Source Video">

<t>If the video source is some portion of a computer screen (e.g., desktop or
application sharing), then the considerations in this section also apply.</t>

<t>Because screen-sourced video can change resolution (due to, e.g., window
resizing and similar operations), WebRTC video recipients MUST be prepared
to handle mid-stream resolution changes in a way that preserves their utility.
Precise handling (e.g., resizing the element a video is rendered in versus
scaling down the received stream; decisions around letter/pillarboxing) is
left to the discretion of the application.</t>

<t>Note that the default video scan format (Y&rsquo;CbCr 4:2:0) is known to be
less than optimal for the representation of screen content produced by
most systems in use at the time of this document&rsquo;s publication, which
generally use RGB with at least 24 bits per sample. In the future, it
may be advisable to use video codecs optimized for screen content for the
representation of this type of content.</t>

<t>Additionally, attention is drawn to the requirements in
<xref target="I-D.ietf-rtcweb-security-arch"/> section 5.2 and the
considerations in <xref target="I-D.ietf-rtcweb-security"/> section 4.1.1.</t>

</section>
</section>
<section anchor="stream-orientation" title="Stream Orientation">

<t>In some circumstances &ndash; and notably those involving mobile devices &ndash; the
orientation of the camera may not match the orientation used by the encoder.
Of more importance, the orientation may change over the course of a call,
requiring the receiver to change the orientation in which it renders the
stream.</t>

<t>While the sender may elect to simply change the pre-encoding orientation of
frames, this may not be practical or efficient (in particular, in cases where
the interface to the camera returns pre-compressed video frames). Note that
the potential for this behavior adds another set of circumstances under which
the resolution of a screen might change in the middle of a video stream, in
addition to those mentioned under &ldquo;Screen Sourced Video,&rdquo; above.</t>

<t>To accommodate these circumstances, RTCWEB implementations that can generate
media in orientations other than the default MUST support generating the R0
and R1 bits of the Coordination of Video Orientation (CVO) mechanism described
in section 7.4.5 of <xref target="TS26.114"/>, and MUST send them for all orientations when
the peer indicates support for the mechanism.  They MAY support sending the
other bits in the CVO extension, including the higher-resolution rotation
bits.  All implementations SHOULD support interpretation of the R0 and R1
bits, and MAY support the other CVO bits.</t>

<t>Further, some codecs support in-band signaling of orientation (for example,
the SEI &ldquo;Display Orientation&rdquo; messages in H.264 and H.265). If CVO has been
negotiated, then the sender MUST NOT make use of such codec-specific mechanisms.
However, when support for CVO is not signaled in the SDP, then such
implementations MAY make use of the codec-specific mechanisms instead.</t>

</section>
<section anchor="mandatory-to-implement-video-codec" title="Mandatory to Implement Video Codec">

<t>For the definitions of &ldquo;WebRTC Browser,&rdquo; &ldquo;WebRTC Non-Browser&rdquo;, and
&ldquo;WebRTC-Compatible Endpoint&rdquo; as they are used in this section, please
refer to <xref target="I-D.ietf-rtcweb-overview"/>.</t>

<t>WebRTC Browsers MUST implement the VP8 video codec as described in <xref target="RFC6386"/>
and H.264 Constrained Baseline as described in <xref target="H264"/>.</t>

<t>WebRTC Non-Browsers that support transmitting and/or receiving video MUST
implement the VP8 video codec as described in <xref target="RFC6386"/> and H.264
Constrained Baseline as described in <xref target="H264"/>.</t>

<t><list style='empty'>
  <t>NOTE: To promote the use of non-royalty bearing video codecs, participants in
the RTCWEB working group, and any successor working groups in the IETF, intend
to monitor the evolving licensing landscape as it pertains to the two
mandatory-to-implement codecs. If compelling evidence arises that one of the
codecs is available for use on a royalty-free basis, the working group plans
to revisit the question of which codecs are required for Non-Browsers, with
the intention being that the royalty-free codec will remain mandatory to
implement, and the other will become optional.</t>
</list></t>

<t><list style='empty'>
  <t>These provisions apply to WebRTC Non-Browsers only. There is no plan to
revisit the codecs required for WebRTC Browsers.</t>
</list></t>

<t>&ldquo;WebRTC-compatible endpoints&rdquo; are free to implement any video codecs they see
fit. This follows logically from the definition of &ldquo;WebRTC-compatible
endpoint.&rdquo; It is, of course, advisable to implement at least one of the video
codecs that is mandated for WebRTC Browsers, and implementors are encouraged
to do so.</t>

</section>
<section anchor="codec-specific-considerations" title="Codec-Specific Considerations">

<t>SDP allows for codec-independent indication of preferred video resolutions
using the mechanism described in <xref target="RFC6236"/>. WebRTC endpoints MAY send an
&ldquo;a=imageattr&rdquo; attribute to indicate the maximum resolution they wish to
receive. Senders SHOULD interpret and honor this attribute by limiting the
encoded resolution to the indicated maximum size, as the receiver may not be
capable of handling higher resolutions.</t>

<t>Additionally, codecs may include codec-specific means of signaling maximum
receiver abilities with regards to resolution, frame rate, and bitrate.</t>

<t>Unless otherwise signaled in SDP, recipients of video streams MUST be able to
decode video at a rate of at least 20 fps at a resolution of at least 320 pixels
by 240 pixels.
These values are selected based on the recommendations in <xref target="HSUP1"/>.</t>

<t>Encoders are encouraged to support encoding media with at least the same
resolution and frame rates cited above.</t>

<section anchor="vp8" title="VP8">

<t>For the VP8 codec, defined in <xref target="RFC6386"/>, endpoints MUST support
the payload formats defined in <xref target="I-D.ietf-payload-vp8"/>.</t>

<t>In addition to the <xref target="RFC6236"/> mechanism, VP8 encoders MUST limit the
streams they send to conform to the values indicated by receivers in the
corresponding max-fr and max-fs SDP attributes.</t>

<t>Unless otherwise signaled, implementations that use VP8 MUST encode and
decode pixels with a implied 1:1 (square) aspect ratio.</t>

</section>
<section anchor="h264" title="H.264">

<t>For the <xref target="H264"/> codec, endpoints MUST support the payload formats
defined in <xref target="RFC6184"/>. In addition, they MUST support Constrained Baseline
Profile Level 1.2, and they SHOULD support H.264 Constrained High Profile
Level 1.3.</t>

<t>Implementations of the H.264 codec have utilized a wide variety of optional
parameters.  To improve interoperability the following parameter settings are
specified:</t>

<t><list style='hanging'>
  <t hangText='packetization-mode:'>
  Packetization-mode 1 MUST be supported. Other modes MAY be negotiated and
used.</t>
  <t hangText='profile-level-id:'>
  Implementations MUST include this parameter within SDP and MUST interpret
it when receiving it.</t>
  <t hangText='max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br:'>
  These</t>
  <t>parameters allow the implementation to specify that they can support
certain features of H.264 at higher rates and values than those signalled by
their level (set with profile-level-id).  Implementations MAY include these
parameters in their SDP, but SHOULD interpret them when receiving them,
allowing them to send the highest quality of video possible.</t>
  <t hangText='sprop-parameter-sets:'>
  H.264 allows sequence and picture information to be sent both in-band, and
out-of-band.  WebRTC implementations MUST signal this information in-band.
This means that WebRTC implementations MUST NOT include this parameter in
the SDP they generate.</t>
</list></t>

<t>H.264 codecs MAY send and MUST support proper interpretation of SEI &ldquo;filler
payload&rdquo; and &ldquo;full frame freeze&rdquo; messages. &ldquo;Full frame freeze&rdquo; messages are
used in video switching MCUs, to ensure a stable decoded displayed picture
while switching among various input streams.</t>

<t>When the use of the video orientation (CVO) RTP header extension is not
signaled as part of the SDP, H.264 implementations MAY send and SHOULD support
proper interpretation of Display Orientation SEI messages.</t>

<t>Implementations MAY send and act upon &ldquo;User data registered by Rec. ITU-T
T.35&rdquo; and &ldquo;User data unregistered&rdquo; messages. Even if they do not act on them,
implementations MUST be prepared to receive such messages without any ill
effects.</t>

<t>Unless otherwise signaled, implementations that use H.264 MUST encode and
decode pixels with a implied 1:1 (square) aspect ratio.</t>

</section>
</section>
<section anchor="security-considerations" title="Security Considerations">

<t>This specification does not introduce any new mechanisms or security concerns
beyond what is in the other documents it references. In WebRTC, video is protected
using DTLS/SRTP. A complete discussion of the security considerations can be found in
<xref target="I-D.ietf-rtcweb-security"/> and <xref target="I-D.ietf-rtcweb-security-arch"/>.
Implementors should consider whether the use of variable bit rate video codecs
are appropriate for their application, keeping in mind that the degree of
inter-frame change (and, by inference, the amount of motion in the frame) may
be deduced by an eavesdropper based on the video stream&rsquo;s bit rate.</t>

<t>Implementors making use of H.264 are also advised to take careful note of the
&ldquo;Security Considerations&rdquo; section of <xref target="RFC6184"/>, paying special regard to the
normative requirement pertaining to SEI messages.</t>

</section>
<section anchor="iana-considerations" title="IANA Considerations">

<t>This document requires no actions from IANA.</t>

</section>
<section anchor="acknowledgements" title="Acknowledgements">

<t>The author would like to thank Gaelle Martin-Cocher, Stephan Wenger, and
Bernard Aboba for their detailed feedback and assistance with this document.
Thanks to Cullen Jennings for providing text and review, and to Russ Housley
for a careful final review. This draft includes text from draft-cbran-rtcweb-codec.</t>

</section>


  </middle>

  <back>

    <references title='Normative References'>

&RFC2119;
<reference anchor="H264" target="http://www.itu.int/rec/T-REC-H.264">
  <front>
    <title>Advanced video coding for generic audiovisual services (V9)</title>
    <author >
      <organization>ITU-T Recommendation H.264</organization>
    </author>
    <date year="2014" month="February"/>
  </front>
</reference>
<reference anchor="HSUP1" target="http://www.itu.int/rec/T-REC-H.Sup1">
  <front>
    <title>Application profile - Sign language and lip-reading real-time conversation using low bit rate video communication</title>
    <author >
      <organization>ITU-T Recommendation H.Sup1</organization>
    </author>
    <date year="1999" month="May"/>
  </front>
</reference>
&RFC6184;
&RFC6236;
&RFC6386;
&I-D.ietf-payload-vp8;
&I-D.ietf-rtcweb-overview;
<reference anchor="SRGB" target="http://www.colour.org/tc8-05/Docs/colorspace/61966-2-1.pdf">
  <front>
    <title>Multimedia systems and equipment - Colour measurement and management - Part 2-1: Colour management - Default RGB colour space - sRGB.</title>
    <author >
      <organization>IEC 61966-2-1</organization>
    </author>
    <date year="1999" month="October"/>
  </front>
</reference>
<reference anchor="IEC23001-8" target="http://standards.iso.org/ittf/PubliclyAvailableStandards/c062088_ISO_IEC_23001-8_2013.zip">
  <front>
    <title>Coding independent media description code points</title>
    <author >
      <organization>ISO/IEC 23001-8:2013/DCOR1</organization>
    </author>
    <date year="2013"/>
  </front>
</reference>
<reference anchor="TS26.114" target="http://www.3gpp.org/DynaReport/26114.htm">
  <front>
    <title>3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; IP Multimedia Subsystem (IMS); Multimedia Telephony; Media handling and interaction (Release 12)</title>
    <author >
      <organization>3GPP TS 26.114 V12.8.0</organization>
    </author>
    <date year="2014" month="December"/>
  </front>
</reference>


    </references>

    <references title='Informative References'>

&I-D.ietf-rtcweb-rtp-usage;
&I-D.ietf-rtcweb-security;
&I-D.ietf-rtcweb-security-arch;


    </references>



  </back>
</rfc>

