Minutes: NETVC BOF; March 24, 2015; Dallas, TX Note: This file contains official meeting minutes at the top, which capture those points that had non-trivial bearing on the proposed charter and questions around whether to form a working group. These official minutes are followed by the raw notes provided by the two scribes who took notes during the meeting. Richard Barnes (AD): Introduction and Scoping --------------------------------------------- The area director set context by drawing parallels between this work and the CODEC working group that resulted in the successful development of the RF OPUS audio codec. No substantive discussion ensued. Chairs: Goals and Progress -------------------------- See meeting slides for description of goals and progress. No substantive discussion ensued. Mo Zanaty: Codec Considerations ------------------------------- See meeting slides for key considerations about developing an internet video codec. Ensuing conversation resulted in a request to include a statement regarding transcoding in the proposed charter text. Timothy Terriberry: Daala Coding Tools and Progress --------------------------------------------------- See meeting slides for Daala Coding Tools and Progress. Ensuing discussion focused predominantly on questions around measurement methodologies. Chairs: Charter Discussion -------------------------- See meeting slides for charter text presented in BOF. Some discussion ensued regarding whether the WG charter should aim to be "competitive with" or "better than" current video codecs. On the balance, there appeared to be weak support for keeping the text as proposed ("competitive with"). There were some points raised about codec work taking place in other SDOs, including the RF video work that has been attempted in MPEG. A request for an expanded liaison list was made. Refinements to the charter were proposed to indicate a focus on real-time encoding and to indicate that mobile platforms were explicitly a target for the codec. It was proposed, and generally supported, that the statement regarding not wishing to work on very low bandwidth behavior should be removed. Those present engaged in a relatively broad conversation regarding the wisdom of delivering proposed IPR terms as a WG item, with the general sense of the room being that this is not something the proposed WG should work on. A change to the codec deliverable was requested, so that the WG would normatively define the bitstream itself, rather than normatively defining the encoder operation. Support for this proposal was unanimous. An uncontroversial request was made to define "evaluation "criteria" rather than "evaluation metrics." Chairs: Questions for Interested Parties ---------------------------------------- Q1: Is there a problem that needs solving? Consensus: Yes (unanimous and strong) Q2: Is the scope of the problem well defined and understood? Is there agreement on what a WG would need to deliver? Consensus: Yes, strong, with some dissent. Q3: Are there people willing to do the work? Q3.1: Who will write the drafts? - Seven people indicated they would. Q3.2: Who will review the drafts? - Approximately a dozen people indicated they would. Q3.3: Who will implement, test, and characterize a reference implementation? - Approximately six people indicated they would. Q4: How many people feel that a WG should not be formed at the IETF? Consensus: Very little opposition to forming a working group. =========================================================================== Raw notes follow =========================================================================== [Notes from Varun Singh] netvc: Internet Video Codec slideset 1: Considerations for Internet Video Codec Presenter: Mo Zanaty Frank/Tim: bit patterns ...? Bernard Aboba: not just describe the bitstream and the coding tools, but look at the overall architecture. for example send configuration parameters in-band, also things like in-band FEC. Also to think about I-frames and avoid sending them in some simple cases, and thus avoiding FIRs. Jean-Marc Valin: pursue RTP in parallel with the bitstream evolution, we did this for Opus and should do it again Slideset 2: Daala coding tools Presenter: Tim Terribery Frank: What sequences did you use? Tim: uses x265 and all sequence are 1s long, and 6 sequences? it uses Multiscale FastSSIM. Frank: we know this metric is broken, as it downscales, etc. Frank: how often do you look at the video? Tim: we play video side-by-side. Mo: Evaluation results and methodology, will need a robust framework. Because the external people will look at this and gauge performance based on that. We should focus on real deployments, especially different modes for example: real-time, video on demand (DASH-like) would need to be done so that it has universal applicability. Frank: arewecompresedyet.com? give us git repository and it will take the code and run it in comparison to other works. We use the same platform to run H.264, H.265, VP8, and VP9. so it can be used for other codec work as well. Frank: Tim, you said the HQ video is 6Mbps, this looks really high bitrate and compares poorly to the current generation which does the same at 2-3Mbps Tim: .. Slideset 3: Proposed Charter Text Charter: Process (1/2) - Bernard: Add some text in the process part which claims the WG will do something better in some area (the WG is free to pick what that is) Harald: Likes the current 5 bullets and no disagrees with the views to claim additional - ?: Should say something along the lines of low-latency, may be different from real-time - ekr: clarify if mobile hardware is in scope or not Charter: Non-Goals - Mo: remove < 256kbps (Tim: agrees to remove this) Charter: Collaboration - Ted/Joe: We have a general liason with ITU-T, but would need a specific liason to Study Group 16. (Stephan Wenger volunteer to do the needful.) - It would be good to add liason to 3GPP. Charter: Deliverables - Bernard: Why not add RTP? Chairs: Will pursue it in Payload. Deliverable 1 - Ted Hardie: Do not do Item 1. Cullen: There was a definite feeling to have a smaller set of IPR licensing than read the individual IPRs. In Opus we did this later, and it was hard to get them onboard, so there is desire to do this earlier. Kieth Drage: we should go with what is the norm in the IETF than redefine it. Ted Hardie: The document may attempt to define If you use one these license then we know what the outcome of that decision is in the WG. Tim: Encourage Ted to wordsmith this, as our intention was to get that spirit. Stephan Wenger: My recommendation is to stop the effort for the drafts however it is acceptable to do this for the code. Harald: BCP79 does not contain any licensing terms. (advise to strike the first deliverable). Chairs: REMOVE deliverable 1. - Mo: Instead of 1, do an IPR survey? Chair: perhaps a living document, not something delivered to the IESG. Deliverable 3 - Mo and Frank: define encoder bitstream and decoder operations instead how it is described currently in point 3. Deliverable 5 - Mo: we need an evaluation criteria in addition to the deliverable 5 on test results. Charter: Milestones (Dates TBD) - Frank: what timelines are we looking at? Tim: Optimistically, we are looking for 1 year, it could be more. Chairs: How long did Opus take? Tim: 3 years Chairs: Is this bigger work or smaller? Tim: It is bigger, but there are more people at Daala working on this. Questions: Is there a problem that needs solving? - Strong and unanimous. Is the scope of the problem well defined and understood? Is there agreement on what a WG would need to deliver? - Strong support with some dissent. Are there people willing to do the work? - show of hands for various questions: contribute 6 people, review: 12 people, implement: 6 people How many people feel that a WG should not be formed at the IETF? - majority in favor for forming the WG. --------------------------------------------------------------------------- [Notes from Cullen Jennings] Chairs - Jon Peterson, Adam Roach Note takers - Varun Singh, Cullen Jennings AD Set Context At IETF75, we tried to decide if we should do an audio codec. That resulted in OPUS an widely successful audio codec. Later we decided to start looking at video codec. Today we are trying to decide what we are going to a video codec WG. Chairs -------- - see slide for clear set of goals of what we want to accomplish AD & Chairs clarified purpose of today was to decide if we should do something in this space and what it should be Key Consideration presentation by Mo -------------------------------------------------- Mo discussed several of the issues that are bit different for an codec meant for the Internet. Discussed how our need for fast fine rate control is a different set of requirements than what most video codec are optimized for. Algorithm Agility - brilliant ideas solicited for way to avoid IPR risk and to do that technically to allow algorithm agility to minimize IPR risk Question Q what resolutions and bit depth? A: Very rough requirements draft mentions this but the WG needs to Q. Keith - new codecs make new problems (transcoding). Should be in requirements that don't make transcoding more difficult. Concern that we can address the delay problem in transcoding Q. Tim T - IPR around spatial scaling is nasty, do you think it is possible to more than? A. Q. Bernard - in many cases spatial scaling is less efficient than simulcast Opus had an ease of configuration that made it successful. Do we want in band FEC, are there are other things like this. When doing switching, you send FIR and get huge hit in bandwidth, and spacial support in bitstream might be able to help with that. Q Jean-Marc - opus was designed for RTP and we need to do the same thing here Tim T presentation on the Daala project --------------------------------------- Strategy for getting RF free - replace major chunks of the codec with very different technology to avoid IPR - this is high risk but high reward - helps avoid large swaths of patents including ones we are not aware of - moves us area that is much less patent s Explained several areas that are huge parts of the key way most codecs work today and then went on to explain the very different tackiness that daala is exploring to to avoid theses techniques. Example given in sides that show an example where image looks better but PSNR is worse. Contributions to daala code from 37 people - many different companies - many of them IETF participants Q - Frank Bosson - Questions about the type of encoding and which 265. A - Using just one I frame on sequence 1 second long and trying to set up for an interactive type codec. Q - what about other codec A - yes and don't look as good as this but like some other metrics too A - Time suggested taking results with full shaker of salt Q- Mo Z. - External view of this work means that we need to really focus on evaluation. We need metrics to focus on true real time performance. Traditional codecs have fallen way below reference implementation when used in real time practical implementations. Q. Eric Rescorla - ask about graph A. For 1080p you tube around 6 and low quality video conferencing around 1mbps or 0.5 Q about continuous improvement A. have web site (https://arewecompressedyet.com/) that shows how compression is going on data sets provided by anyone on many codecs Q. how fast does the test framework need to be A. run of a bunch of video sequences takes about 20 minutes Q Frank Bosson - Probably want to be at around 2-3 mbps for 1080p not the 6 mbps listed before. Q. Ross Finlayson - has everyone signed the equivalent of Note Well A. no Charter Discussion ------------------------- Chairs presented objective and the charter to the room Q. Keith D asked about used for interactive or non interactive A. yes this is for interactive A. pointed at point 2 and 3 and processes 1/2 slide Keith D felt what was in chart was pretty good but might send a few words to slightly more mention real time Bernard - We should say where it is better. He thinks it has to be better. Opus is a quantum leap better than stuff before it. Would like it to be better than existing stuff in way other that RF. He wants something better like auto configuration to do in band FEC. Some area can say just want to be competitive with, but in some area we want to say we are way better - and not just "freer" - We should be very generic in how we address this because we are not the HTA - disagrees with Bernard. If we start saying things like MUST be better than popular stuff and goal is 2 years out, . Opus is just a little bit better than AMR-WB. Like charter text as is. Richard Barnes - the must be better is not really relevant to charter. Its the consensus of WG to say it is done that is key thing. The IESG is not evaluating if it is better Bernard A. - In band FEC is something that has not been done. Just need something to Keith Drage - If the only thing we can say about it is a RF codec, then we have failed. RF is just a part of it. Charter should say we want to develop a better codec Steve Bosco - we might want to add the words "low latency" somewhere to indicate where we are focusing it to be better. Good to a Gaelle Martin-Cocher - Q - compare to codecs when it is done is vague. It should say it needs to compare to something specific, VP9, 265 etc Q. have you considered allow a core profile that is RF and non core profile that is not RF Q. ITU has been trying to do RF codec for awhile now Eric Rescorla - need to remove a } on some bullet - would be nice to have some word around diversity of software like if we want to be able to do software implementation on mobile phones we should say that. Is the assumptions software on Thomas Edwards with box - might be worth doing gap analysis on existing codecs - people are working on jpeg2000 profile that can provide a low latency video codec - even with mepg2 we are seeing improvements over time as people learn to use the tools better. - with H265 we are seeing it is not appropriate for software for all resolutions and frame rates. It is only now that we are getting H265 Tim Terriberry - responding to Gaelle questions - we have tried the approach of RF but not as good - theora for example - not as successful as we want - with 264 we had plan to make baseline RF and then other profiles non RF but baseline did not end up RF Gaelle - We should reach out to other places Chairs - we try and look and see if we expertise. Same thing with opus, many people doubted this is right place but we took that risk. Downside is we could have RFC that does not light world on fire. Gaelle - If you want to mitigate risk of IPR, you will want to have the ITU and may have better change to get a broader set of companies to signal IPR on it. Gaelle - how do you know what IRP risk are acceptable Chairs - IETF process leaves that the participants of the WG Ted Hardie - some of these issues are dealt with on further slides Mo Z - Might be good to have a list of useful IPR Randall Jesup - Better is a rat hole. RF and competitive for interactive / real time is what we need. Thomas - There is always the risk that there is IPR you will not know about. Chairs - this is true about all the stuff we work on Bernard - it will be better to get very low bit rates because of lower price HTMl phones in Africa. Mo - propose strike the 256 kbps text. Tim support **** Chair ask if anyone want to strike the text on 256 kbps lower bow. No one objected. Joe - we have general liaison to ITU but do we need specific liaison to to SQ16 **** Chair will follow up offline to see what need Stephan Wenger - currently liaison to mpeg. Most currently work is joint work between ITU and MPEG. Stephen Bernard - no payload format. Answer - we will do that in payload at same time Ted Hardie - first document are huge rat whole and mostly unless Mo - be good to have a survey of relevant IPR that is expired or soon expiring Chairs - might what as living doc not a deliverable that IETF consensus evaluation metrics should be evaluation methodology perhaps we should define encoded bitstream and decoder operation instead of defining encoder and decoder Frank - right now read as we will normatively define encoder. is that really what we want Cullen - explain desire for the point 1 text Keith - point one on IRP could be read as changing the way this WG will deal IPR Ted - will have to have argument about what licensing terms are before they know what they will contribute We could have the WG try to say if some of the well known licenses are acceptable to WG. Tim - What ted was proposing is in line with what intended here and could ted send text Stephen Wenger - done patent work for decade. When you say you have a goal of RF, people can dream up something. Recommendation is to stop this effort. Go ahead and do this for open source. Harald A. - BCP 79 does not have licensing term. Eric Rescorla - don't feel strongly about it. . One proposal would move it down to item 4. Another possible possibility to rephrase as example terms. ***** No one felt strongly the IPR number one bullet point should stay in the charter. On to milestones slides - could strike the IPR milestone as well Frank - Question about dates - how long are we talking Tim T - hope to freeze bitstream by end of year. A year is optimistic but that type of range. Opus took 3 years from point is chartering. This is a larger chunk works. More people working on it. Gaelle - ? Answer - invite anyone who show up and contribute encoder / decoder Keith - take a pass though and check milestones match Moving into Questions The question - Is there a problem worth solving ? Thomas - ask about what the problem is Strong and unanimous for the minutes in favor of YES Is the scope of problem well defined and understood? Is there agreement on what a WG would need to deliver? Strongly consensus with a little bit of decidedly in favor of this Are there people will do do the work 7 people willing to writ the documents lots of people willing to review drafts big chunk of people willing to implement How many people feel that a WG should not be formed at the IETF ? Heard maybe 1 person Majority of room strongly in favor of forming a WG.