RMCAT Agenda at IETF-88 (Vancouver) Wednesday, November 6, 2013 Meeting Chair: Lars Eggert; Mirja Kuehlewind (remote) Notetaker: Gorry Fairhurst Administrativa, Note Well, & WG Status Agenda was agreed Lars asked for committed reviewers for the congestion control requirements. People who agreed, include: Zahed Dave Taht Michael Welzl Bill VerSteeg - Reviews would be required to be received to allow this proceed before end of year. The next milestone is to discuss app interactions. Individual drafts are invited. Hasna?: Is the milestone linked to other WGs, does it include signalling protocols? Lars: RMCAT primarilly targets the RTCweb WG, and how this interacts with the RTCweb application. if you know more, please send an email to the list. Individual Drafts Evaluating Congestion Control for Interactive Real-time Media Varun Singh draft-singh-rmcat-cc-eval (milestone eval-criteria) Randal Jessup: I think a model for internet traffic would be better based on web clients. There needs to be reasonable models. Colin Perkins: The modesl I've seen are quite different. Gorry: There's pacmime if you use ns2 - if you are looking for SPDY I do not know of models (it is quite different). Mozilla ?: We can help provide traffic info. Michael Ramalho: You could choose a web page and script what happened for that web page. Lars: Send email to the list on how to model the short TCP flows. Chairs: Thanks for your work on the inidvidual draft. This is probably not the correct time to adopt as a WG draft. Z Sarker: We could write-up scenarios and whether the use-cases belongs to this draft. Varun: We could consider writing a new draft. RMCAT Video Quality Evaluation and Double Bottleneck Test Scenario Geert Van der Auwera draft-vanderauwera-rmcat-video-quality (milestone eval-criteria) Colin Perkins: Subjective quality evaluation is good, and desirable at the end stages, once we have solid proposals. It is hard during development, since it is labour intensive. Are there ways to evaluate results (e.g. better than PSNR). Geert: I think the output needs to be viewed. The proposal starts the process. Colin: Providing test sequences as input is a good step forward to let people to provide common evaluations. Lars: We foresaw the need to check the EXP specs for safety, and later we would choose to standardise some of the specs for specific use. Could this come at the later stage? Z Sarker: It's not about comparing codecs, more about achieving a minimum acceptable performance. Varun: How do you do subjective quality? Do you do a side-by-side comparison? Geert: You show the video to the expert reviewers the test results and see if they see the plot is OK. Z Sarker: I think this is a good start. Michael Ramalho: The slide looks like it shows multiple bottleneck, rather than asymmetry. What are we trying to model? Geert: Not sure. Michael Ramalho: I'd like to see this looking at 2 competing flows one agressive Bill VerSteeg (Xiaoqing Zhu): Slide 4 - is different to the google site. Why are there differences? R Jessup: This topology maps to an access link at both ends. Shibat?: If this is an access link, it may be OK. A cellular link or wireless requires a MAC model. Jitter will be very different. Lars: I think we currently have specs in several places, it would be good to have this all in one palce so people can see this. Lars: I think we need to see the problem as also enclosing the behaviour in massively deployed web browsers. What should we tell the RTCweb people to generate data on actual deployment experience. Collin: I have drafts in the XR group and would value inputs. R Jessup: We have ways to collect large anomonous data across the deployment. S Wenger: Current methods for automated evalaution do not work well when we have loss or large variations. Automatic measurments in the browser clients could report the QP that could be plotted. R Jessup: +1. I intended to say we could gather network path characteristics and how well we meet delivery targets. S Wenger: The QP scales nicely to the perceived video quality. It also scales non-linearly to the bit rate for the video, and you know how to calculate this. Geert: I think you finally need to view the picture. -: Why not consider an objective metric also for the video? S Wenger: I've yet to see a quality metric that works well under high loss or high rate changes. Lars: It would be good to start email on this, and if we could link this to quality it would be good either in browser reports or smaller more-controlled test. Modeling of Live Video Encoding in NADA Xiaoqing Zhu (presented by Michael Ramalho) -: Is this a general model? Michael: This is profiling a real encoder. R Jessup: The X-axes is a large period of time, what is the frame rate? Bill: This is 30 f/s. R Jessup: I was surprised how little frame-to-frame rate variability. Bill: This is indicative for video encoders when the door opens. Z Sarker: If there is no movement, then this is typical. Colin: Is this the bits on the wire? Michael: It's output of the codec showing the measured size of one frame. There is also the NADA control loop that sets the rate on the wire, it is input to the smooting buffer. R Jessup: Shifting the resolution (with a refresh over a number of frames). This explains the spike. R Jessup: This model looks reasonable. I'd like to adapt-down quickly. Michael: We agree, this is the most important but easiset direction to change. Encoder/real-time data source model Zaheduzzaman Sarker Mozilla : Changes in resolution tends to not be common. You could use 10-layer SVC, but they often do not. Most people drop some enhancement levels to adjust the rate and can make fine changes in the rate. Varun: It would be good to validate this in other cases, can you send to the list? Bill: (channeling Xiaoqing Zhu) Is the scale factor arbitrary? R Jessup: In larger differences this may not scale perfectly. D Taht: Do you have code that can be published? Varun: Not yet. Update on coupled congestion control for RTP media Michael Welzl draft-welzl-rmcat-coupled-cc (milestone group-cc) Lars:Have you thought about jittering (e.g. priotity or time). Michael W: We are looking at various systems. Michael R: You could use a fixed offset or de-correlate the effects. Michael W: Here, synchronisation helps us. Lars: This may also be facet of a perfect simulated world. Michael W: That could be so. Varun: When does the alg run? Michael W: When the congestion controller updates its rate. Here, when it got an ACK. Lars: I think in RFC 2140 you get something different, this always looks like one connection (one congestion controller). Michael W: Not sure, let's talk-off line. -: The slow starts are pretty slow? Michael W: You can use the unused capacity in application-limited methods. Lars: All your flows seem to have the same RTT? (yes). Are you going to try this with some of the RMCAT methods? Michael W: They're a moving target, we're planning to do RAP (as an AIMD example), TFRC (as a media-oriented example), LEDBAT (as a delay-based example). Lars: Is this a candidate work item? -: I would like to see something that shows with the RMCAT methods. Michael R: I would like to see this work directed to RT flows. It's best that all these flows compete differently. -: I would like to see data channel also included here. D Taht: I think audio and video have different properties. R Jessup: People traditionally regarded audio as separate and fixed. If we want to add the data channel stuff, we should talk about this. A Brunstrom: If we have a mix of flow types then the synchronization problem you see may no longer be an issue. The sharing is the main issue. We should see what happens when we mix. Toerless Eckhart: To me there seems to be one layer on top that is trying to allocate into the capacity. This will typically put audio before video. James Polk: Dave Taht talked about separation of audio and video and data versus realtime. Michael W: They all travel over one path and share a bottleneck. James: You don't differentiate the packets? Michael W: Yes you can assign priorities, that's the whole point, you have better control here. R Jessup: Can they see three separate bottlenecks for the different locations? Michael W: That would need the shared bottleneck detection mechanism, we're working on that. Initial Results for Google's congestion control Varun Singh Bill: What causes loss in the graphs? Varun: This could be queue length or could be late arrival. Lars: Why did you not plot all lines in the table? Varun: 0% was the same. -: What is residual loss? Varun: This wa late arrival. -: What was the jitter buffer? Varun: I think the deadline was 300ms. Harald: The algorithm responds to variation in delay. -: The candidates try to reject synthetic loss. Varun: We used a Markov chain model. In loss mode it falls back to TFRC. Michael R: What does "sense RTT" mean? Varun: This is different if delay increases. R Jessup: Is there 0.5 S of RTT? This is not a very usable call. Varun: It is 150ms one way. Michael R: How do you measure RTT? Varun: Measured at the endpoints in RTCP, ganuality 1 sec. -: Can you also plot the tau or the delta? This may belp show why this is this way. R Jessup: Did the TCP flow start at the same time? -: TCP filled the queue, the reaction is on variance. Lars: It is good to see work of this type to examine how this works. S Wenger offered copies of standard videoconference files for testing. Lars said that he would make these available via the IETF web site. Meeting closed. No time was remaining to present: Video Quality metrics in evaluation criteria for RMCAT Hassnaa Moustafa Overview on Mechansims for Preferential Packet Dropping Toerless Eckert