RTP Media Congestion Avoidance Techniques (RMCAT) at IETF 86, Orlando Monday March 11 2013, 9:00-11:30 Presentation: Lars Eggert (LE), Administrativa & WG Status -> We're running late on Requirements, we have one draft to choose from; We're running late on Evaluation Criteria and group-cc -> We aren't sure we need to do rtcp-requirements and application interactions milestones -> We're going to talk with AD about milestones - proposing 1 year slip for submission to IESG milestones -> Nothing happened on the mailing list over winter holidays, picking up a little speed now Harald Alvestrand (HA): You said "add a year to milestones". I would hope that we could get adoption done within the first year. LE: Yes, this was meant for finishing the documents, not adopting them. Presentation: Randell Jesup (RJ), Congestion Control Requirements For RMCAT draft-jesup-rmcat-reqs-01 (milestone cc-requirements) # -> What is fair? No correct definition exists. -> Risk of definition is boiling the ocean. Michael Ramalho (MR): I would challenge us to come up with a metric of fairness in terms of self-fairness and types of traffic that they're competing against, and have it independent of whether you're in the core or access network. RJ: I would love to see list discussion on that. HA: I think we know how to detect blatant unfairness. We should say "here's the tests. if something fails one of these tests, it's not fair". I'm happy with a definition of fairness that says "we can't tell whether you're fair or not, but we can tell whether you passed these tests or not" RJ: Agreed. Still comes down to a subjective evaluation, to check if on the whole. LE: This list could get very long. It would be nice if we could find some principles up front. e.g., "you shouldn't start a new flow persistently". Then define cases for checking against those. Fairness is the biggest issue for the group right now. The people who have solution candidates must have thought about this, so these people should make proposals. Authors should explain what fairness criteria they address. If we don't solve it now in the meeting we should see if we can make some progress later in ths meeting. Toerless Eckert (TE): Let's agree on a priority list. It seems to be clear that total blatant unfairness is on the top of the list. Second, how self-friendly or unfriendly is existing real-time traffic already, just as a comparison metric. I would love to see what we can achieve not only over an Internet path with other traffic, but also if there was no other traffic (self-fairness). RJ: This has some deployment issues, e. g., whether core routers separate traffic. LE: It's not only us competing against TCP. rtcweb has a data channel. Matt Mathis: TCP is not fair itself. TCP fairness is a very deep rathole, guaranteed not to converge. I would suggest making a list of fairness criteria, evaluate based on simple principles. Preventing starvation, for instance, would be very important, and that is something you could define rigorously. RJ: I agree LE: We should start a design team on this. Folks on the mic might be a good starting point. Faisan Hasan (FH): Holy grail of congestion control. TCP is not fair, e.g., RTT-unfriendliness. Shall we go down to "friendliness" instead of fairness? Dan Weber (DW): Is is planned to design an test environment? RJ: Depends on the criteria. Probably we want test of algorithms against well-defined criteria. Dan Weber (DW): MOS scores as in audio quality or something like that might be good enough. RJ: Problematic to evaluate. Video quality could be measured. Mathematically difficult. PSNR does a bad job. LE: It's a bit harder, if you produce poor audio quality that's only your own problem, but a bad congestion controller harms everyone. Colin Perkins (CP): Another way is: What do we want to "compete reasonably with" - TCP. But we could decide initially that that's not a goal and only compete fairly with ourselves. RJ: TCP does come in here, e.g., when switching. Short TCP flows are also a well-known issue. For instance, how long do we compete to TCP? When competing with TCP, at some point you have to transition to loss based mode. CP: Decision, what do we want to give up? RJ: In the short term, we definitely want to compete with TCP. Longer term, either give up or transition. Pat (Broadcom): One has to judge the importance of fairness over other characteristics. Reaction time, throughput loss, might be more problematic. Don't want to lose throughput because of fairness constraints. It is really difficult to test fairness. Randell: This is one of the hard problems of computer science. LE: ... and we will solve it before Berlin! At some point we have to make a decision. If you want to actively help work on this stay here, we will try to form a design team and meet this week. Presentation: Varun Singh (VS), Evaluating Congestion Control for Interactive Real-time Media draft-singh-rmcat-cc-eval-02 (milestone eval-criteria) -> Open issue: two metrics under discussion, other metrics seem ok -> identified 8 evaluation guidelines RJ: Discard rate = maximum beyond which we shouldn't resume. I assume that this is under the assumption of a low delay environment? my real question is: is this relative to shortest delay or is it an absolute value? VS: Absolute value. 300ms in the document. Most of the scenarios are about 200ms. MR: What you should do is to compile a histogram of delay values. TCP will fill up the buffers. Should have some user related metric. like find out where the tail is, wait a reasonable amount of time for a packet to come in. What's a reasonable amount of time? VS: Most RTP implementations have dejitter buffer. Playout delay when you receive packets. MR: smart jitter buffer will consider if it's audio or video. Smart jitter buffer doesn't use a fixed value. Zaheduzzaman Sarker (ZS): There will be some queuing in the bottlenecks. You have to look into this delay figure and compare with RTT etc. CDF of queuing delay: you get an idea of values. I think we shouldn't get rid of delay. VS: Discard rate to check if trade-off make sense or not. ZS: But then it shouldn't be a fixed value. Hardware-acceleration issues exist. They have to be factored into this. DW: You can have buffering somewhere, e.g. on the video card, so how does this fit in? VS: The way I use it is that you get a feedback value from the receiver to calculate. It's only for evaluation case, not for deploying it. ZS: We need a minimum set of guidelines, just those you really care about for real-time interactive communication. We can put this all into three/four scenarios. VS: So is this list sufficient? ZS: We have to discuss. On the mailing list I had 3 more suggestions to this one. Lars: Feedback from solution authors needed. ZS: Is this related to the requirements draft. MR: when you say video frame rate etc., are you creating something an artificial 128kbps source with constant bit rate or are you using real video sources? Real video sources are much more intelligent. VS: That's an open question. Do we use a packet generation or real measurement of video streams? MR: Suggest to simplify to something we can characterize. DW: Video is - unlike audio - highly inconsistent. Average bitrate may exist over a certain bitrate, but there may be additional control. Using any form of constant bitrate creates serious issues in terms of evaluation. I-frame can blast 100kbytes / second. VS: Should we remove this and say each app starts with different values? DW: Pick five, use the same streams for all tests. We need to compress it before we even test this. Then we can focus on network related constraints. VS: But with congestion control you change the rates on the fly. It's not a fixed stream. Depends on the encoder. Congestion control can be fed into encoder. A start value has to be fixed. Everybody should start with the same rate. RJ: We'll have to add cases to this as we go along. But must have some understanding of edge cases. Single set of parameters will not help us determine those. Another issue, I would suggest to keep the number of variables down to something reasonable. Suggest: encoder can use all bits it has available, starts with a given rate. When congestion control gives feedback, controller changes rate (and sticks with it), with some defined delay based on the frame rate. Trying to characterize all the encoder behaviors will make too many problems. I-Frames etc. should be avoided anyway, I would not try to model that. Simple packet generator would be better. Varun: Is there an available traffic generator. How do we model I-frames etc.? Randell: I would suggest an arbitrary byte stream that starts at a given rate. It has to stick withing the rate. HA: I suggest that these are two different tests. Real video streams increase complexity. Not try to use real video streams for the tests we do when we develop things. But we need test scenario that can show that cc. is not ruining the video experience. Varun: Packet generation initially? Gorry Fairhurst (GF): Maximum end-to-end delay of 300ms stands out, what is it? Application expectation? Network requirement? Is RMCAT going to break the circuit beyond 300ms? VS: No it's not a circuit breaker. This is application expectation. The packet would not be fed into the decoder, would be discarded upon arrival. GF: RMCAT might be used for various things. What is RMCAT then doing when it is exceeded? What are the transport semantics? What happens to the transport level? let's be clear to separate audio / video from transport part of it. ZS: We're talking about the wrong things here now. We should focus on test scenarios, not values. Evaluation draft should take this approach, to test if guidelines have been followed, but fixed parameters won't work. Can the requirement draft define the test suite? MR: I agree, we have to take out some uncertainty. Packet generator that should follow some rule of the cc that you want to test. E.g., a UDP packet generator with variable rate. We have to specify some initial test cases, e.g., topology, RTT. A good algorithm will have notion of bandwidth, RTT, etc. TCP tracks bits in flight which is a round trip measure. It worked long, so our algorithm is more likely if it has a measure of capacity, measure of bits in flight. We need to have a mechanism that's stable, behaves well. let's not worry about video MOS score yet. Not start with a real video source, but packet generator fit to test a cc. mechanism. Mo Zanaty (MZ): We're looking at it from different perspectives. Application perspective vs. network perspective. Now we're mixing all those, that's not good. In the requirements and in the evaluation draft, let's have a section for application-level and network-level objective metrics. Need to start putting down, what are the real needs of video applications - start rate, loss tolerance, then network-centric flow behavior. RJ: Good to separate network parameters, video parameters, application parameters. What are the parameters of the networks that connect the nodes for evaluation? Critical for determining the results of the algorithm. e.g. low-delay network, WiFi network … not a single set of points, a multi-dimensional surface of results. Magnus Westerlund (MW1): Followup to Gorry. Important to evaluate that algorithms are stable. Minimum path delay is important. An algorithm must be stable for 1ms delay and 500ms delay. Video is dynamic, this is the challenge here. For evaluation, let's get as many bits across as possible is also a possible evaluation for these cases. Slide Evaluation Scenarios 1/3 MR: is this delay the propagation delay or the queuing delay? Queuing delay is rather important (buffer bloat). Lars: It is propagation delay. ???: Buffers will not be defined in terms of delay. Lars: Buffer scenarios have to be defined. We want at least FIFO, but the charter mentions more. Slide Evaluation Scenarios 2/3 ZS: I don't understand this variable capacity link. For example, in wireless, each user will have different capacity. Where is the bottleneck? Capacity changes. Varun: My assumption is that there is a trace that defines mobility events etc. ZS: Capacity changes even without doing anything. Lars: We can't discuss all scenarios in the meeting. Authors, please comment on this on the mailing list. MM: single queue assumption or using separate queues for throughput-maximizing traffic? VS: single MM: it should be stated for cases where throughput maximizing and delay-based mechanisms competing are guaranteed to fail. MR: We could design a topology such that we're only competing against ourselves and stay in a delay-based mode. But maybe I'm wrong, I thought this group was about competing against other traffic too. LE: assumption is correct. I think we want to have a set of isolated scenarios but also cover running on the Internet with other traffic. Please help Varun. MM: Have to ask: when you have to compete, and when you have separate queues, and when mode switching works. Hum on WG acceptance: Randells draft: 20 hands in favor, 0 against LE: We can call that rough consensus. Varun's Evaluation criteria draft: 10 hands in favor, 4 or 5 against. So not clear consensus. LE: What do people want to see in the document to be "yes"? Presentation: Michael Welzl (MW2), Coupled congestion control for RTP media draft-welzl-rmcat-coupled-cc-00 (milestone group-cc) LE: Regarding algorithms: it is not really about priority, it is more about share. Fairness is not priorities (slide 5) MW2: Proportional share MW: How do you handle mix of RTP, SCTP flows etc. Can you handle all these cases. MW2: Current version can handle 5-tuples, beyond that need shared bottleneck detection, false negatives will not be a problem. Integrating rate-based and window-based is a challenge. Could have multiple FSEs. MW: Another alternative is to apply RMCAT to SCTP. In RTCweb, we are going to have prioritites. LE: If we assume that kernel implementations of congestion control doesn't change on short time-scale, the flow table gives the browser some means to control and application could do more quick adaptation. DW: Would it make sense to broadcast state table on local LAN so everyone can see what goes on? MW2: Interesting research topic, but raises too many issues for rmcat. ZS: Do the flows using the FSE run the same cong control algorithm? Could cover more scenarios than rmcat MW2: in my tests yes, but we could use different ones. LE: Are more poeple working on coupled cong control? Presentation: Colin Perkins (CP), On the Use of RTP Control Protocol (RTCP) Feedback for Unicast Multimedia Congestion Control draft-perkins-rmcat-rtp-cc-feedback (related to milestone rtcp-requirements) -> Thinking about how existing feedback channel works (with AVPF) (Per-packet, RTCP feedback, per video frame) MZ: I am surprised that you are using full size. I always assumed that we would do RSize extension. CP: I was picking pessimistic assumption. MZ: I think optimization is even doable in the per-packet case. CP: Depends on the bandwidth. But there's maybe not even a point. Hannes Tschofenig (HT): Sending feedback quickly has its origin in cc. for traffic that is very quickly adaptable. This doesn't apply to audio/video, there are reasons in video/audio codecs why you can't change it so fast anyway. And you may not want to do anyway because of the user experience. CP: I agree that changing quality frequently is problematic for user quality. But you do need enough feedback so you can tell when problems are happening, to look at change in queuing delay for example. Frame feedback seems like a reasonable compromise, with also sending feedback faster when problems are developing. HT: Agreed. I think RTCP is in fact a suitable mechanism for cc. feedback. MR: Draft is very well written, thank you. Per-frame or per-packet feedback is a sampling problem. Ideally you do every single packet because backchannel is unreliable. Feedback has to be often enough for the control system to work. The fact that video encoders are limited in how fast they can change the rate is somehow irrelevant for the question of convergence, stability etc. of the controller. CP: Agreed, this is a balancing act. We need to figure out what is appropriate. If something roughly in the order of an RTT is enough I think we can do that with RTCP very well. MR: My point is that it's a control theory problem wrt. what types of feedback we want in the system. RJ: Agreed. But these are just worst case scenarios, best case scenarios are much better. You may need to change rates dramatically in certain situations, to recover appropriately and deal with bursty situations. TE: Is there anything video specific here? Can we ignore that it's video? People talk about VBR audio codecs that can change arbitrarily. Is the feedback for a video codec not totally decorrelated from congestion feedback? CP: With audio, you may quickly run into situations where you send as much RTCP feedback as data. RJ: Just because RTCP-fair feedback will work, doesn't mean that an algorithm should use RTCP feedback. This feedback is important and needed for certain cases where a reverse flow does not exisit. Piggybacking may be a useful technique in conjunction with RTCP. CP: Point is, RTCP is a valid player in the space, potentially the right sort of feedback. It was claimed in the past that RTCP is not possible per packet. MZ: I agree with Randell: Even if RTCP feedback could do it, from an efficiency point of view the packet rate often matters more than the total bandwidth consumption, especially because of the per-packet overhead in wireless networks. Especially piggybacking could help. CP: This is the regular RTCP feedback. Not sending any RTCP packets that you wouldn't send anyway. MR: These are extreme cases. There is something in-between. Feedback could be very low in terms of bandwidth, but it needs to come often enough, but perhaps not more often than necessary. I see it as a sampling problem. Kevin Gross (KG): There are two issues: How often do you sample, how often do you process that data? In control theory, you can cause control system instability by having too much latency in feedback. CP: People developing CC. algorithms must answer that for their algorithm. I'm trying to show that for reasonable amounts of feedback, RTCP can work. Don't discard it because of the assumption it's too slow. The main point here is: Do not exclude RTCP now. ZS: Do we know what the minimum delay interval or frequency of feedback is? It depends on how we design our controller, that is not answered anywhere yet. RTCP is already there. MR: TCP does it via clocking. You have to take the RTT into consideration. I want to make that point really clear, in order to get the control loop right, you have to think about that. LE: What is the intended goal of the draft? CP: Create awareness, raise discussion, and to get feedback. As such, this discussion is already a success. Not proposing to progress it any further. LE: I could see the material in an appendix of a mechanism, to explain design choices that were made. Presentation: Rong Pan (RP), NADA: A Unified Congestion Control Scheme for Real-Time Media draft-zhu-rmcat-nada -> Design goals: limit self-inflicted delay, leverage a suite of network feedback mechanisms (loss-based, delay-based, ECN, PCN) -> Sender does linear prediction to accommodate feedback latency -> Simulation of NADA with ECN shows improvement in avoiding queue build; PCN allows zero standing queue ZS: PCN based signaling is something new that you want to design, it's not there, right? LE: PCN standards are out. ZS: Yes, but not used yet. We're going with things for the current Internet. LE: We do a mechanism for the current Internet, but good if it works better in the future Internet. ZS: I have similar results, and shown in the CC. workshop. Second thing: you have a rate-shaping buffer at the sender. Does it have any relationship with the encoder/ to certain video frames? It's per packet, right? RP: Video frames are there, buffer is basically smoothing the traffic into the network. ZS: Packet shaper on the packet level that delays packets may affect the subjective quality, e.g. when doing it with I-frame. Importance of packet is missing. RJ: (Non-ECN case, bottleneck queue length) Queue is a fair number of packets beyond a certain window in your simulation. Once you're overloaded, do you end up with delay even when competing with itself? RP: Depends on where your settle, on how heavily congested things are. RJ: Loss rates on one slide were in the 3-5% range. That's a pretty high loss rate for video. RP: Yes, but FEC is possible. If that's the way the network gives the feedback then that's how it is. But algorithm could be tuned to be more sensitive to delay. RJ: Latency is a critical parameter. Throughput includes any recovery or FEC or whatever is needed to make for a high loss rate. Must be factored in. FEC has delay issues. KR: Have you looked at Codel? RP: Future work, should be evaluated in particular for PCN. KR: Codel works without ECN or PCN. RP: But it does drop packets. KR: But the way it drops packets based on delay measurements. Jim Gettys (JG): Codel is happy to do ECN. RP: Have to look at it. If it's ECN that's fine. Algorithm can be adopted here. As a result you'll have less losses. KR: How does it compete with TCP? RP: Good question. Here we look at self-fairness. If TCP is the concern, we have a similar design where, we can respond with sqrt(p) when competing with loss based. HA: Next time you update the draft, please include a reference for PCN. TE: Agree with ZS, bringing in more codec knowledge might be used to optimize the behavior. The adaptation layer is important to distinguish between algorithm and hardware codecs. ECN might provide benefits, but we don't know. ZS: Encoder would have to know that this packet has been delayed or lost. TE: Showed that it is fair. Test should take into account whether ECN has some benefit over delay variation, especially when there is other traffic. Do we have clear evidence that we could ask network operators to enable it? That's still the open question now. MZ: In Nada, losses are translated into delays, in Dflow, delays are translated into losses, in Google yet different. what is the right way to mix the signals? Observation is that all drafts mix delay, loss, etc. Should the evaluation draft comment on this? What will be evaluated against? Otherwise, all drafts will use own heuristics to deal with delay and loss. LE: Evaluation criteria are more about objective quality measures of the result. Up to the mechanism to decide what's best. MZ: There should be objective criteria on how to mix the signals - e.g. this combination of delay and loss is better than that combination of delay and loss. LE: agreed. RJ: I agree. This is part of the evaluation criteria. If you take in more loss you effectively reduce your bit rate because you have to add FEC. Subjective quality is more important in the end. ZS: Different algorithms handle delay and loss rate requirements in different ways. Let's define how many % loss rate are okay. Some say 5%, Google's mechanism says 10% is okay. LE: It's not so easy. In a shared bottleneck you're not going to be the one who sees all the losses. What matters is what you cause to others. Lars: Design team: Please come to the front.