Subject: [iccrg] Very fundamental congestion control questions...

From: "Bless, Roland (TM)" <[email-masked]>
Date: Mon, 18 Feb 2019 14:50:22 +0100

Hi all,

I think I can second Keith Winstein's statement
"we lack a real understanding and common language about
some of the fundamental concepts and hard parts..."
(https://mailarchive.ietf.org/arch/msg/iccrg/LHRAWIjbFktaqLC5hFqgG37M9zk)
and want to open a
new discussion thread for this.

One of the problems that we have is (IMHO), that there
is currently no commonly agreed definition of what
_congestion_ is actually. So every congestion control
developing people should at least have clear answers
for themselves to these questions:

1) what is congestion (or congestion situation)?

2) how to detect a congestion situation
   (or even severity levels of congestion)?

3) how to react on a detected congestion situation?

Letting the related fairness discussion aside, I think that even
RFCs do not give a clear definition of what congestion is and
the views, Internet usage, and requirements have considerably
changed over time.

So in former times, the term congestion was mainly
driven by the congestion collapse experience.
Consequently, most RFCs refer to the prevention of
that congestion collapse when discussion congestion control
(or congestion avoidance), e.g., RFC 7567:
"It is primarily these TCP congestion avoidance
algorithms that prevent the congestion collapse of today's Internet."

Nowadays, the interest has probably shifted towards
systematically reducing queueing delay and congestion could
be seen as being related to the level of waiting time in
bottleneck queues (this would also allow for the definition
of different levels of congestion).

While the analogy to ordinary car traffic scenarios is not always
right, in this case I find it useful: you do not consider the
waiting time in front of traffic lights as congestion
unless you are waiting more than x min in addition
to what you normally experience during non-congested
times (e.g., waiting two traffic light phases is probably
considered being okay, but more than that may be already
viewed as being traffic congestion).
However, both traffic broadcast and navigation systems
nowadays announce congestion by indication of expected
additional waiting/queueing time (which is more useful than
just mentioning the length of the congestion in kilometers).
In this regard, I would heavily recommend to reread
Kathie's and Van's "Controlling Queue Delay" Paper:
https://queue.acm.org/detail.cfm?id=2209336
and the introduction of good/bad queues, standing queues,
and sojourn times, which could be quite helpful in the
congestion definition discussion.

Similarly, the answer to 2) should not be "packet loss"
alone anymore, but probably trying to distinguish
between congestion induced packet loss and short-term (burst)
loss that is not caused by a real congestion situation.
So in lack of ECN or other explicit congestion signals,
multiple measured parameters may be combined for better
congestion detection than relying on packet loss alone.

I think it could be useful to discuss a bit along
the above listed questions in order to move CC research
forward by probably achieving a better understanding and
common language about the fundamental concepts... :-)

Regards
 Roland

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 20 Feb 2019 08:33:56 +0000

Roland,

A better word for the symptoms of temporarily approaching capacity would 
be great, but it's a hard problem to come up with a useful single term. 
I don't have an answer, but I can add a couple of requirements:

Consider the case of a virtual queue giving out ECN signals (as in 
RFC5670's pre-congestion or Alizadeh's "phantom queue" in HULL, or VCP's 
lower threshold). There is no queuing at all here - no delay and 
capacity is not exceeded. But we still need a better word for these 
signals than explicit /congestion/ notification.

It would also be useful if the word could be easily substituted in all 
the places where we currently say 'congestion' (i.e. beginning with 'c' 
would help, so abbreviations can be re-defined).

It may help to first try to find a set of terms, then try to find a 
single term for the set.

Bob

On 18/02/2019 13:50, Bless, Roland (TM) wrote:
> _______________________________________________
> iccrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: Kilkki Matti <[email-masked]>
Date: Wed, 20 Feb 2019 10:13:03 +0000

Hi all,

I agree with Bob that we need at least two distinct terms.

Maybe we could define congestion as a situation where system performance is reduced due to excessive load. 

Then we need a separate term to illustrate a situation where load exceeds the system’s capacity, e.g., overload situation. Overload situation does not necessarily generate congestion if the system is properly designed and congestion can occur even without excessive load if the system is badly designed.  
 
This kind of separation of concepts might be useful, because there also are two types of actions (or reactions) to the situation. First, the system tries to maintain its (maximum) service capacity even during overload situations. Secondly, the system tries to control the offered load by providing incentives for users (and applications) to adjust the traffic they are generating during overload situation (this is actually what I think is the main task of QoS or traffic control in general, see https://arxiv.org/abs/1901.06867).

Regards,

Kalevi

________________________________________
From: iccrg <[email-masked]> on behalf of Bob Briscoe <[email-masked]>
Sent: Wednesday, February 20, 2019 10:33 AM
To: Bless, Roland (TM); [email-masked]
Subject: Re: [iccrg] Very fundamental congestion control questions...

Roland,

Bob

---

From: David Hayes <[email-masked]>
Date: Wed, 20 Feb 2019 11:44:58 +0100

Hi Bob, Roland, and all,

Congestion and its control is difficult to define because it 
depends on the context and the time intervals of interest. I've 
written down some thoughts that may help the discussion.

I think the fundamental base idea (and building block) of 
congestion for packets is when a packet has to be delayed or 
dropped because there is not immediate capacity available for it 
to be sent. So as Bob indicated, it has to do with capacity being 
momentarily exceeded. This fundamental base idea can then be 
combined at multiple network points (depending on the context) and 
averaged over different time intervals, roughly becoming the 
rather plural definitions of "congestion".

For example:

At the time scale of packet send times, packets can be temporarily 
buffered or dropped. Mitigating congestion is then a matter of 
queueing and scheduling. Note, this is in the context of somewhat 
random packet arrivals where the average arrival rate is usually 
significantly less than the capacity. If packets are arriving 
perfectly paced (CBR), then there is no congestion until the 
arrival rate exceeds the capacity. Only then will there "suddenly" 
be queueing; in this context buffering and scheduling may be of 
limited help. Load is a good measure, but as shown, it does not 
directly relate to congestion, as "congestion" also depends on the 
arrival distribution (also the departure distribution though 
probably less important standard packets on a wire, but very 
relevant in wireless scenarios). Load can, with knowledge of the 
distributions, be used as a predictor of "congestion".

"Congestion" measured by averaging this base idea over slightly 
longer time scales, may be alleviated by changes in routing at a 
particular point, i.e. by sending some of the packets on more 
lightly loaded paths.

At longer time scales of averaging and combining of base instances 
along a path, the measured "congestion" can be mitigated (in a 
distributed way) by senders reducing their send rates. At best 
this is on the time scale of the RTT from sender to congestion 
point, at "worst" it is on the time scale of the RTT from sender 
to receiver. This is the time scale of transport congestion 
control. Currently the Internet mainly uses the "worst" option as 
the best option requires network support. (of course there are 
even worse ways of doing it...)

At flow/connection times scales, congestion measured by averaging 
and combining the base congestion instances over these time scales 
and over a broader network context, "congestion" can be alleviated 
by some sort of admission control preventing new connections/flows 
from entering the network.

At even longer time scales (broadest context, time scales of 
months-years) measured "congestion" can only really be helped by 
network dimensioning (i.e. building more capacity).

Although this is still a rather simplistic view, and there are 
additional options to those mentioned above, perhaps it helps to 
highlight the complexity of the term "congestion" before we dive 
down any rat holes. As Bob writes, perhaps we are looking more at 
a set of terms.

Kind regards,

David

-- 
David Hayes
SimulaMet
https://www.simula.no/people/davidh

---

From: Ingemar Johansson S <[email-masked]>
Date: Wed, 20 Feb 2019 12:27:54 +0000

Hi

Great discussion. I believe that it can be good to distinguish between congestion and overload. 
My personal definitions are, heavily biased by my preference to solve issues and challenges in 3GPP style cellular access, and I probably simplify things too much.

+ Congestion : Queues are built up mainly due to users data traffic, transport protocols congestion control and AQM behavior (with/without ECN or L4S) can to a more or less good job to keep the queues at an acceptable level
+ Overload : The traffic pressure becomes so high that no AQM or ECN or L4S can save you, a fantasy example is a festival out in the boondocks  where operators perhaps deployed some cellular coverage underestimated the number of attendees or perhaps completely missed the important fact that there are only teenagers there who upload 1MegaInstagramPictures/hour 😊.

/Ingemar

-----Original Message-----
From: David Hayes <[email-masked]> 
Sent: den 20 februari 2019 11:45
To: Bob Briscoe <[email-masked]>
Cc: Bless, Roland (TM) <[email-masked]>; [email-masked]
Subject: Re: [iccrg] Very fundamental congestion control questions...

Hi Bob, Roland, and all,

Congestion and its control is difficult to define because it depends on the context and the time intervals of interest. I've written down some thoughts that may help the discussion.

I think the fundamental base idea (and building block) of congestion for packets is when a packet has to be delayed or dropped because there is not immediate capacity available for it to be sent. So as Bob indicated, it has to do with capacity being momentarily exceeded. This fundamental base idea can then be combined at multiple network points (depending on the context) and averaged over different time intervals, roughly becoming the rather plural definitions of "congestion".

For example:

At the time scale of packet send times, packets can be temporarily buffered or dropped. Mitigating congestion is then a matter of queueing and scheduling. Note, this is in the context of somewhat random packet arrivals where the average arrival rate is usually significantly less than the capacity. If packets are arriving perfectly paced (CBR), then there is no congestion until the arrival rate exceeds the capacity. Only then will there "suddenly" 
be queueing; in this context buffering and scheduling may be of limited help. Load is a good measure, but as shown, it does not directly relate to congestion, as "congestion" also depends on the arrival distribution (also the departure distribution though probably less important standard packets on a wire, but very relevant in wireless scenarios). Load can, with knowledge of the distributions, be used as a predictor of "congestion".

"Congestion" measured by averaging this base idea over slightly longer time scales, may be alleviated by changes in routing at a particular point, i.e. by sending some of the packets on more lightly loaded paths.

At longer time scales of averaging and combining of base instances along a path, the measured "congestion" can be mitigated (in a distributed way) by senders reducing their send rates. At best this is on the time scale of the RTT from sender to congestion point, at "worst" it is on the time scale of the RTT from sender to receiver. This is the time scale of transport congestion control. Currently the Internet mainly uses the "worst" option as the best option requires network support. (of course there are even worse ways of doing it...)

At flow/connection times scales, congestion measured by averaging and combining the base congestion instances over these time scales and over a broader network context, "congestion" can be alleviated by some sort of admission control preventing new connections/flows from entering the network.

At even longer time scales (broadest context, time scales of
months-years) measured "congestion" can only really be helped by network dimensioning (i.e. building more capacity).

Although this is still a rather simplistic view, and there are additional options to those mentioned above, perhaps it helps to highlight the complexity of the term "congestion" before we dive down any rat holes. As Bob writes, perhaps we are looking more at a set of terms.

Kind regards,

David

On Wed, Feb 20 2019 at 09:33, Bob Briscoe <[email-masked]>
wrote:

> Roland,
>
> A better word for the symptoms of temporarily approaching capacity 
> would be great, but it's a hard problem to come up with a useful 
> single term.
> RFC5670's pre-congestion or Alizadeh's "phantom queue" in HULL, or 
> VCP's lower threshold). There is no queuing at all here - no delay and 
>> I think I can second Keith Winstein's statement "we lack a real 
>> understanding and common language about some of the fundamental 
>> concepts and hard parts..."
>> One of the problems that we have is (IMHO), that there is currently 
>> no commonly agreed definition of what _congestion_ is actually. So 
>> every congestion control developing people should at least have clear 
>> answers for themselves to these questions:
>>
>> Letting the related fairness discussion aside, I think that even RFCs 
>> do not give a clear definition of what congestion is and the views, 
>> Internet usage, and requirements have considerably changed over time.
>>
>> So in former times, the term congestion was mainly driven by the 
>> congestion collapse experience.
>> Consequently, most RFCs refer to the prevention of that congestion 
>> collapse when discussion congestion control (or congestion 
>> avoidance), e.g., RFC 7567:
>> "It is primarily these TCP congestion avoidance algorithms that 
>> prevent the congestion collapse of today's Internet."
>>
>> Nowadays, the interest has probably shifted towards systematically 
>> reducing queueing delay and congestion could be seen as being related 
>> to the level of waiting time in bottleneck queues (this would also 
>> allow for the definition of different levels of congestion).
>>
>> right, in this case I find it useful: you do not consider the waiting 
>> time in front of traffic lights as congestion unless you are waiting 
>> more than x min in addition to what you normally experience during 
>> non-congested times (e.g., waiting two traffic light phases is 
>> probably considered being okay, but more than that may be already 
>> However, both traffic broadcast and navigation systems nowadays 
>> announce congestion by indication of expected additional 
>> waiting/queueing time (which is more useful than just mentioning the 
>> length of the congestion in kilometers).
>> In this regard, I would heavily recommend to reread Kathie's and 
>> Van's "Controlling Queue Delay" Paper:
>> and the introduction of good/bad queues, standing queues, and sojourn 
>> times, which could be quite helpful in the congestion definition 
>> discussion.
>>
>> alone anymore, but probably trying to distinguish between congestion 
>> induced packet loss and short-term (burst) loss that is not caused by 
>> a real congestion situation.
>> So in lack of ECN or other explicit congestion signals, multiple 
>> measured parameters may be combined for better congestion detection 
>> than relying on packet loss alone.
>>
>> I think it could be useful to discuss a bit along the above listed 
>> questions in order to move CC research forward by probably achieving 
>> a better understanding and common language about the fundamental 
>> concepts... :-)
>>
>> Regards
>>   Roland
>>

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Wed, 20 Feb 2019 13:35:16 +0100

Hi Kalevi,

> Maybe we could define congestion as a situation where system performance is reduced due to excessive load. 

Yep, the question is though, what is "excessive"? Temporary overload
will be quite normal due to bursty arrivals, capacity probing, etc.
However, "persistent" overload is bad. Now, what is persistent?
Probably, longer than one RTT (because the sender usually cannot react
much faster given the current feedback loop), but RTTs vary (letting
queuing delay aside for the moment) depending on the distance between
sender and receiver. The time-dependency makes it difficult.

> Then we need a separate term to illustrate a situation where load exceeds the system’s capacity, e.g., overload situation. Overload situation does not necessarily generate congestion if the system…

Yes, that's what buffers are for. They serve to mitigate the
consequences of such temporary overload situations. However,
if the load exceeds the buffer's capacity for longer periods
or even if the buffer is permanently occupied too much we may
speak of persistent overload and thus congestion.

Also "load" can be quite different: a) You simply may have too
many arriving flows for the bottlenecks capacity, so that each flow
has only a single packet in flight. In this case the ordinary CC loop
probably does not kick in. b) only a few flows have too much data in
flight so that the amount of excess inflight data enters the buffer
(which is ok if it is not persistent, i.e., resulting in the mentioned
standing queue).

> This kind of separation of concepts might be useful, because there also are two types of actions (or reactions) to the situation. First, the system tries to maintain its (maximum) service capacity…

Ah, yes, that was on my "should read soon" paper pile already :-)

Regards,
 Roland

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Wed, 20 Feb 2019 13:36:53 +0100

Hi Bob,

> be great, but it's a hard problem to come up with a useful single term.

Do you mean (full) link utilization or link saturation when considering
a single (bottleneck) node?
Indeed, under some circumstances one can reach full link utilization
without filling the bottleneck queue. However, it's clear that we
need buffers in order to allow for good/high link utilization in case
of short-term mismatch in arrival and departure rates. Once traffic
enters the queue, the behavior is a bit different than when forwarded
w/o queue.

One problem may also be related to the term "temporarily":
how long would that be? this time period may actually be seen
quite different among various people.

> signals than explicit /congestion/ notification.

I would say it's explicit network feedback. I'm convinced that we
actually get better solutions when end-to-end congestion control
can make its decisions based on such explicit network feedback
(yes, there were already XCP, FCP and so on...).

> It may help to first try to find a set of terms, then try to find a

Hmm, let's see what progress this discussion achieves. Looking forward
to discuss....

Regards,
 Roland

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Wed, 20 Feb 2019 14:44:30 +0100

Hi David,

Am 20.02.19 um 11:44 schrieb David Hayes:
> Congestion and its control is difficult to define because it depends on
> the context and the time intervals of interest. I've written down some
> thoughts that may help the discussion.

Yep.

> I think the fundamental base idea (and building block) of congestion for
> packets is when a packet has to be delayed or dropped because there is
> not immediate capacity available for it to be sent. So as Bob indicated,
> it has to do with capacity being momentarily exceeded. This fundamental
> base idea can then be combined at multiple network points (depending on
> the context) and averaged over different time intervals, roughly
> becoming the rather plural definitions of "congestion".
> 
> For example:
> 
> buffered or dropped. Mitigating congestion is then a matter of queueing
> and scheduling. Note, this is in the context of somewhat random packet

That point of view surprises me a bit, since I don't think
that queueing and scheduling can actually "mitigate" congestion
(maybe due to my current view that congestion is some kind of
persistent overload; so short-term overload is not congestion for me).
I think that queueing and scheduling can only influence the
utilization on shorter time scales.
IMHO congestion mitigation requires some kind of
push back signal to the sources to slow down (e.g., reduce the
number of data in flight) or to back off (e.g.,
if the number of sources is too high).

> arrivals where the average arrival rate is usually significantly less
> than the capacity. If packets are arriving perfectly paced (CBR), then
> there is no congestion until the arrival rate exceeds the capacity. Only
> then will there "suddenly" be queueing; in this context buffering and

Even CBR shaped traffic doesn't help here, since it
actually depends on the concrete scenario. Imagine a simple
switch that has two incoming interfaces A and B and an outgoing
interface C, all using the same constant link rate of 100 Mbps.
Now, let's assume that A and B have each a traffic arrival rate
of 50 Mbps and we assume uniform fixed packet lengths.
So even if both incoming flows are perfectly paced (CBR),
you may end up with either no resource conflict (because they can
be served perfectly alternating) achieving 100% link utilization
or a permanent conflict situation (perfect simultaneous arrivals
at A and B) that would lead to only 50% link utilization if the
switch does not buffer. Under the assumption of mean arrival rates
of 50 Mbps (not CBR) the effective utilization may lie somewhere
in between (without buffers, my simulation currently
tells me 27,7% packet loss at C for exponentially distributed arrivals
at A and B (average at @50Mbps)).

> scheduling may be of limited help. Load is a good measure, but as shown,
> it does not directly relate to congestion, as "congestion" also depends
> on the arrival distribution (also the departure distribution though
> probably less important standard packets on a wire, but very relevant in
> wireless scenarios). Load can, with knowledge of the distributions, be
> used as a predictor of "congestion".

Yes, you are right the arrival/departure distributions are important,
see above.

> "Congestion" measured by averaging this base idea over slightly longer
> time scales, may be alleviated by changes in routing at a particular
> point, i.e. by sending some of the packets on more lightly loaded paths.

That tries to shift load away from resource bottlenecks (or increase
the utilized capacity).

> At longer time scales of averaging and combining of base instances along
> a path, the measured "congestion" can be mitigated (in a distributed
> way) by senders reducing their send rates. At best this is on the time
> scale of the RTT from sender to congestion point, at "worst" it is on
> the time scale of the RTT from sender to receiver. This is the time
> scale of transport congestion control. Currently the Internet mainly
> uses the "worst" option as the best option requires network support. (of
> course there are even worse ways of doing it...)

Yep.

> At flow/connection times scales, congestion measured by averaging and
> combining the base congestion instances over these time scales and over
> a broader network context, "congestion" can be alleviated by some sort
> of admission control preventing new connections/flows from entering the
> network.

Right, since admission control (together with policing) is a tool
to control the traffic load entering a network :-)

> months-years) measured "congestion" can only really be helped by network
> dimensioning (i.e. building more capacity).
>
> highlight the complexity of the term "congestion" before we dive down
> any rat holes. As Bob writes, perhaps we are looking more at a set of
> terms.

> David

Thanks for sharing your thoughts,
 Roland

> On Wed, Feb 20 2019 at 09:33, Bob Briscoe <[email-masked]> wrote:
> 
>> Roland,
>>
>> single term. I don't have an answer, but I can add a couple of
>> requirements:
>>
>> the places where we currently say 'congestion' (i.e. beginning with
>> 'c' would help, so abbreviations can be re-defined).
>>

---

From: Wesley Eddy <[email-masked]>
Date: Wed, 20 Feb 2019 08:49:15 -0500

> for themselves to these questions:

Some people may remember this was an early topic discussed a lot when 
the ICCRG was created, and I think the result of those conversations is 
captured in the introduction to RFC 6077:

    Congestion can be defined as a state or condition that occurs when
    network resources are overloaded, resulting in impairments for
    network users as objectively measured by the probability of loss
    and/or delay.  The overload results in the reduction of utility in
    networks that support both spatial and temporal multiplexing, but no
    reservation [Keshav07  <https://tools.ietf.org/html/rfc6077#ref-Keshav07>].  Congestion control is a (typically
    distributed) algorithm to share network resources among competing
    traffic sources.

    ...

https://tools.ietf.org/html/rfc6077

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Wed, 20 Feb 2019 15:10:13 +0100

Hi Wes,

see below.

> https://tools.ietf.org/html/rfc6077

Thanks for remembering this and providing a pointer
to this RFC.

While I completely agree to this basic definition
of congestion, my question was a bit more targeted
to more concrete definitions that also allow to
derive measures like the severity of congestion etc.

For instance:
* the previously mentioned standing queue may serve as
  congestion level indicator (and would also match
  the above definition w.r.t. delay impairment)

* the BBR paper mentions: "Congestion is just sustained
  operation to the right of the BDP line, and congestion
  control is some scheme to bound how far to the right a
  connection operates on average.", and finally

* loss-based CC schemes use packet loss as congestion
  indication (which causes self-inflicted congestion and
  impairment w.r.t. delay).

Regards,
 Roland

---

From: Szilveszter Nadas <[email-masked]>
Date: Thu, 21 Feb 2019 11:23:07 +0000

Hi all,

I uploaded a relevant old presentation on Traffic Management (~ 3 minutes read).
https://www.researchgate.net/publication/331249301_On_Traffic_Management

A related nice property of the Core-Stateless solutions we are working on is that they provide a good congestion metric (Congestion Threshold Value, see more at http://ppv.elte.hu/ ), which can be used to harmonize mechanisms into strategies. :)

Sz.

-----Original Message-----
From: iccrg <[email-masked]> On Behalf Of Bless, Roland (TM)
Sent: Wednesday, February 20, 2019 15:10
To: Wesley Eddy <[email-masked]>; [email-masked]
Subject: Re: [iccrg] Very fundamental congestion control questions...

Hi Wes,

see below.

Am 20.02.19 um 14:49 schrieb Wesley Eddy:
> the ICCRG was created, and I think the result of those conversations 
> is captured in the introduction to RFC 6077:
> 

Thanks for remembering this and providing a pointer to this RFC.

While I completely agree to this basic definition of congestion, my question was a bit more targeted to more concrete definitions that also allow to derive measures like the severity of congestion etc.

For instance:

Regards,
 Roland

---

From: Kilkki Matti <[email-masked]>
Date: Fri, 22 Feb 2019 09:10:55 +0000

Hi Roland,

So the current definition seems to be:

“Congestion can be defined as a state or condition that occurs when network resources are overloaded, resulting in impairments for network users as objectively measured by the probability of loss and/or delay.” 

As I already mention in an earlier mail, I would prefer a definition that depends only on the observable behavior of the system, not on the users of the system. If we accept this as a starting point, I see three options:

1)	Congestion is a situation in which excessive load impairs the throughput of the system (regardless of packet loss ratio and delay).
2)	Congestion is a situation in which the packet loss ratio and/or delay exceeds given thresholds. 
3)	Congestion is a situation in which the system needs or tries to decrease the traffic load in order to improve the service quality.

But the third option is somewhat circular and thus not really reasonable (in my opinion).

The second option has many challenges as already discussed: what are reasonable thresholds and, in particular, what is the timescale for measuring packet losses and delay? Anyway, this is the way the control systems for reducing congestion usually work. 

Then as to the first option, it we use a car traffic analogy, a highway is congested only when the throughput of the highway (or speed of the cars) is decreased due to excessive traffic (regardless of the number of cars waiting to access the highway). Normally, when the load exceeds a limit the speed of cars is considerably reduced and then the highway certainly is congested. As there typically is a hysteresis in this process, the problem of defining timescale is alleviated. 
  
So I would prefer option 1 (anyway, it is in line with the definition of congestion in the context of car traffic). 

Still, it might be that a kind of combination of options 2 and 3 is easier to accept, but I cannot really formulate any feasible and simple enough defition for congestion based on those. I think that we first need a general definition of congestion (whatever it will be) and then maybe one specific engineering definition with several parameters (or even several definitions for different types of systems and networks).

Regards,
Kalevi

________________________________________
From: iccrg <[email-masked]> on behalf of Bless, Roland (TM) <[email-masked]>
Sent: Wednesday, February 20, 2019 4:10 PM
To: Wesley Eddy; [email-masked]
Subject: Re: [iccrg] Very fundamental congestion control questions...

Hi Wes,

see below.

For instance:

Regards,
 Roland

---

From: "Bless, Roland (TM)" <[email-masked]>
Date: Fri, 22 Feb 2019 13:51:34 +0100

Hi Kalevi,

just a quick first reply inline...

> Then as to the first option, it we use a car traffic analogy, a highway is congested only when the throughput of the highway (or speed of the cars) is decreased due to excessive traffic (regardless…

I think the analogy does not hold here very well, see below...

> So I would prefer option 1 (anyway, it is in line with the definition of congestion in the context of car traffic). 

Option 1 has IMHO the problem that it does not cover the bufferbloat
problem: From the point of view of the bottleneck, its throughput
with a standing queue is the same as without (letting the CC differences
aside for a moment), but the additional delay is an impairment of the
flows in terms of performance (for some ACK-clocked window-based CCs
even in throughput in the sense that throughput= CWnd/RTT_effective will
decrease due to the increase queuing delay if CWnd is kept unchanged,
because RTT_effective includes queueing delay).

But it's exactly these subtle differences that I wanted to discuss...

Regards,
 Roland

---

From: Mario Hock <[email-masked]>
Date: Fri, 22 Feb 2019 14:45:48 +0100

> Then as to the first option, it we use a car traffic analogy, a highway is congested only when the throughput of the highway (or speed of the cars) is decreased due to excessive traffic (regardless…

Thanks for sharing this analogy. I think it just points to another 
aspect that's kind of fuzzy with the current "congestion" term.

Let's assume we have a good working congestion control and we have a 
situation in which the congestion control is actually need and does its 
job well. Do we consider this situation as congestion (since congestion 
control is active/needed). Or do we say, this situation would result in 
congestion, if the congestion control were not there?

- If we call this "congestion", a network with zero packet loss and zero 
queuing delay, can still be congested.

- If we say it's no congestion, how do we call a situation that requires 
a congestion control to take action, if not "congestion"?

Similar questions apply for terms like "overload". If the demand is 
larger than the capacity and the congestion control manages this well, 
is there a overload or not?

- If we call it "overload", we can have a overloaded network with zero 
packet loss and zero queuing delay (if we have a really good congestion 
control).

- Otherwise, we would denote the network as not overloaded even though 
the demand might be much higher than the actual throughput (or even output)

Bottom line: How do we factor-in the effects of the congestion control 
feedback loop in our terminology?

---

From: Christian Huitema <[email-masked]>
Date: Fri, 22 Feb 2019 10:07:08 -0800

The term that is well defined is "congestion collapse", which happens when each additional input causes more overload and fewer good output as if the additional input had not been present.

We call algorithms like Reno or Cubic congestion control because one of their functions is to prevent congestion collapse. The basic insight is that when the network appears congested, it is a good idea to slow down, rather than for example  aggressively repeat old packets as pre 88 TCP used to do.

The other function of these algorithms is to adapt the offered load to the available resource, and navigate between two extremes: not slowing enough and cause congestion, versus not accelerating enough and cause resource to go unused. We call that congestion control too for historical reasons.

Then there is the definition of overload and its consequences. One definition of overload is "more load than the network can carry", translating of course in packet losses. But buffering in network queues smoothes the load over time, so there is also a time component there, leading to another characteristic of overload, "long queues in the network that contribute to poor response time" (a.k.a. buffer bloat).

Some of our congestion control algorithms only care about packet losses (Reno, Cubic). Other care about transmission time (BBR) or about both time and losses (Vegas, LEDBAT).

-- Christian Huitema 

> On Feb 22, 2019, at 5:45 AM, Mario Hock <[email-masked]> wrote:
> 
> Thanks for sharing this analogy. I think it just points to another aspect that's kind of fuzzy with the current "congestion" term.
> 
> Let's assume we have a good working congestion control and we have a situation in which the congestion control is actually need and does its job well. Do we consider this situation as congestion (since congestion control is active/needed). Or do we say, this situation would result in congestion, if the congestion control were not there?
> 
> - If we call this "congestion", a network with zero packet loss and zero queuing delay, can still be congested.
> 
> - If we say it's no congestion, how do we call a situation that requires a congestion control to take action, if not "congestion"?
> 
> 
> Similar questions apply for terms like "overload". If the demand is larger than the capacity and the congestion control manages this well, is there a overload or not?
> 
> - If we call it "overload", we can have a overloaded network with zero packet loss and zero queuing delay (if we have a really good congestion control).
> 
> - Otherwise, we would denote the network as not overloaded even though the demand might be much higher than the actual throughput (or even output)
> 
> Bottom line: How do we factor-in the effects of the congestion control feedback loop in our terminology?
>

---

From: "Schwarz Albrecht (ETAS/ESY1)" <[email-masked]>
Date: Sat, 23 Feb 2019 07:52:50 +0000

Mario,

my understanding:
> Or do we say, this situation would result in congestion, if the congestion control were not there?

Because the metric *load* is effectively related to the *offered traffic load*, i.e., the product (let's say A) of the "observed traffic arrival process" and the "expected traffic service process".
A rough rule of thumb for differentiation between overload and congestion was:
a) A > 1 Erlang = Overload
b) 0.x < A < 1 Erlang = Congestion ("in case of queueing effects ... e.g. due to variance")

But that's just a coarse, stationary consideration for the classification between overload and congestion.
Any useful definition needs to take into account the assumption of a time-scale (as the thread points out) in order to cover the non-stationarity effects.

Regards,
Albrecht

-----Original Message-----
From: iccrg <[email-masked]> On Behalf Of Christian Huitema
Sent: 22 February 2019 19:07
To: Mario Hock <[email-masked]>
Cc: [email-masked]
Subject: Re: [iccrg] Very fundamental congestion control questions...

-- Christian Huitema

---

From: Mario Hock <[email-masked]>
Date: Wed, 27 Feb 2019 18:45:25 +0100

Thanks Christian for your explanations. In my understanding "congestion 
collapse" is actually called like this for historical reasons. 
Therefore, I perceive that the term "congestion" should not be defined 
with respect to congestion collapse. Let me explain:

> The term that is well defined is "congestion collapse", which happens when each additional input causes more overload and fewer good output as if the additional input had not been present.

As far as I understand it, congestion collapses essentially happened 
because of bad retransmission timers. The throughput is high (link fully 
utilized) but goodput is low (most packets are duplicates of 
actually/eventually delivered data).

> We call algorithms like Reno or Cubic congestion control because one of their functions is to prevent congestion collapse. The basic insight is that when the network appears congested, it is a good…

BBR impressively shows that aggressively keep sending under high packet 
loss does not necessarily cause congestion collapse. A vital difference 
to pre 88 TCP is that we now have better retransmission timers, SACK, 
etc. Thus, while having lots of retransmissions in certain scenarios, 
BBR mainly retransmits actually lost packets. With respect to the 
bottleneck link this means that throughput is high, but goodput is also 
high. (Of course there is a considerable overhead due to retransmissions 
on non-bottleneck links -- in our experiments around 10% of the goodput 
-- but not on the bottleneck link.)

This basically means, that overload and severe packet loss (let's call 
it "congestion") does not _cause_ a congestion collapse. In pre 88 TCP, 
bad loss recovery mechanisms caused these collapses, triggered by 
"congestion".

> The other function of these algorithms is to adapt the offered load to the available resource, and navigate between two extremes: not slowing enough and cause congestion, versus not accelerating en…

In this scenario my question actually boils down to the question, where 
the offered load is considered. A) Demand of the applications (towards 
the congestion control) or B) Actually sent data (granted to be 
transmitted by the congestion control)

- in variant A: The network would be called "congested" when the 
congestion control has to adapt the offered load to suit the network 
capacity.

- in variant B: The network would be called "congested" when the 
congestion control fails to adapt the offered load to suit the network 
capacity.

Both variants are reasonable. But I think these interpretations are 
sometimes mixed up. Therefore, I would suggest that we need a clear 
terminology to distinguish them.

> Then there is the definition of overload and its consequences. One definition of overload is "more load than the network can carry", translating of course in packet losses. But buffering in network…

Same discussion applies to the term "overload".

- in variant A: "Overload" would translate to "the application is slowed 
down by the congestion control" (if the congestion control works fine).

Your descriptions fits variant B, here. However, in this case, I think 
we would need another term for the situation stated above.

Best, Mario

---

From: Mario Hock <[email-masked]>
Date: Wed, 27 Feb 2019 19:16:22 +0100

Albrecht,

thanks for sharing your understanding. But I'm not sure if I can follow 
you entirely.

> Because the metric *load* is effectively related to the *offered traffic load*, i.e., the product (let's say A) of the "observed traffic arrival process" and the "expected traffic service process".

As I just wrote in my other mail, the question is where do we consider 
this load. Demand of the application to send data or actually 
transmitted data.

> b) 0.x < A < 1 Erlang = Congestion ("in case of queueing effects ... e.g. due to variance")

I would say the task of the congestion control is maintain the situation 
you describe in b). If the application demand already fulfills this 
requirement, congestion control is not really needed. But if the 
application demand is as in (a), i.e., would fulfill your "overload" 
criterion, the offered load has to be limited by the congestion control.

Based on your rule of thumb I understand you as follows: "Congestion is 
when the congestion control needs to limit the application demand to the 
available capacity." And maybe even: "If the congestion control fails to 
do this, it's not congestion anymore but overload." Is that right?

Best, Mario