Subject: [re-ECN] CONEX IESG comments #1

From: Lars Eggert <[email-masked]>
Date: Mon, 1 Mar 2010 07:32:44 -0800

Hi, CONEX group,

I'm going to forward a few issues that were raised during the IESG discussion of the charter to the list, and CC the IESG so that the discussion can happen directly, without me being the bottleneck in the middle.

(I'd like to thank the ADs who wrote up their issues in this way - this will make it much easier to progress the discussion!)

Lars

Begin forwarded message:
> In no particular order...
> 
> 1. This looks like really interesting research work with a potential outcome 
> that could provide substantial benefit. But why is this not proposed as work 
> in the IRTF?
> 
> 2. The charter seems to be trying to say that the work will be Experimental 
> (which sounds like a grand goal), but fails dismally! I see...
>  The output of the CONEX WG are Informational and Experimental
>  documents, apart from work item #2 which is tentatively slated for
>  Standards Track. Depending on the bits that are proposed to be used
>  in the IP header and their semantics (e.g., Bit 48 in IPv4, "ECN
>  Nonce"), there may need to be further Standards Track or Informational
>  documents to reassign those bits from their current definitions
>  (which would be subject to appropriate IESG and IETF review as
>  necessary).
> But #2 is...
>  Specification of IPv4 and IPv6 packet structures to carry CONEX
>  information (header bits, interpretation)
> 
> And, in fact, no other documents listed would be appropriate as 
> Experimental.
> Why are the protocol specifications not also listed as Experimental? If (and 
> this is clearly not known yet because the protocol specs have not been 
> written) it is necessary to use some bits, bytes, or codepoints that 
> currently require standards action, then it would be appropriate to write a 
> standards action document to reassign those fields as experimental. This 
> document would be distinct from the protocol specification.
> 
> 3. Deployment and usage models
> According to the charter, it appears that the deployment and usage work is 
> intended to follow on from the protocol specification and BCPs. I feel that 
> there is considerable lack of clarity about how/where the proposed 
> mechanisms would be deployed. That is, it is unclear whether "...any node 
> along the network path..." is meant to mean what it says.
> - Why is the deployment model not part of the problem spec?
> - Is there an assumption that there is IP forwarding at every hop?
> - Is there an implication that MPLS extensions will also be developed?
> - Can you show some deployment examples?
> 
> 4. ISP support
> Somewhat tied to the previous point is the issue I cannot find supporters in 
> the operational departments of ISPs. I have been trying, because I feel it 
> is important to understand their views. But the most I can get back is "We 
> will watch it in case it goes somewhere." Unfortunately, I also hear a lot 
> of "Our company does not support this work. We believe it will waste 
> industry resources and cause confusion. We do not understand how we could 
> deploy it within our business model, and have no plans to deploy it."
> - Where is the ISP support for this?
> 
> 5. Security
> The charter does give a passing nod to the existence of security and trust 
> issues. These need to be addressed up front. An architecture that relies on 
> ISPs not gaming the system and not lying is critical in this area. The 
> security and veracity of the information shared will also be critical.
> - Where is the security and trust model described?

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 04 Mar 2010 12:27:20 +0000

Phil, Leslie,

As suggested by Phil, I've chopped the IESG from the cc for now, 
while we collate responses. My suggestions for text in response to 
some of the items are below. I'm also keen to see other people's thoughts...

At 15:32 01/03/2010, Lars Eggert wrote:
>Hi, CONEX group,
>
>I'm going to forward a few issues that were raised during the IESG 
>discussion of the charter to the list, and CC the IESG so that the 
>discussion can happen directly, without me being the bottleneck in the middle.
>
>(I'd like to thank the ADs who wrote up their issues in this way - 
>this will make it much easier to progress the discussion!)

Yes - thanks from me too - direct dialogue is much preferable to indirect.

> > that could provide substantial benefit. But why is this not 

[BB]: Research on the protocol is all done. It started in 2003 and 
it's baked enough for standards - at least experimental (see below).
For instance, collected on the re-ECN project home page there's:
- 18 papers on re-ECN, including
   - 4 assessments by other researchers or organisations
     (the protocol went through two evolutions in response to attacks 
proposed by researchers in these papers and more informally on mailing lists)
- 2 implementations plus simulator implementations
- 8 articles from the technical media

Yes, there is research going on the IRTF (ICCRG) on the implications 
for congestion control of ConEx. But the protocol itself is not 
research any more. It is a pre-requisite for a branch of the ICCRG's 
research, hence bringing it the IETF.

> > document would be distinct from the protocol specification.

We should ask for the main protocol docs to be IETF experimental 
track which may require necessary stds track docs to change the 
status of relevant fields in existing RFCs (e.g. from reserved to 
experimental).

Assessment of whether adoption will happen is one of the most common 
uses for the experimental track.
IRTF isn't appropriate, as the protocol has been researched to death.

> > - Can you show some deployment examples?

I know of at least two groups of folks are already working on 
deployment scenarios, and incentives. Both are part of 
industry-academic consortia:
- one hosted by David Clark's group at MIT.
- the other is part of the Trilogy project.

These have both been helped a lot by Matt Mathis's suggestions on this list.

If this milestone was brought forward, I'm sure material created by 
both/either of these groups would be ready in time for inclusion. 
Plus, of course, additional IETF work itself - and any other people 
working in this space that I'm not aware of.

> > - Where is the ISP support for this?

If I were an ISP, I wouldn't necessarily have developed a policy on 
using ConEx at this stage. For comparison (without implying any 
equivalence between Diffserv & ConEx) at the Diffserv BoF in April 
1997 few of the operational people in ISPs had policies on deploying Diffserv.

However, on this list it would be great to see messages of 
interest/support from operational/commercial people in ISPs. I have 
pointed this list at the interest expressed in the assessment of 
ConEx at <http://giic.org/>

> > - Where is the security and trust model described?

There's been a huge amount of work on the integrity of the info 
provided by ConEx / re-ECN. The main doc on this is 
draft-briscoe-tsvwg-re-ecn-tcp-motivation

In addition, I have prepared a comprehensive report giving pseudocode 
of policers/droppers etc, attack models, defences, simulations of the 
defence mechanisms etc. But I still cannot publish it because my 
company's IPR department (still) haven't given me permission to 
publish. Rest assured it will be available for when the w-g needs it tho.

I am also aware of others who are working on their own designs for 
the security elements of a ConEx system.

HTH

Bob

>_______________________________________________
>re-ECN mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/re-ecn

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 9 Mar 2010 14:02:12 +0100

Hi,

see comments inline...

> proposed by researchers in these papers and more informally on mailing
> lists) - 2 implementations plus simulator implementations

Yes, I think so too. Research at least on re-ECN as a starting point for a 
congestion exposure protocol is done. To form a working group which will get 
a protocol ready for standardization would be a good next step at this point. 
Furthermore, I think will should make clear that a conex protocol is a 
signaling protocol. Such a protocol will signal congestion information to 
network component. This will not change any network behavior so far. 
Just a soon as a policer comes into action by e.g. dropping packets or the 
application developers use new congestion control mechanisms the traffic will 
change. 
So from my view the deployment and usage work (see #3) might be a topic for 
IRTF research. But how to do research on this without having a protocol 
and/or framework which provides any kind of congestion accountibility...?

>
> > > 2. The charter seems to be trying to say that the work will be
> >
> > Experimental
> >
> > > standards action document to reassign those fields as experimental.
> > > This document would be distinct from the protocol specification.
>

This is another point for forming a working group. We probably need a standard 
track doc to change the status of the relevant bits. But it should be more 
clear what we want to achieve; Standard track or Experimental? As there has 
been already discussion about this on the list, I have the feeling we do not 
really have an agreement about that on the list, do we?

> > > According to the charter, it appears that the deployment and usage work
> > > is intended to follow on from the protocol specification and BCPs. I
> >
> > feel that
> >
> > > mechanisms would be deployed. That is, it is unclear whether "...any
> > > node along the network path..." is meant to mean what it says.

Maybe we should state clearly why the usage models are not part of the 
charter. (I would say because its a too wide field...) But we might want to 
focus more strongly on the benefits of conex in charter on a higher level. 
This might come along with the points Alissa stated:
- enable 'better' or appropriate congestion control, e.g. LEDBAT
- congestion accountability  between ISPs

And there is one further point we did not talk about so far:
re-ECN will enable the use of ECN. Today no one wants to use ECN because 
(beside that it might no work at all) you will have a disadvantage against 
someone not using ECN. Or say it the other way around; you might have an 
advantage not using ECN. re-ECN will creating an advantage for using ECN.

>
> > > 4. ISP support
> > > Somewhat tied to the previous point is the issue I cannot find
> >
> > supporters in
> >
> > > the operational departments of ISPs. I have been trying, because I feel
> > > it is important to understand their views. But the most I can get back
> > > is "We will watch it in case it goes somewhere." Unfortunately, I also
> > > hear a lot of "Our company does not support this work. We believe it
> > > will waste industry resources and cause confusion. We do not understand
> > > how we could deploy it within our business model, and have no plans to
> > > deploy it." - Where is the ISP support for this?
>

Probably ISPs are more interested in the deployment and usage models of conex. 
As we separated this from the protocol, I can imagined that the benefits of 
conex are not clear. This seems to make a lot of people conex automatically 
connected to congestion charging/pricing. I still think we should explicit 
try to break this link. I'm still not sure how to do this. Maybe enumerate 
the benefits of conex explicit as basically suggested by Alissa. (I wouldn't 
wanne talk about 'selling points'... ;-) )

>
> > > 5. Security
> > > The charter does give a passing nod to the existence of security and
> > > trust issues. These need to be addressed up front. An architecture that
> >
> > relies on
> >

As a conex protocol is a signaling protocol it will not change any traffic 
characteristics. Many effects I have seen when investigating re-ECN are based 
on ECN or RED mechanism. When using the conex information you have to be 
aware of the mechanisms the conex protocol will be based on and you have to 
consider influences of other inter-working protocol. But enabling conex or 
re-ECN alone in the endsystem will not change the traffic.

Another point is to make sure the conex information is declared honestly. 
Therefore you might need a dropper or something in the network or at network 
egress. This elements needs to considered when designing a conex protocol. At 
least requirements on such a element need to be phrased, from my point of 
view.

However, policer design will be part of the deployment cases. We should not 
mention the policer in respect to any security analysis.

BR,
Mirja

---

From: <[email-masked]>
Date: Tue, 16 Mar 2010 10:26:58 -0000

Some comments in-line from my personal perspective.
And thanks to the IESG for their comments
phil

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Lars Eggert
> Sent: 01 March 2010 15:33
> To: [email-masked] list
> Cc: The IESG
> Subject: [re-ECN] CONEX IESG comments #1
> 
> 
> Hi, CONEX group,
> 
> I'm going to forward a few issues that were raised during the 
> IESG discussion of the charter to the list, and CC the IESG 
> so that the discussion can happen directly, without me being 
> the bottleneck in the middle.
> 
> (I'd like to thank the ADs who wrote up their issues in this 
> way - this will make it much easier to progress the discussion!)
> 
> Lars
> 
> Begin forwarded message:
> > 1. This looks like really interesting research work with a 
> potential 
> > outcome

I agree with others that enough research has been done. I don't think
further research in the IRTF would help. Personally I think "it's now or
never" - either there's consensus that there's enough motivation and
momentum to form an IETF WG, or there isn't. 

> > document would be distinct from the protocol specification.

Several points here:
The draft charter lists items as
- INFO, for #1 (Applicability statement) and #4 (Use cases)
- STD, for #2 (spec of v4 & v6 packet structure)
- EXPT, for #3 (spec of TCP mod for transporting cong info from
destination to sender; and spec of one mechanism to ensure
trustworthiness of conex info)

So the items under #3 are listed as Experimental.

The packet structure (#2) is listed as Standards track. To me redefining
the IP header feels like standards track rather than experimental
(compare DSCP definition in rfc2474). But if everyone else is happy with
Experimental, then that's ok with me. (Side-note: a question is how to
confine the experiment. The re-ecn proposal seems to naturally
self-limit since bit48 should be ignored by non-aware routers [and some
preliminary expts showed this was ~ true].)

> > - Can you show some deployment examples?

Several points:

The deployment and usage work was intended to be in parallel with the
protocol spec. The final Milestone for Use cases is a little later, to
allow capture of the "results" of the experiments that use conex info -
we figured that the results would be the last thing to be completed. But
I agree this should be clearer. 

"Problem spec" = ? There isn't a Milestone on a problem statement. Work
item #1 is an applicability statement: "including its architectural
features, limitations and assumptions..." - so it's nearer to an
Architecture doc that a Problem statement. 
[Personally I think that the purpose of problem statement docs is to
motivate the work and setting up of a WG, but they tend to be a
distraction within a WG.]

The draft charter limits the scope by assuming IP forwarding at every
hop. As Bob detailed, MPLS nodes can already ECN-mark pkts, using
RFC5129, or drop them; but it would need work beyond the charter if MPLS
nodes were to see the rest-of-path (or whole path) congestion.
Conclusion: you're right, the text should be corrected: "...any <IP>
node along the network path..."

Finally, (to repeat a comment I made earlier on the ML) I'd distinguish
two things:

- the conex "building block" - the ability for any node to see the
rest-of-path congestion (& the whole path congestion)
- the solutions to various aspects of 'congestion mgt' that can be built
using the conex "building block"

The conex draft charter proposes work on the first of these (ie to
standardise the IP header bits so that whole-path congestion info is
exposed (visible) to all nodes on the path, and to define a TCP-based
mechanism that allows a TCP destination to tell the source the
whole-path info, & hence the IP header can be set by the source)

The conex draft charter only proposes a little work on the second of
these. This work item is to encourage experiments on some "use cases"
and document about them. I guess the "use cases" would be tests towards
a solution (to some aspect of congestion mgt) that might get deployed by
an operator. 

We should be clear: the charter doesn't propose to standardise a full
solution that would get deployed. Rather it standardises a capability
which would be used to build a solution. 

> > trust

If I get the question right:
- item #1 includes describing the threats [= trust model?] 
- item #3 includes work to ensure trustworthiness of the conex info

Best wishes
phil

---

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 18 Mar 2010 02:31:03 -0400

I definitely need to comment on "ISP support".

Please note that I am expressing a personal opinion here, not any
official Comcast position.

> supporters in the operational departments of ISPs. I have been trying,

> because I feel it is important to understand their views. But the most

> I can get back is "We will watch it in case it goes somewhere." 
> Unfortunately, I also hear a lot of "Our company does not support this

> work. We believe it will waste industry resources and cause confusion.

> We do not understand how we could deploy it within our business model,
and have no plans to deploy it."
> - Where is the ISP support for this?

In the IESG comments, I see three different topics intertwined: (1) how
would an ISP leverage Conex, (2) when would an ISP roll out Conex
technologies ("we will watch it in case it goes somewhere"), and (3) how
would Conex fit within the current ISP business model or change it ("we
do not understand how we could deploy it within our business model").

Let me start with 'how an ISP could leverage Conex'. I believe there are
two potential use cases that could apply to our current network: using
congestion volume instead of absolute volume as input to a congestion
management system (aka FairShare), and discovering and diagnosing
congestion issues outside of the broadband access network. In the first
use case, the intent is to create an incentive for applications to use
LEDBAT congestion control through the (future) behavior of congestion
management. In the second use case, it would be helpful to capture
information about congestion conditions that occur on our backbone
(particularly our interconnects with other backbones), in other ISP
networks (as they relate to our customers' traffic), and perhaps in
remote data centers belonging to content and application providers. Note
that it is tricky enough to diagnose application performance issues as
they occur, and almost impossible to diagnose after the fact
(particularly when it is off-network). There may be additional use cases
like DDOS mitigation, which require further investigation. (Note that I
did not mention 'congestion pricing' here on purpose -- see below.)

Next let me tackle 'when would an ISP roll out Conex'. This is a
difficult question to answer directly. Assuming that Conex becomes an
IETF standards activity, there are two Conex ecosystems that need to be
created: a software implementation ecosystem (operating systems,
appliance firmware, network elements including routers and broadband
access equipment, network management systems, etc), and a network
deployment ecosystem (ISPs, application and content providers, consumer
networks including home gateways perhaps, etc). If Conex relies on ECN
(one of the current proposals), then the network deployment ecosystem
needs to address ECN network deployment as well. Participants in both
ecosystems will need their own business justifications about how they
benefit from Conex implementation and deployment, of course, and there
need to be prototypes, lab testing, and trial deployments before full
Conex production. This could easily take 2-3 years. On a related note,
I'm not sure that further academic research is going to speed up the
timeline.

Finally, let me tackle 'how Conex relates to the current ISP business
model'. Note that my use cases above are strongly correlated to my
employer's current business model. I also know that several folks have
proposed using Conex for 'congestion pricing', and there are some solid
technical and economic arguments for why the ISP business model should
embrace congestion pricing, e.g. aligning 'cost' allocations to
infrastructure capacity upgrades. But I still have a few reasons to
hesitate before embracing this model (personally). While volume
accounting is fairly straightforward as an engineer, many ISPs still
find it difficult to communicate to customers how application usage
translates to volume. (As an ironic example, I can explain how the
Comcast Usage Meter works, but I have no idea why my own January
household usage was three times higher than February's. Except I have
teenage kids. :) I don't know how to start to communicate what
congestion volume and pricing means to customers and their service.
There are also potentially consumer policy and regulatory concerns. As
an example, the typical peak usage time for broadband access providers
also happens to be 'prime time' for TV viewing, so some might wonder
what impact congestion pricing will have on Internet video distribution
(even if that has nothing to do with the intent).

I can discuss this during the Conex session next week.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Lars Eggert
Sent: Monday, March 01, 2010 10:33 AM
To: [email-masked] list
Cc: The IESG
Subject: [re-ECN] CONEX IESG comments #1

Hi, CONEX group,

I'm going to forward a few issues that were raised during the IESG
discussion of the charter to the list, and CC the IESG so that the
discussion can happen directly, without me being the bottleneck in the
middle.

(I'd like to thank the ADs who wrote up their issues in this way - this
will make it much easier to progress the discussion!)

Lars

Begin forwarded message:
> 1. This looks like really interesting research work with a potential 
> outcome that could provide substantial benefit. But why is this not 
> proposed as work in the IRTF?
> 
> Experimental (which sounds like a grand goal), but fails dismally! I
see...

> Why are the protocol specifications not also listed as Experimental? 
> If (and this is clearly not known yet because the protocol specs have 
> not been
> currently require standards action, then it would be appropriate to 
> write a standards action document to reassign those fields as 
> experimental. This document would be distinct from the protocol
specification.
> According to the charter, it appears that the deployment and usage 
> work is intended to follow on from the protocol specification and 
> BCPs. I feel that there is considerable lack of clarity about 
> how/where the proposed mechanisms would be deployed. That is, it is 
> unclear whether "...any node along the network path..." is meant to
mean what it says.
> supporters in the operational departments of ISPs. I have been trying,

> because I feel it is important to understand their views. But the most

> I can get back is "We will watch it in case it goes somewhere." 
> Unfortunately, I also hear a lot of "Our company does not support this

> work. We believe it will waste industry resources and cause confusion.

> We do not understand how we could deploy it within our business model,
and have no plans to deploy it."
> trust issues. These need to be addressed up front. An architecture 
> that relies on ISPs not gaming the system and not lying is critical in

> this area. The security and veracity of the information shared will
also be critical.