[rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper

Robin Whittle <rw@firstpr.com.au> Fri, 12 February 2010 12:12 UTC

Return-Path: <rw@firstpr.com.au>
X-Original-To: rrg@core3.amsl.com
Delivered-To: rrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 81B0B3A7834 for <rrg@core3.amsl.com>; Fri, 12 Feb 2010 04:12:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.416
X-Spam-Level:
X-Spam-Status: No, score=-1.416 tagged_above=-999 required=5 tests=[AWL=-0.121, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, J_CHICKENPOX_63=0.6]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tVaKP+Ui-4y5 for <rrg@core3.amsl.com>; Fri, 12 Feb 2010 04:12:06 -0800 (PST)
Received: from gair.firstpr.com.au (gair.firstpr.com.au [150.101.162.123]) by core3.amsl.com (Postfix) with ESMTP id EEFC33A782F for <rrg@irtf.org>; Fri, 12 Feb 2010 04:12:03 -0800 (PST)
Received: from [10.0.0.6] (wira.firstpr.com.au [10.0.0.6]) by gair.firstpr.com.au (Postfix) with ESMTP id E381E175B44; Fri, 12 Feb 2010 23:13:20 +1100 (EST)
Message-ID: <4B7545E5.2070909@firstpr.com.au>
Date: Fri, 12 Feb 2010 23:13:25 +1100
From: Robin Whittle <rw@firstpr.com.au>
Organization: First Principles
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
MIME-Version: 1.0
To: RRG <rrg@irtf.org>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 8bit
Cc: Michael R Meisel <meisel@cs.ucla.edu>, Beichuan Zhang <bzhang@cs.arizona.edu>
Subject: [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper
X-BeenThere: rrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IRTF Routing Research Group <rrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/rrg>
List-Post: <mailto:rrg@irtf.org>
List-Help: <mailto:rrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Feb 2010 12:12:10 -0000

Short version:   I believe the GLI-Split designers were mistaken to
                 class their proposal as a Core-Edge Separation (CES)
                 architecture.  I am sure it is a CEE (Core-Edge
                 Elimination) architecture.

                 Comments on Lixia's and colleagues' 2008 paper
                 on Core-Edge Separation vs. Core-Edge Elimination.

                 I think they were mistaken to class GSE as a
                 Core-Edge Separation architecture.

                 I think this paper refers generally to the
                 core-edge "separation" being of the routing
                 systems of the edge networks from the core - but
                 the real distinguishing thing about CES is the
                 separation of one subset of global unicast address
                 space into a "edge" subset for scalably supporting
                 end-user networks with multihoming etc. from the
                 remaining "core" addresses.

                 The authors were in general the designers of APT,
                 which separated not just the address space into
                 two subsets, but had a long-term goal of actually
                 separating all the edge networks from the core, so
                 no edge host could physically send a packet to
                 any core address.  AFAIK, no other CES architecture
                 is intended to separate the routing systems like
                 this.

                 I think this double sense of the term "separate"
                 plus the mistaken classification of GSE as CES
                 contributed to the GLI-Split authors considering
                 their proposal to be a CES architecture too.  Yet
                 it is clearly CEE, since it implements the
                 Locator/Identification Separation naming model.

                 These terminological and classification problems
                 are compounded by LISP being inappropriately named -
                 since it is a CES architecture which does not
                 implement Locator/Identifier Separation.

                 Six/One Router is a translation-based CES
                 architecture.

                 GSE and GLI-Split are translation-based CEE
                 architectures.  They are unusual among CEE
                 architectures in that they only require the
                 new router functions and changes to the stack -
                 they do not require applications to be modified.
                 (However I just learnt that GLI-Split could
                 support new application functionality via a new
                 API.)

                 Comparison charts of:

                 CES: LISP-ALT/NERD, APT, Ivip, TRRP, TIDR,
                      IRON-RANGER and Six/One Router.

                 CEE: GSE, GLI-Split, ILNP, Name-Based Sockets.


Here is some discussion further to my understanding of the
distinctions between Core-Edge Elimination (CEE) architectures and
Core-Edge Separation (CES) architectures:

  CES & CEE are completely different (graphs)
  http://www.ietf.org/mail-archive/web/rrg/current/msg05865.html

This paper:

  Towards a Future Internet Architecture: Arguments for Separating
  Edges from Transit Core
  Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang
  http://conferences.sigcomm.org/hotnets/2008/papers/18.pdf

is really important to the RRG discussions, because it formalises the
architectural terminology for two important classes of scalable
routing solution.  To see my attempt to trace the origins of these
concepts and terms, please see:

  Re: [rrg] Terminology
  http://www.ietf.org/mail-archive/web/rrg/current/msg05966.html

I support this paper, but have some comments to make:

  1 - The definition of exactly what core and edge things are
      being separated or eliminated could be improved - to
      refer to separation into two subsets of addresses.

  2 - The suggestion that GSE is a CES architecture is mistaken.

  3 - The paper doesn't mention what I consider to be the biggest
      objection to CEE: that it implements the Locator/Identifier
      Separation naming model which will generally slow down the
      establishment of communication sessions and place extra
      burdens on all hosts - especially those on slow, unreliable
      wireless links.


The paper refers in general to:

    "separating edge networks from the transit core"

but really both CEE and CES maintain the already existent separation
between edge networks from the transit core.  (See discussion of
section 4.3 below for my understanding of why the authors may have
emphasised the separation of networks rather than the separation of a
new subset of scalable "edge" addresses from the remaining "core"
addresses.  The intended APT to ultimately separate the two systems
so much that an edge host would be physically unable to send a packet
to a core router.)

What is being separated in a CES architecture is the "edge" subset of
addresses from the remainder of the global unicast addresses, which
is then known as "core" addresses.  This is a different separate
concept from "separating edge networks from the core", since the
networks are already separated.  Its just that at present, these PI
prefix-using End User Networks (EUNs) use address space which all
core (DFZ) routers need to be concerned with.

>From the abstract, it can be seen that the CES definition is not the
opposite of the CEE definition:

CES:    The first direction, which we dub separation, calls for
        separating edge networks from the transit core, and
        engineering a control and management layer in between.

CEE:    The other direction, which we dub elimination, calls for edge
        networks to adopt multiple provider-assigned addresses to
        enable provider-based address aggregation.

(See below for discussion of the paper's more formal description of CEE.)

In fact, the CEE definition could reasonably include CES
architectures, since CES architectures generally use multiple ETRs,
each on a PA ("provider-assigned") address, for the purposes of
enabling the DFZ to only handle prefixes which are subject to
"provider-based address aggregation".

The most important architectural distinction between CES and CEE is:

  CES involves end-user networks' hosts using global unicast
  addresses from a specially handled "edge" subset of the global
  unicast address space, with the remainder being known as the
  "core".  IP addresses in both subsets will continue to be used, as
  are all today, both for Identifying hosts and for the Routing
  system to decide how to forward packets (i.e. "Locator).  But note
  that CES has a new, fancy, way of doing this for this subset of
  "edge" addresses - different from today's plain DFZ approach -
  which is highly scalable and allows these "edge" addresses to be
  used by EUNs for portability, multihoming inbound TE and perhaps
  for mobility.

  CEE involves end-user networks' hosts using something other
  than a global unicast IP address to Identify themselves.  Thus
  CEE implements the Locator/Identifier Separation naming model.
  Applications use this new Identifier, which is in a different
  namespace to IP addresses and any Locators used in the system,
  to identify each host.  Hosts can be reached via one or more
  potentially unstable Locators, and the applications are unaffected
  due to their reliance purely on Identifiers to specify which hosts
  are communicating.

CES preserves the existing IP naming model:

       Role             Level

       Text name  <---- FQDN
       Identifier <---] IP address
       Locator    <---]

while CEE creates one of several models, in which the Locator and
Identifier roles are played by different levels - different objects
in different namespaces.

CEE architectures always implement Locator/Identifier Separation.

No CES architecture implements Locator/Identifier Separation.  What
they separate is two subsets of the global unicast address space:
"edge" from the remaining "core".  Both types of address are used
both as Locators and as Identifiers.

Both subsets are used as Identifiers by all hosts.  Hosts make no
distinction between addresses in the two subsets.

Most routers make no distinction between how they handle packets with
destination addresses in these two subsets.  The exception is ITRs,
which make a big distinction.  ITRs implement the Locator
functionality of "edge" addresses in a totally different way: by
looking up mapping and tunneling to an ETR, rather than by forwarding
the packet to a neighbour.

More on naming models:

  Today's "IP addr. = ID = Loc" naming model should be retained
  http://www.ietf.org/mail-archive/web/rrg/current/msg05864.html

CES architectures never require changes to host stacks or applications.

Many CEE architectures require changes to both stacks and apps, but
at least two - GLI-Split and I think GSE - only require changes to
the stacks.


Here are some cases:

CES:  LISP, APT, Ivip, TRRP, TIDR and IRON-RANGER.

      There's no debate about this - they all create a special
      "edge" subset of the global unicast address space for EUNs
      to use for portability, multihoming, TE and perhaps mobility
      - and add things to the routing system to scalably get packets
      addressed to "edge" addresses to the destination networks
      without excessive burdens on the DFZ control plane.

      Except for Ivip's long-term "Modified Header Forwarding" modes,
      all these use encapsulation in order to tunnel packets from
      an ITR or ITR-like device to an ETR or ETR-like device.  This
      raises some thorny PMTUD problems.

      These CES architectures are all intended to work for both IPv4
      and IPv6.  The descend directly or indirectly from the original
      map-and-encaps architecture.  No host stack or application
      changes are required.  Things are added to the routing system -
      but in general routers stay the same.


CES:  Six/One Router - by Christian Vogt, though AFAIK it is no
      longer under development.

http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf

      This is clearly a CES architecture, because the end-user
      networks operate from a single prefix drawn from an "edge"
      subset of the global unicast address range.  (Sect. 2.1.)

         Figure 1 illustrates this setup and addressing for an
         edge network that is multi-homed with two providers.  This
         edge network has one border link to each provider, so two
         Six/One routers are in use. The edge addresses are from the
                                         ----
         prefix ABC::/64. The Six/One router on the border link to
         provider 1 translates between those and transit addresses
         from the prefix 1000::/64; the Six/One router on the border
         link to provider 2 translates between the same edge
         addresses and transit addresses from the prefix 2000::/64.

      A host HA in one Six/One Router network, addresses packets to
      a host HB in another Six/One Router network another using the
      same IP address in the destination field as HB uses as its
      own "edge" address.

         Edge addresses are for use within this and other upgraded
         edge networks.  They are globally unique, but do not appear
                                  ---------------
         in the global routing table because they cannot be
         efficiently aggregated.

      All hosts in Six/One routers use a form of DNS which returns
      these "edge" addresses for any host in any Six/One Router
      network.  So this is a genuine scalable "edge" subset of the
      global unicast address space.

      However, because Six/One Router has no equivalent to DITRs
      (Ivip) or PTRs (LISP), hosts in ordinary IPv6 networks can't
      send packets to the Six/One Router hosts using these scalable
      "edge" addresses.   So, from a non-Six/One Router network, a
      DNS lookup on the same FQDN of host HB will not return its
      "edge" address, but one or more another addresses, based on
      one or more addresses within the PA prefixes HB's network
      obtains from its one or more ISPs.  The address-rewriting
      router at the border of HB's network will rewrite the
      destination address of a  packet coming from some ordinary IPv6
      host HX, to the "edge" address of HB.

      This is explained in section 2.4.  No other CES architecture
      has this "split-horizon" (I think this is the right term) DNS
      arrangement.  I guess it was impossible to implement an
      equivalent of DITRs and PTRs.

      The hosts in Six/One Router behave as usual.  There are no
      stack or application changes.  The naming model used in the
      packets they send and receive is the same as with today's IPv4
      and IPv6, as illustrated above.

      Rather than tunnel packets across the DFZ with encapsulation or
      Modified Header Forwarding, Six/One Router uses address
      rewriting (AKA address translation) at EUN BRs or perhaps
      in a router in their ISP(s).

      Unfortunately, Six/One Router can't be used for IPv4 due
      (I guess) to the more chaotic EUN prefix lengths and (I am
      sure) due to the need for a 2 ISP multihomed EUN to consume 3
      times the amount of global unicast address space it really
      needs.  (One for its "edge" prefix and then the same amount
      of PA space from each of its ISPs.)  Also, when Christian and
      I last discussed it on the RRG, he wrote that he needed a
      flag bit from the IPv6 header, which would require a modified
      IPv6 header format and so require updates to all DFZ and many
      other routers.



CEE: GSE - Mike O'Dell, 1997

       http://ietfreport.isoc.org/idref/draft-ietf-ipngwg-gseaddr/

     This is an extension of his earlier 8+8 architecture - both are
     for IPv6 only.

     GSE is clearly a CEE architecture because it has separate
     objects for Identifying hosts and for Locating them.  There is
     no separate "edge" subset of the global unicast address space by
     which hosts in EUNs identify themselves.  The Identifier is the
     64 bit ESD.

     There needs to be new things in the network: an address
     rewriting function in the BRs of EUNs which adopt GSE.

     Host stacks needs to be somewhat modified, but I think this
     is mainly so header checksums are computed only on the ESD
     and not on other parts of the IPv6 address.

     As far as I know, applications do not need to be modified.

     Multihomed EUNs get a prefix from each of their two or more
     ISPs, and the identity of hosts is unrelated to this, or to
     any IP address.  These prefixes are PA, and the ISP may have
     a single large (short) prefix which it splits into many such
     prefixes for many such EUNs.  Host Identity - and so all
     communications establishment and session continuity - is
     determined by the value each host uses for the 64 bit ESD,
     which exists in a separate namespace from IPv6 addresses.


CEE: GLI-Split

http://www3.informatik.uni-wuerzburg.de/~menth/Publications/papers/Menth-GLI-Split.pdf


     I believe GLI-Split is a CEE architecture, because it introduces
     a separate "ID" object (Figure 2), with a separate namespace
     from IPv6 addresses or anything else, to Identify each host
     - which is different from the objects used to Locate them: the
     GL or LL Locators.

     The fact that both objects are combined with other things to
     form an IPv6 address does not affect the fact that these objects
     for Identifier and Locator are independent, and in separate
     namespaces, just as they are in ILNP, GSE and the other CEE
     architectures.

     GLI-Split implements Locator/Identifier Separation, as do all
     other CEE architectures.  From the Abstract:

         It implements the locator/identifier split and makes routing
         in the core of the Internet more scalable.

    More on GLI-Split at the end of this message.  The authors
    explicitly state that it is a CES architecture, but I am sure
    it is a CEE architecture.  I think they are following a mistake
    made which I believe was made in the the 2008 sep./elim. paper.


CEE: ILNP - Ran Atkinson.

     This is clearly a CEE proposal, because it has separate levels
     for Identifier and Locator.  It is a purely host-based
     descendent of GSE.

     There is no need for new routers, but the host stacks and
     applications all need to be upgraded, because the applications
     use Identifiers and Locators separately via the API to the
     stack (or do the applications use only Identifiers?).

     As with GSE and all other CEE architectures, a multihomed
     network uses two or more PA prefixes from its two or more
     ISPs, and the PA nature of those prefixes means each ISP can
     supply many such prefixes from a single prefix the ISP
     advertises in the DFZ.


CEE: Name Based Sockets - Christian Vogt.

     This is a CEE architecture because the Identity of each host
     is a FQDN and its one or more Locators are IPv6 addresses,
     again drawn from PA prefixes of the EUNs one or more ISPs.

     Name Based Sockets is unusual for a CEE architecture in that
     it appears to have a two-level naming model.  However, I am
     not sure how it could have this and still be able to support
     DNS lookups leading to multiple separate hosts.  I hope
     Christian will clarify this.


CEE? RANGI and Name Overlay Service

     I am yet to read these properly, but my impression is that they
     are both CEE architectures.


???: hIPv4 - Patrick Frejborg

     I think this was an attempt to create a new, larger, address
     space while enabling scalable routing and transport of the
     new format packets across the DFZ.  The ELOC part of the
     new address is not globally unique. (If it was, it would be
     an Identifier.)  The combination of the ALOC and ELOC provides
     the extended equivalent of an IP address today - meaning both
     a host Identifier and including a routing locator within the
     one object.

     I previously thought it was a CEE but I think it is neither
     CEE or CES.  Unfortunately, hIPv4 is not practical since DFZ
     routers are not ready to handle packets with the required
     option header.


???: Aggregation with Increasing Scopes.

     I don't yet understand this.



Here are some charts:

CES            IPv4   IPv6   Tunneling  Local            Mapping
architecture                            full-            speed
                                        database
                                        mapping
                                        servers?

LISP-NERD        v4     v6   Encaps.    ITRs are full    Slow
                                        database

LISP-ALT         v4     v6   Encaps.    No               Global
                                                         lookup

ALT              v4     v6   Encaps.    Yes              Slow

Ivip             v4     v6   Encaps.    Yes              Fast
                             or MHF

TRRP             v4     v6   Encaps.    No               Global DNS
                                                         lookup

TIDR             v4     v6   Encaps.    ITR-like         Slow - via
                                        DFZ routers      DFZ control
                                        have full DB.    plane.

IRON-RANGER      v4     v6   Encaps.    (Neither question applies
                                         directly to IRON-RANGER.)

Six/One Router          v6   Address    (Six/One Router's mapping

                             rewriting   system was never specified.)




CEE            IPv4   IPv6   App       Stack      Router
                             changes?  changes?   changes?

GSE                     v6   -         Stack      Router rewrites

GLI-Split               v6   -*        Stack      Router rewrites

ILNP                    v6   App       Stack      -

Name Based
Sockets                 v6   App       Stack      -


* GLI-Split does not require host changes, but Michael Menth just
  wrote to me offlist that new applications could be used, for
  instance for multipath protocols, if the stack had a suitable
  new API:

     "Applications usually communicate with identifier addresses,
      but also a more advanced API must be offered in addition to
      allow active multipath routing through applications or at
      least by transport layer protocols.

  He indicated that this was not in the current documentation but
  would be added in the future.



Back to the 2008 paper: on page 2, 2nd last paragraph:

     There are also other types of separation
     solutions besides Map & Encap. For example,
     Six-One Router [23] and GSE [19] use address
     rewriting, which rewrites the packet header
     to include information about the
     destination’s attachment point to the transit
     core.

For the reasons listed above, I believe GSE is a CEE architecture.

The fact that the CES architecture Six/One Router has routers
rewriting addresses does not mean that GSE is a CES architecture
simply because it too has routers rewriting addresses.


In Section 3, there is a more formal description of CEE:

     In order to achieve routing scalability, the
     elimination approach enforces provider-based
     address aggregation by eliminating all PI
     prefixes and de-aggregated PA prefixes.

I wouldn't state it so strongly.  I think it is more accurate to say
that when all hosts in all networks adopt CEE, the CEE mechanisms can
be relied upon entirely for portability, multihoming and inbound TE.
 Then, and only then, this means that no network needs PI space to
achieve these things.  This doesn't mean that PI space (the current,
unscalable, form of "edge" space) would automatically be eliminated -
just that it could be eliminated without any network missing out on
portability, multihoming or inbound TE for all its hosts and
communications.

     Each multihomed edge network will receive
     from each of its providers an address block out
     of a larger, aggregated block announced by the
     provider.

I would rephrase it slightly.  Each EUN which adopts CEE would
generally use space like this.  Each such EUN *could* use PI space.
Its just that PI space is no better than PA space for use with a CEE
architecture.  (Likewise a EUN using a prefix it gets as PA space
from one ISP via another ISP as a "more specific PA prefix" - which
also has the effect of adding another prefix to the DFZ control plane.)

The degree to which they can multihome etc. depends on how many of
the hosts they are communicating with are also using the CEE
architecture, as noted in 4.1.

     The multihomed site does not inject PI prefixes
     or more specific PA prefixes into the routing
     system. Instead, each host in a multihomed site
     is given multiple PA addresses. For example, as
     shown in Figure 2, the host obtains two addresses,
     one from each of its network’s ISPs.

Typically, this is what would happen.

The "elimination" in this description refers to the elimination of a
class of address space, the current unscalable "edge" subset which is
primarily PI prefixes and also, to some extent (I don't know how
much) these "more specific PA prefixes".

The paper's initial use of "separation" was not in terms of address
space, but in terms of separating edge *networks* from the core.
Edge networks are always separate from the core.

I think that a more correct use of "separation" would be to refer to
the deliberate creation of a subset of "edge" space which can provide
what EUNs need, but in a scalable manner.  Then, there will be more
EUNs getting the portability, multihoming and inbound TE they need,
with less and less of them using PI prefixes or "more specific PA
prefixes" to get it - which are also "edge" uses of address space,
but the kind we are trying to discourage because they are unscalable.


In section 4.3, I think I see why the authors refer more to
separating edge networks from the core.  I think what they are
referring to is the end-point of APT adoption:

  1 - All EUNs use APT, and so use the scalable "edge" subset of
      the global unicast address space.

  2 - Therefore, no EUN needs PI space or "more specific PA
      prefixes".  So these are all converted either to the new
      scalable "edge" subset of space or to the remainder -
      the "core" subset.

  3 - Therefore, ISPs advertise prefixes into the DFZ which are
      all in the "core" subset.

  4 - Because all ISPs and EUNs use APT, there is no need for
      APT's equivalent of LISP's Proxy Tunnel Routers or Ivip's
      DITRs.  (APT's arrangement was never well documented, but
      it too could provide full support for packets sent from
      non-upgraded networks.)  Therefore, there is no need to
      advertise any "edge" space in the DFZ.

  5 - Now, if it was desired - as the APT designers desired -
      it would be possible, in theory, to prevent any host
      in an EUN from being able to send a packet to any host
      or router on a "core" address.  Hosts in EUNs only need
      to communicate with other hosts in EUNs.  (Traceroute
      and ping wouldn't concern DFZ routers, since all packets
      between EUN hosts would be tunneled across the DFZ.)

Most of the APT designers wrote this 2008 paper.

AFAIK, no other CES architecture had this complete, physical,
separation of EUN hosts from being able to send or receive packets to
or from "core" addresses as a goal.  I specify it as a non-goal for Ivip.

>From section 4.3:

   Separating edges from the transit core provides
   additional features that are sorely missing in
   today’s Internet.  With separation, an end host
   can send packets through the transit core, but
   can no longer address a packet to any specific
   device inside the transit core. Although the
   separation does not eliminate any specific
   security threat, it raises the bar against
   malicious attacks targeted at the global routing
   infrastructure. In addition, the mapping layer
   between edge and core networks can serve as a
   mounting point for badly-needed control and
   protection mechanisms, and can also act as a
   cushion layer between the edge and core,
   allowing each side to deploy innovations without
   any involvement of the other side. We now
   elaborate on each of these benefits.

They go on to discuss "Rolling out new protocols." and "DDoS
mitigation." which they argue would be possible or easier only with
complete separation of the "edge" networks from the "core" networks.

I don't agree with these benefits, and I doubt it would be feasible
to achieve such complete separation.  I won't mention why, because
that is not the point of this discussion.  I am trying to explain why
the authors of this important paper seemed to equated, in my terms:

   Separation of a subset of the global unicast space into a
   scalable "edge" subset.

with:

   Separation of the "edge" networks so their hosts couldn't send
   packets to, or receive them from, devices in the "core".

They also mention a third argument for this "separation": "Ingress
traffic engineering.".  I have two comments about this.

Firstly, inbound TE works fine due to any amount of adoption of the
CES architecture (that is separation of the scalable "edge" addresses
into a subset) and does not depend on the complete separation of
"edge" from "core" networks, which is what this section 4.3 concerns.
 As long as the CES architecture supports all packets sent from
non-upgraded networks, inbound TE will be for all the adopting
networks incoming packets, no matter how many other networks have
so-far adopted the CES architecture.

Secondly, what they write in the second paragraph about Site1 having
preferences for which of Site2's two or more ETRs to use doesn't
strike me as part of most CES architectures.  I think the aim of most
CES architectures is to to have the recipient site control which of
its ISPs the packets arrive via.  This is inbound TE.  Maybe APT had
some concept of the sending EUN controlling this, but I guess most
sending EUN wouldn't care which ISP the packets went to.


I think the discussion in section 5 about multipath being used with
CES is not really about multipath as some other people conceive of
it.  My understanding of true multipath, such as MPTCP, is the
sending host (or perhaps a router)and perhaps the receiving host (or
its router?) being able to choose which of multiple ISPs in the
sending host's network packets may go out from, at the same time as
choosing which of multiple ISPs in the recipient host's network the
packets will travel through.   With CES, I don't see how a sending
host could affect either.

The routers in the sending host's network can choose which ISP link
to send out the packets on (assuming the ISPs accept packets with
"edge" addresses, which they will when CES is widely used).  If those
routers are ITRs, then in some or many CES architectures (but not
Ivip) then perhaps the ITR might be locally programmed to prefer one
of multiple destination ETRs over others - so in principle the ITRs
in the sending network could affect the incoming load balance of the
destination network.  However, as far as I know, the intention with
all CES architectures is to allow the sending network to choose the
outgoing ISP and the recipient network to choose (inbound TE) its
incoming ISP(s).  In the case of Ivip, the recipient network only
provides a single ETR address, so there is no option for the sending
network to have its ITRs tunnel traffic to some other ETR of the
destination network.


I think this paper doesn't mention what I regard as the greatest
objection to CEE architectures - their adoption of the
Locator/Identifier Separation naming model, as I discuss in (msg05864).


I the GLI-Split authors were mistaken to state that GLI-Split is a
CES architecture.

The summary:

http://tools.ietf.org/html/draft-irtf-rrg-recommendation-04#section-10.1

includes text indicating that perhaps the authors consider GLI-Slit
to be a CES architecture:

    GLI-Split implements a separation between global routing
    (in the global Internet outside edge networks) and local
    routing (inside edge networks) and using global and local
    locators (GLs, LLs).

Yes, but all EUNs have separate routing from the DFZ - so this is not
 a function of GLI-Split.

     o  Hierarchical aggregation of routing information in the
        global Internet through separation of edge and core
        routing.

These phrases seem to imply that GLI-Split is a Core-Edge Separation
architecture.

Back to the GLI-Split paper, there is explicit recognition of how it
involves the separation of Locator and Identifier functions.  This is
a feature of all CEE architectures, and is not (despite LISP's name)
something which CES architectures do:

    Separating current IP addresses into two independent
    pieces of reachability and identification information
    helps to reduce this growth and is called locator/
    identifier split (Loc/ID split) [4]. The stable
    identifier (ID) gives a global name to a node. A
    changeable locator (Loc) describes how the node can
    currently be reached through the global Internet.
    Furthermore, a mapping system (MS) is needed to map
    locators to identifiers. This principle makes routing
    in the stable Internet core more scalable because core
    routing is not affected by changed attachment points
    and multihoming of edge networks. The deployment of
    Loc/ID split in the Internet requires modifications to
    the current routing and addressing architecture.

In section 6, there is a further mention of "separation" of
"networks" rather than of the scalable "edge" address subset.  This
is a reference [32] to the 2008 sep./elim. paper and so simply
repeats its terminology:

    The authors of [32] identified two different
    strategies: separation of core and edge networks
    and elimination of de-aggregated provider-independent
    and provider-aggregatable addresses from BGP routing
    tables.

The next paragraph continues:

    Proposals implementing separation can be subdivided
    into address rewriting, map-and-encaps, and source
    routing approaches.

I am not sure what the "source routing" "separation" proposals are,
but so far this repeats the 2008 paper's taxonomy of considering a
proposal which does address rewriting as a "separation" (CES)
architecture: both Six/One Router and GSE.

But I am sure it is a mistake to characterize GSE as a CES
architecture, which is stated two sentences later.

The GLI-Split designers state that their architecture was "evolved
from the early ideas of GSE".  (Quotation below.)  They then assert
that ILNP and Six/One router are likewise "evolved from the early
ideas of GSE".  I agree this is the case with ILNP - and Ran states
this clearly: http://ilnp.cs.st-andrews.ac.uk/

However, I disagree that Six/One Router "evolved" from GSE.  There is
no mention of GSE in Christian's paper.  While both Six/One Router
and GSE involve address rewriting at the borders of EUNs, there are
fundamental differences between Six/One Router and any CEE
architecture, as listed above.

GSE is clearly a CEE architecture and is implementing
"Locator/Identifier Separation".

Six/One Router is not.  The word "separation" only appears in the
references of the Six/One Router paper.  Likewise "locator".
"Identifier" doesn't appear at all.


    With address rewriting, border routers add global
    locator information to packets destined for a
    different domain by coding this information into
    source and destination addresses for transit purposes.

    GLI-Split falls into that class.

GLI-Split does address rewriting, as does GSE (CEE, but mistakenly
classified as a CES architecture) and Six/One Router, which is
correctly classified as a CES architecture.

But this does not mean that GLI-Split is a CES architecture.  It is
clearly a CEE architecture.


    Also Six/One Router [9, 33] uses address rewriting.

This is true.

    Identifiers are only locally routable addresses.

This statement appears to be about Six/One Router, but it cannot be
true to say this of Six/One Router.  The term "Identifier" does not
appear in the Six/One Router paper I am quoting from, which I recall
is the latest version.  The last time we discussed Six/One Router on
the RRG was in August 2008, referring to this no-longer existent file:

http://users.piuha.net/chvogt/pub/2008/vogt-2008-six-one-router-design.pdf

I don't seem to have an electronic copy of this, but Christian
announced it on 2008-07-12:

  http://psg.com/lists/rrg/2008/msg01801.html

and the MobiArch08 paper I am quoting from:

http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf

was last updated on 2008-07-24, so I figure this is the final version.

I think it is wrong to state: "Identifiers are only locally routable
addresses.".

The "edge" addresses used by hosts in Six/One Router networks (which
I guess is what the GLI-Split authors were referring to) are indeed
only "locally routable" in a direct sense, but they are globally
unique, are returned by any DNS query in the world made from within a
Six/One Router network.  In a sense these addresses are "globally
routable" in that packets with these source and destination addresses
can be transported to any Six/One Router network in the world, via
the writing and rewriting of the upper 64 bits.  This in no way makes
them "Identifiers" - they are IP addresses and hosts and at least
some routers treat them just like IP addresses today: both as
Identifiers and Locators.

   When communicating with nodes in different domains,
   the addresses are rewritten 1-to-1 through stateless
   NAT in border routers to globally routable transit
   addresses. A major focus of Six/One Router is
   improved multi-homing support.

This is all fine.

Then there is some discussion of ILNP, which looks fine to me.  Then:

   GLI-Split, ILNP, and Six/One Router have evolved from the
   early ideas of GSE (global, site, and end-system address
   elements) [36, 37].

This is true of GLI-Split and ILNP, but not of Six/One Router.

   It essentially codes a global locator, a local
   locator, and an identifier into an IPv6 address.
   Addresses are dynamically combined from these parts.
   It uses only the identifier for TCP checksum
   calculation and requires host upgrades for deployment.

This is a brief description of GLI-Slit, but it is a mistake to think
 GSE, ILNP or GLI-Split are Core-Edge Separation architectures.

 - Robin