[rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper (v3)
Robin Whittle <rw@firstpr.com.au> Wed, 24 February 2010 06:39 UTC
Return-Path: <rw@firstpr.com.au>
X-Original-To: rrg@core3.amsl.com
Delivered-To: rrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id E1B4C3A8446 for <rrg@core3.amsl.com>; Tue, 23 Feb 2010 22:39:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.437
X-Spam-Level:
X-Spam-Status: No, score=-1.437 tagged_above=-999 required=5 tests=[AWL=-0.142, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, J_CHICKENPOX_63=0.6]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7aU6bZYuQbLn for <rrg@core3.amsl.com>; Tue, 23 Feb 2010 22:39:39 -0800 (PST)
Received: from gair.firstpr.com.au (gair.firstpr.com.au [150.101.162.123]) by core3.amsl.com (Postfix) with ESMTP id 2FB203A8445 for <rrg@irtf.org>; Tue, 23 Feb 2010 22:39:36 -0800 (PST)
Received: from [10.0.0.6] (wira.firstpr.com.au [10.0.0.6]) by gair.firstpr.com.au (Postfix) with ESMTP id D7B83175A69; Wed, 24 Feb 2010 17:41:40 +1100 (EST)
Message-ID: <4B84CA27.3050703@firstpr.com.au>
Date: Wed, 24 Feb 2010 17:41:43 +1100
From: Robin Whittle <rw@firstpr.com.au>
Organization: First Principles
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
MIME-Version: 1.0
To: RRG <rrg@irtf.org>
References: <4B7545E5.2070909@firstpr.com.au> <4B814AE2.5010603@firstpr.com.au>
In-Reply-To: <4B814AE2.5010603@firstpr.com.au>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: 8bit
Cc: Randall Atkinson <rja@extremenetworks.com>
Subject: [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; 2008 sep./elim. paper (v3)
X-BeenThere: rrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IRTF Routing Research Group <rrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/rrg>
List-Post: <mailto:rrg@irtf.org>
List-Help: <mailto:rrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Wed, 24 Feb 2010 06:39:48 -0000
This is a revised version of my msg06060 and msg06089 (v2) message, to correct my mistaken statement that ILNP requires upgraded applications. Please see (msg06103) and (msg06105) for Ran and I discussing this. I have also added a note that ILNP optionally allows routers to rewrite Locator fields in ILNP packets. I have added RANGI to the CEE architecture table and added some notes about a forthcoming, fully decentralised, real-time, mapping system for Ivip and potentially LISP. Changes are marked '!'. - Robin SPI Scalable Provider Independent: "Edge" space handled by a CES architecture's ITRs, ETRs etc. and so which is Provider Independent (portable) and suitable for multihoming and inbound TE. MAB Mapped Address Block: A prefix advertised in the DFZ which covers a typically large amount of SPI space which is typically used for many (tens to hundreds of thousands) of individual SPI micronets (Ivip) or EID prefixes (LISP). Dino recently used the term "coarse prefix" to refer to the same thing in LISP. EUN End User Network: Any user's single host or network, from an IP-connected cellphone, to a single IPv4 DSL service to a university, corporate or government network. EUN-PI: End User Network using (unscalable) PI space. EUN-PA: End User Network using PA space - including home & SOHO DSL etc. end-users. EUN-SPI: End User Network using SPI space = "edge" space in a CES architecture. CES Core-Edge (address) Separation: a subset of the global unicast address space is separated from the remaining "core" global unicast addresses and is supported by the CES system (ITRs, mapping system and ETRs) to be used as "SPI" space. Also a class of scalable routing architectures in which this is done - including LISP, APT, Ivip, TIDR and IRON-RANGER. CEE Core-Edge (address) Elimination: A class of scalable routing architectures in which the Locator / Identity Separation naming model is introduced, replacing the current IPv4/v6 model in which the IP address plays both the Identifier and Locator roles. Includes ILNP, GSE, GLI-Split, Name-Based Sockets. (Does not include LISP - LISP does not use Locator / Identity Separation.) Hosts have portability of Identifiers and are able to do multihoming and inbound TE on PA IP addresses, which are also known as "core" addresses. Therefore there is no need for any "edge" addresses - the unscalable PI addresses which are currently the only way of achieving portability, multihoming and inbound TE, or any other type of IP address with special characteristics for EUNs. EUN-SPI Full Adoption: (Applicable only to a CES architecture.) All EUNs which want or need portability, multihoming and/or inbound TE have adopted SPI space. This includes any EUN-PI networks - so there are no more PI prefixes. The CES architecture has achieved 100% of the routing scalability improvements it could possibly achieve. Many hosts are still on ISP's prefixes - ISP's hosts and all EUN-PA networks - but these in no way constitute a scalable routing problem, since they do not burden the DFZ or result in any loss of portability, multihoming and/or inbound TE to any EUN which wants these benefits. (Those which do are already EUN-SPI networks.) Universal SPI Adoption: Every host - including those of ISPs and of those in what were formerly EUN-PI or EUN-PA networks - uses SPI "edge" space. This is well beyond "EUN-SPI Full Adoption", and does not involve any further improvement to routing scalability. CENI Core-Edge Network Isolation: Once Universal SPI Adoption has been achieved, the routing system is altered so no host can send packets to any "core" address: any address of a DFZ or some other (ISP) routers, or of any ITR or ETR. (This may not make much sense.) Only APT had this as a goal, but it seems APT's designers, who made up many of the authors of the 2008 paper, thought (and perhaps still think) that other CES architectures such as LISP and Ivip had this goal too - but this is not the case. Short version: Core-Edge Separation / Elimination \________/ \______________________/ Adjectives Verbs Are the adjectives referring to: addresses? networks? In "Core-Edge Elimination", the adjectives refer to addresses: there being no more need for "edge" addresses, meaning today's unscalable PI addresses, or any other kind of address for EUNs. If the adjectives referred to "networks", it would make no sense, since neither the core network nor edge networks are being eliminated - nor are the distinctions between them being eliminated If the adjectives in "Core-Edge Separation" are assumed to refer to "addresses" then this is a meaningful and useful term: Separation of global unicast addresses into "edge" and "core" subsets makes sense and is a useful distinction for categorising the CES class of scalable routing proposals, the other class (CEE) eliminates the need for "edge addresses". If the adjectives referred to "networks", then "Core-Edge Separation" makes no sense, since these networks are already separate from each other and will always remain so. However, it seems that the term "Core-Edge Separation" is sometimes used in a way which conflates: Separation of a global unicast addresses into a subset of scalable "edge" addresses, leaving the remainder as "core" addresses. with: Isolation of all "edge" networks from the "core" network (the DFZ, ISP routers, ITRs and ETRs) so that hosts in "edge" networks cannot send packets to "core" addresses. This leads to lots of confusion. Comments on Lixia's and colleagues' 2008 paper on Core-Edge Separation vs. Core-Edge Elimination. I think they were mistaken to class GSE as a Core-Edge Separation architecture. I think this paper refers generally to the core-edge "separation" being the separation of the global unicast *address* space into "core" and "edge" subsets. This is the defining characteristic of CES architectures. However, I believe they were also thinking of, and sometimes referring to with the same "core-edge separation" term, something very different: Core-Edge *network* "separation", but really meaning "Core-Edge Network *Isolation*". This was a goal of APT, but is not a goal of other CES architectures - but the authors may have thought that other CES architectures did have this goal. Many of the authors of this 2008 paper were the designers of APT, which separated not just the address space into two subsets, but had a long-term goal of actually "separating" (really *isolating*) all the edge networks as a set, from the core, so no edge host could physically send a packet to any core address. I think this double sense of the term "separate": Separation of global unicast address space into "edge" and "core" subsets. ("Separation" = Isolation) of all edge networks from the core network. plus the paper's mistaken classification of GSE as CES contributed to the GLI-Split authors considering their proposal to be a CES architecture too. Yet GLI-Split is clearly CEE, since it implements the Locator/Identification Separation naming model. These terminological and classification problems are compounded by LISP being inappropriately named - since it is a CES architecture which does not implement Locator/Identifier Separation. Six/One Router has some similarities to a CES architecture, but uses address rewriting (AKA "translation") rather than tunneling. I think it is not a proper CES architecture, in part because hosts in upgraded networks can't be reached by hosts in non-upgraded networks on their real addresses, requiring DNS tricks to return different IP addresses to hosts in non-upgraded networks, which is the only way the latter type of host can initiate communications with a host in a Six/One Router network. GSE and GLI-Split are translation-based CEE architectures. Like ILNP (which I understand will ! optionally allow routers to rewrite Locator fields ! in ILNP packets) GSE and GLI-Split they do not ! require IPv6 applications to be modified. ! (However I just learnt that GLI-Split could support new application functionality via a new API.) Comparison charts of: CES: LISP-ALT/NERD, APT, Ivip, TRRP, TIDR, and IRON-RANGER. CEE: GSE, GLI-Split, ILNP, Name-Based Sockets, RANGI. ! I believe the GLI-Split designers were mistaken to class their proposal as a Core-Edge Separation (CES) architecture. I am sure it is a CEE (Core-Edge Elimination) architecture. Here is some discussion further to my understanding of the distinctions between Core-Edge Elimination (CEE) architectures and Core-Edge Separation (CES) architectures: CES & CEE are completely different (graphs) http://www.ietf.org/mail-archive/web/rrg/current/msg05865.html Please also see these messages for discussion of the current IP naming model, why I believe the IP address plays both the Identifier and Locator roles - and how this is still 100% true when a CES architecture such as LISP or Ivip etc. is used and the destination address is an EID (LISP) or SPI address (Ivip): LEIDs, SPI & ordinary IP addresses as both IDs & Locs http://www.ietf.org/mail-archive/web/rrg/current/msg06070.html http://www.ietf.org/mail-archive/web/rrg/current/msg06082.html and other messages in this thread. This paper: Towards a Future Internet Architecture: Arguments for Separating Edges from Transit Core Dan Jen, Lixia Zhang, Lan Wang, Beichuan Zhang http://conferences.sigcomm.org/hotnets/2008/papers/18.pdf is really important to the RRG discussions, because it formalises the architectural terminology for two important classes of scalable routing solution. To see my attempt to trace the origins of these concepts and terms, please see: Re: [rrg] Terminology http://www.ietf.org/mail-archive/web/rrg/current/msg05966.html I support this paper, but have some comments to make: 1 - The definition of exactly what core and edge things are being separated or eliminated could be improved - to refer to separation into two subsets of addresses and to use a different term for what I refer to as "CENI - Core- Edge Network Isolation". 2 - The suggestion that GSE is a CES architecture is mistaken. 3 - The paper doesn't mention what I consider to be the biggest objection to CEE: that it implements the Locator/Identifier Separation naming model which will generally slow down the establishment of communication sessions and place extra burdens on all hosts - especially those on slow, unreliable wireless links. The paper refers in general to: "separating edge networks from the transit core" but really both CEE and CES maintain the already existent separation between edge networks from the transit core. (See discussion of section 4.3 below for my understanding of why the authors may have emphasised the "separation of networks" (really CENI) rather than the separation of a new subset of scalable "edge" addresses from the remaining "core" addresses. They intended APT to ultimately "separate" (really *isolate*) the two systems so much that an edge host would be physically unable to send a packet to a core router. They also seem to have assumed that LISP and other CES architectures were intended to do this as well, which is not the case What is being separated in a CES architecture is a new, scalable "edge" subset of *addresses* (SPI addresses in Ivip - EID addresses in LISP) from the remainder of the global unicast addresses, which are then known as "core" addresses. This is a different concept from "separating edge *networks* from the core", since the edge networks are already separate from each other and separate from the DFZ, and this use of "separation" really means isolation between things which are already separate. At present, any EUN which is actually getting portability, multihoming and inbound TE is getting it with PI prefixes - which are advertised in the core routing system (DFZ) just like ISP's prefixes. While these PI prefixes are for "edge" networks (since these are EUNs, rather than ISP or transit provider networks), these PI prefixes are part of the "core" address space, since all DFZ routers need to develop best paths for each such PI prefix. >From the abstract, it can be seen that the CES definition is not the opposite of the CEE definition: CES: The first direction, which we dub separation, calls for separating edge networks from the transit core, and engineering a control and management layer in between. CEE: The other direction, which we dub elimination, calls for edge networks to adopt multiple provider-assigned addresses to enable provider-based address aggregation. (See below for discussion of the paper's more formal description of CEE.) In fact, the CEE definition could reasonably include CES architectures, since CES architectures generally use multiple ETRs, each on a PA ("provider-assigned") address, for the purposes of enabling the DFZ to only handle prefixes which are subject to "provider-based address aggregation". The most important architectural distinction between CES and CEE is: CES CES involves end-user networks' hosts using global unicast addresses from a specially handled "edge" (AKA SPI, AKA EID) subset of the global unicast address space, with the remainder being known as "core" addresses. Global unicast IP addresses in both the "core" and "edge" subsets will continue to be used, as are all today, both for Identifying hosts and for the Routing system to decide how to forward packets (i.e. as a "Locator"). See msg06070 and msg06082. But note that CES has a new, fancy, way of implementing the Locator semantics of destination addresses whose IP address is in the new scalable "edge" (SPI, EID) subset. msg06082 has a detailed example of this 2nd algorithm, which is only performed by ITRs. The original 1st algorithm, as used by all routers except ITRs, still works as normal on these "edge" destination addresses. CEE CEE involves end-user networks' hosts using something other than a global unicast IP address to Identify themselves. Thus CEE implements the Locator/Identifier Separation naming model. Hosts (either upgraded stack and ordinary IPv6 application, or ! upgraded stack and upgraded applications) use this new Identifier, ! which is in a different namespace to IP addresses and any Locators ! used in the system, to identify each host. Hosts can be reached ! via one or more potentially unstable Locators, and the applications ! are unaffected due to either: ! ! 1 - The non-upgraded application is unaware of the changed ! Locators, due to the actions of the upgraded stack and ! potentially the actions of upgraded routers which rewrite ! Locator fields in packets, or: ! ! 2 - The application sees or generates the different Locator ! values, but continues the session without disruption ! because the session is bound to the Identifier of the ! remote host. ! CES preserves the existing IP naming model: Role Level Text name <---- FQDN Identifier <---] IP address Locator <---] while CEE creates one of several models, in which the Locator and Identifier roles are played by different levels - different objects in different namespaces. CEE architectures always implement Locator/Identifier Separation. No CES architecture implements Locator/Identifier Separation. What they separate is two subsets of the global unicast address space: "edge" from the remaining "core". Both types of address are used both as Locators and as Identifiers. Both subsets are used as Identifiers by all hosts. Hosts make no distinction between addresses in the two subsets. Most routers make no distinction between how they handle packets with destination addresses in these two subsets. The exception is ITRs, which make a big distinction. ITRs implement the Locator functionality of destination "edge" addresses in a totally different way: by looking up mapping and tunneling the packet to an ETR, rather than by forwarding the packet to a neighbour. More on naming models: Today's "IP addr. = ID = Loc" naming model should be retained http://www.ietf.org/mail-archive/web/rrg/current/msg05864.html CES architectures never require changes to host stacks or applications. Many CEE architectures require changes to both stacks and apps, but at least two - GLI-Split and I think GSE - only require changes to the stacks. Here are some cases: CES: LISP, APT, Ivip, TRRP, TIDR and IRON-RANGER. There's no debate about this - they all create a special "edge" subset of the global unicast address space for EUNs to use for portability, multihoming, TE and perhaps mobility - and add things to the routing system to scalably get packets addressed to "edge" addresses to the destination networks without excessive burdens on the DFZ control plane. Except for Ivip's long-term "Modified Header Forwarding" modes, all these use encapsulation in order to tunnel packets from an ITR or ITR-like device to an ETR or ETR-like device. This raises some thorny PMTUD problems. These CES architectures are all intended to work for both IPv4 and IPv6. The descend directly or indirectly from the original map-and-encaps architecture. No host stack or application changes are required. Things are added to the routing system - but in general routers stay the same. These will work in principle and in practice for both IPv4 and IPv6. CES-like? - Six/One Router. See footnote. CEE: GSE - Mike O'Dell, 1997 http://ietfreport.isoc.org/idref/draft-ietf-ipngwg-gseaddr/ This is an extension of his earlier 8+8 architecture - both are for IPv6 only due to the way multiple items are combined to create the final IP address. GSE is clearly a CEE architecture because it has separate objects for Identifying hosts and for Locating them. There is no separate "edge" subset of the global unicast address space by which hosts in EUNs identify themselves. The Identifier is the 64 bit ESD. There needs to be new things in the network: an address rewriting function in the BRs of EUNs which adopt GSE. Host stacks needs to be somewhat modified, but I think this is mainly so header checksums are computed only on the ESD and not on other parts of the IPv6 address. As far as I know, applications do not need to be modified. Multihomed EUNs get a prefix from each of their two or more ISPs, and the identity of hosts is unrelated to this, or to any IP address. These prefixes are provided by the ISP as PA space. The ISP typically has a single large (short) prefix which it splits into many such PA prefixes for many such EUNs. Host Identity - and so all communications establishment and session continuity - is determined by the value each host uses for the 64 bit ESD, which exists in a separate namespace from IPv6 addresses. CEE: GLI-Split http://www3.informatik.uni-wuerzburg.de/~menth/Publications/papers/Menth-GLI-Split.pdf I believe GLI-Split is a CEE architecture, because it introduces a separate "ID" object (Figure 2), with a separate namespace from IPv6 addresses or anything else, to Identify each host. This is different from the objects used to Locate them: the GL or LL Locators. The fact that both objects are combined with other things to form an IPv6 address does not affect the fact that these objects for Identifier and Locator are independent, and in separate namespaces, just as they are in ILNP, GSE and the other CEE architectures. GLI-Split implements Locator/Identifier Separation, as do all other CEE architectures. From the Abstract: It implements the locator/identifier split and makes routing in the core of the Internet more scalable. More on GLI-Split at the end of this message. The authors explicitly state that it is a CES architecture, but I am sure it is a CEE architecture. I think they are following a mistake made which I believe was made in the the 2008 sep./elim. paper. See also my draft critique of GLI-Split and what the designers will hopefully soon write in response: (msg06056). CEE: ILNP - Ran Atkinson. This is clearly a CEE proposal, because it has separate levels for Identifier and Locator. As currently specified, it is a ! purely host-based descendent of GSE. However, according to ! (msg06103, see also msg06105), ILNP has, or will have, optional ! router rewriting of Locator fields. There is no need for new routers (but see previous sentence), ! but the host stacks need to be upgraded. These ILNP upgraded ! hosts work with ordinary IPv6 applications - but see (msg06105) ! where I ask if with a new API, upgraded applications could work ! directly with ILNP's Locator / Identifier Separation naming ! model. ! As with GSE and all other CEE architectures, a multihomed network uses two or more PA prefixes from its two or more ISPs, and the PA nature of those prefixes means each ISP can supply many such prefixes from a single prefix the ISP advertises in the DFZ. CEE: Name Based Sockets - Christian Vogt. This is a CEE architecture because the Identity of each host is a FQDN and its one or more Locators are IPv6 addresses, again drawn from PA prefixes of the EUNs one or more ISPs. Name Based Sockets is unusual for a CEE architecture in that it appears to have a two-level naming model. However, I am not sure how it could have this and still be able to support DNS lookups leading to multiple separate hosts. I hope Christian will clarify this. CEE: RANGI See my recent questions and Xiaohu Xu's reply. ???: Name Overlay Service (NOL) See (msg06062). Neither CEE nor CES. It is NAT-like and I do not think it is at all practical. ???: hIPv4 - Patrick Frejborg Now in version 05, which is different from what I critiqued. See: "hIPv4-05: new header" thread. I previously thought it was a CEE but I think it is neither CEE or CES. ???: Aggregation with Increasing Scopes. See my notes in (msg06093). The authors state that the ! end-result somewhat resembles their previous APT CES ! architecture. ! Here are some charts: CES architectures ----------------- IPv4 IPv6 Tunneling Local or Mapping ! "nearby" speed ! full database ! mapping servers ! LISP-NERD v4 v6 Encaps. ITRs are full Slow database LISP-ALT v4 v6 Encaps. No Global lookup ALT v4 v6 Encaps. Yes Slow Ivip v4 v6 Encaps. Local or Fast - with real- ! or MHF "nearby" * time updates to ! ITRs which need it. ! TRRP v4 v6 Encaps. No Global DNS lookup TIDR v4 v6 Encaps. ITR-like Slow - via DFZ routers DFZ control have full DB. plane. IRON-RANGER v4 v6 Encaps. (Neither question applies directly to IRON-RANGER.) * I am about to write a message "DRTM - Distributed Real Time Mapping ! for Ivip and LISP" which extends the new arrangements of (msg05975) ! to add an intermediate arrangement which will probably be an ! enduring one, though it will still be possible for ISPs and EUNs to ! run their own local (within their network) full-database QSD mapping ! query server. The new arrangement will rely on typically "nearby" ! query servers provided directly or indirectly by the companies which ! run the MABs. Typically, these query servers will be within a few ! thousand km of the ISP or EUN's Map Resolvers, through which ITRs ! query these new query servers. So this is no longer "local", but ! it will typically "close enough to be fast and reliable" - so avoiding ! the objections to global query server systems such as LISP-ALT. At ! the same time, the new arrangement will not involve any single server ! being required to hold the full mapping database - so overcoming, I ! hope, some objections to Ivip concerning "globally synchronised ! mapping database" and the like. ! CEE architectures ----------------- Please see (msg06105) where I respond to Ran Atkinson's assertion ! that his ILNP proposal is not a CEE architecture. So far, I think ! he has not supported his assertion with any arguments - so for now ! I consider ILNP to be a CEE architecture. ! IPv4 IPv6 Do IPv6 Stack Router upgrades ! applicat upgrades required - for ! ions need required? rewriting Locator ! to be or other fields? ! upgraded? ! ! GSE v6 No Yes Yes ! ! GLI-Split v6 No* Yes Yes ! ! ILNP v6 No^ Yes Optional router rewriting !!! of Locator fields. !!! Name Based ! Sockets v6 Yes Yes No ! ! RANGI v6 Yes Yes Only for proxies to ! IPv6 hosts. ! * GLI-Split does not require application changes, but Michael Menth ! wrote to me off-list that new applications could be used, for ! instance for multipath protocols, if the stack had a suitable new ! API: ! "Applications usually communicate with identifier addresses, but also a more advanced API must be offered in addition to allow active multipath routing through applications or at least by transport layer protocols. He indicated that this was not in the current documentation but would be added in the future. ^ Please see (msg06105) where I ask Ran whether ILNP could include ! a new API so upgraded applications could directly work with ILNP's ! "Locator / Identifier Separation" naming model. ! Back to the 2008 paper: on page 2, 2nd last paragraph: There are also other types of separation solutions besides Map & Encap. For example, Six-One Router [23] and GSE [19] use address rewriting, which rewrites the packet header to include information about the destination’s attachment point to the transit core. For the reasons listed above, I believe GSE is a CEE architecture. The fact that the somewhat CES-like architecture Six/One Router has routers rewriting addresses does not mean that GSE is a CES architecture simply because it too has routers rewriting addresses. In Section 3, there is a more formal description of CEE: In order to achieve routing scalability, the elimination approach enforces provider-based address aggregation by eliminating all PI prefixes and de-aggregated PA prefixes. Note that the verb "eliminate" has an object: a class of *address*. This is congruent with my claim that the important CES and CEE distinctions concern separation or elimination of classes of address - not "separation" (meaning isolation between) two classes of *network*. I wouldn't state it so strongly - a CEE architecture doesn't "enforce" anything or forcibly "eliminate" PI prefixes. I think it is more accurate to say that when all hosts in all networks adopt CEE, the CEE mechanisms can be relied upon entirely for portability, multihoming and inbound TE. Then, and only then, this means that no network needs PI space to achieve these things. This doesn't mean that PI space (the current, unscalable, form of "edge" space) would automatically be eliminated - just that it could be eliminated without any network missing out on portability, multihoming or inbound TE for all its hosts and communications. Each multihomed edge network will receive from each of its providers an address block out of a larger, aggregated block announced by the provider. I would rephrase it slightly. Each EUN which adopts CEE would generally use space like this. Each such EUN *could* use PI space. Its just that PI space is no better than PA space for use with a CEE architecture. (I am including in my definition of "PI" usage a EUN using a prefix it gets as PA space from one ISP via another ISP as a "more specific PA prefix" - which also has the effect of adding another prefix to the DFZ control plane.) The degree to which they can multihome etc. depends on how many of the hosts they are communicating with are also using the CEE architecture, as noted in 4.1. The multihomed site does not inject PI prefixes or more specific PA prefixes into the routing system. Instead, each host in a multihomed site is given multiple PA addresses. For example, as shown in Figure 2, the host obtains two addresses, one from each of its network’s ISPs. Typically, this is what would happen. The "elimination" in this description refers to the elimination of a class of address space, the current unscalable "edge" subset which is primarily PI prefixes and also, to some extent (I don't know how much) these "more specific PA prefixes". The paper's initial use of "separation" was not in terms of address space, but in terms of separating edge *networks* from the core. >From the abstract: The first direction, which we dub separation, calls for separating edge networks from the transit core, and engineering a control and management layer in between. Edge networks are already separate from the core. This initial use of "separation" probably was intended to mean "isolating all edge networks from being able to send packets to any core address" - which is clearly something the APT designers intended for APT. It seems the authors also thought that all other CES architectures had the same intention - but this is not the case. So I think the authors at times conflate two completely different things: CES Core-Edge (address) Separation CENI Core-Edge Network Isolation referring to both with the same term "Core-Edge Separation" and furthermore imply that CES architectures other than APT - LISP, Ivip and others - also intend to do both of the above. Nonetheless, most of their usage of "Core-Edge Separation" seems to refer to the separation of a new, scalable, "edge" subset of the global unicast *address* space from what remains as the "core" - and does not refer to the "Separation" (isolation between) core and edge *networks*. In section 4.3, I think I see why the authors refer more to separating edge networks from the core. I think what they are referring to is the end-point of APT adoption: 1 - All EUNs use APT, and so use the scalable "edge" subset of the global unicast address space. Furthermore, all hosts in ISPs which need to communicate with hosts in EUNs are also using SPI space. This is not just what I now call: "EUN-SPI Full Adoption". It goes well beyond this - well beyond what is required to maximise scalable routing benefits - to what I call "Universal SPI Adoption". 2 - Therefore, no EUN needs PI space or "more specific PA prefixes". So any space which was previously a PI prefix has been converted either to the new scalable "edge" subset of space or to the remainder - the "core" subset. 3 - Therefore, ISPs advertise prefixes into the DFZ which are all in the "core" subset. ISPs use these addresses for ITR and ETRs, and for internal and DFZ routers. 4 - Because all ISPs and EUNs use APT, there is no need for APT's equivalent of LISP's Proxy Tunnel Routers or Ivip's DITRs. (APT's arrangement was never well documented, but it too could provide full support for packets sent from non-upgraded networks.) Therefore, there is no need to advertise any "edge" space in the DFZ. 5 - Now, if it was desired - as the APT designers desired - it would be possible, in theory, to prevent any host in an EUN from being able to send a packet to any host or router on a "core" address. Hosts in EUNs only need to communicate with other hosts in EUNs. (Traceroute and ping wouldn't concern DFZ routers, since all packets between EUN hosts would be tunneled across the DFZ.) This is what I now call "CENI - Core-Edge Network Isolation". Most of the APT designers wrote this 2008 paper. AFAIK, no other CES architecture had this complete, physical, (separation) isolation of EUN hosts or routers in EUNs - all on scalable (SPI = EID) "edge" addresses - from being able to send or receive packets to or from "core" addresses as a goal. I specify it as a non-goal for Ivip. >From section 4.3: Separating edges from the transit core provides additional features that are sorely missing in today’s Internet. With separation, an end host can send packets through the transit core, but can no longer address a packet to any specific device inside the transit core. Although the separation does not eliminate any specific security threat, it raises the bar against malicious attacks targeted at the global routing infrastructure. In addition, the mapping layer between edge and core networks can serve as a mounting point for badly-needed control and protection mechanisms, and can also act as a cushion layer between the edge and core, allowing each side to deploy innovations without any involvement of the other side. We now elaborate on each of these benefits. They go on to discuss "Rolling out new protocols", "DDoS mitigation" and "Ingress Traffic Engineering" - all of which they argue would be possible or easier only with complete "separation" of the "edge" networks from the "core" networks. I think "Ingress Traffic Engineering" is possible with any level of adoption of APT (assuming that APT, like LISP with PTRs and Ivip with DITRs supports packets sent from non-upgraded networks, which it can) - so this depends on Core-Edge (address) Separation, but not on "EUN-SPI Full Adoption". Support for "Ingress Traffic Engineering" certainly doesn't depend on "Universal SPI Adoption" or "CENI Core-Edge Network Isolation". Side-note on Ingress Traffic Engineering: Inbound TE works fine due to any amount of adoption of the CES architecture. As long as the CES architecture supports all packets sent from non-upgraded networks, inbound TE will be for packets arriving in all the adopting networks, no matter how many other networks have so-far adopted the CES architecture. What the authors write in the second paragraph about Site1 having preferences for which of Site2's two or more ETRs to use doesn't strike me as part of most CES architectures. I think the aim of most CES architectures is to to have the recipient site control which of its ISPs the packets arrive via. This is inbound TE. Maybe APT had some concept of the sending EUN controlling this, but I guess most sending EUN wouldn't care which ISP used by the destination EUN the packets went through. In their discussion of the other two goals: "Rolling out new protocols" and "DDoS mitigation", to the extent that these goals are facilitated by any kind of core-edge separation, they too are dependent on "Core-Edge (address) Separation" - and do not depend at all on "EUN-SPI Full Adoption", "Universal SPI Adoption" or "CENI Core-Edge Network Isolation". So I think the phrase at the start of this paragraph "Separating edges from the transit core" is misleading if it means "isolating" edge networks from being able to send packets to core addresses. It would be better to use the phrase: "Deploying an architecture in which a scalable subset of 'edge' space is separated from the remaining 'core' global unicast addresses" - AKA "Deploying a CES architecture". I think the discussion in section 5 about multipath being used with CES is not really about multipath as some other people conceive of it. My understanding of true multipath, such as MPTCP, is the sending host (or perhaps a router)and perhaps the receiving host (or its router?) being able to choose which of multiple ISPs in the sending host's network packets may go out from, at the same time as choosing which of multiple ISPs in the recipient host's network the packets will travel through. With CES, I don't see how a sending host could affect either. The routers in the sending host's network can choose which ISP link to send out the packets on (assuming the ISPs accept packets with "edge" addresses, which they will have to when CES is widely used). If those routers are ITRs, then in some or many CES architectures (but not Ivip) then perhaps the ITR might be locally programmed to prefer one of multiple destination ETRs over others - so in principle the ITRs in the sending network could affect the incoming load balance of the destination network. However, as far as I know, the intention with all CES architectures is to allow the sending network to choose the outgoing ISP and the recipient network to choose (inbound TE) its incoming ISP(s). In the case of Ivip, the recipient network only provides a single ETR address, so there is no option for the sending network to have its ITRs tunnel traffic to some other ETR of the destination network. I think this paper doesn't mention what I regard as the greatest objection to CEE architectures - their adoption of the Locator/Identifier Separation naming model, as I discuss in (msg05864). Many of the authors of the 2008 paper also wrote the 2009 RRG proposal: AIS / Evolution AKA "Aggregation with Increasing Scopes: An Evolutionary Path Towards Global Routing Scalability". In this they seem also to be conflating CES (address) with what I now call CENI - Core-Edge Network Isolation. I will write more about that in my forthcoming discussion / critique of the AIS proposal. I think the GLI-Split authors were mistaken to state that GLI-Split is a CES architecture. The summary: http://tools.ietf.org/html/draft-irtf-rrg-recommendation-04#section-10.1 includes text indicating that perhaps the authors consider GLI-Slit to be a CES architecture: GLI-Split implements a separation between global routing (in the global Internet outside edge networks) and local routing (inside edge networks) and using global and local locators (GLs, LLs). Yes, but all EUNs have separate routing from the DFZ - so this is not a function of GLI-Split. o Hierarchical aggregation of routing information in the global Internet through separation of edge and core routing. These phrases seem to imply that GLI-Split is a Core-Edge Separation architecture. Back to the GLI-Split paper, there is explicit recognition of how it involves the separation of Locator and Identifier functions. This is a feature of all CEE architectures, and is not (despite LISP's name) something which CES architectures do: Separating current IP addresses into two independent pieces of reachability and identification information helps to reduce this growth and is called locator/ identifier split (Loc/ID split) [4]. The stable identifier (ID) gives a global name to a node. A changeable locator (Loc) describes how the node can currently be reached through the global Internet. Furthermore, a mapping system (MS) is needed to map locators to identifiers. This principle makes routing in the stable Internet core more scalable because core routing is not affected by changed attachment points and multihoming of edge networks. The deployment of Loc/ID split in the Internet requires modifications to the current routing and addressing architecture. In section 6, there is a further mention of "separation" of "networks" rather than of the scalable "edge" address subset. This is a reference [32] to the 2008 sep./elim. paper and so simply repeats its terminology: The authors of [32] identified two different strategies: separation of core and edge networks and elimination of de-aggregated provider-independent and provider-aggregatable addresses from BGP routing tables. The next paragraph continues: Proposals implementing separation can be subdivided into address rewriting, map-and-encaps, and source routing approaches. Except for the mention of "source routing" "separation" proposals (which I don't understand at all), this repeats the 2008 paper's taxonomy of considering a proposal which does address rewriting as a "separation" (CES) architecture: both Six/One Router and GSE. But I am sure it is a mistake to characterize GSE as a CES architecture, which is stated two sentences later. The GLI-Split designers state that their architecture was "evolved from the early ideas of GSE". (Quotation below.) They then assert that ILNP and Six/One router are likewise "evolved from the early ideas of GSE". I agree this is the case with ILNP - and Ran Atkinson states this clearly: http://ilnp.cs.st-andrews.ac.uk/ . However, I disagree that Six/One Router "evolved" from GSE. There is no mention of GSE in Christian Vogt's paper. While both Six/One Router and GSE involve address rewriting at the borders of EUNs, there are fundamental differences between Six/One Router and any CEE architecture, as mentioned in the footnote below. GSE is clearly a CEE architecture and is implementing "Locator/Identifier Separation". Six/One Router is not. The word "separation" only appears in the references of the Six/One Router paper. The words "Locator" and "Identifier" don't appear at all. With address rewriting, border routers add global locator information to packets destined for a different domain by coding this information into source and destination addresses for transit purposes. GLI-Split falls into that class. GLI-Split does address rewriting, as does GSE (CEE, but mistakenly classified as a CES architecture) and Six/One Router. But this does not mean that GLI-Split is a CES architecture. It is clearly a CEE architecture. Also Six/One Router [9, 33] uses address rewriting. This is true. Identifiers are only locally routable addresses. This statement appears to be about Six/One Router, but it cannot be true to say this of Six/One Router. The term "Identifier" does not appear in the Six/One Router paper I am quoting from, which I recall is the latest version. The last time we discussed Six/One Router on the RRG was in August 2008, referring to this no-longer existent file: http://users.piuha.net/chvogt/pub/2008/vogt-2008-six-one-router-design.pdf I don't seem to have an electronic copy of this, but Christian announced it on 2008-07-12: http://psg.com/lists/rrg/2008/msg01801.html and the MobiArch08 paper I am quoting from: http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf was last updated on 2008-07-24, so I figure this is the final version. I think it is wrong to state: "Identifiers are only locally routable addresses.". The "edge" addresses used by hosts in Six/One Router networks (which I guess is what the GLI-Split authors were referring to) are indeed only "locally routable" in a direct sense, but they are globally unique, are returned by any DNS query from within any Six/One Router network. In a sense these addresses are "globally routable" in that packets with these source and destination addresses can be transported to any Six/One Router network in the world, via the writing and rewriting of the upper 64 bits. This in no way makes them "Identifiers" - they are IP addresses and hosts and at least some routers treat them just like IP addresses today: both as Identifiers and Locators. When communicating with nodes in different domains, the addresses are rewritten 1-to-1 through stateless NAT in border routers to globally routable transit addresses. A major focus of Six/One Router is improved multi-homing support. This is all fine. Then there is some discussion of ILNP, which looks fine to me. Then: GLI-Split, ILNP, and Six/One Router have evolved from the early ideas of GSE (global, site, and end-system address elements) [36, 37]. This is true of GLI-Split and ILNP, but not of Six/One Router. It essentially codes a global locator, a local locator, and an identifier into an IPv6 address. Addresses are dynamically combined from these parts. It uses only the identifier for TCP checksum calculation and requires host upgrades for deployment. This is a brief description of GLI-Split, but it is a mistake to think GSE, ILNP or GLI-Split are Core-Edge Separation architectures. - Robin Footnote on Six/One Router: CES: Six/One Router - by Christian Vogt, though AFAIK it is no longer under development. http://conferences.sigcomm.org/sigcomm/2008/workshops/mobiarch/papers/p13.pdf I previously stated that this was "clearly a CES architecture" but now I think it is not one, despite some resemblances. Between hosts in upgraded networks, it appears these end-user networks operate from a single prefix drawn from an "edge" subset of the global unicast address range. (Sect. 2.1.) Figure 1 illustrates this setup and addressing for an edge network that is multi-homed with two providers. This edge network has one border link to each provider, so two Six/One routers are in use. The edge addresses are from the ---- prefix ABC::/64. The Six/One router on the border link to provider 1 translates between those and transit addresses from the prefix 1000::/64; the Six/One router on the border link to provider 2 translates between the same edge addresses and transit addresses from the prefix 2000::/64. A host HA in one Six/One Router network, addresses packets to a host HB in another Six/One Router network another using the same IP address in the destination field as HB uses as its own "edge" address. Edge addresses are for use within this and other upgraded edge networks. They are globally unique, but do not appear --------------- in the global routing table because they cannot be efficiently aggregated. All hosts in Six/One routers use a form of DNS which returns these "edge" addresses for any host in any Six/One Router network. So this is a genuine scalable "edge" subset of the global unicast address space. However, because Six/One Router has no equivalent to DITRs (Ivip) or PTRs (LISP), hosts in ordinary IPv6 networks can't send packets to the Six/One Router hosts using these scalable "edge" addresses. So, from a non-Six/One Router network, a DNS lookup on the same FQDN of host HB will not return its "edge" address, but one or more another addresses, based on one or more addresses within the PA prefixes HB's network obtains from its one or more ISPs. The address-rewriting router at the border of HB's network will rewrite the destination address of a packet coming from some ordinary IPv6 host HX, to the "edge" address of HB. This is explained in section 2.4. No other CES architecture has this "split-horizon" (I think this is the right term) DNS arrangement. I guess it was impossible to implement an equivalent of DITRs and PTRs. So there isn't a single address by which a host in a Six/One Router network can be contacted or communicated with by both other such hosts, or hosts in ordinary IPv6 networks. To those hosts in ordinary networks, the "edge" subset of addresses doesn't work. No other CES architecture is like this, and I now think it is not correct to think of Six/One Router as being as much a CES architecture as LISP, APT, Ivip, TRRP, TIDR or IRON-RANGER. The hosts in Six/One Router behave as usual. There are no stack or application changes. The naming model used in the packets they send and receive is the same as with today's IPv4 and IPv6, as illustrated above. Rather than tunnel packets across the DFZ with encapsulation or Modified Header Forwarding, Six/One Router uses address rewriting (AKA address translation) at EUN BRs or perhaps in a router in their ISP(s). Unfortunately, Six/One Router can't be used for IPv4 due (I guess) to the more chaotic EUN prefix lengths and (I am sure) due to the need for a 2 ISP multihomed EUN to consume 3 times the amount of global unicast address space it really needs. (One for its "edge" prefix and then the same amount of PA space from each of its ISPs.) Also, when Christian and I last discussed it on the RRG, I recall that he wrote that he needed a flag bit from the IPv6 header, which would require a modified IPv6 header format and so (I guess) require updates to all DFZ and many other routers.
- [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; … Robin Whittle
- Re: [rrg] CES & CEE: GLI-Split; GSE, Six/One Rout… Robin Whittle
- [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; … Robin Whittle
- [rrg] CES & CEE: GLI-Split; GSE, Six/One Router; … Robin Whittle