[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[RAM] ViP: Anycast ITRs in the DFZ & mobile tunnels
Here are some ideas which as far as I know are novel. I will write
them up as an I-D if anyone thinks they might be useful. I may well
have made some blunders - please let me know where I am mistaken.
The name for this proposal is ViP - Versatile redIrection of
Packets. "ViP" is pretty lame, but it worked well for Rock Hudson
in 1961 (Lover Come Back To Me, with Doris Day and Tony Randall).
- Robin
This proposal is intended to be easily incrementally deployable,
while I think LISP would be difficult or impossible to incrementally
deploy. I am assuming LISP 2 or higher - since 1 or 1.5 has no
benefits for the global BGP routing table size, as far as I know.
There's no mention of Identifiers or Locators here - just IPv4
addresses. I am thinking in IPv4 terms, but the same scheme may be
useful for IPv6.
I am borrowing from LISP the concepts of the Ingress Tunnel Router
(ITR), the Egress Tunnel Router (ETR), the IP-in-IP tunnelling of
packets from ITR to ETR and ETRs being either border routers or
internal routers. I assume a centralised database of information
which determines how a packet's destination IP address controls the
packet being encapsulated to be sent to a specific ETR address. I
assume all the ITRs have a full up-to-date copy of this database,
via some chained or tree-structured, real-time, "push" update system.
The ITR functions in this scheme are performed by "V-routers",
meaning "Versatile". In some instances a V-router may also perform
something like an ETR function, which I will call a TTR
(Translation Tunnel Router) but that is a separate issue.
The major differences with respect to LISP are:
1 - LISP-mapped prefixes and individual IP addresses are not
within any prefix which is advertised in the BGP system.
ViP-mapped prefixes and individual addresses and are always
part of a prefix which is advertised in BGP.
2 - With LISP, the ITRs are always located in edge networks.
They may be interior routers or the border routers. They
accept incoming packets on interfaces which are not connected
to eBGP peers.
With ViP, the ITR function is performed by a "V-router" which
is always a BGP router. The V-router may be a border router
- single-homed or multi-homed - or a transit router. If the
V-router is a border router, it may accept incoming packets
for encapsulation on any of its interfaces: those connecting
to eBGP peers and those connecting to internal routers.
It makes most sense if the V-router ITR function is implemented
as part of the workload of a multihomed border router or a
transit router. If this is the case, ViP places all ITRs in
the DFZ.
However, it would be technically possible to have the ITR
function performed by a single-homed border router or by
specialised router which has only a single interface which
connects to a transit router or to any border router. In
principle, the ITR function could be performed by the
host which originates the packet, but I intend that the
ITR function have a full copy of the global mapping database
so this would be extremely costly.
3 - For LISP-mapped addresses to be universally reachable, all
edge networks need to implement ITR functions, either at
their border router(s) or at one or probably many internal
routers. (Alternatively, edge networks would need to change
their routing system to forward all packets not covered by a
BGP advertised prefix to some external router which would
provide the ITR function.)
A ViP-mapped address will be universally reachable as long
as there is a single reachable V-router which can receive
packets from the original sender and send encapsulated packets
to the ETR for this address.
4 - Point 3 has a profound effect on how the two proposals may
be incrementally deployed. With LISP, there is a serious
and perhaps insurmountable barrier to anyone deciding to
connect a host to a LISP-mapped address. The first people
to do this will find their addresses generally unreachable
since virtually no edge networks will have upgraded any
of their internal or border routers to perform LISP ITR
functions. Why should they? It is a huge investment to
enable communications with a handful of rebels who are
voluntarily opting out of the BGP system (albeit for the
benefit of humanity).
I can't imagine how LISP would ever get off the ground,
since early adopters would have extremely patchy
connectivity. No-one would put a crucial service on a
LISP-mapped address. Since all crucial services would
be available on ordinary BGP-routed addresses, why should
edge networks spend money on a new or upgraded router?
ViP requires no changes in the edge network of the sender.
It can be implemented with a single ITR on some BGP router
somewhere in the world, and a single ETR in the edge
network of the destination host.
5 - LISP assumes a single system. I will assume a single
ViP system in most of the discussion below, but there
could be any number of independent ViP systems in
operation, without conflict. Indeed, a ViP system could
be a profitable enterprise.
6 - Because LISP needs to be a single system, it needs to be
standardised by the IETF and IANA. I think ViP could be
implemented without any new IETF standardised protocols
or IANA assignments. I am not suggesting this be done -
it would be best to create either a single unified ViP
system or standards by which multiple ViP systems could
all work in the same way.
7 - One of the primary benefit of ViP, together with providing
reachability from the outset without requiring sending
edge networks to do anything, is that there would be a smaller
number of ITRs in the system. These will tend to be bigger
than with LISP and will have more intensive work to do.
I am thinking of direct RAM-based lookup approaches, involving
two or so memory cycles, to speed this operation in hardware.
Perhaps one or more models of current high-end router,
such as the CRS-1, could perform these functions nicely,
with an extensive firmware upgrade.
8 - With ViP, packets to be encapsulated reach the edge of the
BGP system or are forwarded within it, either to a single
ITR (if the ViP system only has one V-router) or to one
of multiple ITR functions in separate V-routers all of
which have the same IP address.
ViP uses Anycast to provide multiple redundant paths
within the BGP system for packets to find their way to
an ITR. Anycast is not normally used for TCP
communications, but I am hoping it will be appropriate
here, since the individual ITRs only encapsulate the
packet and tunnel it to the ETR.
Assuming the sender Host A (HA) has a BGP-mapped address and
the Host B (HB) destination address is ViP-mapped, the
packets in the opposite direction use ordinary BGP routers
to make their way from HB to HA. If HA also has a ViP-mapped
address, the same process occurs in the reverse direction,
with the packets leaving HB, and being directed towards the
nearest anycast V-router, which performs the ITR function and
tunnels the packet to the ETR for HA.
If the border routers of both edge networks are V-routers,
which perform ITR functions and these same routers are also
the ETRs for their hosts, then the paths of the packets will be
the same in both directions.
9 - LISP and what I will call "ViP-basic" both have their ETRs
in edge networks, or at the border router. These ETRs must
have an interface with an IP address which is reachable via the
global BGP system. These only handle packets one way - from
the sender HA to the LISP/ViP-mapped destination address of HB.
Packets in the return direction HA <- HB are sent normally,
without relying on tunnelling, if the destination HA's address
is not LISP/ViP-mapped. Otherwise, a similar process occurs
with the packet being intercepted by an ITR and tunnelled to
the ETR which handles HA's address.
I will also describe a ViP-mobile system where the tunnel
endpoint is an individual host, which establishes the
tunnel with either the ITR or a TTR (Translating Tunnel
Router) both of which are BGP routers with BGP-routable
addresses. The host requires suitable tunnelling software
in its operating system or application program, but can
be located anywhere on the Internet, including behind NAT.
It is conceivable that a fully developed approach to this
mobility idea could provide multiple tunnels over different
upstream links from the target to one or more TTRs. The
tunnel between the TTR and the host would be bi-directional
so the mobile host's current ISP doesn't need to know about
the mobile host's ViP address, for instance for allowing
upstream packets from this address.
10 - With LISP, there are an unknown number of ITRs, since there
must be one or more in every edge network if LISP-mapped
addresses are to be fully reachable. With a single ViP system,
there are a finite and generally smaller number of known ITRs,
all part of the BGP network. This makes it practical, or more
practical than with LISP, to run a tightly meshed system for
distributing updates of the mapping database to each ITR in
real-time.
If there was a single global ViP system then the ITRs would
be operated by multiple ASes. So a considerable amount of
coordination would be required. If there were multiple ViP
systems, then the ITRs of each system may or may not be run
by the one AS.
11 - Since ViP has a smaller number of routers performing the ITR
functions, it is more practical to have them contain a complete
up-to-date version of the mapping database. This means that
there are no delays, as in a LISP variant 2.x or 3.x "pull"
model, which needs to request mapping information for packets
with destination addresses it does not have mapping information
for. LISP variants are listed at:
http://www1.ietf.org/mail-archive/web/ram/current/msg01289.html
The delay in getting that mapping will usually result in such
delays in delivering the packet that the host application or
whatever it was which sent it is likely to assume that the
packet was dropped. Although ViP could work with a "pull"
query and cache system rather than a full database in each ITR,
I am proposing every ITR have a database which is up-to-date
within a few seconds of any changes being made to the master
database in some central, or distributed, control system.
There are no-doubt many pitfalls in all this. I am only considering
unicast packets, not multicast.
Example of ViP-basic
--------------------
ASCII Art only works with fixed width fonts:
Edge Network A
[ITR]
HA---(IR)----(BR1NA)------(VR1)-----(TR2)--(TR4) Edge Network B
\ \ \
\ \ \ [ETR]
(TR1)----(VR2)---(TR3)---(BR1NB)---(IR)---HB
\ /
\ /
(TR5)----(TR6)
\
\ Edge Network C
\ [ETR]
(BR1NC)----HC
\
---HD
HA's address is 11.11.11.11, which is part of a prefix which is
advertised in BGP and which is not covered by ViP mapping.
HB's address is 22.22.0.1, which is part of 22.22.0.0/16, which is
also advertised in BGP and which is ViP-mapped. The other hosts'
addresses are:
HC = 22.22.4.1
HD = 22.22.4.2
There are three border routers and two V-routers. In this initial
example of HA sending a packet to HB, the ITR function is performed
by VR1. Both VR1 and VR2 also perform normal BGP transit router
functions.
Edge Network A has no ViP-equipped internal or border routers.
Edge Networks B and C have a router which performs ETR functions.
In this example, Edge Network B uses an internal router and Edge
Network C uses its border router.
Edge Networks A and C are singlehomed and B is multihomed.
The packet makes its way through Edge Network 1 to BR1NA which
recognises it's destination address as being within the BGP
advertised prefix 22.22.0.0/16.
Both VR1 and VR2 advertise this prefix 22.22.0.0/16.
BR1NA forwards the packet to VR1, because this is the closest.
If VR1 had become unreachable through that direct link, BR1NA would
have forwarded the packet to TR1 instead, which would have forwarded
the packet to VR2, which would have performed the same ITR function
as I will describe for VR1.
VR1 recognises that the destination address of this packet is for a
BGP advertised prefix which is covered by ViP mapping. It therefore
uses the IP address to look up the IP address of the ETR the packet
should be tunnelled to.
While it would be possible to ViP-map an entire BGP-advertised
prefix to a single ETR, this is not how I intend it operate. To do
so would not achieve any reduction in the global BGP routing table.
Instead, I intend that ViP-mapping be applied generally to rather
large (in terms of number of addresses) prefixes (that is, a short
prefix) and that each such prefix should have many fine divisions in
terms of which edge networks the ETRs are located in.
It is technically possible, although perhaps hard work for the ITR,
to have a different ETR for every IP address. In this example, a
/16 has 65,536 IP addresses. The mapping is, in part:
22.22.0.0 to 22.22.0.15 ---> Internal router of Edge Network B.
22.22.4.0 to 22.22.4.3 ---> Border router of Edge Network C.
In this way, a single BGP prefix could have its packets tunnelled to
thousands of edge networks.
VR1 IP-in-IP encapsulates the packet and forwards the packet
according to the BGP routing system for the address of the IR
internal router in Edge Network B.
When it arrives there, the Internal Router recognises it as a
tunnelled packet, strips off the outer IP header and then processes
it as it does any other packet. This causes the packet to be
forwarded to HB.
In the return direction, it is vital that BR1NB allow packets with
the source address of HB into the BGP network.
Assuming there is an already existing system of V-routers to perform
ITR functions (actually, only one is required) then here is what the
administrators of Edge Network B need to do in order to gain
ViP-mapped addresses for hosts such as HB.
1 - They need to trust whoever runs the V-routers, which effectively
means whoever has been assigned the prefix 22.22.0.0/16 - in
this example AS7777.
In terms of long-term stability, portability etc. of HB's IP
address, everything depends on AS7777 retaining this prefix,
and controlling the V-routers so that HB's IP address is
correctly handled.
AS7777 may charge a fee per IP address or via traffic volume
to cover its costs - which are primarily running a bunch of
fast, expensive, V-routers at various points in the DFZ, to
provide robust coverage for all sources and destinations,
without excessively longer routes than would be required if the
packets were not being handled with ViP.
2 - They pay a fee, make an arrangement etc. by which they can
tell AS7777's system the IP address of the ETR which they want
packets tunnelled to, whenever those packets are addressed to
HB's new ViP-mapped IP address. In this case, Edge Network B
gets 16 IP addresses from AS7777: 22.22.0.0/28.
How the ViP-mapping works within a BGP-mapped subnet is a
matter for AS7777 to determine. There is not necessarily any
reason to stick to binary boundaries.
3 - They set up a router to perform ETR functions. The size and
location of this router depends on how much traffic they are
expecting on ViP-mapped tunnels.
4 - They configure their routers to handle this 22.22.0.0/28
prefix. They configure HB and any other hosts to have the
appropriate addresses. Part of this is ensuring that their
border router will allow packets with this source address out to
the BGP routing system.
HB's ViP-mapped address is reachable by all edge networks, since
every edge network will forward the packets to one of the V-routers.
Edge Network A is multihomed, and this multihoming applies to the
way the packets will reach a nearby V-router in the most efficient
fashion.
Edge Network B is multihomed, and this applies to how tunnelled
packets are delivered to its ETR internal router.
Similar principles apply to Edge Network C, except that it is not
multihomed.
So far, unless I have made a mistake, we have seen how ViP can map a
single IP addresses and/or longer prefixes of a single, shorter
BGP-advertised prefix to hosts in a very large number of edge
networks, each of which needs at least one BGP-advertised prefix in
order that its ETR be reachable, and for packets in the reverse
direction.
Now imagine that HB is owned by company Big Inc. The above example
would be the same, except that the actions and responsibilities
would be different.
1 - Edge Network B would have already installed an ETR, so its
customers could use ViP-mapped addresses.
2 - Big Inc. would have dealt with AS7777 to obtain lasting rights
to the prefix 22.22.0.0/28. Edge Network B may have helped them
do this.
3 - Big Inc. (perhaps with the help of Edge Network B) registers the
address of Edge Network B's ETR with AS7777 as being the ETR
for all its 16 IP addresses.
Now let's say Big Inc wants to move its HB to another Edge Network
E. This involves:
1 - Edge Network E already having an ETR, and configuring it and its
network to handle 22.22.0.0/28.
2 - Big Inc. using some kind of real-time, cryptographically
secured, update system to change AS7777's system's ViP mapping
for 22.22.0.0/28 to the address of the ETR of Edge Network E.
Edge Network B doesn't have to do anything.
So as long as AS7777 run their V-routers and their database as
promised, and as long as Big Inc. can find an ISP which will run an
ETR and configure their network accordingly, they can take their
AS7777 provided subnet to whichever such ISP they like.
Example of ViP-basic on a host-by-host basis
--------------------------------------------
It would be possible to implement the ETR function in HB directly.
This would enable HB to have its ViP-mapped IP address in any
supportive ISP. This doesn't necessarily achieve much, since HB
still needs an ordinary BGP-mapped and routed IP address so it can
be its own ETR.
However, Big Inc does gain completely (at least to ETR supportive
ISPs) portable IP addresses, with multihoming (assuming the ISP is
multihomed) without burdening the global BGP routing table with
another advertised prefix.
It can also have a single host, performing its own ETR functions,
but operating multiple ViP mapped IP addresses, if that serves any
purpose.
Traffic Engineering
-------------------
In the first example, I think Edge Network B and Big Inc. could work
together to fine-tune the ViP-mapping of the one or more IP
addresses which are carrying a lot of traffic. Edge Network B may
want to balance the load of incoming traffic across its two upstream
links.
One way of doing this would be to have two ETRs, each with an IP
address on a separately advertised BGP prefix. One prefix would be
advertised on the link to TR4 and the other on the link to TR3.
Then, assuming both ETRs could send packets to HB, it would be
possible to control incoming traffic by altering the ViP mapping for
HB to one or the other of these ETRs.
ViP-Mobile
----------
The purpose of ViP mobile is to enable a single ViP mapped IP
address to be reached by virtually any Internet-connected host,
including any host which:
1 - Has a direct BGP-routable IP address, or:
2 - Is behind one or more NAT firewalls, but can initiate a
tunnel session with one or more TTRs, or:
3 - Is on an IP address which is mapped with ViP-basic.
The idea is that the mobile host creates a two-way encrypted tunnel
to a TTR, with some mechanisms for:
1 - Choosing a single nearby TTR.
2 - Perhaps maintaining two tunnels with two TTRs, with a fancier
ViP mapping system which adapts to tunnel from the ITRs to
whichever TTR currently can reach the mobile host.
The TTR would function as an ETR for the purposes of packets being
sent to the mobile host, but it would send them to the host by a
second, encrypted and possibly compressed, VPN tunnel.
The TTR would receive packets from the host via the VPN tunnel, and
as a first instance, forward them so they were handled by the global
BGP routing system. There, they would either be sent directly to
their destination. Alternatively, if the destination was a
ViP-mapped address, they would be forwarded to the nearest V-router
and tunnelled from there.
The TTR could also function as a ViP ITR for packets to ViP-mapped
addresses. However, being a ViP ITR is an onerous task, even for
small packet volumes, unless it uses a pull approach to mapping,
which enables it to maintain only a partial version of the mapping
database.
There is obviously a lot more to think about with mobility.
Ordinarily the mobile host can use a tunnel to one active TTR.
Ideally this is near the mobile device - which is why a global
network of V-routers which can be ITRs and TTRs would be a good thing.
The idea is to have very light-weight mobility, with the mobile
device automatically finding a nearby TTR, authenticating itself,
and being online without any arrangements with whichever ISP it is
currently connected to.
Placement of V-routers (ITRs)
-----------------------------
One way of implementing ViP is to have a single global system, in
which case, there could be two models, at least:
1 - No fee associated with the mapping of IP addresses or traffic.
In this case, there needs to be some reason why people such as
transit providers and ISPs want to run V-routers without being
paid for the traffic they must handle.
2 - A single global system with some kind of fee for addresses
and/or traffic.
This raises questions of charging and billing, and distribution
of monies.
Although a single ViP system for the whole world is attractive in
terms of elegance, if the idea is a good one, there is nothing I
know of which would stop any organisation setting up its own ViP
network. All it needs is an RIR's agreement to use assigned IP
addresses in this way, and then the resources to install and run
bunch of V-routers around the world.
There could be significant revenue opportunities in privately run
ViP systems, as long as there was a well standardised approach to
the ETR function, which needs to operate on the border routers or
internal routers of many ISPs, before multiple ViP systems would be
viable.
With many or most ISPs running one or more ETRs for free, as part of
their general service to customers, it would be possible for
multiple competing ViP systems to work alongside each other, each
with one or more largish assignments of BGP-routable address space
to split up.
If there were four such competing systems, with similar or identical
technical principles of operation, there would be opportunities for
other companies, such as ISPs, to run a single router which performs
the ETR functions for one, two, three or all four of the competing
global systems - probably for a fee.
An ISP may want to create its own ETR function at its border routers
or very close to them, in order to minimise or eliminate outgoing
and incoming traffic for ViP-mapped addresses within its own
network. For instance, an ISP in Australia might have a ViP-mapped
host on its Sydney network, but doesn't want to make all the routers
in its other cities aware of that IP address. Hosts in other cities
would communicate with the Sydney host via a V-router ITR function,
so it would be best if the ISP itself had a V-router in each city,
so the packets don't escape to the wider Net en-route to the nearest
anycast V-router.
_______________________________________________
RAM mailing list
RAM at iab.org
https://www1.ietf.org/mailman/listinfo/ram