[rrg] IPv4 & IPv6 routing scaling problems

Robin Whittle <rw@firstpr.com.au> Thu, 04 February 2010 02:37 UTC

Return-Path: <rw@firstpr.com.au>
X-Original-To: rrg@core3.amsl.com
Delivered-To: rrg@core3.amsl.com
Received: from localhost (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id D446F3A6878 for <rrg@core3.amsl.com>; Wed, 3 Feb 2010 18:37:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.495
X-Spam-Level:
X-Spam-Status: No, score=-1.495 tagged_above=-999 required=5 tests=[AWL=0.085, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, SARE_MILLIONSOF=0.315]
Received: from mail.ietf.org ([64.170.98.32]) by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ktpCi7NMgqFC for <rrg@core3.amsl.com>; Wed, 3 Feb 2010 18:37:04 -0800 (PST)
Received: from gair.firstpr.com.au (gair.firstpr.com.au [150.101.162.123]) by core3.amsl.com (Postfix) with ESMTP id 990C83A6876 for <rrg@irtf.org>; Wed, 3 Feb 2010 18:37:03 -0800 (PST)
Received: from [10.0.0.6] (wira.firstpr.com.au [10.0.0.6]) by gair.firstpr.com.au (Postfix) with ESMTP id ED8CF175AA6; Thu, 4 Feb 2010 13:37:46 +1100 (EST)
Message-ID: <4B6A32F8.4080800@firstpr.com.au>
Date: Thu, 04 Feb 2010 13:37:44 +1100
From: Robin Whittle <rw@firstpr.com.au>
Organization: First Principles
User-Agent: Thunderbird 2.0.0.23 (Windows/20090812)
MIME-Version: 1.0
To: RRG <rrg@irtf.org>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Cc: Scott Brim <swb@employees.org>, Noel Chiappa <jnc@mercury.lcs.mit.edu>
Subject: [rrg] IPv4 & IPv6 routing scaling problems
X-BeenThere: rrg@irtf.org
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: IRTF Routing Research Group <rrg.irtf.org>
List-Unsubscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=unsubscribe>
List-Archive: <http://www.irtf.org/mail-archive/web/rrg>
List-Post: <mailto:rrg@irtf.org>
List-Help: <mailto:rrg-request@irtf.org?subject=help>
List-Subscribe: <http://www.irtf.org/mailman/listinfo/rrg>, <mailto:rrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Feb 2010 02:37:06 -0000

Short version:    If the RRG recommends an IPv6-only solution,
                  this would be analogous to solving the
                  planetary warming problem only on Mars, on
                  the basis that Earth will cope for the
                  immediate future - and is so overcrowded
                  that most of humanity will need to move to
                  Mars anyway, real soon now.

                  CEE architectures are only viable on IPv6,
                  since a multihomed /16 network needs at least
                  two /16 PI prefixes - and so chews address
                  space in a manner which really can't work
                  on IPv4.

                  So if the RRG chose a CEE solution, this will
                  not help with the IPv4 scaling problem at all
                  until and unless IPv6 becomes a viable
                  alternative to IPv4 for mass adoption.

                  Current levels and growth rates in the DFZ
                  advertised prefixes for IPv4 and IPv6.

                  At current growth rates, the IPv6 number will
                  reach today's IPv4 figure of ~300k in 11 or 12
                  years.  By then, at current growth rates, the
                  IPv4 figure would be 1.46 million.

                  Discussion impact of mass-market IPv6 use for
                  cellphone and other mobile IP uses - and how
                  this would not directly drive the scaling
                  problem.  Only when large numbers of end-user
                  networks want their own IPv6 PI space would a
                  problem arise.

                  Replies to messages from Joel, Noel, Eric,
                  Brian, and Scott - all from the "SIP will work
                  fine with CES, CEE won't work with NAT" thread.


Hi Joel,

In "Re: [rrg] Multiple critiques, choice of single proposal, ..."
(msg05846) you wrote:

> The question of IPv4 support is an example of something the RG has
> not converged on.

In later messages in the "SIP will work fine with CES, CEE won't work
with NAT" thread you and others also discussed IPv4 and IPv6 scaling
problems.  I respond to those messages below.

One measure of the IPv4 scaling problem is is ~300k prefixes in the
DFZ with a doubling time of about 5 years.

  http://bgp.potaroo.net/

Another measure, which is hard to quantify, would be how many
end-user networks are not getting portability, multihoming and
inbound TE when they want / need it.

The IPv6 Internet has no scaling problem:

  http://bgp.potaroo.net/v6/as2.0/
  http://bgp.potaroo.net/v6/as6447/

These indicate about 2.6k prefixes with a doubling time of about 20
months.  At these rates it would take 11.4 years (2021) for this
measure of the IPv6 scaling problem to reach the level of today's
IPv4 scaling problem. By then, at the current growth rates, the IPv4
figure would be about 1.46 million.

I expect the rate of growth in the number of IPv4 routes in the DFZ
to to increase, since there will be increasing pressure to slice and
dice currently used address space into more, smaller, chunks once the
fresh unused expanses of space run out.  I think this will be
required to increase the number of global unicast addresses which can
be actively used.

I expect that a sparsely used prefix (whether of an ISP or a PI
prefix of an end-user network) will have its hosts, DSL services or
whatever moved to one half.  Then the other half will be advertised
as a separate prefix - perhaps by another organisation - and used
similarly intensively.

Also, I imagine that the rate of IPv6 adoption will increase.

However, the only scenario I can foresee for mass adoption of IPv6
involves large mobile phone companies using it.  This would give each
cell-phone a public IPv6 address, rather than the only other option -
putting each one behind an IPv4 NAT box.  They could still give NATed
IPv4 access via tunneling.  In a footnote "MNs with public
addresses?" below, I consider the potential problems of having a
public IPv4 address or IPv6 /64, including via a TTR Mobility.


While this could involve hundreds of millions of handsets, I guess
the number of such mobile networks would probably be small, and that
each one would only advertise one or a few prefixes.  So this scenario
wouldn't lead to a significant scaling problem.  Nor would a number
of large ISPs - such as those with millions of DSL, fibre, HFC cable
or WiMAX connected customers giving their millions of customers IPv6
space - lead to such a number of prefixes large enough to make a
significant contribution to any scaling problem.

I think we all agree that unscalable growth in the number of prefixes
advertised in the DFZ is caused by large numbers of end-user networks
advertising their PI prefixes.

I guess few end-user networks want or need IPv6 connectivity at
present.  However, if there were millions or billions of cell-phone
users with direct IPv6 connectivity, but limited (behind NAT) IPv4
connectivity, then there could be a substantial number of end-user
networks wanting their own portable, multihomable, PI space in the
IPv6 Internet so they can sell services to these millions of
end-users.  I guess this would start to drive the IPv6 scaling problem.

If an end-user network had a bunch of its staff and/or customers with
IPv6 cellphones, it would probably want its own portable,
multihomable, IPv6 space so it could communicate directly with these
people's mobile devices - so that would lead to general adoption of
IPv6 by more and more end-user networks which were not in the
business of selling things to mobile users.  Until we develop and
deploy a scalable way of providing these end-user spaces, any such
networks which can get PI space will be advertising it in the IPv6
DFZ and so further driving the future IPv6 routing scaling problem.


Joel, in (msg05925), you wrote:

> Robin Whittle wrote:
> ...
>>       IPv4 will be _central_ for a long time to come.
> ...

That was me quoting, in agreement, Patrick Frejborg, in (msg05895),
agreeing with Noel, who wrote:

    IPv4 will be around for a very long time - especially in the
    old economies

> IPv4 will clearly be around for a LONG time. IPv4 will, I expect,
> continue to be used for a LOT of content for a LONG time.

OK - we agree on this.


> However, I am not sure that matters for the RRG work.
>
> The IPv4 Internet works. The routers, and the routing system, cope
> with the current pressures. To my way of looking at things, the
> question is how will the Internet routing system cope with growth.
> But, definitionally, there really is not that much growth left in
> IPv4.

I disagree for two reasons.  I believe the number of prefixes in the
IPv4 DFZ will grow at an increasing rate - until we have a scaling
solution widely adopted.

  1 - It is possible to make a much larger proportion of IPv4
      space used for actual services, whether they be a host
      or a DSL router or whatever.   Some prefixes are pretty
      full but many others are sparsely populated.  By the
      mechanism I described above many more addresses can be
      used: consolidating a scattered bunch of used addresses
      into one end of a prefix and then advertising this end as
      and the other end as separate prefixes, where the other
      end prefix is now empty and ready to be intensively used,
      or divided into still more prefixes which themselves can
      be intensively used by a greater number of smaller
      ISP or end-user networks.

      See messages quoted below from Brian, Scott and Noel -
      who appear to agree with this argument.

  2 - The above process accelerates the growth in the number
      of prefixes advertised in the DFZ, which is the most
      tangible measure of the routing scaling problem.


> On the other hand, although IPv6 is tiny now, unless the entire
> Internet stops growing, v6 will become massive.

I suspect that with squeezing more use of the IPv4 space, and
increasing use of NAT, that another 10 or 20 years of natural growth
in usage can be accommodated in the IPv4 Internet - IF mass adoption
mobile IP gives each device an address behind NAT AND if most of
those mobile devices do not get their own IPv4 address of SPI space,
such as by TTR Mobility.

I guess there will be 5 billion or 8 billion or whatever cell-phones
(or whatever they might be called) with IP connectivity by 2020 or
2025.  They could all have IPv4 connectivity behind NAT, without port
forwarding.  They couldn't all have their own portable SPI space -
because there are only 3.7 billion global unicast IPv4 addresses.

They could all have their own global unicast PI IPv6 address or /64
prefix, which would be different for each access network they use,
and could vary within one large access network.  They could all have
their own portable /64 of SPI space via TTR Mobility.

Over a long period of time, I think something other than IPv4 will
have billions of hosts on it, and I think the most likely initial
settlers in this second mass-adoption Internet will be mobile
devices.  The most obvious second mass-adoption Internet is IPv6.


> If we do not work out an
> architecture that can cope with growth, when that pass the current
> IPv4 size, and keeps growing faster, we will be up the proverbial
> creek without any control whatsoever.

I agree, but the IPv4 scaling problem is likely to remain more
serious than the IPv6 one for a long time, such as a decade or two -
especially due to the increasing pressure to slice and dice IPv4
space into multiple smaller prefixes in order to cram as many
actively used addresses into the system as possible.

> Hence, I think it is
> actually quite reasonable to have an architecture and approach
> which only addresses IPv6 Internet Scaling. Without denying that
> IPv4 will be here, and important, for a very, very, long time.

I completely disagree.  As noted above, the IPv4 scaling problem is
bigger now - there is no IPv6 scaling problem today.  It is
reasonable to expect that the IPv4 scaling problem will remain bigger
than IPv6's for a decade or more.  IPv4 routing scaling difficulties
will grow at a greater rate *because* of the shortage of fresh
address space, since this is the only way to accommodate increased
growth in the number of utilized addresses.

I think this idea that the RRG should recommend an IPv6-only solution
is analogous to the UN setting up a 3 year taskforce on global
warming, and them coming back with a recommendation:

   We have a solution for the greenhouse effect warming problems
   on Mars.  We recommend that all resources be devoted to
   implementing it.  Earth will cope for the time-being, and is
   already way-overcrowded - so it won't be long before most of
   humanity moves to Mars.


Noel, in (msg05930) you wrote, quoting Joel:

>> definitionally, there really is not that much growth left in IPv4.
>
> Not necessarily; mind, I don't _know_ that there is going to be
> more growth, but I can see plausible circumstances where there is
> continued substantial growth in the number of routes in the DFZ.
>
> E.g. if some organizations which currently have large chunks of the
> globally-visible IPv4 namespace drop back to using smaller chunks
> of the that namespace, and the chunks they return are broken up
> into many smaller pieces for use by a number of currently
> non-connected entities.

OK - we agree entirely on this.  (However, I would have written
"address range" rather than "namespace".)


> The exact shape of the future can be awfully hard to predict. If
> you'd asked people back in 1995 if we'd be here today, people would
> probably have gawked at you... Never say 'Never'... :-)

Even without growth in the human population, it is easy to see that
the adoption of cell-phones won't stop until there's about one for
every person.  So I think we need to plan for up to 10 billion
IP-capable cellphones.  Ideally, each one would have its own global
unicast IPv4 or IPv6 address, to be used no matter what sort of
connection it has.  The TTR Mobility architecture can do this - but
obviously only a fraction of these 10 billion devices could have
their own IPv4 SPI address.


Joel, in (msg05932), you wrote:

> I'll readily grant that judgment calls such as I am making are
> subject to the whims of reality.
>
> In fact, if we do not get a reasonable, deployable, usable,
> multihoming and multiconnectivity solution for IPv6, then it won't
> matter whether folks bury v4 in prefixes, or switch to v6.

Not necessarily.  Maybe the only viable user base for mass IPv6
adoption will be mobile devices, starting in 2015 or whatever - and
maybe that would not lead to a 300k+ IPv6 scaling problem until 2025
or so.

In all those years, the scaling problem of IPv4 is going to get a lot
worse, and will remain worse than IPv6's for a long time after.

The IPv4 Internet matters.

It is the Internet which everyone uses and depends upon.

Only when everyone has their own IPv6 space, connectivity, host
stacks and applications will no-one rely on IPv4.  That is not going
to happen in the foreseeable future, such as by 2025.

> Conversely, if we do have a good solution, that is less painful
> than the various de-aggregation drivers for IPv4, we have a good
> shot at moving people because the pain is going to be quite high.

The IPv4 scaling problem will be a growing burden, but it is hard to
foresee a time when it will be possible to sell anyone an IPv6-only
service, except perhaps as noted above for cell-phones.

In the foreseeable future, I am sure it would always be profitable to
sell NATed IPv4 connectivity to homes (DSL, HFC, fibre, WiMAX etc.)
and to cell-phone users than to not give them this and only give them
IPv6 global unicast space.

For better or worse, this would just prolong our dependence on IPv4
as the only Internet which everyone uses.  There may well be another
Internet such as IPv6, but why would anyone bother paying for a
service on that if they couldn't get at least NATed IPv4 connectivity
as well?


> I would be happy to see an answer that also helped v4. But I think
> that is a distinctly secondary consideration.

I think your perspective is based on a long history of IETF optimism
that IPv6 will be widely adopted real soon.  This optimism has proven
to be unrealistic for ten or fifteen years.  I think the current wave
of IPv6 boosterism based on the purported imminent collapse of IPv4
is just a continuation of this pattern.

All that is happening with IPv4 is that the last fresh paddocks of
unused land are now being built upon.  There's plenty of scope for
squishing houses closer together - and then for building multi-storey
houses (NAT).   Without NAT, some kind of crunch would occur with
IPv4 if there was ever a need to connect more than about 3 or 3.5
billion hosts at any one time.  With NAT, there's no limit.

NAT is not the ideal of any-to-any Internet connectivity, just like
the crowded suburbs with multi-storey dwellings don't provide all the
benefits of living in an uncrowded piece of countryside.  But just as
crowded cities get more crowded, so will IPv4.  The attraction is the
same for cities as for the choice of which Internet use - most people
want and need to be where all the other people are.


Eric, you wrote, in (msg05933):

> I'm sure that Noel's insight below is not contentious to this
> list.

(Quoting Noel in msg05876):

  >> Because of the inertia of the IPv4 service interface, with
  >> the amazingly huge ball-and-chain of deployed base which
  >> speaks it, I think IPv4 addresses will be around for a very
  >> long time - even if, inside the network, we move to something
  >> else, e.g. between xTRs. In other words, TCP will continue to
  >> see IPv4 addresses - even if those are mapped into 'something
  >> else' shortly after leaving TCP.

> However, his observation was reinforced in my own thinking when I
> observed how many embedded systems have been using IPv4 for some
> time now. This taught me that my thinking has been far too small:
> IPv4 has become truly ubiquitous in the literal meaning of the word
> and now includes so many things that never previously were
> networked.

I agree.

> Another aspect of Noel's insight was reinforced to me when I noted
> that many environments (especially those operating in lower
> bandwidth situations) had chosen frame sizes to correspond with
> IPv4 packet sizes to the extent that deploying IPv6 in those
> environments has to overcome significant performance problems
> unless header compression is used.

Could header compression always compress headers sufficiently?

> However, the bottom line remains as it always has been: there is
> a strong business case for the end user to *not* support
> non-value-added protocol diversity.  I wish that RFC 1687 were no
> longer true for IPv6, but, unfortunately, it still is. The key
> question is for how much longer?

  A Large Corporate User's View of IPng
  E. Fleischman Boeing Computer Services  August 1994
  http://tools.ietf.org/html/rfc1687

Here are 3 of the 5 points in the executive summary:

    1)  Large corporate users generally view IPng with disfavor.

    2)  Industry and the IETF community have very different values
        and viewpoints which lead to orthogonal assessments
        concerning the desirability of deploying IPng.

    3)  This paper provides insight into the mindset of a large
        corporate user concerning the relevant issues surrounding an
        IPng deployment.  The bottom line is that a new deployment of
        IPng runs counter to several business drivers.  A key point
        to highlight is that end users actually buy applications --
        not networking technologies.


> The argument that we are rapidly running out of IPv4 addresses has
> always been a significant concern for me personally.  New
> deployments (e.g., civil aviation's ATN IPS) and certain industries
> that have vast numbers of networked devices (e.g., electrical power
> industry) are excellent candidates to adopt IPv6 for this very
> reason. However, for the majority of end users, I expect us to
> prefer to indefinitely use IPv4 by leveraging map-and-encaps
> techniques such as RANGER, despite the fact that RANGER is part of
> the effort to support a clean migration to IPv6.

I agree with all this, except I would use the term Core-Edge
Separation architectures, and at present I think LISP and Ivip are a
better solution than RANGER for either IPv4 or IPv6.


Brian, in (msg05934) you wrote, responding to Noel's (msg05930):

> That is the true unknown for IPv4 - now that we have reached the
> end game in prefix allocation, will the resulting food fight
> recreate a swamp of a few million tiny prefixes that can't be
> aggregated?

I think this is an evocative description!

However, if I was standing on railway tracks and saw a freight-train
coming, I wouldn't regard its imminent arrival as a "true unknown".


> This is the main reason that my CCR paper says: don't extrapolate
> the graphs. We really don't know if the number of prefixes per
> origin AS will remain stable or grow signficantly.

As I argued above, I think there are strong reasons to believe the
rate of growth will accelerate.  However, I haven't yet read your paper:

   Observed Relationships between Size Measures of the Internet
   Brian E. Carpenter       April 2009
   Department of Computer Science, University of Auckland
   ACM SIGCOMM Computer Communication Review Volume 39, Number 2,



Scott, in (msg05935) you wrote, responding to Joel's (msg05925):

> Joel, I don't quite get it.  First, if you only deal with v6
> then would you create two parallel Internets?  If so you need
> routing and addressing interworking.

I agree.  The protocols I can think of which directly inter-work are
email and DNS.

Also, as Patrick and I discussed in recent days in the "SIP will work
fine with CES, CEE won't work with NAT" thread, there is also a lot
of scope for IPv4/IPv6 agnostic client-server communications based on
FQDNs and HTTP.

> Second, there will be pressure for growth in the number of IPv4
> _prefixes_ and if we don't design a way to support it, people will
> find ways we don't like.

Yes - I agree entirely.

  - Robin


MNs with public addresses?

There are potential problems with having a cellphone with a global
unicast IPv4 address or an IPv4 /64  There would be no way of
controlling unwanted streams of packets, from targeted attacks or
from random or port scanning for vulnerable hosts.  These would cost
money and clog up the downstream link.

Behind NAT, the cellphone only has to receive packets which are
returned by hosts to which it has opened up a session.  A single,
reasonably random, IPv6 address might be less of a problem since it
can't be plastered with packets sent out to sequential or random
addresses.

TTR Mobility is intended to provide a portable IPv4 address (or
multiple of them) to each mobile device - or an IPv6 /64.  If
unwanted packets were a problem, then the TTR company would need to be
responsive to each customer's preferences and block unwanted packets
at the TTR, so they are not sent on the tunnel to the mobile device
itself.