idnits 2.17.1 draft-donley-nat444-impacts-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 2, 2013) is 4040 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC4689' is defined on line 1149, but no explicit reference was found in the text == Outdated reference: A later version (-09) exists of draft-donley-behave-deterministic-cgn-05 == Outdated reference: A later version (-06) exists of draft-shirasaki-nat444-02 Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group C. Donley, Ed. 3 Internet-Draft CableLabs 4 Intended status: Informational L. Howard 5 Expires: October 4, 2013 Time Warner Cable 6 V. Kuarsingh 7 Rogers Communications 8 J. Berg 9 CableLabs 10 J. Doshi 11 University of Colorado 12 April 2, 2013 14 Assessing the Impact of Carrier-Grade NAT on Network Applications 15 draft-donley-nat444-impacts-06 17 Abstract 19 NAT444 is an IPv4 extension technology being considered by Service 20 Providers to continue offering IPv4 service to customers while 21 transitioning to IPv6. This technology adds an extra Carrier-Grade 22 NAT ("CGN") in the Service Provider network, often resulting in two 23 NATs. CableLabs, Time Warner Cable, and Rogers Communications 24 independently tested the impacts of NAT444 on many popular Internet 25 services using a variety of test scenarios, network topologies, and 26 vendor equipment. This document identifies areas where adding a 27 second layer of NAT disrupts the communication channel for common 28 Internet applications. This document was updated to also include 29 Dual-Stack Lite impacts. 31 Status of this Memo 33 This Internet-Draft is submitted in full conformance with the 34 provisions of BCP 78 and BCP 79. 36 Internet-Drafts are working documents of the Internet Engineering 37 Task Force (IETF). Note that other groups may also distribute 38 working documents as Internet-Drafts. The list of current Internet- 39 Drafts is at http://datatracker.ietf.org/drafts/current/. 41 Internet-Drafts are draft documents valid for a maximum of six months 42 and may be updated, replaced, or obsoleted by other documents at any 43 time. It is inappropriate to use Internet-Drafts as reference 44 material or to cite them other than as "work in progress." 46 This Internet-Draft will expire on October 4, 2013. 48 Copyright Notice 49 Copyright (c) 2013 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents 54 (http://trustee.ietf.org/license-info) in effect on the date of 55 publication of this document. Please review these documents 56 carefully, as they describe your rights and restrictions with respect 57 to this document. Code Components extracted from this document must 58 include Simplified BSD License text as described in Section 4.e of 59 the Trust Legal Provisions and are provided without warranty as 60 described in the Simplified BSD License. 62 Table of Contents 64 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4 65 2. Testing Scope . . . . . . . . . . . . . . . . . . . . . . . . 5 66 2.1. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . 5 67 2.1.1. Case1: Single Client, Single Home Network, Single 68 Service Provider . . . . . . . . . . . . . . . . . . . 5 69 2.1.2. Case2: Two Clients, Single Home Network, Single 70 Service Provider . . . . . . . . . . . . . . . . . . . 6 71 2.1.3. Case3: Two Clients, Two Home Networks, Single 72 Service Provider . . . . . . . . . . . . . . . . . . . 7 73 2.1.4. Case4: Two Clients, Two Home Networks, Two Service 74 Providers Cross ISP . . . . . . . . . . . . . . . . . 8 75 2.2. General Test Environment . . . . . . . . . . . . . . . . . 8 76 2.3. Test Metrics . . . . . . . . . . . . . . . . . . . . . . . 10 77 2.4. Test Scenarios Executed . . . . . . . . . . . . . . . . . 11 78 2.5. General Test Methodologies . . . . . . . . . . . . . . . . 11 79 3. Observed CGN Impacts . . . . . . . . . . . . . . . . . . . . . 12 80 3.1. Dropped Services . . . . . . . . . . . . . . . . . . . . . 13 81 3.2. Performance Impacted Services . . . . . . . . . . . . . . 14 82 3.3. Improvements since 2010 . . . . . . . . . . . . . . . . . 15 83 3.4. Additional CGN Challenges . . . . . . . . . . . . . . . . 16 84 4. 2011 Summary of Results . . . . . . . . . . . . . . . . . . . 16 85 4.1. NAT444 . . . . . . . . . . . . . . . . . . . . . . . . . . 17 86 4.2. DS-Lite . . . . . . . . . . . . . . . . . . . . . . . . . 19 87 5. 2010 Summary of Results . . . . . . . . . . . . . . . . . . . 21 88 5.1. Case1: Single Client, Single Home Network, Single 89 Service Provider . . . . . . . . . . . . . . . . . . . . . 22 90 5.2. Case2: Two Clients, Single Home Network, Single 91 Service Provider . . . . . . . . . . . . . . . . . . . . . 24 92 5.3. Case3: Two Clients, Two Home Networks, Single Service 93 Provider . . . . . . . . . . . . . . . . . . . . . . . . . 24 94 5.4. Case4: Two Clients, Two Home Networks, Two Service 95 Providers Cross ISP . . . . . . . . . . . . . . . . . . . 25 97 6. CGN Mitigation . . . . . . . . . . . . . . . . . . . . . . . . 25 98 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 99 8. Security Considerations . . . . . . . . . . . . . . . . . . . 26 100 9. Informative References . . . . . . . . . . . . . . . . . . . . 26 101 Appendix A. Acknowledgements . . . . . . . . . . . . . . . . . . 27 102 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 104 1. Introduction 106 IANA, APNIC, and RIPE exhausted their IPv4 address space in 2011- 107 2012. Current projections suggest that ARIN may exhaust its free 108 pool of IPv4 addresses in 2013. IPv6 is the solution to the IPv4 109 depletion problem; however, the transition to IPv6 will not be 110 completed prior to IPv4 exhaustion. NAT444 [I-D.shirasaki-nat444] 111 and Dual-Stack Lite ([RFC6333]) are transition mechanisms that will 112 allow Service Providers to multiplex customers behind a single IPv4 113 address, which will allow many legacy devices and applications some 114 IPv4 connectivity. While both NAT444 and Dual-Stack Lite do provide 115 basic IPv4 connectivity, they impact a number of advanced 116 applications. This document describes suboptimal behaviors of NAT444 117 and DS-Lite in our test environments. 119 In July-August 2010, CableLabs, Time Warner Cable, and Rogers 120 Communications tested the impact of NAT444 on common applications 121 using Carrier Grade NAT (CGN) devices. This testing was focused on a 122 wide array of real time usage scenarios designed to evaluate the user 123 experience over the public Internet using NAT444, in both single ISP 124 and dual ISP environments. The purpose of this testing was to 125 identify applications where the technology either breaks or 126 significantly impacts the user experience. The outcome of the 127 testing revealed that applications such as video streaming, video 128 gaming and peer-to-peer file sharing are impacted by NAT444. 130 From June - October 2011, CableLabs conducted additional testing of 131 CGN technologies, including both NAT444 and Dual-Stack Lite. The 132 testing focused on working with several vendors including A10, 133 Alcatel-Lucent, and Juniper to optimize the performance of those 134 applications that experienced negative impacts during earlier CGN 135 testing and to expand the testing to DS-Lite. 137 Applications that were tested included but were not necessarily 138 limited to the following: 140 1. Video/Audio streaming, e.g. Silverlight-based applications, 141 Netflix, YouTube, Pandora 2. 143 2. Peer-to-peer applications, e.g. video gaming, uTorrent 145 3. On line gaming, e.g. Xbox 147 4. Large file transfers using File Transfer Protocol (FTP) 149 5. Session Initiation Protocol (SIP) calls via X-Lite, Skype 150 6. Social Networking, e.g. Facebook, Webkinz 152 7. Video chat, e.g. Skype 154 8. Web conferencing 156 2. Testing Scope 158 2.1. Test Cases 160 The diagrams below depict the general network architecture used for 161 testing NAT444 and Dual Stack-Lite co-existence technologies at 162 CableLabs. 164 2.1.1. Case1: Single Client, Single Home Network, Single Service 165 Provider 167 ^^^^^^^^ 168 (Internet) 169 vvvvvvvv 170 | 171 | 172 +---------------+ 173 | CGN | 174 +---------------+ 175 | 176 +---------------+ 177 | CMTS | 178 +---------------+ 179 | 180 +---------------+ 181 | CM | 182 +---------------+ 183 | 184 +-------------------------+ 185 | Home Router | 186 +-------------------------+ 187 | 188 +---------------+ 189 | Client | 190 +---------------+ 192 This is a typical case for a client accessing content on the 193 Internet. For this case, we focused on basic web browsing, voice and 194 video chat, instant messaging, video streaming (using YouTube, Google 195 Videos , etc.), Torrent leeching and seeding, FTP, and gaming. 197 2.1.2. Case2: Two Clients, Single Home Network, Single Service Provider 199 ^^^^^^^^ 200 (Internet) 201 vvvvvvvv 202 | 203 | 204 +---------------+ 205 | CGN | 206 +---------------+ 207 | 208 +---------------+ 209 | CMTS | 210 +---------------+ 211 | 212 +---------------+ 213 | CM | 214 +---------------+ 215 | 216 +-------------------------+ 217 | Home Router | 218 +-------------------------+ 219 | | 220 +---------------+ +---------------+ 221 | Client | | Client | 222 +---------------+ +---------------+ 224 This is similar to Case 1, except that two clients are behind the 225 same LSN and in the same home network. This test case was conducted 226 to observe any change in speed in basic web browsing and video 227 streaming. 229 2.1.3. Case3: Two Clients, Two Home Networks, Single Service Provider 231 ^^^^^^^^ 232 (Internet) 233 vvvvvvvv 234 | 235 | 236 +---------------+ 237 | CGN | 238 +---------------+ 239 | 240 +---------------+ 241 | CMTS | 242 +---------------+ 243 | 244 ---------------------------------------- 245 | | 246 +---------------+ +---------------+ 247 | CM | | CM | 248 +---------------+ +---------------+ 249 | | 250 +-------------------------+ +-------------------------+ 251 | Home Router | | Home Router | 252 +-------------------------+ +-------------------------+ 253 | | 254 +---------------+ +---------------+ 255 | Client | | Client | 256 +---------------+ +---------------+ 258 In this scenario, the two clients are under the same LSN but behind 259 two different gateways. This simulates connectivity between two 260 residential subscribers on the same ISP. We tested peer-to-peer 261 applications. 263 2.1.4. Case4: Two Clients, Two Home Networks, Two Service Providers 264 Cross ISP 266 ^^^^^^^^ ^^^^^^^^ 267 ( ISP A ) ( ISP B ) 268 Vvvvvvvv vvvvvvvv 269 | | 270 +---------------+ +---------------+ 271 | LSN | | LSN | 272 +---------------+ +---------------+ 273 | | 274 +---------------+ +---------------+ 275 | CMTS | | CMTS | 276 +---------------+ +---------------+ 277 | | 278 +---------------+ +---------------+ 279 | CM | | CM | 280 +---------------+ +---------------+ 281 | | 282 +-------------------------+ +-------------------------+ 283 | Home Router | | Home Router | 284 +-------------------------+ +-------------------------+ 285 | | 286 +---------------+ +---------------+ 287 | Client | | Client | 288 +---------------+ +---------------+ 290 This test case is similar to Case 1 but with the addition of another 291 identical ISP. This topology allows us to test traffic between two 292 residential customers connected across the Internet. We focused on 293 client-to-client applications such as IM and peer-to-peer. 295 2.2. General Test Environment 297 The lab environment was intended to emulate multiple service provider 298 networks with a CGN deployed, and with connectivity to the public 299 IPv4 or IPv6 internet (as dictated by the co-existence technology 300 under test). This was accomplished by configuring a CGN behind 301 multiple CMTSes and setting up multiple home networks for each ISP. 302 Testing involved sending traffic to and from the public internet in 303 both single and dual ISP environments, using both single and multiple 304 home networks. The following equipment was used for testing: 306 o CGN 308 o CMTS 309 o IP sniffer 311 o RF sniffer 313 o Metrics tools (for network performance) 315 o CPE gateway devices 317 o Laptop or desktop computers (multiple OS used) 319 o Gaming consoles 321 o iPad or tablet devices 323 o other CE equipment, e.g. BluRay players supporting miscellaneous 324 applications 326 One or more CPE gateway devices were configured in the home network. 327 One or more host devices behind the gateways were also configured in 328 order to test conditions such as multiple users on multiple home 329 networks in the CGN architecture, both in single and dual ISP 330 environments. 332 The scope of testing was honed down to the specific types of 333 applications and network conditions that demonstrated a high 334 probability of diminishing user experience based on prior testing. 335 The following use cases were tested: 337 1. Video streaming over Netflix 339 2. Video streaming over YouTube 341 3. Video streaming over Joost 343 4. On line gaming with Xbox (one user) 345 5. Peer to Peer gaming with Xbox (two users) 347 6. Bit Torrent/uTorrent file seeding/leeching 349 7. Pandora internet radio 351 8. FTP server 353 9. Web conferencing (GTM, WebEx) 355 10. Social Networking - Facebook, Webkinz (chat, YouTube, file 356 transfer) 358 11. Internet Archive - Video and Audio streaming; large file 359 downloads 361 12. Video streaming using iClips 363 13. SIP Calls - X-Lite, Skype, PJSIP 365 14. MS Smooth Streaming (Silverlight) 367 15. Video chat - Skype, OoVoo 369 The following CPE devices were used for testing these applications on 370 one or more home networks: 372 1. Windows 7, XP and Vista based laptops 374 2. MAC OS X laptop 376 3. iPad 378 4. Xbox gaming consoles 380 5. iPhone and Android smartphones 382 6. LG Blu-Ray player (test applications such as Netflix, Vudu, etc.) 384 7. Home routers - Netgear, Linksys, D-Link, Cisco, Apple 386 2.3. Test Metrics 388 Metrics data that were collected during the course of testing were 389 related to throughput, latency, and jitter. These metrics were 390 evaluated under three conditions: 392 1. Initial finding on the CGN configuration used for testing 394 2. Retest of the same test scenario with the CGN removed from the 395 network 397 3. Retest with a new configuration (optimized) on the CGN (when 398 possible) 400 In our testing, we found only slight differences with respect to 401 latency or jitter when the CGN was in the network versus when it was 402 not present in the network. It should be noted that we did not 403 conduct any performance testing and metrics gathered were limited to 404 single session scenarios. Also, bandwidth was not restricted on the 405 DOCSIS network. Simulated homes shared a single DOCSIS upstream and 406 downstream channel. 408 +---------+---------+---------+---------+-----------------+---------+ 409 | Case | Avg | Min | Max | RFC4689 | Max | 410 | | Latency | Latency | Latency | Absolute Avg | Jitter | 411 | | | | | Jitter | | 412 +---------+---------+---------+---------+-----------------+---------+ 413 | With | 240.32 | 233.77 | 428.40 | 1.86 us | 191.22 | 414 | CGN | us | us | us | | us | 415 +---------+---------+---------+---------+-----------------+---------+ 416 | Without | 211.88 | 190.39 | 402.69 | 0.07 us | 176.16 | 417 | CGN | us | us | us | | us | 418 +---------+---------+---------+---------+-----------------+---------+ 420 CGN Performance 422 Note: Performance testing as defined by CableLabs includes load 423 testing, induction of impairments on the network, etc. This type of 424 testing was out of scope for CGN testing. 426 2.4. Test Scenarios Executed 428 The following test scenarios were executed using the aforementioned 429 applications and test equipment: 431 1. Single ISP, Single Home Network with Single User 433 2. Single ISP, Two Home Networks With One User on Each Network 435 3. Dual ISPs, Single Home Network with Single User on each ISP 437 4. Dual ISPs, One Home Network With One User ISP-A; Two Home 438 Networks with one user on each for ISP-B 440 These test scenarios were executed for both NAT444 and DS-Lite 441 technologies. 443 2.5. General Test Methodologies 445 The CGN was configured for optimal setting for the specific test 446 being executed for NAT444 or DS-Lite. Individual vendors provided 447 validation of the configuration used for the co-existence technology 448 under test prior to the start of testing. Some NAT444 testing used 449 private [RFC1918] IPv4 space between the CGN and CPE router; other 450 tests used public (non-[RFC1918]) IPv4 space between the CGN and CPE 451 router. With the exception of 6to4 ([RFC3056]) traffic, we observed 452 no difference in test results whether private or public address space 453 was used. 6to4 failed when public space was used between the CGN and 454 CPE router was public, but CPE routers did not initiate 6to4 when 455 private space was used. 457 CPE gateways and client devices were configured with IPv4 or IPv6 458 addresses using DHCP or manual configuration as required by each of 459 the devices used in the test. 461 All devices were brought to operational state. Connectivity of CPE 462 devices to provider network and public Internet were verified prior 463 to start of each test. 465 IP sniffers and metrics tools were configured as required before 466 starting tests. IP capture and metrics data was collected for all 467 failed test scenarios. Sniffing was configured behind the home 468 routers, north and south of the CMTS, and north and south of the CGN. 470 The test technician executed test scenarios listed above, for single 471 and dual ISP environments, testing multiple users on multiple home 472 networks, using the applications described above, where applicable to 473 the each specific test scenario. Results checklists were compiled 474 for all tests executed and for each combination of devices tested. 476 3. Observed CGN Impacts 478 CGN testing revealed that basic services such as e-mail and web 479 browsing worked normally and as expected. However, there were some 480 service affecting issues noted for applications that fall into two 481 categories; dropped service and performance impacted service. In 482 addition, for some specific applications in which the performance was 483 impacted, throughput, latency and jitter measurements were taken. We 484 observed that performance often differs from vendor to vendor and 485 from test environment to test environment, and the results are 486 somewhat difficult to predict. So as to not become a comparison 487 between different vendor implementations, these results are presented 488 in summary form. When issues were identified, we worked with the 489 vendors involved to confirm the specific issues and explore 490 workarounds. Except where noted, impacts to NAT444 and DS-Lite were 491 similar. 493 In 2010 testing, we identified that IPv6 transition technologies such 494 as 6to4 [RFC3056] and Teredo [RFC4380]) fail outright or are subject 495 to severe service degradation. We did not repeat transition 496 technology testing in 2011. 498 Note: While e-mail and web browsing operated as expected within our 499 environment, there have been reports that anti-spam/anti-abuse 500 measures limiting the number of connections from a single address can 501 cause problems in a CGN environment by improperly interpreting 502 address sharing as too many connections from a single device. Care 503 should be taken when deploying CGN to mitigate the impact of address 504 sharing when configuring anti-spam/anti-abuse measures. See Section 505 3.4. 507 3.1. Dropped Services 509 Several peer-to-peer applications, specifically peer-to-peer gaming 510 using Xbox and peer-to-peer SIP calls using the PJSIP client, failed 511 in both the NAT444 and Dual-Stack Lite environments. Many CGN 512 devices use "full cone" NAT so that once the CGN maps a port for 513 outbound services, it will accept incoming connections to that port. 514 However, some applications did not first send outgoing traffic and 515 hence did not open an incoming port through the CGN. Other 516 applications try to open a particular fixed port through the CGN; 517 while service will work for a single subscriber behind the CGN, it 518 fails when multiple subscribers try to use that port. 520 PJSIP and other SIP software worked when clients used a registration 521 server to initiate calls, provided that the client inside the CGN 522 initiated the traffic first and that only one SIP user was active 523 behind a single IPv4 address at any given time. However, in our 524 testing, we observed that when making a direct client-to-client SIP 525 call across two home networks on a single ISP, or when calling from a 526 single home network across dual ISPs, calls could neither be 527 initiated nor received. 529 In the case of peer-to-peer gaming between two Xbox 360 users in 530 different home networks on the same ISP, the game could not be 531 connected between the two users. Both users shared an outside IP 532 address, and tried to connect to the same port, causing a connection 533 failure. There are some interesting nuances to this problem. In the 534 case where two users are in the same home network and the scenario is 535 through a single ISP, when the Xbox tries to register with the Xbox 536 server, the server sees that both Xboxes are coming through the same 537 public IP address and directs the devices to connect using their 538 internal IP addresses. So, the connection ultimately gets 539 established directly between both Xboxes via the home gateway, rather 540 than the Xbox server. In the case where there are two Xbox users on 541 two different home networks using a single ISP, and the CGN is 542 configured with only one public IPv4 address, this scenario will not 543 work because the route between the two users cannot be determined. 544 However, if the CGN is configured with two public NAT IP addresses 545 this scenario will work because now there is a unique IP address to 546 communicate with. This is not an ideal solution, however, because it 547 means that there is a one-to-one relationship between IP addresses in 548 the public NAT and the number of Xbox users on each network. 550 Update: in December, 2011, Microsoft released an update for Xbox. 551 While we did not conduct thorough testing using the new release, 552 preliminary testing indicates that Xboxes that upgraded to the latest 553 version can play head-to-head behind a CGN, at least for some games. 555 Other peer-to-peer applications that were noted to fail were seeding 556 sessions initiated on Bittorent and uTorrent. In our test, torrent 557 seeding was initiated on a client inside the CGN. Leeching was 558 initiated using a client on the public Internet. It was observed 559 that direct peer-to-peer seeding did not work. However, the torrent 560 session typically redirected the leeching client to a proxy server, 561 in which case the torrent session was set up successfully. 562 Additionally, with the proxy in the network, re-seeding via 563 additional leech clients worked as would be expected for a typical 564 torrent session. Finally, uTorrent tries to use STUN to identify its 565 outside address. In working with vendors, we learned that increasing 566 the STUN timeout to 4 minutes improved uTorrent seeding performance 567 behind a CGN, resulting in the ability for the uTorrent client to 568 open a port and successfully seed content. 570 FTP sessions to servers located inside the home (e.g. behind two 571 layers of NAT) failed. When the CGN was bypassed and traffic only 572 needed to flow through one layer of NAT, clients were able to 573 connect. Finally, multicast traffic was not forwarded through the 574 CGN. 576 3.2. Performance Impacted Services 578 Large size file transfers and multiple video streaming sessions 579 initiated on a single client on the same home network behind the CGN 580 experienced reduced performance in our environment. We measured 581 these variations in user experience against a baseline IPv4 582 environment where NAT is not deployed. 584 In our testing, we tried large file transfers from several FTP sites, 585 as well as downloading sizable audio and video files (750MB - 1.4 GB) 586 from the Internet Archive. We observed that when Dual-Stack Lite was 587 implemented for some specific home router and client combinations, 588 the transfer rate was markedly slower. For example, PC1 using one 589 operating system behind the same home router as PC2 using a different 590 operating system yielded a transfer rate of 120Kbps for PC1, versus 591 250Kbps for PC2. Our conclusion is that varying combinations of home 592 routers and CE client devices may result in a user experience that is 593 less than what the user would expect for typical applications. It is 594 also difficult to predict which combinations of CPE routers and CE 595 devices will produce a reduced experience for the user. We did not 596 analyze the root cause of the divergence in performance across CE 597 devices, as this was beyond the scope of our testing. However, as 598 this issue was specific to Dual-Stack Lite, we suspect that it is 599 related to the MTU. 601 While video streaming sessions for a single user generally performed 602 well, testing revealed that video streaming sessions such as 603 Microsoft Smooth Streaming technology (i.e. Silverlight) or Netflix 604 might also exhibit some service impacting behavior. In particular, 605 this was observed on one older, yet popular and well-known CPE router 606 where the first session was severely degraded when a second session 607 was initiated in the same home network. Traffic from the first 608 session ceased for 8 s once the second session was initiated. While 609 we are tempted to write this off as a problematic home router, its 610 popularity suggests that home router interactions may cause issues in 611 NAT444 deployments (newer routers that support DS-Lite were not 612 observed to experience this condition). Overall, longer buffering 613 times for video sessions were noted for most client devices behind 614 all types of home routers. However, once the initial buffering was 615 complete, the video streams were consistently smooth. In addition, 616 there were varying degrees as to how well multiple video sessions 617 were displayed on various client devices across the CPE routers 618 tested. Some video playback devices performed better than others. 620 3.3. Improvements since 2010 622 Since CableLabs completed initial CGN testing in 2010, there have 623 been quantifiable improvements in performance over CGN since that 624 time. These improvements may be categorized as follows: 626 o Content provider updates 628 o Application updates 630 o Improvements on the CGNs themselves 632 In terms of content provider updates, we have noted improvements in 633 the overall performance of streaming applications in the CGN 634 environment. Whereas applications such as streaming video were very 635 problematic a year ago with regard to jitter and latency, our most 636 recent testing revealed that there is less of an issue with these 637 conditions, except in some cases when multiple video streaming 638 sessions were initiated on the same client using specific types of 639 home routers. Applications such as MS Smooth Streaming appear to 640 have addressed these issues to some degree. 642 As far as application updates, use of STUN and/or proxy servers to 643 offset some of the limitations of NAT and tunneling in the network 644 are more evident as workarounds to the peer-to-peer issues. 645 Applications appear to have incorporated other mechanisms for 646 delivering content faster, even if buffering times are somewhat 647 slower and the content is not rendered as quickly. 649 CGN vendors have also upgraded their devices to mitigate several 650 known issues with specific applications. With regard to addressing 651 peer-to-peer SIP call applications, port reservations appear to be a 652 workaround to the problem. However, this approach has limitations 653 because of there are limited numbers of users that can have port 654 reservations at any given time. For example, one CGN implementation 655 allowed a port reservation to be made on port 5060 (default SIP port) 656 but this was the only port that could be configured for the SIP 657 client. This means that only one user can be granted the port 658 reservation. 660 3.4. Additional CGN Challenges 662 There are other challenges that arise when using shared IPv4 address 663 space, as with NAT444. Some of these challenges include: 665 o Loss of geolocation information - Often, translation zones will 666 cross traditional geographic boundaries. Since the source 667 addresses of packets traversing an LSN are set to the external 668 address of the LSN, it is difficult for external entities to 669 associate IP/Port information to specific locations/areas. 671 o Lawful Intercept/Abuse Response - Due to the nature of NAT444 672 address sharing, it will be hard to determine the customer/ 673 endpoint responsible for initiating a specific IPv4 flow based on 674 source IP address alone. Content providers, service providers, 675 and law enforcement agencies will need to use new mechanisms 676 (e.g., logging source port and timestamp in addition to source IP 677 address) to potentially mitigate this new problem. This may 678 impact the timely response to various identification requests. 679 See [RFC6269]. 681 o Antispoofing - Multiplexing users behind a single IP address can 682 lead to situations where traffic from that address triggers 683 antispoofing/DDoS protection mechanisms, resulting in 684 unintentional loss of connectivity for some users. We have 685 received reports of such antispoofing/DDoS mechanisms affecting 686 email and web services in some instances, but did not experience 687 them in our environment. 689 4. 2011 Summary of Results 690 4.1. NAT444 692 +--------------+--------+--------+--------+--------------+----------+ 693 | Test | Single | Single | Dual | Dual ISP, | Notes | 694 | Scenario | ISP, | ISP, | ISP, | One HN+One | | 695 | (per Test | Single | Two | One HN | User on | | 696 | Plan) | HN, | HN, | with | ISP-A, Two | | 697 | | Single | Single | One | HN with One | | 698 | | User | User | User | User on Each | | 699 | | | on | on | on ISP-B | | 700 | | | Each | Each | | | 701 | | | | ISP | | | 702 +--------------+--------+--------+--------+--------------+----------+ 703 | Video | Pass | Pass | Pass | Pass | fails | 704 | streaming | | | | | behind | 705 | over Netflix | | | | | one | 706 | | | | | | router | 707 +--------------+--------+--------+--------+--------------+----------+ 708 | Video | Pass | Pass | Pass | Pass | | 709 | streaming | | | | | | 710 | over YouTube | | | | | | 711 +--------------+--------+--------+--------+--------------+----------+ 712 | Video | Pass | Pass | Pass | Pass | | 713 | streaming | | | | | | 714 | over Joost | | | | | | 715 +--------------+--------+--------+--------+--------------+----------+ 716 | Online | Pass | Pass | Pass | NT | | 717 | gaming with | | | | | | 718 | one user | | | | | | 719 +--------------+--------+--------+--------+--------------+----------+ 720 | Peer to Peer | Pass | Fail | Pass | NT | fails | 721 | gaming with | | | | | when | 722 | two users | | | | | both | 723 | | | | | | users | 724 | | | | | | NAT to | 725 | | | | | | same | 726 | | | | | | address | 727 +--------------+--------+--------+--------+--------------+----------+ 728 | Bit Torrent | Fail | Fail | Fail | Fail | | 729 | uTorrent | | | | | | 730 | file seeding | | | | | | 731 +--------------+--------+--------+--------+--------------+----------+ 732 | Bit Torrent | Pass | Pass | Pass | Pass | | 733 | uTorrent | | | | | | 734 | file | | | | | | 735 | leeching | | | | | | 736 +--------------+--------+--------+--------+--------------+----------+ 737 +--------------+--------+--------+--------+--------------+----------+ 738 | Pandora | Pass | Pass | Pass | Pass | | 739 | internet | | | | | | 740 | radio | | | | | | 741 +--------------+--------+--------+--------+--------------+----------+ 742 | FTP server | Pass | Pass | Pass | Pass | | 743 +--------------+--------+--------+--------+--------------+----------+ 744 | Web | Pass | Pass | Pass | Pass | | 745 | conferencing | | | | | | 746 | GTM | | | | | | 747 +--------------+--------+--------+--------+--------------+----------+ 748 | Social | Pass | Pass | Pass | Pass | | 749 | Networking | | | | | | 750 | Facebook | | | | | | 751 +--------------+--------+--------+--------+--------------+----------+ 752 | Social | Pass | Pass | Pass | Pass | | 753 | Networking | | | | | | 754 | Webkinz | | | | | | 755 +--------------+--------+--------+--------+--------------+----------+ 756 | X-Lite for | Pass | Pass | Pass | Pass | | 757 | SIP calls | | | | | | 758 | with proxy | | | | | | 759 +--------------+--------+--------+--------+--------------+----------+ 760 | X-Lite for | Fail | Fail | Fail | Fail | | 761 | SIP calls no | | | | | | 762 | proxy | | | | | | 763 +--------------+--------+--------+--------+--------------+----------+ 764 | Skype text | Pass | Pass | Pass | Pass | | 765 | chat | | | | | | 766 +--------------+--------+--------+--------+--------------+----------+ 767 | Skype video | Pass | Pass | Pass | Pass | | 768 | chat | | | | | | 769 +--------------+--------+--------+--------+--------------+----------+ 770 | Oovoo | Pass | Pass | Pass | Pass | | 771 +--------------+--------+--------+--------+--------------+----------+ 772 | MS Smooth | Pass | Pass | Pass | Pass | | 773 | streaming | | | | | | 774 +--------------+--------+--------+--------+--------------+----------+ 775 | Internet | Pass | Pass | Pass | Pass | | 776 | Archive | | | | | | 777 | video | | | | | | 778 | streaming | | | | | | 779 +--------------+--------+--------+--------+--------------+----------+ 780 | Internet | Pass | Pass | Pass | Pass | | 781 | Archive | | | | | | 782 | audio | | | | | | 783 | streaming | | | | | | 784 +--------------+--------+--------+--------+--------------+----------+ 785 +--------------+--------+--------+--------+--------------+----------+ 786 | Internet | Pass | Pass | Pass | Pass | | 787 | Archive file | | | | | | 788 | download | | | | | | 789 +--------------+--------+--------+--------+--------------+----------+ 790 | Iclips | Pass | Pass | Pass | Pass | | 791 +--------------+--------+--------+--------+--------------+----------+ 793 NAT-444 795 4.2. DS-Lite 797 +---------------+--------+---------+-------------+------------------+ 798 | Test Scenario | DS-Lit | Duratio | Description | General | 799 | (per Test | eTest | nof Tes | of Test | Observations/Not | 800 | Plan) | Resul | tPerfor | Execution | es | 801 | | ts | med | | | 802 +---------------+--------+---------+-------------+------------------+ 803 | Video | Pass | 15 | | | 804 | streaming | | | | | 805 | over Netflix | | | | | 806 +---------------+--------+---------+-------------+------------------+ 807 | Video | Pass | 10 | | | 808 | streaming | | | | | 809 | over YouTube | | | | | 810 +---------------+--------+---------+-------------+------------------+ 811 | Video | Pass | 10 | | | 812 | streaming | | | | | 813 | over Joost | | | | | 814 +---------------+--------+---------+-------------+------------------+ 815 | On line | Pass | 15 | | | 816 | gaming (one | | | | | 817 | user) | | | | | 818 +---------------+--------+---------+-------------+------------------+ 819 | Peer to Peer | Fail | NA | user inside | Users inside | 820 | gaming (two | | | HN1 playing | both HN are not | 821 | users) | | | game | able to connect. | 822 | | | | against | The error shown | 823 | | | | user inside | on console- "The | 824 | | | | HN2 | game session is | 825 | | | | | no longer | 826 | | | | | available" | 827 +---------------+--------+---------+-------------+------------------+ 828 +---------------+--------+---------+-------------+------------------+ 829 | Bit | Fail | 12 | user on the | | 830 | Torrent/uTorr | | | internet is | | 831 | ent file | | | able to | | 832 | seeding | | | download | | 833 | | | | file using | | 834 | | | | proxy | | 835 | | | | server and | | 836 | | | | not | | 837 | | | | peer-to-pee | | 838 | | | | r | | 839 +---------------+--------+---------+-------------+------------------+ 840 | Bit | Pass | 10 | | | 841 | Torrent/uTorr | | | | | 842 | ent file | | | | | 843 | leeching | | | | | 844 +---------------+--------+---------+-------------+------------------+ 845 | Pandora | Pass | 10 | | | 846 | internet | | | | | 847 | radio | | | | | 848 +---------------+--------+---------+-------------+------------------+ 849 | FTP server | Pass | 700 Mb | | | 850 +---------------+--------+---------+-------------+------------------+ 851 | Web | Pass | 10 | | | 852 | conferencing | | | | | 853 | (GTM) | | | | | 854 +---------------+--------+---------+-------------+------------------+ 855 | Social | Pass | NA | | | 856 | Networking - | | | | | 857 | Facebook | | | | | 858 +---------------+--------+---------+-------------+------------------+ 859 | Social | Pass | NA | | | 860 | Networking - | | | | | 861 | Webkinz | | | | | 862 +---------------+--------+---------+-------------+------------------+ 863 | X-Lite (for | Pass | 10 | | | 864 | SIP calls) | | | | | 865 | (proxy given) | | | | | 866 +---------------+--------+---------+-------------+------------------+ 867 | X-Lite (for | Fail | NA | | | 868 | SIP calls) | | | | | 869 | (proxy not | | | | | 870 | given) | | | | | 871 +---------------+--------+---------+-------------+------------------+ 872 | Skype text | Pass | NA | | | 873 | chat | | | | | 874 +---------------+--------+---------+-------------+------------------+ 875 +---------------+--------+---------+-------------+------------------+ 876 | Skype video | Pass | 20 | | | 877 | chat | | | | | 878 +---------------+--------+---------+-------------+------------------+ 879 | Oovoo | Pass | 15 | | | 880 +---------------+--------+---------+-------------+------------------+ 881 | MS Smooth | Pass | 10 | | | 882 | streaming | | | | | 883 +---------------+--------+---------+-------------+------------------+ 884 | Internet | Pass | 10 | | | 885 | Archive - | | | | | 886 | video | | | | | 887 | streaming | | | | | 888 +---------------+--------+---------+-------------+------------------+ 889 | Internet | Pass | 5 | | | 890 | Archive - | | | | | 891 | audio | | | | | 892 | streaming | | | | | 893 +---------------+--------+---------+-------------+------------------+ 894 | Internet | Pass | 80 Mb | | | 895 | Archive - | | | | | 896 | file download | | | | | 897 +---------------+--------+---------+-------------+------------------+ 898 | Iclips | Pass | 10 | | | 899 +---------------+--------+---------+-------------+------------------+ 901 DSLite 903 5. 2010 Summary of Results 905 The tables below summarize results from 2010 NAT444 testing at 906 CableLabs, Time Warner Cable, and Rogers Communications. They are 907 included for comparison with 2011 results, documented above. 909 5.1. Case1: Single Client, Single Home Network, Single Service Provider 911 +--------------+---------------+------------------------------------+ 912 | Test Case | Results | Notes | 913 +--------------+---------------+------------------------------------+ 914 | Web browsing | pass | | 915 +--------------+---------------+------------------------------------+ 916 | Email | pass | | 917 +--------------+---------------+------------------------------------+ 918 | FTP download | pass | performance degraded on very large | 919 | | | downloads | 920 +--------------+---------------+------------------------------------+ 921 | Bittorrent | pass | | 922 | leeching | | | 923 +--------------+---------------+------------------------------------+ 924 | Bittorrent | fail | | 925 | seeding | | | 926 +--------------+---------------+------------------------------------+ 927 | Video | pass | | 928 | streaming | | | 929 +--------------+---------------+------------------------------------+ 930 | Voice chat | pass | | 931 +--------------+---------------+------------------------------------+ 932 | Netflix | pass | | 933 | streaming | | | 934 +--------------+---------------+------------------------------------+ 935 | Instant | pass | | 936 | Messaging | | | 937 +--------------+---------------+------------------------------------+ 938 | Ping | pass | | 939 +--------------+---------------+------------------------------------+ 940 | Traceroute | pass | | 941 +--------------+---------------+------------------------------------+ 942 | Remote | pass | | 943 | desktop | | | 944 +--------------+---------------+------------------------------------+ 945 | VPN | pass | | 946 +--------------+---------------+------------------------------------+ 947 | Xbox live | pass | | 948 +--------------+---------------+------------------------------------+ 949 | Xbox online | pass | Blocked by some LSNs. | 950 +--------------+---------------+------------------------------------+ 951 | Xbox network | fail | Your NAT type is moderate. For | 952 | test | | best online experience you need an | 953 | | | open NAT configuration. You | 954 | | | should enable UPnP on the router. | 955 +--------------+---------------+------------------------------------+ 956 +--------------+---------------+------------------------------------+ 957 | Nintendo Wii | pass behind | | 958 | | one LSN, fail | | 959 | | behind | | 960 | | another | | 961 +--------------+---------------+------------------------------------+ 962 | Playstation | pass | | 963 | 3 | | | 964 +--------------+---------------+------------------------------------+ 965 | Team | fail | pass behind one LSN, but | 966 | Fortress 2 | | performance degraded | 967 +--------------+---------------+------------------------------------+ 968 | Starcraft II | pass | | 969 +--------------+---------------+------------------------------------+ 970 | World of | pass | | 971 | Warcraft | | | 972 +--------------+---------------+------------------------------------+ 973 | Call of Duty | pass | performance degraded behind one | 974 | | | LSN | 975 +--------------+---------------+------------------------------------+ 976 | Slingcatcher | fail | | 977 +--------------+---------------+------------------------------------+ 978 | Netflix | fail | pass behind one LSN | 979 | Party (Xbox) | | | 980 +--------------+---------------+------------------------------------+ 981 | Hulu | pass | performance degraded behind one | 982 | | | LSN | 983 +--------------+---------------+------------------------------------+ 984 | AIM File | pass | performance degraded | 985 | Tranfer | | | 986 +--------------+---------------+------------------------------------+ 987 | Webcam | fail | | 988 +--------------+---------------+------------------------------------+ 989 | 6to4 | fail | | 990 +--------------+---------------+------------------------------------+ 991 | Teredo | fail | | 992 +--------------+---------------+------------------------------------+ 994 Case1 996 5.2. Case2: Two Clients, Single Home Network, Single Service Provider 998 +-----------------+---------+---------------------------------------+ 999 | Test Case | Results | Notes | 1000 +-----------------+---------+---------------------------------------+ 1001 | Bittorrent | pass | | 1002 | leeching | | | 1003 +-----------------+---------+---------------------------------------+ 1004 | Bittorrent | fail | | 1005 | seeding | | | 1006 +-----------------+---------+---------------------------------------+ 1007 | Video streaming | fail | | 1008 +-----------------+---------+---------------------------------------+ 1009 | Voice chat | pass | | 1010 +-----------------+---------+---------------------------------------+ 1011 | Netflix | pass | performance severely impacted, | 1012 | streaming | | eventually failed | 1013 +-----------------+---------+---------------------------------------+ 1014 | IM | pass | | 1015 +-----------------+---------+---------------------------------------+ 1016 | Limewire | pass | | 1017 | leeching | | | 1018 +-----------------+---------+---------------------------------------+ 1019 | Limewire | fail | | 1020 | seeding | | | 1021 +-----------------+---------+---------------------------------------+ 1023 Case2 1025 5.3. Case3: Two Clients, Two Home Networks, Single Service Provider 1027 +-------------------+---------+-------+ 1028 | Test Case | Results | Notes | 1029 +-------------------+---------+-------+ 1030 | Limewire leeching | pass | | 1031 +-------------------+---------+-------+ 1032 | Limewire seeding | fail | | 1033 +-------------------+---------+-------+ 1034 | Utorrent leeching | pass | | 1035 +-------------------+---------+-------+ 1036 | Utorrent seeding | fail | | 1037 +-------------------+---------+-------+ 1039 Case3 1041 5.4. Case4: Two Clients, Two Home Networks, Two Service Providers Cross 1042 ISP 1044 +------------------+---------+-------+ 1045 | Test Case | Results | Notes | 1046 +------------------+---------+-------+ 1047 | Skype voice call | pass | | 1048 +------------------+---------+-------+ 1049 | IM | pass | | 1050 +------------------+---------+-------+ 1051 | FTP | fail | | 1052 +------------------+---------+-------+ 1053 | Facebook chat | pass | | 1054 +------------------+---------+-------+ 1055 | Skype video | pass | | 1056 +------------------+---------+-------+ 1058 Case4 1060 6. CGN Mitigation 1062 Our testing did not focus on mitigating the impact of Carrier Grade 1063 NAT, as described above. As such, mitigation is not the focus of 1064 this document. However, there are several approaches that could 1065 lessen the impacts described above. 1067 +-----------------+-------------------------------------------------+ 1068 | Challenge | Potential Workaround(s) | 1069 +-----------------+-------------------------------------------------+ 1070 | Peer-to-peer | Use a proxy server; [I-D.ietf-pcp-base] | 1071 +-----------------+-------------------------------------------------+ 1072 | Gaming | [I-D.ietf-pcp-base] | 1073 +-----------------+-------------------------------------------------+ 1074 | Negative impact | Deploy CGN close to the edge of the network; | 1075 | to geo-location | use regional IP and port assignments. | 1076 | services | | 1077 +-----------------+-------------------------------------------------+ 1078 | Logging | Deterministic Logging | 1079 | requirements | [I-D.donley-behave-deterministic-cgn]; data | 1080 | for lawful | compression [I-D.sivakumar-behave-nat-logging]; | 1081 | intercept | bulk port logging | 1082 +-----------------+-------------------------------------------------+ 1084 CGN mitigation 1086 Other mitigation techniques that are currently being researched, such 1087 as [I-D.tsou-stateless-nat44], may also improve performance. 1089 7. IANA Considerations 1091 This document has no IANA considerations. 1093 8. Security Considerations 1095 Security considerations are described in [RFC6264] and [RFC6269]. 1097 In general, since a CGN device shares a single IPv4 address with 1098 multiple subscribers, CGN devices may provide an attractive target 1099 for denial of service attacks. In addition, as described in 1100 [I-D.donley-behave-deterministic-cgn], abuse attribution is more 1101 challenging with CGN, and requires content providers to log IP 1102 address, source port, and time to correlate with service provider CGN 1103 logs. Also, if a CGN public IP address is added to a blacklist (e.g. 1104 for SPAM) or if a server limits the number of connections per IP 1105 address, it could negatively impact legitimate users. 1107 9. Informative References 1109 [I-D.donley-behave-deterministic-cgn] 1110 Donley, C., Grundemann, C., Sarawat, V., Sundaresan, K., 1111 and O. Vautrin, "Deterministic Address Mapping to Reduce 1112 Logging in Carrier Grade NAT Deployments", 1113 draft-donley-behave-deterministic-cgn-05 (work in 1114 progress), January 2013. 1116 [I-D.ietf-pcp-base] 1117 Wing, D., Cheshire, S., Boucadair, M., Penno, R., and P. 1118 Selkirk, "Port Control Protocol (PCP)", 1119 draft-ietf-pcp-base-29 (work in progress), November 2012. 1121 [I-D.shirasaki-nat444] 1122 Yamagata, I., Shirasaki, Y., Nakagawa, A., Yamaguchi, J., 1123 and H. Ashida, "NAT444", draft-shirasaki-nat444-02 (work 1124 in progress), July 2010. 1126 [I-D.sivakumar-behave-nat-logging] 1127 Sivakumar, S. and R. Penno, "IPFIX Information Elements 1128 for logging NAT Events", 1129 draft-sivakumar-behave-nat-logging-06 (work in progress), 1130 January 2013. 1132 [I-D.tsou-stateless-nat44] 1133 Tsou, T., Liu, W., Perreault, S., Penno, R., and M. Chen, 1134 "Stateless IPv4 Network Address Translation", 1135 draft-tsou-stateless-nat44-02 (work in progress), 1136 October 2012. 1138 [RFC1918] Rekhter, Y., Moskowitz, R., Karrenberg, D., Groot, G., and 1139 E. Lear, "Address Allocation for Private Internets", 1140 BCP 5, RFC 1918, February 1996. 1142 [RFC3056] Carpenter, B. and K. Moore, "Connection of IPv6 Domains 1143 via IPv4 Clouds", RFC 3056, February 2001. 1145 [RFC4380] Huitema, C., "Teredo: Tunneling IPv6 over UDP through 1146 Network Address Translations (NATs)", RFC 4380, 1147 February 2006. 1149 [RFC4689] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, 1150 "Terminology for Benchmarking Network-layer Traffic 1151 Control Mechanisms", RFC 4689, October 2006. 1153 [RFC6264] Jiang, S., Guo, D., and B. Carpenter, "An Incremental 1154 Carrier-Grade NAT (CGN) for IPv6 Transition", RFC 6264, 1155 June 2011. 1157 [RFC6269] Ford, M., Boucadair, M., Durand, A., Levis, P., and P. 1158 Roberts, "Issues with IP Address Sharing", RFC 6269, 1159 June 2011. 1161 [RFC6333] Durand, A., Droms, R., Woodyatt, J., and Y. Lee, "Dual- 1162 Stack Lite Broadband Deployments Following IPv4 1163 Exhaustion", RFC 6333, August 2011. 1165 Appendix A. Acknowledgements 1167 Thanks to the following people for their testing, guidance, and 1168 feedback: 1170 Paul Eldridge 1172 Abishek Chandrasekaran 1174 Vivek Ganti 1176 Joey Padden 1178 Lane Johnson 1180 Authors' Addresses 1182 Chris Donley (editor) 1183 CableLabs 1184 858 Coal Creek Circle 1185 Louisville, CO 80027 1186 USA 1188 Email: c.donley@cablelabs.com 1190 Lee Howard 1191 Time Warner Cable 1192 13241 Woodland Park Rd 1193 Herndon, VA 20171 1194 USA 1196 Email: william.howard@twcable.com 1198 Victor Kuarsingh 1199 Rogers Communications 1200 8200 Dixie Road 1201 Brampton, ON L6T 0C1 1202 Canada 1204 Email: victor.kuarsingh@rci.rogers.com 1206 John Berg 1207 CableLabs 1208 858 Coal Creek Circle 1209 Louisville, CO 80027 1210 USA 1212 Email: j.berg@cablelabs.com 1214 Jinesh Doshi 1215 University of Colorado 1217 Email: jinesh.doshi@colorado.edu