idnits 2.17.1 draft-ietf-v6ops-nat64-experience-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 21 instances of too long lines in the document, the longest one being 2 characters in excess of 72. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (December 9, 2013) is 3791 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'I-D.anderson-siit-dc' is defined on line 807, but no explicit reference was found in the text == Unused Reference: 'RFC6269' is defined on line 877, but no explicit reference was found in the text == Unused Reference: 'RFC6883' is defined on line 896, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) ** Obsolete normative reference: RFC 6145 (Obsoleted by RFC 7915) ** Obsolete normative reference: RFC 6555 (Obsoleted by RFC 8305) == Outdated reference: A later version (-05) exists of draft-chen-sunset4-cgn-port-allocation-02 == Outdated reference: A later version (-09) exists of draft-donley-behave-deterministic-cgn-06 == Outdated reference: A later version (-10) exists of draft-ietf-softwire-4rd-07 == Outdated reference: A later version (-06) exists of draft-ietf-softwire-map-deployment-03 == Outdated reference: A later version (-08) exists of draft-ietf-softwire-map-t-04 == Outdated reference: A later version (-05) exists of draft-ietf-v6ops-ula-usage-recommendations-01 -- Obsolete informational reference (is this intentional?): RFC 6036 (Obsoleted by RFC 9386) Summary: 4 errors (**), 0 flaws (~~), 11 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force G. Chen 3 Internet-Draft Z. Cao 4 Intended status: Informational China Mobile 5 Expires: June 12, 2014 C. Xie 6 China Telecom 7 D. Binet 8 France Telecom-Orange 9 December 9, 2013 11 NAT64 Operational Experience 12 draft-ietf-v6ops-nat64-experience-05 14 Abstract 16 This document summarizes NAT64 function deployment scenarios and 17 operational experience. Both NAT64 Carrier Grade NAT (NAT64-CGN) and 18 NAT64 server Front End (NAT64-FE) are considered in this document. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on June 12, 2014. 37 Copyright Notice 39 Copyright (c) 2013 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 3. NAT64 Networking Experience . . . . . . . . . . . . . . . . . 4 57 3.1. NAT64-CGN Consideration . . . . . . . . . . . . . . . . . 4 58 3.1.1. NAT64-CGN Usages . . . . . . . . . . . . . . . . . . 4 59 3.1.2. DNS64 Deployment . . . . . . . . . . . . . . . . . . 4 60 3.1.3. NAT64 Placement . . . . . . . . . . . . . . . . . . . 5 61 3.1.4. Co-existence of NAT64 and NAT44 . . . . . . . . . . . 5 62 3.2. NAT64-FE Consideration . . . . . . . . . . . . . . . . . 6 63 4. High Availability . . . . . . . . . . . . . . . . . . . . . . 7 64 4.1. Redundancy Design . . . . . . . . . . . . . . . . . . . . 7 65 4.2. Load Balancing . . . . . . . . . . . . . . . . . . . . . 9 66 5. Source Address Transparency . . . . . . . . . . . . . . . . . 9 67 5.1. Traceability . . . . . . . . . . . . . . . . . . . . . . 9 68 5.2. Geo-location . . . . . . . . . . . . . . . . . . . . . . 10 69 6. Quality of Experience . . . . . . . . . . . . . . . . . . . . 11 70 6.1. Service Reachability . . . . . . . . . . . . . . . . . . 11 71 6.2. Resource Reservation . . . . . . . . . . . . . . . . . . 12 72 7. MTU Considerations . . . . . . . . . . . . . . . . . . . . . 13 73 8. ULA Usages . . . . . . . . . . . . . . . . . . . . . . . . . 13 74 9. Security Considerations . . . . . . . . . . . . . . . . . . . 14 75 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 76 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 15 77 12. Additional Author List . . . . . . . . . . . . . . . . . . . 15 78 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 15 79 13.1. Normative References . . . . . . . . . . . . . . . . . . 16 80 13.2. Informative References . . . . . . . . . . . . . . . . . 17 81 Appendix A. Testing Results of Application Behavior . . . . . . 19 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 84 1. Introduction 86 IPv6 is the only sustainable solution for numbering nodes on Internet 87 due to the IPv4 depletion. Network operators have to deploy 88 IPv6-only networks in order to meet the needs of the expanding 89 internet without available IPv4 addresses. 91 Single stack IPv6 network deployment can simplify network's 92 provisioning. Some justifications have been described in 464xlat 93 [RFC6877]. As an example, IPv6-only connectivity confers some 94 benefits to mobile operators. In such mobile context, it enables the 95 use of a single IPv6 Packet Data Protocol(PDP) context or Evolved 96 Packet System (EPS) bearer if Long Term Evolution (LTE) network is 97 considered, which eliminates significant network costs caused by 98 doubling the number of PDP contexts in some cases and the need of 99 IPv4 addresses to be assigned to customers. In broadband networks 100 overall, it can allow for the scaling of edge-network growth 101 decoupled from IPv4 numbering limitations. 103 In a transition scenario, some existing networks are likely to be 104 IPv4-only configured for quite a long time. IPv6 networks and hosts 105 will need to coexist with IPv4 numbered resources. Widespread dual- 106 stack deployments have not materialized at the anticipated rate over 107 the last 10 years, one possible conclusion being that legacy networks 108 will not make the jump quickly. The Internet will include nodes that 109 are dual-stack, nodes that remain IPv4-only, and nodes that can be 110 deployed as IPv6-only nodes. A translation mechanism based on a 111 NAT64[RFC6146] [RFC6145]function is likely to be a key element of the 112 Internet for IPv6-IPv4 interoperability. 114 [RFC6036] reports at least 30% of operators plan to run some kind of 115 translator (presumably NAT64/DNS64). Advice on NAT64 deployment and 116 operations are therefore of some importance. [RFC6586] documents the 117 implications for IPv6 only networks. This document intends to be 118 specific to NAT64 network planning. 120 2. Terminology 122 In regards to IPv4/IPv6 translation, [RFC6144] has described a 123 framework of enabling networks to make interworking possible between 124 IPv4 and IPv6 networks. This document has further categorized 125 different NAT64 function locations and use cases. The principle 126 distinction of location is if the NAT64 is located in a Carrier Grade 127 NAT or server Front End. The terms of NAT-CGN/FE are understood to be 128 a topological distinction indicating different features employed in a 129 NAT64 deployment. 131 NAT64 Carrier Grade NAT (NAT64-CGN): A NAT64-CGN is placed in an ISP 132 network. IPv6 subscribers leverage the NAT64-CGN to access 133 existing IPv4 internet services. The ISP as an administrative 134 entity takes full control on the IPv6 side, but has limited or no 135 control on the IPv4 side. NAT64-CGN may have to consider the IPv4 136 Internet environment and services to make appropriate 137 configurations. 139 NAT64 server Front End (NAT64-FE): A NAT64-FE is generally a device 140 with NAT64 functionality in a content provider or data center 141 network. It could be for example a traffic load balancer or a 142 firewall. The operator of the NAT64-FE has full control over the 143 IPv4 network within the data center, but only limited influence or 144 control over the external IPv6 network. 146 3. NAT64 Networking Experience 148 3.1. NAT64-CGN Consideration 150 3.1.1. NAT64-CGN Usages 152 Fixed network operators and mobile operators may locate NAT64 in 153 access networks or in mobile core networks. It can be built into 154 various devices, including routers, gateways or firewalls in order to 155 connect IPv6 users to the IPv4 Internet. With regard to the numbers 156 of users and the shortage of public IPv4 addresses, stateful 157 NAT64[RFC6146] is more adapted to perform some maximal sharing of 158 public IPv4 addresses. The usage of stateless NAT64 can be seen with 159 better transparency features 160 [I-D.ietf-softwire-stateless-4v6-motivation], while it has to be 161 coordinated with A+P[RFC6346] processes as specified in 162 [I-D.ietf-softwire-map-t]and [I-D.ietf-softwire-4rd] in order to cope 163 with IPv4 shortage. 165 3.1.2. DNS64 Deployment 167 DNS64[RFC6147] is recommended for use in combination with stateful 168 NAT64, and will likely be an essential part of an IPv6 single-stack 169 network that couples to the IPv4 Internet. 464xlat[RFC6877] is 170 proposed to enable access of IPv4 only applications or applications 171 that call IPv4 literal addresses. Using DNS64 will help 464xlat to 172 automatically discover NAT64 prefix through [RFC7050]. Berkeley 173 Internet Name Daemon (BIND) software supports the function. It's 174 important to note that DNS64 generates the synthetic AAAA reply when 175 services only register A records. Operators should not expect to 176 access IPv4 parts of a dual-stack server using NAT64/DNS64. The 177 traffic is forwarded on IPv6 paths if dual-stack servers are 178 targeted. IPv6 traffic may be routed not going through NAT64. Only 179 the traffic going to IPv4-only service would traverse NAT64. In some 180 sense, it encourages IPv6 transmission and restrains NAT uses 181 compared to NAT44(if used), on which all traffic flows have to be 182 traversed and translated. In some cases, NAT64-CGN may serve double 183 roles, i.e. a translator and IPv6 forwarder. In mobile networks, 184 NAT64 is likely deployed as the default gateway serving for all the 185 IPv6 traffic. The traffic heading to a dual-stack server is only 186 forwarded on the NAT64. Therefore, both IPv6 and IPv4 are suggested 187 to be configured on the Internet faced interfaces of NAT64. We 188 tested on Top100 websites (referring to [Alexa] statistics). 43% of 189 websites are connected and forwarded on the NAT64 since those 190 websites have both AAAA and A records. With expansion of IPv6 191 supports, the translation process on NAT64 will likely be faded. 193 3.1.3. NAT64 Placement 195 All connections to IPv4 services from IPv6-only clients must traverse 196 the NAT64-CGN. It can be advantageous from the vantage-point of 197 troubleshooting and traffic engineering to carry the IPv6 traffic 198 natively for as long as possible within an access network and 199 translate packets only at or near the network egress. NAT64 can be 200 considered as a feature of the Autonomous System (AS) border in fixed 201 networks. And, it is likely to be deployed in an IP node beyond the 202 Gateway GPRS Support Node (GGSN) or Public Data Network- Gateway 203 (PDN-GW) in mobile networks or directly in the gateway itself in some 204 situations. This allows consistent attribution and traceability 205 within the service provider network. It has been observed that the 206 process of correlating log information is problematic from multiple- 207 vendor's equipment due to inconsistent formats of log records. 208 Placing NAT64 in a centralized location may reduce diversity of log 209 format and simplify the network provisioning. Moreover, since NAT64 210 is only targeted at serving traffic flows from IPv6 to IPv4-only 211 services, the user traffic volume should not be as high as in a NAT44 212 scenario, and therefore, the gateway's capacity in such location may 213 not be as much of a concern or a hurdle to deployment. On the other 214 hand, the placement in a centralized way would require more strict 215 high availability (HA) design. It would also make geo-location based 216 on IPv4 addresses rather inaccurate as it is currently the case for 217 NAT44 CGN already deployed in ISP networks. More considerations or 218 workarounds on HA and traceability could be found at Section 4 and 219 Section 5. 221 3.1.4. Co-existence of NAT64 and NAT44 223 NAT64 could likely co-exist with NAT44 in a dual-stack network mostly 224 because IPv4 private addresses are allocated to customers. The 225 coexistence has already appeared in mobile networks, in which dual 226 stack mobile phones normally initiate some dual-stack PDN/PDP 227 Type[RFC6459] to query both IPv4/IPv6 address and IPv4 allocated 228 addresses are very often private ones. [RFC6724] always prioritizes 229 IPv6 connections regardless of whether the end-to-end path is native 230 IPv6 or IPv6 translated to IPv4 via NAT64/DNS64. Conversely, Happy 231 Eyeballs[RFC6555] will direct some IP flows across IPv4 paths. The 232 selection of IPv4/IPv6 paths may depend on particular implementation 233 choices or settings on a host-by-host basis, and may differ from an 234 operator's deterministic scheme. Our tests verified that hosts may 235 find themselves switching between IPv4 and IPv6 paths as they access 236 identical service, but at different times 237 [I-D.kaliwoda-sunset4-dual-ipv6-coexist]. Since the topology on each 238 path is different, it may cause unstable user experiences and some 239 degradation of Quality of Experience (QoE) when fallback to the other 240 protocol is not powerful enough. It's also difficult for operators 241 to find a solution to make a stable network with optimal resource 242 utilization. In general, it's desirable to figure out the solution 243 that will introduce IPv6/IPv4 translation service to IPv6-only hosts 244 connecting to IPv4 servers while making sure dual-stack hosts to have 245 at least one address family accessible via native service if it's 246 possible. With the end-to-end native IPv6 environment is available, 247 hosts should be upgraded aggressively to migrate to IPv6-only. There 248 is an ongoing effort to detect host connectivity and propose new 249 DHCPv6 option[I-D.wing-dhc-dns-reconfigure] to convey appropriate 250 configuration information to the hosts. 252 3.2. NAT64-FE Consideration 254 Some Internet Content Providers (ICPs) may locate NAT64 in front of 255 an Internet Data Center (IDC), for example co-located with load 256 balancing function. Load balancers are employed to connect different 257 IP family domains, meanwhile distribute workloads across multiple 258 domains or internal servers actually. In some cases, IPv4 addresses 259 exhaustion may not be a problem in some IDC's networks. IPv6 support 260 for some applications may require some investments and workloads so 261 IPv6 support may not be a priority. The use of NAT64 may be served 262 to support widespread IPv6 adoption on the Internet while maintaining 263 IPv4-only applications access. 265 Different strategy has been described in [RFC6883]referred to as 266 "inside out" and "outside in". An IDC operator may implement the 267 following practices in the NAT64-FE networking. 269 o Some ICPs who already have satisfactory operational experiences 270 would adopt single stack IPv6 operations to build up their data 271 center network, servers and applications since it allows new 272 services delivery without having to integrate consideration of 273 IPv4 NAT and address limitations of IPv4 networks. Stateless 274 NAT64[RFC6145]is used to provide services for IPv4-only 275 subscribers. [I-D.anderson-siit-dc]has provided further 276 descriptions and guidelines. 278 o ICPs who attempt to offer customers IPv6 support in their 279 application farms at an early stage may likely run some proxies, 280 which are configured to handle incoming IPv6 flows and proxy them 281 to IPv4 back-end systems. Many load balancers have already 282 integrated some proxy functionality. IPv4 addresses configured in 283 the proxy can be multiplexed like a stateful NAT64 performs. A 284 similar challenge exists once increasingly numerous users in IPv6 285 Internet access an IPv4 network. High loads on load-balancers may 286 be apt to cause additional latency, IPv4 pool exhaustion, etc. 287 Therefore, this approach is only reasonable at an early stage. 288 ICPs may learn from the experiences and move on to dual-stack or 289 IPv6 single stack in a further stage, since the native IPv6 is 290 always more desirable than transition solutions. 292 [RFC6144] recommends that AAAA records of load-balancers or 293 application servers can be directly registered in the authoritative 294 DNS servers requiring to populate these servers with corresponding 295 AAAA records. In this case, there is no need to deploy DNS64 296 servers. Those AAAA records can be some native IPv6 addresses or 297 some IPv4-converted IPv6 addresses[RFC6052]. The type of IPv6 298 address does not give the possibility to nodes to get any information 299 about NAT64 presence on communication path and the possibility to 300 prefer IPv4 path or the IPv6 path in dual-stack networks. For the 301 testing purpose, operators could use an independent sub domain e.g. 302 ipv6exp.xxx.xxx to identify experimental ipv6 services to users. How 303 to design the FQDN for the IPv6 service is out-of-scope of this 304 document. 306 4. High Availability 308 4.1. Redundancy Design 310 High Availability (HA) is a major requirement for every service and 311 network services. The deployment of redundancy mechanism is an 312 essential approach to avoid single-point failure and significantly 313 increase the network reliability. It's not only useful to stateful 314 NAT64 cases, but also to stateless NAT64 gateways. 316 Three redundancy modes are mainly used hereafter: cold standby, warm 317 standby and hot standby. 319 o Cold standby can't replicate the NAT64 states from the primary 320 equipment to the backup. Administrators switch on the backup 321 NAT64 only if the primary NAT64 fails. As the results, all the 322 existing established sessions will be disconnected. The internal 323 hosts are required to re-establish sessions to the external hosts. 324 Since the backup NAT64 is manually configured to switch over to 325 active NAT64, it may have unpredictable impacts to the ongoing 326 services. Normally, the handover would take several minutes so as 327 to wait for the whole process of NAT64 bootstrap loader. 329 o Warm standby is a flavor of the cold standby mode. Backup NAT64 330 would keep running once the primary NAT64 is working. This makes 331 warm standby less time consuming during the traffic failover. 332 Virtual Router Redundancy Protocol (VRRP)[RFC5798] can be a 333 solution to enable automatic handover in the warm standby. It was 334 tested that the handover takes as maximum as 1 minute if the 335 backup NAT64 needs to take over routing and re-construct the 336 Binding Information Bases (BIBs) for 30 million sessions. In 337 deployment phase, operators could balance loads on distinct NAT64s 338 devices. Those NAT64s make a warm backup of each other. 340 o Hot standby must synchronize the BIBs between the primary NAT64 341 and backup. When the primary NAT64 fails, backup NAT64 would take 342 over and maintain the state of all existing sessions. The 343 internal hosts don't have to re-connect the external hosts. The 344 handover time has been extremely reduced. Thanks to Bidirectional 345 Forwarding Detection (BFD) [RFC5880] combining with VRRP, a delay 346 of only 35ms for 30 million sessions handover was observed during 347 testing. In some sense, it could guarantee the session continuity 348 for every service. In order to timely transmit session states, 349 operators may have to deploy extra transport links between primary 350 NAT64 and distant backup. The scale of synchronization data 351 instance is depending on the particular deployment. For example, 352 If a NAT64-CGN is served for 200,000 users, the average amount of 353 800, 000 sessions per second is roughly estimated for new created 354 and expired sessions. A 10Gbps transport link should be used for 355 the sync data transmission. 357 In general, cold-standby and warm-standby is simpler and less 358 resource intensive, but it requires clients to re-establish sessions 359 when a fail-over occurs. Hot standby doubles resource's consumption 360 to synchronize the states, but it achieve seamless handover. The 361 consideration of redundancy mode for stateless NAT64 is simple, 362 because it doesn't have to consider time consuming for states 363 maintenance. The warm standby is sufficient for stateless NAT64. In 364 regards to stateful NAT64, it may be useful to investigate 365 performance tolerance of applications and the traffic characteristics 366 in a particular network. Some testing results are shown in the 367 Appendix A. 369 Our statistics in a mobile network shown that almost 91.21% of amount 370 of traffic is accounted by browsing services. Those services don't 371 require session continuity. The handover time of warm standby is 372 qualified to the delay tolerance. Hot-standby does not offer much 373 benefit for those sessions on this point. In a fixed network, HTTP 374 streaming, p2p and online games would be the major 375 traffic[Cisco-VNI]. Consideration should be given to the importance 376 of maintaining bindings for those sessions across failover. 377 Operators may also consider the Average Revenue Per User (ARPU) 378 factors to deploy suitable redundancy mode. Warm standby may still 379 be adopted to cover most services while hot standby could be used to 380 upgrade Quality of Experience (QoE) using DNS64 with different 381 synthetic responses for limited traffic. Further considerations are 382 discussed at Section 6. 384 4.2. Load Balancing 386 Load balancing is used to accompany redundancy design so that better 387 scalability and resiliency could be achieved. Stateless NAT64s allow 388 asymmetric routing while anycast-based solutions are recommended in 389 [I-D.ietf-softwire-map-deployment]. The deployment of load balancing 390 may make more sense to stateful NAT64s for the sake of single-point 391 failure avoidance. Since the NAT64-CGN and NAT64-FE have distinct 392 facilities, the following lists the considerations for each case. 394 o NAT64-CGN equipment doesn't implement load balancer functions on a 395 board card. Therefore, the gateways have to resort to DNS64 or 396 internal host's behavior. Once DNS64 is deployed, the load 397 balancing can be performed by synthesizing AAAA response with 398 different IPv6 prefixes. For the applications not requiring DNS 399 resolver, internal hosts could learn multiple IPv6 prefixes 400 through the approaches defined in[RFC7050] and then select one 401 based on a given prefix selection policy. 403 o A dedicated Load Balancer could be deployed at front of a NAT64-FE 404 farm. Load Balancer uses proxy mode to redirect the flows to the 405 appropriate NAT64 instance. Stateful NAT64s require a 406 deterministic pattern to arrange the traffic in order to ensure 407 outbound/inbound flows traverse the identical NAT64. Therefore, 408 static scheduling algorithms, for example source-address based 409 policy, is preferred. A dynamic algorithm, for example Round- 410 Robin, may have impacts on applications seeking session 411 continuity, which described in the Table 1. 413 5. Source Address Transparency 415 5.1. Traceability 417 Traceability is required in many cases such as identifying malicious 418 attacks sources and accounting requirements. Operators are asked to 419 record the NAT64 log information for specific periods of time. In 420 our lab testing, the log information from 200,000 subscribers have 421 been collected from a stateful NAT64 gateway for 60 days. 422 Syslog[RFC5424] has been adopted to transmit log message from NAT64 423 to a log station. Each log message contains transport protocol, 424 source IPv6 address:port, translated IPv4 address: port and 425 timestamp. It takes almost 125 bytes long in ASCII format. It has 426 been verified that the volume of recorded information reach up to 427 42.5 terabytes in the raw format and 29.07 terabytes in a compact 428 format. Operators have to build up dedicated transport links, 429 storage system and servers for the purpose. 431 There are also several implementations to mitigate the issue. For 432 example, stateful NAT64 could configure with bulk port allocation 433 method. Once a subscriber creates the first session, a number of 434 ports are pre-allocated. A bulk allocation message is logged 435 indicating this allocation. Subsequent session creations will use 436 one of the pre-allocated port and hence does not require logging. 437 The log volume in this case may be only one thousandth of dynamic 438 port allocation. Some implementations may adopt static port-range 439 allocations [I-D.donley-behave-deterministic-cgn] which eliminates 440 the need for per-subscriber logging. As a side effect, the IPv4 441 multiplexing efficiency is decreased regarding to those methods. For 442 example, the utilization ratio of public IPv4 address is dropped 443 approximately to 75% when NAT64 gateway is configured with bulk port 444 allocation (The lab testing allocates each subscriber with 400 445 ports). In addition, port-range based allocation should also 446 consider port randomization described in [RFC6056] . A trade-off 447 among address multiplexing efficiency, logging storage compression 448 and port allocation complexity should be considered. More 449 discussions could be found in 450 [I-D.chen-sunset4-cgn-port-allocation].Basically, the decision 451 depends on usable IPv4 resource and investments of log systems. 453 5.2. Geo-location 455 IP addresses are usually used as inputs to geo-location services. 456 The use of address sharing will prevent these systems from resolving 457 the location of a host based on IP address alone. Applications that 458 assume such geographic information may not work as intended. The 459 possible solutions listed in [RFC6967] are intended to bridge the 460 gap. However, those solutions can only provide a sub-optimal 461 substitution to solve the problem of host identification, in 462 particular it may not today solve problems with source identification 463 through translation. The following lists current practices to 464 mitigate the issue. 466 o Operators who adopt NAT64-FE may leverage the application layer 467 proxies, e.g. X-Forwarded-For (XFF) 468 [I-D.ietf-appsawg-http-forwarded], to convey the IPv6 source 469 address in HTTP headers. Those messages would be passed on to 470 web-servers. The log parsing tools are required to be able to 471 support IPv6 and may lookup Radius servers for the target 472 subscribers based on IPv6 addresses included in XFF HTTP headers. 473 XFF is the de facto standard which has been integrated in most 474 Load Balancers. Therefore, it may be superior to use in a NAT-FE 475 environment. In the downsides, XFF is specific to HTTP. It 476 restricts the usages so that the solution can't be applied to 477 requests made over HTTPs. This makes geo-location problematic for 478 HTTPs based services. 480 o The NAT64-CGN equipment may not implement XFF. Geo-location based 481 on shared IPv4 address is rather inaccurate in that case. 482 Operators could subdivide the outside IPv4 address pool so an IPv6 483 address can be translated depending on their geographical 484 locations. As consequence, location information can be identified 485 from a certain IPv4 address range. [RFC6967] also enumerates 486 several options to reveal the host identifier. Each solution 487 likely has their-own specific usage. For the geo-location systems 488 relying on a Radius database[RFC5580], we have investigated to 489 deliver NAT64 BIBs and Session Table Entrys (STEs) to a Radius 490 server[I-D.chen-behave-nat64-radius-extension]. This method could 491 provide geo-location system with an internal IPv6 address to 492 identify each user. It can get along with [RFC5580] to convey 493 original source address through same message bus. 495 6. Quality of Experience 497 6.1. Service Reachability 499 NAT64 is providing a translation capability between IPv6 and IPv4 500 end-nodes. In order to provide the reachability between two IP 501 address families, NAT64-CGN has to implement appropriate application 502 aware functions, i.e. Application Layer Gateway (ALG), where address 503 translation is not itself sufficient and security mechanisms do not 504 render it infeasible. Most NAT64-CGNs mainly provide FTP- 505 ALG[RFC6384]. NAT64-FEs may have functional richness on Load 506 Balancer, for example HTTP-ALG, HTTPs-ALG, RTSP-ALG and SMTP-ALG have 507 been supported. It should be noted that ALGs may impact the 508 performance on a NAT64 box to some extent. ISPs as well as content 509 providers might choose to avoid situations where the imposition of an 510 ALG might be required. At the same time, it is also important to 511 remind customers and application developers that IPv6 end-to-end 512 usage does not require ALG imposition and therefore results in a 513 better overall user experience. 515 The service reachability is also subject to the IPv6 support in the 516 client side. We tested several kinds of applications as shown in the 517 below table to verify the IPv6 supports. 519 Table 1: The tested applications 520 +----------------+----------------------------------------------------+ 521 | APPs | Results and Found Issues | 522 +----------------+----------------------------------------------------+ 523 | Webservices |Mostly pass, some failure cases due to IPv4 Literals| 524 +----------------+----------------------------------------------------+ 525 |Instance Message|Mostly fail, softwares don't support IPv6 | 526 +----------------+----------------------------------------------------+ 527 | Games |Mostly pass for web-based games and mostly fail for | 528 | |standalone games due to software supports | 529 +----------------+----------------------------------------------------+ 530 | P2P |Mostly fail, tracker is unable to support IPv6 peer | 531 | |or some software can't use IPv6 connections | 532 +----------------+----------------------------------------------------+ 533 | FTP | Pass | 534 +----------------+----------------------------------------------------+ 535 | Email | Pass | 536 +----------------+----------------------------------------------------+ 537 | VoIP | Fail, due to the lack of SIP-ALG support on NAT64 | 538 +----------------+----------------------------------------------------+ 539 | VPN | Fail, due to incapability of IPsec verification | 540 +----------------+----------------------------------------------------+ 542 The experiences of some applications are still align with [RFC6586]. 543 In addition, we tested several P2P softwares including BitTorrent, 544 Thunder, PPS TV and eMule. Some failures happened because local 545 tracker servers don't setup with IPv6 extension patch or software 546 can't support IPv6. For VoIP services, we mainly tested Voice over 547 LTE services[IR.92], which is the major solution for voice in the 548 fourth generation communication ages. NAT64 is testified with some 549 issues of SIP-ALG supports. In regard to the widespread applications 550 of VoLTE in the near future, SIP-ALG is of great value to be 551 implemented on NAT64-CGN. 553 6.2. Resource Reservation 555 Session status normally is managed by a static timer. For example, 556 the value of the "established connection idle-timeout" for TCP 557 sessions must not be less than 2 hours 4 minutes[RFC5382] and 5 558 minutes for UDP sessions[RFC4787]. In some cases, NAT resource maybe 559 significantly consumed by largely inactive users. The NAT translator 560 and other customers would suffer from service degradation due to port 561 consummation by other subscribers using the same NAT64 device. A 562 flexible NAT session control is desirable to resolve the issues. 563 PCP[RFC6887] could be a candidate to provide such capability. A 564 NAT64-CGN should integrate with a PCP server, to allocate available 565 IPv4 address/port resources. Resources could be assigned to PCP 566 clients through PCP MAP/PEER mode. Such ability can be considered to 567 upgrade user experiences, for example assigning different sizes of 568 port ranges for different subscribers. Those mechanisms are also 569 helpful to minimize terminal battery consumption and reduce the 570 number of keep-alive messages to be sent by mobile terminal devices. 572 Subscribers can also benefit from network reliability. It has been 573 discussed that hot-standby offers satisfactory experience once outage 574 of primary NAT64 is occurred. Operators may rightly be concerned 575 about the considerable investment required for NAT64 equipment 576 relative to low ARPU income. For example, transport links may cost 577 much, because primary NAT64 and backup are normally located at 578 different locations, separated by a relatively large distance. 579 Additional maintenance has to be spent to ensure the connectivity 580 quality. However, that may be necessary to some applications, which 581 are delay-sensitive and seek session continuity, for example on-line 582 games and live-streaming. Operators may be able to get added-values 583 from those services by offering first-class services. It can be pre- 584 configured on the gateway to hot-standby modes depending on 585 subscriber's profile. The rest of other sessions can be covered by 586 cold/warm standby. 588 7. MTU Considerations 590 IPv6 requires that every link in the internet have an Maximum 591 Transmission Unit (MTU) of 1280 octets or greater[RFC2460]. However, 592 in case of NAT64 translation deployment, some IPv4 MTU constrained 593 link will be used in some communication path and originating IPv6 594 nodes may therefore receive an ICMP Packet Too Big (PTB) message, 595 reporting a Next-Hop MTU less than 1280 bytes. The result would be 596 that IPv6 allows packets to contain a fragmentation header, without 597 the packet being fragmented into multiple pieces. A NAT64 would 598 receive IPv6 packets with fragmentation header in which "M" flag 599 equal to 0 and "Fragment Offset" equal to 0. Those packets likely 600 impact other fragments already queued with the same set of {IPv6 601 Source Address, IPv6 Destination Address, Fragment Identification}. 602 If the NAT64 box is compliant with [RFC5722], there is risk that all 603 the fragments have to be dropped. 605 [RFC6946] discusses how this situation could be exploited by an 606 attacker to perform fragmentation-based attacks, and also proposes an 607 improved handling of such packets. It required enhancements on NAT64 608 gateway implementations to isolate packet's processing. NAT64 should 609 follow the recommendation and take steps to prevent the risks of 610 fragmentation. 612 Another approach that potentially avoids this issue is to configure 613 IPv4 MTU more than 1260 bytes. It would forbid the occurrence of PTB 614 smaller than 1280 bytes. Such an operational consideration is hard 615 to universally apply to the legacy "IPv4 Internet" NAT64-CGN bridged. 616 However, it's a feasible approach in NAT64-FE cases, since a IPv4 617 network NAT64-FE connected is rather well-organized and operated by a 618 IDC operator or content provider. Therefore, the MTU of IPv4 network 619 in NAT64-FE case are strongly recommended to set to more than 1260 620 bytes. 622 8. ULA Usages 623 Unique Local Addresses (ULAs) are defined in [RFC4193] to be 624 renumbered within a network site for local communications. Operators 625 may use ULAs as NAT64 prefixes to provide site-local IPv6 626 connectivity. Those ULA prefixes are stripped when the packets going 627 to the IPv4 Internet, therefore ULAs are only valid in the IPv6 site. 628 The use of ULAs could help in identifying the translation 629 traffic.[I-D.ietf-v6ops-ula-usage-recommendations] has provided 630 further guidance for the ULAs usages. 632 We configure ULAs as NAT64 prefixes on a NAT64-CGN. If a host is 633 only assigned with an IPv6 address and connected to NAT64-CGN, when 634 connect to an IPv4 service, it would receive AAAA record generated by 635 the DNS64 with the ULA prefix. A Global Unicast Address (GUA) will 636 be selected as the source address to the ULA destination address. 637 When the host has both IPv4 and IPv6 address, it would initiate both 638 A and AAAA record lookup, then both original A record and 639 DNS64-generated AAAA record would be received. A host, which is 640 compliant with [RFC6724], will never prefer ULA over IPv4. An IPv4 641 path will be always selected. It may be undesirable because the 642 NAT64-CGN will never be used. Operators may consider to add 643 additional site-specific rows into the default policy table for host 644 address selection in order to steer traffic flows going through 645 NAT64-CGN. However, it involves significant costs to change 646 terminal's behavior. Therefore, operators are not suggested to 647 configure ULAs on a NAT64-CGN. 649 ULAs can't work when hosts transit the Internet to connect with 650 NAT64. Therefore, ULAs are inapplicable to the case of NAT64-FE. 652 9. Security Considerations 654 his document presents the deployment experiences of NAT64 in CGN and 655 FE scenarios. In general, RFC 6146[RFC6146] provides TCP-tracking, 656 address-dependent filtering mechanisms to protect NAT64 from 657 Distributed Denial of Service (DDoS). In NAT64-CGN cases, operators 658 also could adopt unicast Reverse Path Forwarding (uRPF)[RFC3704] and 659 black/white-list to enhance the security by specifying access 660 policies. For example, NAT64-CGN should forbid establish NAT64 BIB 661 for incoming IPv6 packets if uRPF in Strict or Loose mode check does 662 not pass or whose source IPv6 address is associated to black-lists. 664 The stateful NAT64-FE creates state and maps that connection to an 665 internally-facing IPv4 address and port. An attacker can consume the 666 resources of the NAT64-FE device by sending an excessive number of 667 connection attempts. Without a DDoS limitation mechanism, the 668 NAT64-FE is exposed to attacks. Load Balancer is recommended to 669 enable the capabilities of line rate DDOS defense, such as the 670 employment of SYN PROXY-COOKIE. Security domain division is 671 necessary as well in this case. Therefore, Load Balancers could not 672 only serve for optimization of traffic distribution, but also prevent 673 service from quality deterioration due to security attacks. 675 10. IANA Considerations 677 This memo includes no request to IANA. 679 11. Acknowledgements 681 The authors would like to thank Jari Arkko, Dan Wing, Remi Despres, 682 Fred Baker, Hui Deng, Iljitsch van Beijnum, Philip Matthews, Randy 683 Bush, Mikael Abrahamsson, Lorenzo Colitti, Sheng Jiang, Nick Heatley, 684 Tim Chown and Gert Doering for their helpful comments. 686 Many thanks to Wesley George, Lee Howard and Satoru Matsushima for 687 their detailed reviews. 689 The authors especially thank Joel Jaeggli and Ray Hunter for his 690 efforts and contributions on editing which substantially improves the 691 legibility of the document. 693 Thanks to Cameron Byrne who was an active co-author of some earlier 694 versions of this draft. 696 12. Additional Author List 698 The following are extended authors who contributed to the effort: 700 Qiong Sun 701 China Telecom 702 Room 708, No.118, Xizhimennei Street 703 Beijing 100035 704 P.R.China 705 Phone: +86-10-58552936 706 Email: sunqiong@ctbri.com.cn 708 QiBo Niu 709 ZTE 710 50,RuanJian Road. 711 YuHua District, 712 Nan Jing 210012 713 P.R.China 714 Email: niu.qibo@zte.com.cn 716 13. References 717 13.1. Normative References 719 [I-D.ietf-appsawg-http-forwarded] 720 Petersson, A. and M. Nilsson, "Forwarded HTTP Extension", 721 draft-ietf-appsawg-http-forwarded-10 (work in progress), 722 October 2012. 724 [I-D.wing-dhc-dns-reconfigure] 725 Patil, P., Boucadair, M., Wing, D., and T. Reddy, "DHCPv6 726 Dynamic Reconfiguration", draft-wing-dhc-dns- 727 reconfigure-02 (work in progress), September 2013. 729 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 730 (IPv6) Specification", RFC 2460, December 1998. 732 [RFC3704] Baker, F. and P. Savola, "Ingress Filtering for Multihomed 733 Networks", BCP 84, RFC 3704, March 2004. 735 [RFC4193] Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast 736 Addresses", RFC 4193, October 2005. 738 [RFC4787] Audet, F. and C. Jennings, "Network Address Translation 739 (NAT) Behavioral Requirements for Unicast UDP", BCP 127, 740 RFC 4787, January 2007. 742 [RFC5382] Guha, S., Biswas, K., Ford, B., Sivakumar, S., and P. 743 Srisuresh, "NAT Behavioral Requirements for TCP", BCP 142, 744 RFC 5382, October 2008. 746 [RFC5424] Gerhards, R., "The Syslog Protocol", RFC 5424, March 2009. 748 [RFC5580] Tschofenig, H., Adrangi, F., Jones, M., Lior, A., and B. 749 Aboba, "Carrying Location Objects in RADIUS and Diameter", 750 RFC 5580, August 2009. 752 [RFC5722] Krishnan, S., "Handling of Overlapping IPv6 Fragments", 753 RFC 5722, December 2009. 755 [RFC5798] Nadas, S., "Virtual Router Redundancy Protocol (VRRP) 756 Version 3 for IPv4 and IPv6", RFC 5798, March 2010. 758 [RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection 759 (BFD)", RFC 5880, June 2010. 761 [RFC6052] Bao, C., Huitema, C., Bagnulo, M., Boucadair, M., and X. 762 Li, "IPv6 Addressing of IPv4/IPv6 Translators", RFC 6052, 763 October 2010. 765 [RFC6145] Li, X., Bao, C., and F. Baker, "IP/ICMP Translation 766 Algorithm", RFC 6145, April 2011. 768 [RFC6146] Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful 769 NAT64: Network Address and Protocol Translation from IPv6 770 Clients to IPv4 Servers", RFC 6146, April 2011. 772 [RFC6147] Bagnulo, M., Sullivan, A., Matthews, P., and I. van 773 Beijnum, "DNS64: DNS Extensions for Network Address 774 Translation from IPv6 Clients to IPv4 Servers", RFC 6147, 775 April 2011. 777 [RFC6384] van Beijnum, I., "An FTP Application Layer Gateway (ALG) 778 for IPv6-to-IPv4 Translation", RFC 6384, October 2011. 780 [RFC6555] Wing, D. and A. Yourtchenko, "Happy Eyeballs: Success with 781 Dual-Stack Hosts", RFC 6555, April 2012. 783 [RFC6724] Thaler, D., Draves, R., Matsumoto, A., and T. Chown, 784 "Default Address Selection for Internet Protocol Version 6 785 (IPv6)", RFC 6724, September 2012. 787 [RFC6887] Wing, D., Cheshire, S., Boucadair, M., Penno, R., and P. 788 Selkirk, "Port Control Protocol (PCP)", RFC 6887, April 789 2013. 791 [RFC6946] Gont, F., "Processing of IPv6 "Atomic" Fragments", RFC 792 6946, May 2013. 794 [RFC7050] Savolainen, T., Korhonen, J., and D. Wing, "Discovery of 795 the IPv6 Prefix Used for IPv6 Address Synthesis", RFC 796 7050, November 2013. 798 13.2. Informative References 800 [Alexa] Alexa, "http://www.alexa.com/topsites", April 2013. 802 [Cisco-VNI] 803 Cisco, "Cisco Visual Networking Index: Forecast and 804 Methodology, 2012-2017, 805 http://ciscovni.com/forecast-widget/index.html", May 2013. 807 [I-D.anderson-siit-dc] 808 Anderson, T., "Stateless IP/ICMP Translation in IPv6 Data 809 Centre Environments", draft-anderson-siit-dc-00 (work in 810 progress), November 2012. 812 [I-D.chen-behave-nat64-radius-extension] 813 Chen, G. and D. Binet, "Radius Attributes for Stateful 814 NAT64", draft-chen-behave-nat64-radius-extension-00 (work 815 in progress), July 2013. 817 [I-D.chen-sunset4-cgn-port-allocation] 818 Chen, G., "Analysis of NAT64 Port Allocation Method", 819 draft-chen-sunset4-cgn-port-allocation-02 (work in 820 progress), July 2013. 822 [I-D.donley-behave-deterministic-cgn] 823 Donley, C., Grundemann, C., Sarawat, V., Sundaresan, K., 824 and O. Vautrin, "Deterministic Address Mapping to Reduce 825 Logging in Carrier Grade NAT Deployments", draft-donley- 826 behave-deterministic-cgn-06 (work in progress), July 2013. 828 [I-D.ietf-softwire-4rd] 829 Despres, R., Jiang, S., Penno, R., Lee, Y., Chen, G., and 830 M. Chen, "IPv4 Residual Deployment via IPv6 - a Stateless 831 Solution (4rd)", draft-ietf-softwire-4rd-07 (work in 832 progress), October 2013. 834 [I-D.ietf-softwire-map-deployment] 835 Qiong, Q., Chen, M., Chen, G., Tsou, T., and S. Perreault, 836 "Mapping of Address and Port (MAP) - Deployment 837 Considerations", draft-ietf-softwire-map-deployment-03 838 (work in progress), October 2013. 840 [I-D.ietf-softwire-map-t] 841 Li, X., Bao, C., Dec, W., Troan, O., Matsushima, S., and 842 T. Murakami, "Mapping of Address and Port using 843 Translation (MAP-T)", draft-ietf-softwire-map-t-04 (work 844 in progress), September 2013. 846 [I-D.ietf-softwire-stateless-4v6-motivation] 847 Boucadair, M., Matsushima, S., Lee, Y., Bonness, O., 848 Borges, I., and G. Chen, "Motivations for Carrier-side 849 Stateless IPv4 over IPv6 Migration Solutions", draft-ietf- 850 softwire-stateless-4v6-motivation-05 (work in progress), 851 November 2012. 853 [I-D.ietf-v6ops-ula-usage-recommendations] 854 Liu, B., Jiang, S., and C. Byrne, "Recommendations of 855 Using Unique Local Addresses", draft-ietf-v6ops-ula-usage- 856 recommendations-01 (work in progress), October 2013. 858 [I-D.kaliwoda-sunset4-dual-ipv6-coexist] 859 Kaliwoda, A. and D. Binet, "Co-existence of both dual- 860 stack and IPv6-only hosts", draft-kaliwoda-sunset4-dual- 861 ipv6-coexist-01 (work in progress), October 2012. 863 [IR.92] Global System for Mobile Communications Association 864 (GSMA), , "IMS Profile for Voice and SMS Version 7.0", 865 March 2013. 867 [RFC6036] Carpenter, B. and S. Jiang, "Emerging Service Provider 868 Scenarios for IPv6 Deployment", RFC 6036, October 2010. 870 [RFC6056] Larsen, M. and F. Gont, "Recommendations for Transport- 871 Protocol Port Randomization", BCP 156, RFC 6056, January 872 2011. 874 [RFC6144] Baker, F., Li, X., Bao, C., and K. Yin, "Framework for 875 IPv4/IPv6 Translation", RFC 6144, April 2011. 877 [RFC6269] Ford, M., Boucadair, M., Durand, A., Levis, P., and P. 878 Roberts, "Issues with IP Address Sharing", RFC 6269, June 879 2011. 881 [RFC6346] Bush, R., "The Address plus Port (A+P) Approach to the 882 IPv4 Address Shortage", RFC 6346, August 2011. 884 [RFC6459] Korhonen, J., Soininen, J., Patil, B., Savolainen, T., 885 Bajko, G., and K. Iisakkila, "IPv6 in 3rd Generation 886 Partnership Project (3GPP) Evolved Packet System (EPS)", 887 RFC 6459, January 2012. 889 [RFC6586] Arkko, J. and A. Keranen, "Experiences from an IPv6-Only 890 Network", RFC 6586, April 2012. 892 [RFC6877] Mawatari, M., Kawashima, M., and C. Byrne, "464XLAT: 893 Combination of Stateful and Stateless Translation", RFC 894 6877, April 2013. 896 [RFC6883] Carpenter, B. and S. Jiang, "IPv6 Guidance for Internet 897 Content Providers and Application Service Providers", RFC 898 6883, March 2013. 900 [RFC6967] Boucadair, M., Touch, J., Levis, P., and R. Penno, 901 "Analysis of Potential Solutions for Revealing a Host 902 Identifier (HOST_ID) in Shared Address Deployments", RFC 903 6967, June 2013. 905 Appendix A. Testing Results of Application Behavior 906 We test several application behaviors in a lab environment to 907 evaluate the impact when a primary NAT64 is out of service. In this 908 testing, participants are asked to connect a IPv6-only WiFi network 909 using laptops, tablets or mobile phones. NAT64 is deployed as the 910 gateway to connect Internet service. The tested applications are 911 shown in the below table. Cold standby, warm standby and hot standby 912 are taken turn to be tested. The participants may experience service 913 interruption due to the NAT64 handover. Different interruption 914 intervals are tested to gauge application behaviors. The results are 915 illuminated as below. 917 Table 2: The acceptable delay of applications 918 +----------------+------------------------+-------------------------+ 919 | APPs | Acceptable Interrupt | Session Continuity | 920 | | Recovery | | 921 +----------------+------------------------+-------------------------+ 922 | Web Browse |As maximum as 6s | No | 923 +----------------+------------------------+-------------------------+ 924 |Http streaming |As maximum as 10s(cache)| Yes | 925 +----------------+------------------------+-------------------------+ 926 | Gaming | 200ms~400ms | Yes | 927 +----------------+------------------------+-------------------------+ 928 | P2P | 10~16s | Yes | 929 +----------------+------------------------+-------------------------+ 930 |Instant Message |1 minute | Yes | 931 +----------------+------------------------+-------------------------+ 932 |Mail |30 seconds | No | 933 +----------------+------------------------+-------------------------+ 934 |Downloading |1 minutes | No | 935 +----------------+------------------------+-------------------------+ 937 Authors' Addresses 939 Gang Chen 940 China Mobile 941 53A,Xibianmennei Ave., 942 Xuanwu District, 943 Beijing 100053 944 China 946 Email: phdgang@gmail.com 947 Zhen Cao 948 China Mobile 949 53A,Xibianmennei Ave., 950 Xuanwu District, 951 Beijing 100053 952 China 954 Email: caozhen@chinamobile.com 956 Chongfeng Xie 957 China Telecom 958 Room 708 No.118, Xizhimenneidajie 959 Beijing 100035 960 P.R.China 962 Email: xiechf@ctbri.com.cn 964 David Binet 965 France Telecom-Orange 966 Rennes 967 35000 968 France 970 Email: david.binet@orange.com