idnits 2.17.1 draft-nishizuka-cgn-deployment-considerations-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (Mar 29, 2013) is 4043 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-09) exists of draft-donley-behave-deterministic-cgn-05 ** Downref: Normative reference to an Informational draft: draft-donley-behave-deterministic-cgn (ref. 'I-D.donley-behave-deterministic-cgn') == Outdated reference: A later version (-06) exists of draft-ietf-opsawg-lsn-deployment-02 ** Downref: Normative reference to an Informational draft: draft-ietf-opsawg-lsn-deployment (ref. 'I-D.ietf-opsawg-lsn-deployment') ** Downref: Normative reference to an Informational RFC: RFC 2663 ** Downref: Normative reference to an Informational RFC: RFC 2993 ** Downref: Normative reference to an Informational RFC: RFC 6269 == Outdated reference: A later version (-05) exists of draft-chen-sunset4-cgn-port-allocation-01 == Outdated reference: A later version (-13) exists of draft-ietf-softwire-map-04 == Outdated reference: A later version (-06) exists of draft-tsou-behave-natx4-log-reduction-02 Summary: 5 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group K. Nishizuka 3 Internet-Draft NTT Communications 4 Intended status: Standards Track Mar 29, 2013 5 Expires: September 30, 2013 7 Carrier-Grade-NAT (CGN) Deployment Considerations. 8 draft-nishizuka-cgn-deployment-considerations-00 10 Abstract 12 This document provides deployment considerations for Carrier-Grade- 13 NAT (CGN) based on the verification result include the investigation 14 of the number of sessions of applications. The verification was 15 conducted in StarBED which is one of the largest scale network 16 experiment environment in Japan. A million of subscribers was 17 emulated and it revealed the realistic behavior of CGN. 19 Status of this Memo 21 This Internet-Draft is submitted in full conformance with the 22 provisions of BCP 78 and BCP 79. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF). Note that other groups may also distribute 26 working documents as Internet-Drafts. The list of current Internet- 27 Drafts is at http://datatracker.ietf.org/drafts/current/. 29 Internet-Drafts are draft documents valid for a maximum of six months 30 and may be updated, replaced, or obsoleted by other documents at any 31 time. It is inappropriate to use Internet-Drafts as reference 32 material or to cite them other than as "work in progress." 34 This Internet-Draft will expire on September 30, 2013. 36 Copyright Notice 38 Copyright (c) 2013 IETF Trust and the persons identified as the 39 document authors. All rights reserved. 41 This document is subject to BCP 78 and the IETF Trust's Legal 42 Provisions Relating to IETF Documents 43 (http://trustee.ietf.org/license-info) in effect on the date of 44 publication of this document. Please review these documents 45 carefully, as they describe your rights and restrictions with respect 46 to this document. Code Components extracted from this document must 47 include Simplified BSD License text as described in Section 4.e of 48 the Trust Legal Provisions and are provided without warranty as 49 described in the Simplified BSD License. 51 Table of Contents 53 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 54 2. Conventions used in this document . . . . . . . . . . . . . . 3 55 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 56 4. The number of sessions of applications . . . . . . . . . . . . 4 57 5. Feasibility of port assignment methods . . . . . . . . . . . . 6 58 5.1. Port assignment methods . . . . . . . . . . . . . . . . . 6 59 5.2. Efficiency of address saving . . . . . . . . . . . . . . . 6 60 5.3. Logging design . . . . . . . . . . . . . . . . . . . . . . 7 61 5.3.1. Amount of the NAT log . . . . . . . . . . . . . . . . 7 62 5.3.2. Necessity for destination information . . . . . . . . 8 63 6. Scalability of CGN . . . . . . . . . . . . . . . . . . . . . . 9 64 6.1. Performance of CGN . . . . . . . . . . . . . . . . . . . . 9 65 6.2. Redundancy features of CGN . . . . . . . . . . . . . . . . 11 66 6.3. DNS query traffic considerations . . . . . . . . . . . . . 12 67 6.4. Separation of traffic . . . . . . . . . . . . . . . . . . 13 68 7. Tested web sites and applications (Excerpts) . . . . . . . . . 13 69 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 70 9. Security Considerations . . . . . . . . . . . . . . . . . . . 14 71 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 72 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 73 11.1. Normative References . . . . . . . . . . . . . . . . . . . 15 74 11.2. Informative References . . . . . . . . . . . . . . . . . . 15 75 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16 77 1. Introduction 79 IP address sharing is tentative technic to deal with the shortage of 80 IPv4 addresses. As described in [RFC6269], IP address sharing causes 81 many issues such as application failures and security 82 vulnerabilities. A part of these issues is based on the assigned 83 number of sessions per user and port allocation method of CGN. This 84 document lists the number of port consumption of major application 85 and web sites. 87 The efficiency of CGN is based on the number of sessions allocated to 88 each user and the installation location. This document also 89 describes the deployment considerations of CGN to specify the optimum 90 place according to CGN performance. CGN performance was 91 experimentally-verified with realistic traffic generated by amount of 92 emulated users. 94 The growth of IPv6 is continual solution of the shortage of IPv4 95 addresses and frees these issues. By adopting the combination of the 96 IPv4 shared address and native IPv6, the duty of CGN will decrease 97 and as the result, the bad effect on applications which are caused by 98 the limitation of available ports and address translation itself and 99 security vulnerability will be resolved. The most effective way of 100 deploying CGN is examined in this document. Further discussion about 101 the integration of CGN into the existing network is studied in 102 [I-D.ietf-opsawg-lsn-deployment]. 104 2. Conventions used in this document 106 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 107 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 108 document are to be interpreted as described in RFC2119. 110 3. Motivation 112 With a progressive exhaustion of IPv4 addresses, the demands for 113 sharing IPv4 addresses with multiple customers are rapidly rising, 114 thus many proposals are getting much attention include Carrier Grade 115 NAT (CGN, or LSN for Large Scale NAT) 116 [I-D.ietf-behave-lsn-requirements], Dual-Stack Lite [RFC6333], NAT64 117 [RFC6146], Address+Port (A+P) [RFC6346], 464XLAT 118 [I-D.ietf-v6ops-464xlat] and MAP [I-D.ietf-softwire-map]. The 119 practical configuration of these method is based on the same 120 considerations as follows: 122 - Stateful or Stateless 123 - Centralized or Distributed 124 - Dynamic port assignment or Static port assignment 125 - Log reduction strategy 126 - Security considerations 128 The best practice about these considerations should be derived from 129 realistic experiment because there are pros and cons. Though we 130 tested them in NAT444 environment, the result is applicable for other 131 approaches. The investigation of number of sessions is described in 132 this document and it can be also helpful for all of them. 134 4. The number of sessions of applications 136 The number of concurrent sessions of applications is important factor 137 of designing of CGN because there is trade-off between the efficiency 138 of IPv4 address saving and the availability of those applications. 139 In addition, for security and fairness, we should limit the number of 140 sessions per user. As described in [RFC6269], infected devices could 141 rapidly exhaust the available ports of global pool addresses, hence 142 all the rest of users could not through the CGN anymore. So to place 143 the CGN to existing network, we should know how many sessions are 144 sufficient for every user. Here is a list of applications and their 145 average sessions. We selected and tested 50 sites from the list of 146 top sites and remarkable applications. 148 +-------------------------+----------+--------+--------+--------+ 149 | Application | Total | TCP | TCP | UDP | 150 | | sessions | port80 | port443| port53 | 151 +-------------------------+----------+--------+--------+--------+ 152 | Web mail | 65 | 35 | 30 | 20 | 153 | | | | | | 154 | Video | 83 | 77 | 6 | 20 | 155 | | | | | | 156 | Portal site | 47 | 47 | 0 | 13 | 157 | | | | | | 158 | EC site | 45 | 43 | 2 | 11 | 159 | | | | | | 160 | blog | 61 | 59 | 2 | 17 | 161 | | | | | | 162 | Search Engine | 8 | 8 | 0 | 4 | 163 | | | | | | 164 | Online Banking | 20 | 2 | 18 | 4 | 165 | | | | | | 166 | Cloud Service | 29 | 23 | 6 | 6 | 167 | | | | | | 168 | iTunes | 20 | 1 | 19 | 7 | 169 | | | | | | 170 | Twitter | 33 | 1 | 32 | 12 | 171 | | | | | | 172 | Twitter(mobile) | 14 | 2 | 11 | 3 | 173 | | | | | | 174 | facebook | 51 | 40 | 11 | 18 | 175 | | | | | | 176 | facebook(mobile) | 18 | 11 | 7 | 10 | 177 | | | | | | 178 | Game | 95 | 86 | 9 | 19 | 179 | | | | | | 180 +-------------------------+----------+--------+--------+--------+ 181 Figure 1: The number of sessions of applications. 183 Figure 1 185 The number of sessions of these applications are up to 100 sessions. 186 There are no longer high-consumption applications. This observation 187 implies that modern applications such as facebook have changed to use 188 multiplexed requests. Previously, to achieve high-performance web, 189 it abused HTTP sessions. Now, current cutting edge technologies such 190 as WebSocket, SPDY and HTTP2.0 avoid such an abusing. Basically, 191 multiple requests are multiplexed into one TCP connection. However, 192 a kind of game applications still consume many sessions. 194 The last factor of the estimation of number of sessions is how many 195 applications are used simultaneously within a single CPE (Customer 196 Premises Equipment) which includes non-PC devices like gaming 197 devices. Our investigation shows that the average number of session 198 of active subscriber is 400. We daresay the limitation of 1000 199 sessions per user would not affect the most of users while preventing 200 the severe abuse from certain users. 202 5. Feasibility of port assignment methods 204 Basing on the investigation of the number of sessions of 205 applications, the realistic parameter of each port assignment method 206 was estimated by the verification. 208 5.1. Port assignment methods 210 The efficiency of IPv4 saving by CGN is highly depending on how to 211 allocate the ports of pool addresses to each users. There are 2 212 major methods: dynamic assignment and static assignment 213 [I-D.chen-sunset4-cgn-port-allocation]. There are combined problem 214 involving efficiency of address saving and logging information 215 reduction. Typical IP Network Address Translator (NAT) 216 [RFC2663][RFC2993] implementation uses dynamic assignment, so NAT444, 217 NAT64, DS-lite and 464XLAT are originally dynamic assignment 218 approach. To avoid the huge amount of information needed to be 219 recorded, those approaches have variations of static assignment 220 [I-D.donley-behave-deterministic-cgn] and MAP is inherently static 221 assignment approach. For taking advantage of both methods, the 222 hybrid method that is dynamic assignment of port ranges has been 223 implemented in some CGN. The merits of the port block assignment 224 have been referred in [RFC6346], 225 [I-D.donley-behave-deterministic-cgn] and 226 [I-D.chen-sunset4-cgn-port-allocation]. 228 5.2. Efficiency of address saving 230 In the dynamic assignment, the ports of pool address are allocated 231 randomly for active users. This method can use pool addresses and 232 ports most effectively. The average number of port consumption (N) 233 per active subscriber is the key value for dynamic assignment. In 234 the verification, the average number of port consumption (N) was 235 estimated to be 400. At the same time, user-quota of 1000 sessions 236 was set to avoid the abuse. The percentage of the active subscribers 237 (a) was estimated to be 25% at the value during the busy hour of 238 traffic (21:00 pm to 1:00am). In this time, "active" subscriber 239 means who create a new session in certain period of time. Then, when 240 a CGN adopt the dynamic assignment, the required number of the pool 241 address is as follows: 243 # of pool address (P) = # of Subscriber (S) * a * N / (65536 - R) 245 Here, (R) is reserved TCP/UDP port list referred in 246 [I-D.donley-behave-deterministic-cgn]. CGN should eliminate the 247 wellknown ports (0-1023 for TCP and UDP) to avoid the bad 248 interpretation from destination servers. It is natural to translate 249 source port of outgoing packet to ephemeral ports. The ports after 250 32768 is considered to be used without any problems because ephemeral 251 ports are 32768-61000 for Linux and 49152-65535 in the IANA 252 recommendations. As the result, 3,051 pool addresses are sufficient 253 for 1,000,000 subscribers. The feasibility of configuration was 254 confirmed in the verification. 256 On the other hand, in static assignment, the ports are allocated a 257 priori for every users. The pool addresses and ports are reserved to 258 every users, so most of them could be a dead stock because there are 259 light users and heavy users in aspect of port consumption. The max 260 number of port consumption in all subscribers is the key value for 261 static assignment. The true peak number of the session by a heavy 262 user could be over 10,000 sessions. However it is assumed that such 263 a severe consumption of ports to be an abuse, so the number of 264 statically assigned port (M) is controllable parameter by each 265 providers. In the static assignment, the required number of the pool 266 address is as follows: 268 # of pool address (P) = # of Subscriber (S) * M / (65536 - R) 270 Taking account into the investigation of number of sessions of 271 applications, the desirable value of (M) is over 1,000. As the 272 result, no less than 30,517 pool addresses are needed for 1,000,000 273 subscribers. The compression ratio is one tenth of the case of 274 dynamic assignment. 276 5.3. Logging design 278 5.3.1. Amount of the NAT log 280 In the aspect of the efficiency, the dynamic assignment is preferable 281 than the static assignment. However the size of the log is the main 282 consideration. There is a case that providers must identify a user 283 to respond abuse or public safety requests. Conventionally, source 284 IP address and a timestamp are needed. It was possible to identify a 285 user by comparing IP address with authentication logs of the exact 286 time. However, when IP address is shared by the CGN, it is necessary 287 to compare the translated address and port information which are 288 given by the destination host with the NAT log to identify the 289 untranslated IP address. According to the 290 [I-D.ietf-behave-lsn-requirements], following information is 291 recommended to log (for NAT444): 293 - Transport Protocol 294 - Source IP address:port 295 - Source IP address:port after translation 296 - Timestamp 298 In addition, the indicator of the allocation and deallocation are 299 needed because it assures that the identified subscriber certainly 300 had been using the translated IP address and port. Plus, including 301 the index of the CGN host, the average size of NAT log is about 120 302 byte in ASCII format. Every active subscriber generate 400 sessions 303 in average for a certain amount of time. It is assumed that the 304 event happens every 5 minute in the most severe condition. The size 305 of the log (L) for time frame (T) can be estimated as follows: 307 The size of log (L) = # of Subscriber (S) * a * N * 120byte * 2 * ( 308 Time frame(T) / 5 min. ) 310 It should be noted that the log is generated at the timing of NAT 311 table creation and freeing. As the result, for 1,000,000 users, the 312 size of log is piled up to 6.4 terabytes per day. The verification 313 result confirm the existing estimation referred in 314 [I-D.donley-behave-deterministic-cgn]. 316 The size of the log can be reduced without loss of the amount of 317 information. Compact format is the technique of reducing the amount 318 of log by using a notational change (hexadecimal number). It was 319 confirmed by verification that the compact format can reduce amount 320 of log to about 80% as compared with ASCII format. Though it was not 321 tested, theoretically binary format is the smallest notation and 322 amount of log can be reduced to 30%. 324 In static allocation, the amount of log is dramatically reduced even 325 to zero because the untranslated IP address and the translated IP 326 address / port range are mapped a priori. 328 5.3.2. Necessity for destination information 330 In [RFC6269], it is pointed out that only providing information about 331 the external address to a service provider is no longer sufficient to 332 unambiguously identify customers. One of the solutions is the method 333 of recording the source port information (and exact time stamp) 334 additionally by the destination server or FW, which is demanded in 335 [RFC6302]. The other solution is the method of recording destination 336 IP address and port information by CGN of service provider. The both 337 solutions are imperfect. In [I-D.tsou-behave-natx4-log-reduction], 338 it is noted that source port recording is not supported by every 339 application. Thus, to increase the certainty, additional logging of 340 destination address and port is effective measure to deal with the 341 legal request from servers which are not compliant with [RFC6302]. 342 In dynamic assignment, to log destination address is additional. It 343 is confirmed by the verification that by logging destination address, 344 only 4% of amount of log is increased in ASCII format. On the other 345 hand, in static assignment, logging of every session is newly 346 required and it has the same amount of log as the dynamic assignment. 347 It completely breaks the merit of the static assignment. 349 6. Scalability of CGN 351 The estimation of efficiency of address saving and the logging design 352 are depending on the number of subscribers accommodated with a CGN. 353 The scalability of the current CGN was verified by the measurement of 354 the performance. 356 6.1. Performance of CGN 358 According to the experimental results, there are three base 359 capacities to indicate CGN performance as follows: 361 - Through put 362 - MCS: Max Concurrent Sessions 363 - CPS: Connections per Sec 365 These capacities are not independent of each other, but become mixed 366 load for CGN. Each load will be combined in real network traffic, 367 thus using subscriber emulated traffic is important for measuring the 368 performance in realistic way. 370 Through put is forwarding performance of CGN. Currently CGN 371 equipments with an IF of 1GigabitEthernet and 10GigabitEthernet are 372 flagship models of the manufactures, but CGN has an upper limit 373 internally because the performance depends on internal devices such 374 as CPUs. By ON / OFF of ALGs (Application Level Gateway), the 375 forwarding performance will be affected because the traffic process 376 is possibly changed to the path through CPU. 378 MCS shows an upper limit of numbers of records kept in NAT table. 379 Number of holding sessions depends on retention time of NAT table. 380 That is because, even after the end of data transmission, the NAT 381 table is held in a certain period of time to guarantee the behavior 382 of an application. As described in 383 [I-D.ietf-behave-lsn-requirements] REQ-8, if the CGN tracks TCP 384 sessions, NAT tables may be released when RST or FIN of TCP has been 385 observed. In case of TCP session where RST or FIN session has not 386 been observed, and UDP and ICMP communication, NAT table should 387 retain a certain amount of time. Also, in case of Full Cone NAT, a 388 table of Full Cone NAT also should retain a certain time to await 389 communication from outside for a certain period of time. It is 390 effective to shorten the time-out value in order to suppress the 391 overflowing NAT table, but it is needed to be careful not to inhibit 392 the behavior of the application. It is desirable that retention time 393 of NAT table is configurable as time-out value. In the experiment, 394 the time-out values are as follows: 396 +------------------+---------+--------+--------+--------+--------+ 397 | Protocols | TCP | TCP | UDP | DNS | ICMP | 398 | | | SYN | | | | 399 +------------------+---------+--------+--------+--------+--------+ 400 | Time-out Value | 300 | 60 | 300 | 3 | 2 | 401 +------------------+---------+--------+--------+--------+--------+ 402 Figure 2: The time-out values (sec) in the experiment. 404 Figure 2 406 These settings didn't break the behavior of applications. 408 It is very difficult to estimate maximum number of concurrent 409 sessions in the network where traffic is already flowing. Maximum 410 number of concurrent sessions was estimated to be 1M sessions per 411 10,000 users by our assumption as follows: 413 Max Concurrent Sessions (MCS) = # of Subscriber (S) * a * N 415 As the result, it is verified that tested current CGN is able to have 416 16M sessions for 160,000 subscribers with the capability of the 417 dynamic assignment and logging. It means that introducing CGN up to 418 about 15G traffic section is capable, which implies that CGN can be 419 placed to more centralized position of the network. In summary, the 420 settings and the performance result are as follows: 422 +----------------------------------------------+ 423 | | Assumed | 424 | | Values | 425 +---------------------------------+------------+ 426 | average # of sessions(N) | 400 | 427 +---------------------------------+------------+ 428 | % of the active subscribers (a) | 25 | 429 +---------------------------------+------------+ 430 | | Verified | 431 | | Values | 432 +---------------------------------+------------+ 433 | # of Subscriber (S) | 160,000 | 434 +---------------------------------+------------+ 435 | Max Concurrent Sessions(MCS) | 16,000,000 | 436 +---------------------------------+------------+ 437 | Connection Per Sec(CPS) | 30,000 | 438 +---------------------------------+------------+ 439 | # of pool address (P) | 4,000 | 440 +---------------------------------+------------+ 441 | size of log (L) (in 10min) | 7.0GB | 442 +---------------------------------+------------+ 444 Figure 3: The performance results of tested CGN. 446 Figure 3 448 In the verification, session arrival rate by emulated subscribers was 449 not so high because the load of concurrent sessions is noticeable in 450 the equipment used in the experiment. There were no problems in weak 451 load of about 30,000 CPS. In case that traffic flows suddenly change 452 to standby equipment in redundant network, CPS performance becomes 453 rate-limiting, so CPS performance is also important factor to 454 minimize the effect of failures. 456 6.2. Redundancy features of CGN 458 It is often referred that introduction of CGN could create Single 459 Point of Failure(SPOF) (ex. in [RFC6269]). CGN is stateful, in 460 contrast to stateless BR of MAP, so the redundant configuration must 461 be achieved by the synchronization of the NAT table between redundant 462 equipments. Moreover, introduction of CGN creates layer 3 boundary 463 to NATed traffic, so the redundancy features may work with around 464 routers via dynamic routing. Nevertheless, it is verified that 465 current CGN can be configured and introduced to service providers 466 network with the redundancy features. In the verification, CGN can 467 be switched to another CGN with sub-sec loss of traffic even in the 468 situation that they holds 16M concurrent sessions. 470 6.3. DNS query traffic considerations 472 How to deal with the DNS query traffic is unignorable concern for 473 deploiment of CGN. In the test scenario, control experiment was 474 conducted to reveal the impact of the huge amount of DNS queries. 476 Access Node CGN 477 Emulated +-----------+ 478 Subscribers +-------+ +-------+ | | 479 +----+ | | | | | | 480 | --+-----+ R +---------+ CGN +------+ | 481 +----+ | | | | | Core | 482 +---+---+ +-------+ | Network | 483 | | | 484 | | | 485 | +-------+ | +-------+ | 486 | | | | | | | 487 +-------------+ DNS +------+ | DNS | | 488 | | | | | | 489 +-------+ | +-------+ | 490 +-----------+ 491 DNS Forwarder 493 Figure 4: Bypassing of DNS queries using DNS forwarder. 495 In the first case, the original DNS server IP address in the service 496 provider network is distributed to the subscribers. The emulated 497 subscribers use the DNS server to get host IP address by name, so all 498 query packets go through the CGN. The generated DNS query is 12M at 499 the speed of 10k query per sec. In the second case, IP address of a 500 DNS server placed in the bypassing position of CGN is used instead. 501 The second DNS server works as a forwarder, so all queries are 502 forwarded to the first DNS server. Therefore, all DNS queries are 503 bypassed from the CGN while data traffic is still going through the 504 CGN. 506 As the result, it was shown that DNS query almost does not affect the 507 performance of the CGN. The max concurrent sessions of DNS packet 508 was only 40k. NAT table of DNS timeouts in 3 seconds, thus It saves 509 the consumption of NAT tables. However NAT log was generated for 510 every query and it doubled the total amount of the log. It would be 511 rare that the NAT log of DNS is needed to react to a legal request. 512 The impact of the DNS query traffic is relatively small if DNS 513 timeout is adjusted. 515 6.4. Separation of traffic 517 In the existing network, IPv4 communication and IPv6 communication 518 may already be mixed in the dual stack. In this case, by introducing 519 CGN which can route IPv6 and existing IPv4 aside from NAT function, 520 the influence for the network architecture could be suppressed and so 521 a flexible design is possible. However, though current CGN is 522 scalable enough to be deployed in core of the service providers 523 network, the feature of routing is insufficient to replace the 524 exsisting routers. Such a CGN is desirable, otherwise the design 525 which makes IPv6 traffic and traditional IPv4 traffic bypass from CGN 526 is effective choice for providers. In dividing NAT flows and non-NAT 527 flows routers around the CGN, VRF (Virtual Routing and Forwarding) 528 and PBR (policy based routing) are needed in a router under CGN. In 529 that case it is indispensable to configure routers so that the 530 hairpining communication between the NAT user and non-NAT user to be 531 possible. The considerations about the separation of traffic and 532 effective deployment configuration are discussed in detail in 533 [I-D.ietf-opsawg-lsn-deployment]. 535 7. Tested web sites and applications (Excerpts) 537 - Web Mail 538 gmail 539 yahoo mail 540 hot mail 541 - Video 542 ustream 543 youtube 544 nicovideos 545 Hulu 546 dailymotion 547 daum 548 qq 549 fc2 550 xvideos 551 - Portal&EC site 552 yahoo 553 rakuten 554 amazon 555 apple 556 - Blog 557 livedoor blog 558 ameba blog 559 - Search Engine 560 google 561 - Online Banking 562 mizuho bank 563 DC card 564 - Cloud Service 565 drop box 566 Evernote 567 - InstantMessenger & VoIP 568 skype 569 Line 570 - facebook 571 - twitter 572 - google map 573 - Online PC Game 574 aeria games 575 ameba pigg 576 nexon 577 hangame 578 - Consumer Game 579 Armored Core V (Play Station3) 580 Dark Souls 2 (Play Station3) 581 Gundam Extreme VS. (Play Station3) 582 Kinect adventure (XBox) 583 Persona 4 the ultimate in mayonaka arena (XBox) 584 Mingol 4 (WiiU) 585 Monster Hunter 3G (DS-lite) 586 Keri-hime sweets (iOS) 587 PuzzDra (iOS) 589 8. IANA Considerations 591 This document makes no request of IANA. 593 9. Security Considerations 595 TBD 597 10. Acknowledgments 599 This research and experiment are conducted under the great support of 600 Ministry of Internal Affairs and Communications of Japan. Many 601 thanks to MIC, JAIST members and Shin Miyakawa for their ideas and 602 feedback in documentation. 604 11. References 606 11.1. Normative References 608 [I-D.donley-behave-deterministic-cgn] 609 Donley, C., Grundemann, C., Sarawat, V., Sundaresan, K., 610 and O. Vautrin, "Deterministic Address Mapping to Reduce 611 Logging in Carrier Grade NAT Deployments", 612 draft-donley-behave-deterministic-cgn-05 (work in 613 progress), January 2013. 615 [I-D.ietf-behave-lsn-requirements] 616 Perreault, S., Yamagata, I., Miyakawa, S., Nakagawa, A., 617 and H. Ashida, "Common requirements for Carrier Grade NATs 618 (CGNs)", draft-ietf-behave-lsn-requirements-10 (work in 619 progress), December 2012. 621 [I-D.ietf-opsawg-lsn-deployment] 622 Kuarsingh, V. and J. Cianfarani, "CGN Deployment with BGP/ 623 MPLS IP VPNs", draft-ietf-opsawg-lsn-deployment-02 (work 624 in progress), February 2013. 626 [RFC2663] Srisuresh, P. and M. Holdrege, "IP Network Address 627 Translator (NAT) Terminology and Considerations", 628 RFC 2663, August 1999. 630 [RFC2993] Hain, T., "Architectural Implications of NAT", RFC 2993, 631 November 2000. 633 [RFC6269] Ford, M., Boucadair, M., Durand, A., Levis, P., and P. 634 Roberts, "Issues with IP Address Sharing", RFC 6269, 635 June 2011. 637 11.2. Informative References 639 [I-D.chen-sunset4-cgn-port-allocation] 640 Chen, G., "Analysis of NAT64 Port Allocation Method", 641 draft-chen-sunset4-cgn-port-allocation-01 (work in 642 progress), February 2013. 644 [I-D.ietf-softwire-map] 645 Troan, O., Dec, W., Li, X., Bao, C., Matsushima, S., and 646 T. Murakami, "Mapping of Address and Port with 647 Encapsulation (MAP)", draft-ietf-softwire-map-04 (work in 648 progress), February 2013. 650 [I-D.ietf-v6ops-464xlat] 651 Mawatari, M., Kawashima, M., and C. Byrne, "464XLAT: 653 Combination of Stateful and Stateless Translation", 654 draft-ietf-v6ops-464xlat-10 (work in progress), 655 February 2013. 657 [I-D.tsou-behave-natx4-log-reduction] 658 ZOU), T., Li, W., and T. Taylor, "Port Management To 659 Reduce Logging In Large-Scale NATs", 660 draft-tsou-behave-natx4-log-reduction-02 (work in 661 progress), September 2010. 663 [RFC6146] Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful 664 NAT64: Network Address and Protocol Translation from IPv6 665 Clients to IPv4 Servers", RFC 6146, April 2011. 667 [RFC6302] Durand, A., Gashinsky, I., Lee, D., and S. Sheppard, 668 "Logging Recommendations for Internet-Facing Servers", 669 BCP 162, RFC 6302, June 2011. 671 [RFC6333] Durand, A., Droms, R., Woodyatt, J., and Y. Lee, "Dual- 672 Stack Lite Broadband Deployments Following IPv4 673 Exhaustion", RFC 6333, August 2011. 675 [RFC6346] Bush, R., "The Address plus Port (A+P) Approach to the 676 IPv4 Address Shortage", RFC 6346, August 2011. 678 Author's Address 680 Kaname Nishizuka 681 NTT Communications Corporation 682 Granpark Tower 683 3-4-1 Shibaura, Minato-ku 684 Tokyo 108-8118 685 Japan 687 Email: kaname@nttv6.jp