idnits 2.17.1 draft-nishizuka-cgn-deployment-considerations-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (Sep 28, 2013) is 3855 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Informational ---------------------------------------------------------------------------- == Outdated reference: A later version (-09) exists of draft-donley-behave-deterministic-cgn-06 == Outdated reference: A later version (-06) exists of draft-ietf-opsawg-lsn-deployment-03 == Outdated reference: A later version (-05) exists of draft-chen-sunset4-cgn-port-allocation-02 == Outdated reference: A later version (-13) exists of draft-ietf-softwire-map-08 == Outdated reference: A later version (-06) exists of draft-tsou-behave-natx4-log-reduction-04 Summary: 0 errors (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group K. Nishizuka 3 Internet-Draft NTT Communications 4 Intended status: Informational D. Natsume 5 Expires: April 1, 2014 NTT Neomeit 6 Sep 28, 2013 8 Carrier-Grade-NAT (CGN) Deployment Considerations. 9 draft-nishizuka-cgn-deployment-considerations-01 11 Abstract 13 This document provides deployment considerations for Carrier-Grade- 14 NAT (CGN). Due to emerging new web technologies such as Websocket, 15 SPDY and HTTP2.0, the trend of the Internet traffic has been 16 changing. The number of sessions of commonly-used applications were 17 investigated to estimate the efficiency of IPv4 address sharing of 18 CGN. Based on the result of the average number of sessions of 19 subscribers, the verification of CGN was conducted in the large scale 20 network experiment environment with one million emulated subscribers. 21 It revealed that CGN can be used in more centralized location of a 22 provider's network and it arose many considerations. 24 Status of this Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at http://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on April 1, 2014. 41 Copyright Notice 43 Copyright (c) 2013 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (http://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2. Conventions used in this document . . . . . . . . . . . . . . 3 60 3. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 61 4. The number of sessions of applications . . . . . . . . . . . . 4 62 5. Feasibility of port assignment methods . . . . . . . . . . . . 6 63 5.1. Port assignment methods . . . . . . . . . . . . . . . . . 6 64 5.2. Efficiency of address saving . . . . . . . . . . . . . . . 6 65 5.3. Logging design . . . . . . . . . . . . . . . . . . . . . . 7 66 5.3.1. Amount of the NAT log . . . . . . . . . . . . . . . . 7 67 5.3.2. Necessity for destination information . . . . . . . . 9 68 6. Scalability of CGN . . . . . . . . . . . . . . . . . . . . . . 9 69 6.1. Performance of CGN . . . . . . . . . . . . . . . . . . . . 9 70 6.2. Redundancy features of CGN . . . . . . . . . . . . . . . . 11 71 6.3. DNS query traffic considerations . . . . . . . . . . . . . 12 72 6.4. Separation of traffic . . . . . . . . . . . . . . . . . . 13 73 7. Tested web sites and applications (Excerpts) . . . . . . . . . 13 74 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 75 9. Security Considerations . . . . . . . . . . . . . . . . . . . 14 76 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 77 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 78 11.1. Normative References . . . . . . . . . . . . . . . . . . . 15 79 11.2. Informative References . . . . . . . . . . . . . . . . . . 15 80 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 16 82 1. Introduction 84 IP address sharing is tentative technic to deal with the shortage of 85 IPv4 addresses. As described in [RFC6269], IP address sharing causes 86 many issues such as application failures and security 87 vulnerabilities. A part of these issues is based on the assigned 88 number of sessions per user and port allocation method of CGN. How 89 many sessions are sufficient for users is one of the important 90 considerations. Moreover, the efficiency of CGN is based on the 91 average number of sessions of subscribers. To answer to these 92 points, this document lists the number of port consumption of major 93 application and web sites. 95 This document also describes the deployment considerations of CGN to 96 specify the optimum place according to CGN performance. CGN 97 performance was experimentally-verified with realistic traffic 98 generated by amount of emulated users. 100 The growth of IPv6 is continual solution of the shortage of IPv4 101 addresses and frees these issues. By adopting the combination of the 102 IPv4 shared address and native IPv6, the duty of CGN will decrease 103 and as the result, the bad effect on applications which are caused by 104 the limitation of available ports and address translation itself and 105 security vulnerability will be resolved. The most effective way of 106 deploying CGN is examined in this document. Further discussion about 107 the integration of CGN into the existing network is studied in 108 [I-D.ietf-opsawg-lsn-deployment]. 110 2. Conventions used in this document 112 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 113 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 114 document are to be interpreted as described in RFC2119. 116 3. Motivation 118 With a progressive exhaustion of IPv4 addresses, the demands for 119 sharing IPv4 addresses with multiple customers are rapidly rising, 120 thus many proposals are getting much attention include Carrier Grade 121 NAT (CGN, or LSN for Large Scale NAT) [RFC6888], Dual-Stack Lite 122 [RFC6333], NAT64 [RFC6146], Address+Port (A+P) [RFC6346], 464XLAT 123 [RFC6877] and MAP [I-D.ietf-softwire-map]. The practical 124 configuration of these method is based on the same considerations as 125 follows: 127 - Stateful or Stateless 128 - Centralized or Distributed 129 - Dynamic port assignment or Static port assignment 130 - Log reduction strategy 131 - Security considerations 133 The best practice about these considerations should be derived from 134 realistic experiment because there are pros and cons. Though we 135 tested them in NAT444 environment, the result is applicable for other 136 approaches. The investigation of number of sessions is described in 137 this document and it can be also helpful for all of them. 139 4. The number of sessions of applications 141 The number of concurrent sessions of applications is important factor 142 of designing of CGN because there is trade-off between the efficiency 143 of IPv4 address saving and the availability of those applications. 144 In addition, for security and fairness, we should limit the number of 145 sessions per user. As described in [RFC6269], infected devices could 146 rapidly exhaust the available ports of global pool addresses, hence 147 all the rest of users could not through the CGN anymore. In order to 148 place the CGN to existing network, we should know how many sessions 149 are sufficient for every user. Here is a list of applications and 150 their average sessions. We selected and tested 50 sites from the 151 list of top sites and remarkable applications. For web browsing, We 152 used Chrome and Firefox which are capable of SPDY. 154 +-------------------------+----------+--------+--------+--------+ 155 | Application | Total | TCP | TCP | UDP | 156 | | sessions | port80 | port443| port53 | 157 +-------------------------+----------+--------+--------+--------+ 158 | Web mail | 65 | 35 | 30 | 20 | 159 | | | | | | 160 | Video | 83 | 77 | 6 | 20 | 161 | | | | | | 162 | Portal site | 47 | 47 | 0 | 13 | 163 | | | | | | 164 | EC site | 45 | 43 | 2 | 11 | 165 | | | | | | 166 | blog | 61 | 59 | 2 | 17 | 167 | | | | | | 168 | Search Engine | 8 | 8 | 0 | 4 | 169 | | | | | | 170 | Online Banking | 20 | 2 | 18 | 4 | 171 | | | | | | 172 | Cloud Service | 29 | 23 | 6 | 6 | 173 | | | | | | 174 | iTunes | 20 | 1 | 19 | 7 | 175 | | | | | | 176 | Twitter | 33 | 1 | 32 | 12 | 177 | | | | | | 178 | Twitter(mobile) | 14 | 2 | 11 | 3 | 179 | | | | | | 180 | facebook | 51 | 40 | 11 | 18 | 181 | | | | | | 182 | facebook(mobile) | 18 | 11 | 7 | 10 | 183 | | | | | | 184 | Game | 95 | 86 | 9 | 19 | 185 | | | | | | 186 +-------------------------+----------+--------+--------+--------+ 187 Figure 1: The number of sessions of applications. 189 Figure 1 191 The number of sessions of these applications are up to 100 sessions. 192 There are no longer high-consumption applications. This observation 193 implies that modern applications such as facebook have changed to use 194 multiplexed requests. Previously, web technologies for achieving 195 high-performance access consumed many HTTP sessions. Now, current 196 cutting edge technologies such as WebSocket, SPDY and HTTP2.0 avoid 197 such an abusing. Basically, all the requests are multiplexed into 198 one TCP connection. However, a kind of game applications still 199 consume many sessions. 201 The last factor of the estimation of number of sessions is how many 202 applications are used simultaneously within a single CPE (Customer 203 Premises Equipment) which includes non-PC devices like gaming 204 devices. Our investigation shows that the average number of session 205 of active subscriber is 400. We daresay the limitation of 1000 206 sessions per user would not affect the most of users while preventing 207 the severe abuse from certain users. 209 5. Feasibility of port assignment methods 211 Basing on the investigation of the number of sessions of 212 applications, the realistic parameter of each port assignment method 213 was estimated by the verification. 215 5.1. Port assignment methods 217 The efficiency of IPv4 saving by CGN is highly depending on how to 218 allocate the ports of pool addresses to each users. There are 2 219 major methods: dynamic assignment and static assignment 220 [I-D.chen-sunset4-cgn-port-allocation]. There are combined problem 221 involving efficiency of address saving and logging information 222 reduction. Typical IP Network Address Translator (NAT) 223 [RFC2663][RFC2993] implementation uses dynamic assignment, so NAT444, 224 NAT64, DS-lite and 464XLAT are originally dynamic assignment 225 approach. To avoid the huge amount of information needed to be 226 recorded, those approaches have variations of static assignment 227 [I-D.donley-behave-deterministic-cgn] and MAP is inherently static 228 assignment approach. For taking advantage of both methods, the 229 hybrid method that is dynamic assignment of port ranges has been 230 implemented in some CGN. The merits of the port block assignment 231 have been referred in [RFC6346], 232 [I-D.donley-behave-deterministic-cgn] and 233 [I-D.chen-sunset4-cgn-port-allocation]. 235 5.2. Efficiency of address saving 237 In the dynamic assignment, the ports of pool address are allocated 238 randomly for active users. This method can use pool addresses and 239 ports most effectively. The average number of port consumption (N) 240 per active subscriber is the key value for dynamic assignment. In 241 the verification, the average number of port consumption (N) was 242 estimated to be 400. At the same time, user-quota of 1000 sessions 243 was set to avoid the abuse. The percentage of the active subscribers 244 (a) was estimated to be 25% at the value during the busy hour of 245 traffic (21:00 pm to 1:00am). In this time, "active" subscriber 246 means who create a new session in certain period of time. Then, when 247 a CGN adopt the dynamic assignment, the required number of the pool 248 address is as follows: 250 # of pool address (P) = # of Subscriber (S) * a * N / (65536 - R) 252 Here, (R) is reserved TCP/UDP port list referred in 253 [I-D.donley-behave-deterministic-cgn]. CGN should eliminate the 254 wellknown ports (0-1023 for TCP and UDP) to avoid the bad 255 interpretation from destination servers. It is natural to translate 256 source port of outgoing packet to ephemeral ports. Using the 257 equation, 1550 pool addresses are sufficient for 1,000,000 258 subscribers. 260 On the other hand, in static assignment, the ports are allocated a 261 priori for every users. The pool addresses and ports are reserved to 262 every users, so most of them could be a dead stock because there are 263 light users and heavy users in aspect of port consumption. The max 264 number of port consumption in all subscribers is the key value for 265 static assignment. The true peak number of the session by a heavy 266 user could be over 10,000 sessions. However it can be assumed that 267 such a severe consumption of ports to be an abuse, so the number of 268 statically assigned port (M) is controllable parameter by each 269 providers. In the static assignment, the required number of the pool 270 address is as follows: 272 # of pool address (P) = # of Subscriber (S) * M / (65536 - R) 274 Taking account into the investigation of number of sessions of 275 applications, the desirable value of (M) is over 1,000. As the 276 result, no less than 15,501 pool addresses are needed for 1,000,000 277 subscribers. The compression ratio is one tenth of the case of 278 dynamic assignment. 280 The feasibility of dynamic and static assignment configuration was 281 confirmed in the verification. 283 5.3. Logging design 285 5.3.1. Amount of the NAT log 287 The size of the log is important consideration of dynamic assignment 288 because it demands a huge scale of logging ecosystem for CGN. There 289 is a case that providers must identify a user to respond abuse or 290 public safety requests. Conventionally, source IP address and a 291 timestamp are needed. It was possible to identify a user by 292 comparing IP address with authentication logs of the exact time. 293 However, when IP address is shared by the CGN, it is necessary to 294 compare the translated address and port information which are given 295 by the destination host with the NAT log to identify the untranslated 296 IP address. According to the [RFC6888], following information is 297 recommended to log (for NAT444): 299 - Transport Protocol - 1 byte 300 - Source IP address:port - 6 byte 301 - Source IP address:port after translation - 6 byte 302 - Timestamp - 8 byte 304 In addition, the indicator of the allocation and deallocation are 305 needed because it assures that the identified subscriber certainly 306 had been using the translated IP address and port. Plus, some 307 identifier like the index or hostname of the CGN is needed to 308 identify to which realm an address belongs. 310 - Add/Delete - 1 byte 311 - CGN device ID - 4 byte 313 As the result, the minimum size of NAT log is 26 bytes in binary. In 314 ASCII format, the average size of NAT log is about 120 byte . Every 315 active subscriber generate 400 sessions in average for a certain 316 amount of time. It is assumed that the event happens every 5 minute 317 in the most severe condition. The size of the log (L) for time frame 318 (T) can be estimated as follows (for ASCII format): 320 The size of log (L) = # of Subscriber (S) * a * N * 120byte * 2 * ( 321 Time frame(T) / 5 min. ) 323 It should be noted that the log is generated at the timing of NAT 324 table creation and freeing. As the result, for 1,000,000 users, the 325 size of log is piled up to 6.4 terabytes per day. The verification 326 result confirm the existing estimation referred in 327 [I-D.donley-behave-deterministic-cgn]. 329 The size of the log can be reduced without loss of information. 330 Compact format is the technique of reducing the amount of log by 331 using a notational change (hexadecimal number). It was confirmed by 332 verification that the compact format can reduce amount of log to 333 about 80% as compared with ASCII format. Though it was not tested, 334 theoretically binary format is the smallest notation and amount of 335 log can be reduced to 22%. 337 In static allocation, the amount of log is dramatically reduced even 338 to zero because the untranslated IP address and the translated IP 339 address / port range are mapped a priori. 341 5.3.2. Necessity for destination information 343 In [RFC6269], it is pointed out that only providing information about 344 the external address to a service provider is no longer sufficient to 345 identify customers unambiguously . One of the solutions is the 346 method of recording the source port information (and exact time 347 stamp) additionally by the destination server or FW, which is 348 demanded in [RFC6302]. The other solution is the method of recording 349 destination IP address and port information by CGN of service 350 provider. The both solutions are imperfect. In 351 [I-D.tsou-behave-natx4-log-reduction], it is noted that source port 352 recording is not supported by every application. Thus, to increase 353 the certainty, additional logging of destination address and port is 354 effective measure to deal with the legal request from servers which 355 are not compliant with [RFC6302]. In dynamic assignment, to log 356 destination address is additional. It is confirmed by the 357 verification that by logging destination address, only 4% of amount 358 of log is increased in ASCII format. On the other hand, in static 359 assignment, logging of every session is newly required and it has the 360 same amount of log as the dynamic assignment. It completely breaks 361 the merit of the static assignment. 363 6. Scalability of CGN 365 The estimation of efficiency of address saving and the logging design 366 are depending on the number of subscribers accommodated with a CGN. 367 The scalability of the current CGN was verified by the measurement of 368 the performance. 370 6.1. Performance of CGN 372 According to the experimental results, there are three base 373 capacities to indicate CGN performance as follows: 375 - Through put 376 - MCS: Max Concurrent Sessions 377 - CPS: Connections per Sec 379 These capacities are not independent of each other, but become mixed 380 load for CGN. Each load will be combined in real network traffic, 381 thus using subscriber emulated traffic is important for measuring the 382 performance in realistic way. 384 Through put is forwarding performance of CGN. Currently CGN 385 equipments with an IF of 1GigabitEthernet and 10GigabitEthernet are 386 flagship models of the manufactures, but CGN has an upper limit 387 internally because the performance depends on internal devices such 388 as CPUs. By ON / OFF of ALGs (Application Level Gateway), the 389 forwarding performance will be affected because the traffic process 390 is possibly changed to the path through CPU. 392 MCS shows an upper limit of the number of records kept in NAT table. 393 The number of holding sessions depends on retention time of NAT 394 table. That is because, even after the end of data transmission, the 395 NAT table is held in a certain period of time to guarantee the 396 behavior of an application. As described in [RFC6888] REQ-8, if the 397 CGN tracks TCP sessions, NAT tables may be released when RST or FIN 398 of TCP has been observed. In case of TCP session where RST or FIN 399 session has not been observed, and UDP and ICMP communication, NAT 400 table should retain a certain amount of time. Also, in case of Full 401 Cone NAT, a table of Full Cone NAT also should retain a certain time 402 to await communication from outside for a certain period of time. It 403 is effective to shorten the time-out value in order to suppress the 404 overflowing NAT table, but it is needed to be careful not to inhibit 405 the behavior of the application. It is desirable that retention time 406 of NAT table is configurable as time-out value. In the experiment, 407 the time-out values are as follows: 409 +------------------+---------+--------+--------+--------+--------+ 410 | Protocols | TCP | TCP | UDP | DNS | ICMP | 411 | | | SYN | |(port53)| | 412 +------------------+---------+--------+--------+--------+--------+ 413 | Time-out Value | 300 | 60 | 300 | 3 | 2 | 414 +------------------+---------+--------+--------+--------+--------+ 415 Figure 2: The time-out values (sec) in the experiment. 417 Figure 2 419 These settings didn't break the behavior of applications we tested. 421 It is very difficult to estimate maximum number of concurrent 422 sessions in the network where traffic already exists. By our 423 assumption, maximum number of concurrent sessions was estimated to be 424 1M sessions per 10,000 users as follows: 426 Max Concurrent Sessions (MCS) = # of Subscriber (S) * a * N 428 As the result, it is verified that tested current CGN is able to have 429 16M sessions for 160,000 subscribers with the capability of the 430 dynamic assignment and logging. It means that introducing CGN up to 431 about 15G traffic section is capable, which implies that CGN can be 432 placed to more centralized position of the network. In summary, the 433 settings and the performance result are as follows: 435 +----------------------------------------------+ 436 | | Assumed | 437 | | Values | 438 +---------------------------------+------------+ 439 | average # of sessions(N) | 400 | 440 +---------------------------------+------------+ 441 | % of the active subscribers (a) | 25 | 442 +---------------------------------+------------+ 443 | | Verified | 444 | | Values | 445 +---------------------------------+------------+ 446 | # of Subscriber (S) | 160,000 | 447 +---------------------------------+------------+ 448 | Max Concurrent Sessions(MCS) | 16,000,000 | 449 +---------------------------------+------------+ 450 | Connection Per Sec(CPS) | 30,000 | 451 +---------------------------------+------------+ 452 | # of pool address (P) | 4,000 | 453 +---------------------------------+------------+ 454 | size of log (L) (in 10min) | 7.0GB | 455 +---------------------------------+------------+ 457 Figure 3: The performance results of tested CGN. 459 Figure 3 461 In the verification, session arrival rate by emulated subscribers was 462 not so high because the load of concurrent sessions is noticeable in 463 the equipment used in the experiment. There were no problems in weak 464 load of about 30,000 CPS. In case that traffic flows suddenly change 465 to standby equipment in redundant network, CPS performance becomes 466 rate-limiting, so CPS performance is also important factor to 467 minimize the effect of failures. 469 6.2. Redundancy features of CGN 471 It is often referred that introduction of CGN could create Single 472 Point of Failure(SPOF) (ex. in [RFC6269]). CGN is stateful, in 473 contrast to stateless BR of MAP, so the redundant configuration must 474 be achieved by the synchronization of the NAT table between redundant 475 equipments. Moreover, introduction of CGN creates layer 3 boundary 476 to NATed traffic, so the redundancy features may work with routers 477 via dynamic routing. Nevertheless, it is verified that current CGN 478 can be configured and introduced to service providers network with 479 the redundancy features. In the verification, CGN was able to switch 480 to another CGN with sub-sec loss of traffic even in the situation 481 that they holds 16M concurrent sessions. 483 6.3. DNS query traffic considerations 485 How to deal with the DNS query traffic is unignorable concern for 486 deployment of CGN. In the test scenario, a control experiment was 487 conducted to reveal the impact of the huge amount of DNS queries. 489 Access Node CGN 490 Emulated +-----------+ 491 Subscribers +-------+ +-------+ | | 492 +----+ | | | | | | 493 | --+-----+ R +---------+ CGN +------+ | 494 +----+ | | | | | Core | 495 +---+---+ +-------+ | Network | 496 | | | 497 | | | 498 | +-------+ | +-------+ | 499 | | | | | | | 500 +-------------+ DNS +------+ | DNS | | 501 | | | | | | 502 +-------+ | +-------+ | 503 +-----------+ 504 DNS Forwarder 506 Figure 4: Bypassing of DNS queries using DNS forwarder. 508 In the first case, the original DNS server IP address in the service 509 provider network is distributed to the subscribers. The emulated 510 subscribers use the DNS server to get host IP address by name, so all 511 query packets go through the CGN. The generated DNS query is 12M at 512 the speed of 10k query per sec. In the second case, IP address of a 513 DNS server placed in the bypassing position of CGN is distributed to 514 users. The second DNS server works as a forwarder, so all queries 515 are forwarded to the first DNS server. Therefore, all DNS queries 516 are bypassed from the CGN while data traffic is still going through 517 the CGN. 519 As the result, it was shown that DNS query almost does not affect the 520 performance of the CGN. The max concurrent sessions of DNS packet 521 was only 40k. NAT table of DNS (udp/53,tcp/53) timeouts in 3 522 seconds, thus It saves the consumption of NAT tables. However NAT 523 log was generated for every query and it doubled the total amount of 524 the log. It would be rare that the NAT log of DNS is needed to react 525 to a legal request. The impact of the DNS query traffic is 526 relatively small if DNS timeout is adjusted. 528 6.4. Separation of traffic 530 In the existing network, IPv4 communication and IPv6 communication 531 may already be mixed in the dual stack. In this case, by introducing 532 CGN which can route IPv6 and existing IPv4 aside from NAT function, 533 the influence for the network architecture could be suppressed and so 534 a flexible design is possible. However, though current CGN is 535 scalable enough to be deployed in core of the service providers 536 network, the feature of routing is insufficient to replace the 537 exsisting routers. Such a CGN is desirable, otherwise the design 538 which makes IPv6 traffic and traditional IPv4 traffic bypass from CGN 539 is effective choice for providers. In dividing NAT flows and non-NAT 540 flows routers, VRF (Virtual Routing and Forwarding) and PBR (policy 541 based routing) are needed at routers in front of CGN. In that case 542 it is indispensable to configure routers so that the hairpining 543 communication between the NAT user and non-NAT user to be possible. 544 The considerations about the separation of traffic and effective 545 deployment configuration are discussed in detail in 546 [I-D.ietf-opsawg-lsn-deployment]. 548 7. Tested web sites and applications (Excerpts) 550 - Web Mail 551 gmail 552 yahoo mail 553 hot mail 554 - Video 555 ustream 556 youtube 557 nicovideos 558 Hulu 559 dailymotion 560 daum 561 qq 562 fc2 563 xvideos 564 - Portal&EC site 565 yahoo 566 rakuten 567 amazon 568 apple 569 - Blog 570 livedoor blog 571 ameba blog 572 - Search Engine 573 google 574 - Online Banking 575 mizuho bank 576 DC card 577 - Cloud Service 578 drop box 579 Evernote 580 - InstantMessenger & VoIP 581 skype 582 Line 583 - facebook 584 - twitter 585 - google map 586 - Online PC Game 587 aeria games 588 ameba pigg 589 nexon 590 hangame 591 - Consumer Game 592 Armored Core V (Play Station3) 593 Dark Souls 2 (Play Station3) 594 Gundam Extreme VS. (Play Station3) 595 Kinect adventure (XBox) 596 Persona 4 the ultimate in mayonaka arena (XBox) 597 Mingol 4 (WiiU) 598 Monster Hunter 3G (DS-lite) 599 Keri-hime sweets (iOS) 600 PuzzDra (iOS) 602 8. IANA Considerations 604 This document makes no request of IANA. 606 9. Security Considerations 608 TBD 610 10. Acknowledgments 612 This research and experiment are conducted under the great support of 613 Ministry of Internal Affairs and Communications of Japan. Many 614 thanks to MIC, JAIST members and Shin Miyakawa for their ideas and 615 feedback in documentation. 617 11. References 619 11.1. Normative References 621 [I-D.donley-behave-deterministic-cgn] 622 Donley, C., Grundemann, C., Sarawat, V., Sundaresan, K., 623 and O. Vautrin, "Deterministic Address Mapping to Reduce 624 Logging in Carrier Grade NAT Deployments", 625 draft-donley-behave-deterministic-cgn-06 (work in 626 progress), July 2013. 628 [I-D.ietf-opsawg-lsn-deployment] 629 Kuarsingh, V. and J. Cianfarani, "CGN Deployment with BGP/ 630 MPLS IP VPNs", draft-ietf-opsawg-lsn-deployment-03 (work 631 in progress), June 2013. 633 [RFC2663] Srisuresh, P. and M. Holdrege, "IP Network Address 634 Translator (NAT) Terminology and Considerations", 635 RFC 2663, August 1999. 637 [RFC2993] Hain, T., "Architectural Implications of NAT", RFC 2993, 638 November 2000. 640 [RFC6269] Ford, M., Boucadair, M., Durand, A., Levis, P., and P. 641 Roberts, "Issues with IP Address Sharing", RFC 6269, 642 June 2011. 644 [RFC6888] Perreault, S., Yamagata, I., Miyakawa, S., Nakagawa, A., 645 and H. Ashida, "Common Requirements for Carrier-Grade NATs 646 (CGNs)", BCP 127, RFC 6888, April 2013. 648 11.2. Informative References 650 [I-D.chen-sunset4-cgn-port-allocation] 651 Chen, G., "Analysis of NAT64 Port Allocation Method", 652 draft-chen-sunset4-cgn-port-allocation-02 (work in 653 progress), July 2013. 655 [I-D.ietf-softwire-map] 656 Troan, O., Dec, W., Li, X., Bao, C., Matsushima, S., 657 Murakami, T., and T. Taylor, "Mapping of Address and Port 658 with Encapsulation (MAP)", draft-ietf-softwire-map-08 659 (work in progress), August 2013. 661 [I-D.tsou-behave-natx4-log-reduction] 662 Tsou, T., Li, W., Taylor, T., and J. Huang, "Port 663 Management To Reduce Logging In Large-Scale NATs", 664 draft-tsou-behave-natx4-log-reduction-04 (work in 665 progress), July 2013. 667 [RFC6146] Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful 668 NAT64: Network Address and Protocol Translation from IPv6 669 Clients to IPv4 Servers", RFC 6146, April 2011. 671 [RFC6302] Durand, A., Gashinsky, I., Lee, D., and S. Sheppard, 672 "Logging Recommendations for Internet-Facing Servers", 673 BCP 162, RFC 6302, June 2011. 675 [RFC6333] Durand, A., Droms, R., Woodyatt, J., and Y. Lee, "Dual- 676 Stack Lite Broadband Deployments Following IPv4 677 Exhaustion", RFC 6333, August 2011. 679 [RFC6346] Bush, R., "The Address plus Port (A+P) Approach to the 680 IPv4 Address Shortage", RFC 6346, August 2011. 682 [RFC6877] Mawatari, M., Kawashima, M., and C. Byrne, "464XLAT: 683 Combination of Stateful and Stateless Translation", 684 RFC 6877, April 2013. 686 Authors' Addresses 688 Kaname Nishizuka 689 NTT Communications Corporation 690 Granpark Tower 691 3-4-1 Shibaura, Minato-ku 692 Tokyo 108-8118 693 Japan 695 Email: kaname@nttv6.jp 697 Daigo Natsume 698 NTT-Neomeit Corporation 699 3-15 Babacho, Chuo-ku, Osaka-shi 700 Osaka 540-8511 701 Japan 703 Email: daigo.natsume@ntt-neo.co.jp