idnits 2.17.1 draft-lencse-bmwg-benchmarking-stateful-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 2 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). -- The exact meaning of the all-uppercase expression 'MAY NOT' is not defined in RFC 2119. If it is intended as a requirements expression, it should be rewritten using one of the combinations defined in RFC 2119; otherwise it should not be all-uppercase. == The expression 'MAY NOT', while looking like RFC 2119 requirements text, is not defined in RFC 2119, and should not be used. Consider using 'MUST NOT' instead (if that is what you mean). Found 'MAY NOT' in this paragraph: Pseudorandom reading order of the state table MAY NOT be used with unidirectional traffic from the Responder to the Initiator, because if a four tuple is not used until timeout time, then its connection is deleted from the connection tracking table of the DUT and a later use of the given four tuple will cause frame loss. There is no such problem, when bidirectional traffic is used, because then the state table of the Responder is periodically refreshed. -- The document date (May 17, 2021) is 1068 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Duplicate reference: RFC4814, mentioned in 'LEN2020', was also mentioned in 'RFC4814'. -- Duplicate reference: RFC8219, mentioned in 'SIITPERF', was also mentioned in 'RFC8219'. Summary: 0 errors (**), 0 flaws (~~), 4 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Benchmarking Methodology Working Group G. Lencse 3 Internet-Draft Szechenyi Istvan University 4 Intended status: Informational K. Shima 5 Expires: November 18, 2021 IIJ Innovation Institute 6 May 17, 2021 8 Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814 9 Pseudorandom Port Numbers 10 draft-lencse-bmwg-benchmarking-stateful-00 12 Abstract 14 RFC 2544 has defined a benchmarking methodology for network 15 interconnect devices. RFC 5180 addressed IPv6 specificities and it 16 also provided a technology update, but excluded IPv6 transition 17 technologies. RFC 8219 addressed IPv6 transition technologies, 18 including stateful NAT64. However, none of them discussed how to 19 apply RFC 4814 pseudorandom port numbers to any stateful NAT (NAT44, 20 NAT64, NAT66) technologies. We discuss why using pseudorandom port 21 numbers with stateful NAT gateways is a hard problem and recommend a 22 solution. 24 Status of This Memo 26 This Internet-Draft is submitted in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF). Note that other groups may also distribute 31 working documents as Internet-Drafts. The list of current Internet- 32 Drafts is at https://datatracker.ietf.org/drafts/current/. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 This Internet-Draft will expire on November 18, 2021. 41 Copyright Notice 43 Copyright (c) 2021 IETF Trust and the persons identified as the 44 document authors. All rights reserved. 46 This document is subject to BCP 78 and the IETF Trust's Legal 47 Provisions Relating to IETF Documents 48 (https://trustee.ietf.org/license-info) in effect on the date of 49 publication of this document. Please review these documents 50 carefully, as they describe your rights and restrictions with respect 51 to this document. Code Components extracted from this document must 52 include Simplified BSD License text as described in Section 4.e of 53 the Trust Legal Provisions and are provided without warranty as 54 described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 60 2. Pseudorandom Port Numbers and Stateful Translation . . . . . 3 61 3. Test Setup and Terminology . . . . . . . . . . . . . . . . . 4 62 4. Recommended Benchmarking Method . . . . . . . . . . . . . . . 5 63 4.1. Restricted Port Number Ranges . . . . . . . . . . . . . . 5 64 4.2. Preliminary Test Phase . . . . . . . . . . . . . . . . . 6 65 4.3. Control of the Connection Tracking Table Entries . . . . 7 66 4.4. Measurement of the Maximum Connection Establishment Rate 8 67 4.5. Real Test Phase . . . . . . . . . . . . . . . . . . . . . 8 68 4.6. Writing and Reading Order of the State Table . . . . . . 9 69 4.7. Peculiarities of Stateful Testing . . . . . . . . . . . . 10 70 4.7.1. Timeout Budget . . . . . . . . . . . . . . . . . . . 10 71 4.7.2. Special Warning Against Non-zero Frame Loss Testing . 10 72 5. Implementation and Experience . . . . . . . . . . . . . . . . 10 73 6. Limitations of using UDP as Transport Layer Protocol . . . . 11 74 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11 75 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 76 9. Security Considerations . . . . . . . . . . . . . . . . . . . 11 77 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 78 10.1. Normative References . . . . . . . . . . . . . . . . . . 11 79 10.2. Informative References . . . . . . . . . . . . . . . . . 12 80 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 13 81 A.1. 00 . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 82 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 84 1. Introduction 86 [RFC2544] has defined a comprehensive benchmarking methodology for 87 network interconnect devices, which is still in use. It was mainly 88 IP version independent, but it used IPv4 in its examples. [RFC5180] 89 addressed IPv6 specificities and also added technology updates, but 90 declared IPv6 transition technologies out of its scope. [RFC8219] 91 addressed the IPv6 transition technologies, including stateful NAT64. 92 It has reused several benchmarking procedures from [RFC2544] (e.g. 93 throughput, frame loss rate), it has redefined the latency 94 measurement, and added further ones, e.g. the PDV (packet delay 95 variation) measurement. 97 However, none of them discussed, how to apply [RFC4814] pseudorandom 98 port numbers, when benchmarking stateful NATxy (NAT44, NAT64, NAT66) 99 gateways. We are not aware of any other RFCs that address this 100 question. 102 First, we discuss why using pseudorandom port numbers with stateful 103 NATxy gateways is a hard problem. 105 Then we recommend a solution. 107 1.1. Requirements Language 109 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 110 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 111 "OPTIONAL" in this document are to be interpreted as described in 112 BCP14 [RFC2119] [RFC8174] when, and only when, they appear in all 113 capitals, as shown here. 115 2. Pseudorandom Port Numbers and Stateful Translation 117 In its appendix, [RFC2544] has defined a frame format for test frames 118 including specific source and destination port numbers. [RFC4814] 119 recommends to use pseudorandom and uniformly distributed values for 120 both source and destination port numbers. However, stateful NATxy 121 (NAT44, NAT64, NAT66) solutions use the port numbers to identify 122 connections. The usage of pseudorandom port numbers causes different 123 problems depending on the direction. 125 o As for the private to public direction, pseudorandom source and 126 destination port numbers could be used, however, this approach 127 would be a denial of service attack against the stateful NATxy 128 gateway, because it would exhaust its connection tracking table. 129 To that end, let us see some calculations using the 130 recommendations of RFC 4814: 132 * The recommended source port range is: 1024-65535, thus its size 133 is: 64512. 135 * The recommended destination port range is: 1-49151, thus its 136 size is: 49151. 138 * The number of source and destination port number combinations 139 is: 3,170,829,312. 141 We note that section 12 of [RFC2544] also requires testing with 142 256 destination networks, which further increases the number of 143 connection tracking table entries. 145 o As for the public to private direction, the stateful DUT would 146 drop any packets that do not belong to an existing connection, 147 therefore, the direct usage of pseudorandom port numbers from the 148 above-mentioned ranges is not feasible. 150 3. Test Setup and Terminology 152 Our methodology works with any IP version. We use IPv4 in the Test 153 Setup shown in Figure 1 to facilitate its easy understanding based on 154 the well-known stateful NAT44 (also called NAPT: Network Address and 155 Port Translation) solution. 157 +--------------------------------------+ 158 10.0.0.2 |Initiator Responder| 198.19.0.2 159 +-------------| Tester |<------------+ 160 | private IPv4| [state table]| public IPv4 | 161 | +--------------------------------------+ | 162 | | 163 | +--------------------------------------+ | 164 | 10.0.0.1 | DUT: | 198.19.0.1 | 165 +------------>| Sateful NATxy gateway |-------------+ 166 private IPv4| [connection tracking table] | public IPv4 167 +--------------------------------------+ 169 Figure 1: Test Setup for benchmarking stateful NATxy gateways 171 As for transport layer protocol, [RFC2544] recommended testing with 172 UDP, and it was kept also in [RFC8219]. For the general 173 recommendation, we also keep UDP, thus the port numbers in the 174 following text are to be understood as UDP port numbers. We discuss 175 the limitation of this approach in Section 6. 177 We define the most important elements of our proposed benchmarking 178 system as follows. 180 o Connection tracking table: The stateful NATxy gateway uses a 181 connection tracking table to be able to perform the stateful 182 translation in the public to private direction. Its size, policy 183 and content is unknown for the Tester. 185 o Four tuple: The four numbers that identify a connection are source 186 IP address, source port number, destination IP address, 187 destination port number. 189 o Initiator: The port of the Tester that may initiate a connection 190 through the stateful DUT in the private to public direction. 191 Theoretically, it can use any source and destination port numbers: 193 if they do not belong to an existing connection, the DUT will 194 register a new connection into its connection tracking table. 196 o Responder: The port of the Tester that may not initiate a 197 connection through the stateful DUT in the public to private 198 direction. It may send only frames that belong to an existing 199 connection. To that end, it uses four tuples that have been 200 previously extracted from the received test frames and stored in 201 its state table. 203 o State table: The Responder of the Tester extracts the four tuple 204 from each received test frame and stores it in its state table. 205 Recommendation is given for writing and reading order of the state 206 table in Section 4.6. 208 o Preliminary test phase: Test frames are sent only by the Initiator 209 to the Responder through the DUT to fill both the connection 210 tracking table of the DUT and the state table of the Responder. 211 This is a newly introduced operation phase for stateful NATxy 212 benchmarking. The necessity of this phase is explained in 213 Section 4.2. 215 o Real test phase: The actual test (e.g. throughput, latency, etc.) 216 is performed in this phase after the completion of the preliminary 217 test phase. Test frames are sent as required (e.g. bidirectional 218 test or unidirectional test in any of the two directions). 220 4. Recommended Benchmarking Method 222 4.1. Restricted Port Number Ranges 224 The Initiator SHOULD use restricted ranges for source and destination 225 port numbers to avoid the denial of service attack against the 226 connection tracking table of the DUT described in Section 2. The 227 size of the source port number range SHOULD be larger (e.g. in the 228 order of a few times ten thousand), whereas the size of the 229 destination port number range SHOULD be smaller (may vary from a few 230 to several hundreds or thousands as needed). The rationale is that 231 source and destination port numbers that can be observed in the 232 Internet traffic are not symmetrical. Whereas source port numbers 233 may be random, there are a few very popular destination port numbers 234 (e.g. 443, 80, etc., see [IIR2020]) and others hardly occur. And we 235 have found that their role is also asymmetric in the Linux kernel 236 routing hash function [LEN2020]. 238 The product of the sizes of the two ranges can be used as a 239 parameter. The performance of the stateful NATxy gateway MAY be 240 examined as function of this parameter. 242 4.2. Preliminary Test Phase 244 The preliminary phase serves two purposes: 246 1. The connection tracking table of the DUT is filled. It is 247 important, because its maximum connection establishment rate may 248 be lower than its maximum frame forwarding rate (that is 249 throughput). 251 2. The state table of the Responder is filled with valid four 252 tuples. It is a precondition for the Responder to be able to 253 transmit frames that belong to connections exist in the 254 connection tracking table of the DUT. 256 Whereas the above two things are always necessary before the real 257 test phase, the preliminary phase can be used without the real test 258 phase. It is done so, when the maximum connection establishment rate 259 is measured (as described in Section 4.4). 261 A preliminary test phase MUST be performed before all tests performed 262 in the real test phase. In this phase, the following things happen: 264 1. The Initiator sends test frames to the Responder through the DUT 265 at a specific frame rate. 267 2. The DUT performs the stateful translation of the test frames and 268 it also stores the new combinations in its connection tracking 269 table. 271 3. The Responder receives the translated test frames and updates its 272 state table with the received four tuples. The responder 273 transmits no test frames during the preliminary phase. 275 When the preliminary test phase is performed in preparation to the 276 real test phase, the applied frame rate and the duration of the 277 preliminary phase SHOULD be carefully selected so that: 279 o The applied frame rate be safely lower than the maximum connection 280 establishment rate. 282 o The initial transient of the filling of the connection tracking 283 table of the DUT be finished. 285 o Enough four tuples be stored in the state table of the Responder 286 so that it can generate frames with the proper distribution of the 287 four tuples. 289 o The connections do not time out in the DUT even during the 290 beginning of the real test phase. 292 4.3. Control of the Connection Tracking Table Entries 294 The number of the entries in the connection tracking table of the DUT 295 MAY be controlled by using all different source port number 296 destination port number combinations. 298 Let NF and NC denote the number of test frames to send and the number 299 of all possible source port number destination port number 300 combinations, respectively. 302 1. If NF and NC are in the same order of magnitude, then the all 303 different source port number destination port number combinations 304 may be computing efficiently generated by preparing a random 305 permutation of the previously enumerated all possible source port 306 number destination port number combinations using Dustenfeld's 307 random shuffle algorithm [DUST1964]. 309 2. If NF is at least an order of magnitude less than NC, then a 310 simpler solution may be used: the Initiator registers in a table 311 and then it checks if a given source port number destination port 312 number combination was already used (and if yes, then it MUST 313 generate a new one). 315 3. If NF is at least two orders of magnitude less than NC, then 316 mostly different source port number destination port number 317 combinations can be generated without any specific provision. 319 Important warning: in normal router testing, the port number 320 selection algorithm (whether it is pseudo-random or enumerated) does 321 not affect final results. However, our experience with iptables 322 shows that if the connection tracking table is filled using port 323 number enumeration in increasing order, then the maximum connection 324 establishment rate of iptables degrades significantly compared to its 325 performance using pseudorandom port numbers [LEN2021]. 327 [RFC4814] REQUIRES pseudorandom port numbers, which we believe is a 328 good approximation of the distribution of the source port numbers a 329 NATxy gateway on the Internet may face with. 331 The enumeration of the source port number destination port number 332 combinations in increasing order MAY be used as an additional 333 measurement to discover worst case performance. 335 4.4. Measurement of the Maximum Connection Establishment Rate 337 The maximum connection establishment rate is an important 338 characteristic of the stateful NATxy gateway and its determination is 339 necessary for the safe execution of the preliminary test phase 340 (without frame loss) before the real test phase. 342 The measurement procedure of the maximum connection establishment 343 rate is very similar to the throughput measurement procedure defined 344 in [RFC2544]. 346 Procedure: The Initiator sends a specific number of test frames using 347 all (or mostly) different source port number destination port number 348 combinations at a specific rate through the DUT. The Responder 349 counts the frames that are successfully translated by the DUT. If 350 the count of offered frames is equal to the count of received frames, 351 the rate of the offered stream is raised and the test is rerun. If 352 fewer frames are received than were transmitted, the rate of the 353 offered stream is reduced and the test is rerun. 355 The maximum connection establishment rate is the fastest rate at 356 which the count of test frames successfully translated by the DUT is 357 equal to the number of test frames sent to it by the Initiator. 359 Notes: 361 1. All different source port number destination port number 362 combinations SHOULD be used, if the results of the measurement 363 are published. Mostly different source port number destination 364 port number combinations MAY be used, if the results of the 365 measurement are not published, but they are used only to 366 determine a good enough frame rate for the preliminary test phase 367 preparing the test system for the real test phase. 369 2. In practice, we RECOMMEND the usage of binary search. 371 3. As for the successful translation, the Responder MAY (or SHOULD?) 372 check that the source IP address is different than the original 373 source IP address set by the Initiator. 375 4.5. Real Test Phase 377 As for the traffic direction, there are three possible cases during 378 the real test phase: 380 o bidirectional traffic: The Initiator sends test frames to the 381 Responder and the Responder sends test frames to the Initiator. 383 o unidirectional traffic from the Initiator to the Responder: The 384 Initiator sends test frames to the Responder but the Responder 385 does not send test frames to the Initiator. 387 o unidirectional traffic from the Responder to the Initiator: The 388 Responder sends test frames to the Initiator but the Initiator 389 does not send test frames to the Responder. 391 If the Initiator sends test frames, then it uses pseudorandom source 392 port numbers and destination port numbers from the restricted port 393 number ranges. The responder receives the test frames, updates its 394 state table and processes the test frames as required by the given 395 measurement procedure (e.g. only counts them for throughput test, 396 handles timestamps for latency or PDV tests, etc.). 398 If the Responder sends test frames, then it uses the four tuples from 399 its state table. The reading order of the state table may follow 400 different policies (discussed in Section 4.6). The Initiator 401 receives the test frames, and processes them as required by the given 402 measurement procedure. 404 As for the actual measurement procedures, we RECOMMEND to use the 405 updated ones from Section 7 of [RFC8219]. 407 4.6. Writing and Reading Order of the State Table 409 As for writing policy of the state table of the Responder, we 410 RECOMMEND round robin, because it ensures that its entries are 411 automatically kept fresh and thus there is no need to handle timeout. 413 The Responder can read its state table in various orders. We 414 RECOMMEND one of the following ones: 416 o round robin 418 o pseudorandom (with restriction!) 420 o random permutation (no position is repeated until all positions 421 are used). 423 Pseudorandom reading order of the state table MAY NOT be used with 424 unidirectional traffic from the Responder to the Initiator, because 425 if a four tuple is not used until timeout time, then its connection 426 is deleted from the connection tracking table of the DUT and a later 427 use of the given four tuple will cause frame loss. There is no such 428 problem, when bidirectional traffic is used, because then the state 429 table of the Responder is periodically refreshed. 431 We do not see any problem in the round robin reading order, because 432 the state table is filled using pseudorandom port numbers. 434 4.7. Peculiarities of Stateful Testing 436 Stateful testing involves some issues not present in stateless 437 testing. 439 4.7.1. Timeout Budget 441 Even though we do black box testing, one MUST consider timeout and 442 carefully manage timeout budget. For example, if the frame rate is 443 high enough, then every single entry of the state table of the 444 Responder is refreshed within timeout time and it prevents frame 445 sending with a stale four tuple. If the entries of the state table 446 are not refreshed (due to testing with single directional traffic 447 from the Responder to the Initiator) then using all four tuples 448 within timeout time can keep all connection tracking table entries of 449 the DUT alive. 451 Special care should be taken for the lower frame rate in the 452 preliminary phase. 454 If the binary search (or the decreasing of the applied frame rates 455 during the frame loss rate test) results in a frame rate that is too 456 low to keep alive the connection tracking table entries, then it 457 results in the failure of the consecutive tests (the binary search of 458 the throughput test counts down to zero). 460 4.7.2. Special Warning Against Non-zero Frame Loss Testing 462 Several network performance tester vendors include a parameter called 463 "Loss Tolerance" (or similar) for the throughput test and several 464 benchmarking professionals actually use nonzero values [TOL2001]. If 465 frames are lost during stateful testing (especially if it happens 466 during a test with unidirectional traffic from the Responder to the 467 Initiator) the refreshing of the corresponding connection tracking 468 table element is not ensured and it may result in the loss of further 469 frames (not due to the low performance of the DUT, but due to using a 470 stale four tuple). 472 5. Implementation and Experience 474 The "stateful" branch of siitperf [SIITPERF] is a partial 475 implementation of this concept. 477 Our experience is documented in a paper currently under review 478 [LEN2021]. 480 6. Limitations of using UDP as Transport Layer Protocol 482 Stateful NATxy solutions handle TCP and UDP differently, e.g. 483 iptables uses 30s timeout for UDP and 60s timeout for TCP. Thus 484 benchmarking results produced using UDP do not necessarily 485 characterize the performance of a NATxy gateway well enough, when 486 they are used for forwarding Internet traffic. As for the given 487 example, timeout values of the DUT may be adjusted, but it requires 488 extra consideration. 490 Other differences in handling UDP or TCP are also possible. Thus we 491 recommend that further investigations are to be performed in this 492 field. 494 7. Acknowledgements 496 The authors would like to thank ... (TBD) 498 8. IANA Considerations 500 This document does not make any request to IANA. 502 9. Security Considerations 504 We have no further security considerations beyond that of [RFC8219]. 505 Perhaps they should be cited here so that they be applied not only 506 for the benchmarking of IPv6 transition technologies, but also for 507 the benchmarking of stateful NATxy. 509 10. References 511 10.1. Normative References 513 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 514 Requirement Levels", BCP 14, RFC 2119, 515 DOI 10.17487/RFC2119, March 1997, 516 . 518 [RFC2544] Bradner, S. and J. McQuaid, "Benchmarking Methodology for 519 Network Interconnect Devices", RFC 2544, 520 DOI 10.17487/RFC2544, March 1999, 521 . 523 [RFC4814] Newman, D. and T. Player, "Hash and Stuffing: Overlooked 524 Factors in Network Device Benchmarking", RFC 4814, 525 DOI 10.17487/RFC4814, March 2007, 526 . 528 [RFC5180] Popoviciu, C., Hamza, A., Van de Velde, G., and D. 529 Dugatkin, "IPv6 Benchmarking Methodology for Network 530 Interconnect Devices", RFC 5180, DOI 10.17487/RFC5180, May 531 2008, . 533 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 534 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 535 May 2017, . 537 [RFC8219] Georgescu, M., Pislaru, L., and G. Lencse, "Benchmarking 538 Methodology for IPv6 Transition Technologies", RFC 8219, 539 DOI 10.17487/RFC8219, August 2017, 540 . 542 10.2. Informative References 544 [DUST1964] 545 Durstenfeld, R., "Algorithm 235: Random 546 permutation", Communications of the ACM, vol. 7, no. 7, 547 p.420., DOI 10.1145/364520.364540, July 1964, 548 . 550 [IIR2020] Kurahashi, T., Matsuzaki, Y., Sasaki, T., Saito, T., and 551 F. Tsutsuji, "Periodic observation report: Internet trends 552 as seen from IIJ infrastructure - 2020", Internet 553 Infrastructure Review, vol. 49, Dec 2020, 554 . 557 [LEN2020] Lencse, G., "Adding RFC 4814 Random Port Feature to 558 Siitperf: Design, Implementation and Performance 559 Estimation", International Journal of Advances in 560 Telecommunications, Electrotechnics, Signals and Systems, 561 vol 9, no 3, pp. 18-26., DOI 10.11601/ijates.v9i3.291, 562 2020, . 565 [LEN2021] Lencse, G., "Design and Implementation of a Software 566 Tester for Benchmarking Stateful NAT64 Gateways: Theory 567 and Practice of Extending Siitperf for Stateful 568 Tests", under review in Computer Commuications, may be 569 revised or removed without notice, 2021, 570 . 573 [SIITPERF] 574 Lencse, G. and Y. Kadobayashi, "Siitperf: An RFC 8219 575 compliant SIIT (stateless NAT64) tester written in C++ 576 using DPDK", source code, available from GitHub, 2019, 577 . 579 [TOL2001] Tolly, K., "The real meaning of zero-loss testing", IT 580 World Canada, 2001, 581 . 584 Appendix A. Change Log 586 A.1. 00 588 Initial version. 590 Authors' Addresses 592 Gabor Lencse 593 Szechenyi Istvan University 594 Egyetem ter 1. 595 Gyor H-9026 596 Hungary 598 Email: lencse@sze.hu 600 Keiichi Shima 601 IIJ Innovation Institute 602 Iidabashi Grand Bloom, 2-10-2 Fujimi 603 Chiyoda-ku, Tokyo 102-0071 604 Japan 606 Email: keiichi@iijlab.net