idnits 2.17.1 draft-fan-opsawg-packet-loss-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 05, 2013) is 3940 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'RFC4443' is defined on line 638, but no explicit reference was found in the text == Unused Reference: 'RFC792' is defined on line 651, but no explicit reference was found in the text == Unused Reference: 'I-D.chen-coloring-based-ipfpm-framework' is defined on line 656, but no explicit reference was found in the text ** Obsolete normative reference: RFC 2680 (Obsoleted by RFC 7680) == Outdated reference: A later version (-04) exists of draft-tempia-opsawg-p3m-03 Summary: 1 error (**), 0 flaws (~~), 5 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Fan 3 Internet-Draft L. Huang 4 Intended status: Informational China Mobile 5 Expires: January 06, 2014 M. Chen 6 Huawei Technologies 7 N. Kumar 8 Cisco Systems 9 July 05, 2013 11 IP Packet Loss Rate Measurement Testing and Problem Statement 12 draft-fan-opsawg-packet-loss-01 14 Abstract 16 This document describes common methods for measuring packet loss rate 17 and their effectiveness. Issues encountered when using the methods 18 and necessary considerations are also discussed and recommended. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at http://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on January 06, 2014. 37 Copyright Notice 39 Copyright (c) 2013 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (http://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 55 2. Methods for Packet Loss Rate Measurement . . . . . . . . . . 3 56 2.1. Active Approach . . . . . . . . . . . . . . . . . . . . . 3 57 2.1.1. Ping . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1.2. OWAMP and TWAMP . . . . . . . . . . . . . . . . . . . 3 59 2.1.3. Proprietary Tools . . . . . . . . . . . . . . . . . . 4 60 2.2. Passive Approach . . . . . . . . . . . . . . . . . . . . 4 61 2.2.1. Interface Statistics Report . . . . . . . . . . . . . 4 62 2.2.2. Coloring Based Performance Measurement . . . . . . . 5 63 3. Test on Packet Loss Rate Measurement . . . . . . . . . . . . 5 64 3.1. Basic Test Information . . . . . . . . . . . . . . . . . 5 65 3.2. Ping with CLI vs. SNMP . . . . . . . . . . . . . . . . . 6 66 3.3. Ping Behaviors of Routers . . . . . . . . . . . . . . . . 6 67 3.4. Statistics Report of Routers . . . . . . . . . . . . . . 10 68 4. Measurement Issues . . . . . . . . . . . . . . . . . . . . . 10 69 4.1. Issues with Ping . . . . . . . . . . . . . . . . . . . . 10 70 4.2. Issues with OWAMP and TWAMP . . . . . . . . . . . . . . . 11 71 4.3. Issues with Proprietary Tools . . . . . . . . . . . . . . 11 72 4.4. Issues with Interface Statistics Report . . . . . . . . . 12 73 4.5. Issues with Coloring Based Performance Measurement . . . 12 74 5. Considerations and Recommendations . . . . . . . . . . . . . 12 75 6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 76 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 77 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 78 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 79 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 80 9.2. Informative References . . . . . . . . . . . . . . . . . 14 81 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 83 1. Introduction 85 IP packet loss rate is one of the important metrics that are 86 frequently used to measure IP performance of a data path or link. A 87 general framework of IP performance metrics is provided in [RFC2330], 88 including fundamental concepts definition and issues related to 89 defining sound metrics and methodologies. [RFC2680] and [RFC6673] 90 further define metrics for one-way and round-trip packet loss. 92 In practical network operation, a number of methods are used by 93 network engineers to calculate packet loss rate, and one of the 94 common ways is to use ping. By checking ping statistics, people 95 expect to get the idea of traffic transmission condition on the link. 96 This document gives an overview of the frequently used methods for 97 measuring IP packet loss rate, and describes a test on packet loss 98 rate measurement with multiple methods using routers from different 99 vendors. Issues that should be taken into consideration during the 100 measurement using different methods are discussed. Causes analysis 101 and processing mechanisms of routers are also covered. It is 102 expected that an operable measurement scheme with consistent testing 103 results and equal treatment of network components can be reached. 105 2. Methods for Packet Loss Rate Measurement 107 This section describes common methods for measuring packet loss rate. 109 2.1. Active Approach 111 2.1.1. Ping 113 Ping (ICMP echo request/reply) is a useful tool to examine the 114 connectivity and performance of a path between two nodes in the 115 network. The source node generates echo request packets with 116 configured size, interval, count and other settings, and the 117 destination node sends back an echo reply packet once it receives a 118 request. Then we count the packets sent out and received and get the 119 round-trip packet loss rate on the link between source and 120 destination. This approach is clear and convenient, and is 121 frequently used by engineers when packet loss rate is needed. 123 In practical network operation, the ping testing can be initiated 124 manually and directly on the node by engineers, for example through 125 the command line interface (CLI) of a router, or activated indirectly 126 by instructions, for example through SNMP messages sent from network 127 management system. 129 No matter through CLI or SNMP, ping testing can be conducted directly 130 on the endpoint devices of the link to be tested, or other nodes as 131 long as the request/reply packets pass through the link. Those nodes 132 are often referred to as probes, which can be a router or a PC 133 server, directly connected or indirectly reachable to the endpoints. 134 Usually the probes and paths to the endpoints are not supposed to be 135 congested to avoid affecting the ping testing result. 137 2.1.2. OWAMP and TWAMP 139 The One-way Active Measurement Protocol (OWAMP, [RFC4656]) and Two- 140 Way Active Measurement Protocol (TWAMP, [RFC5357]) are defined by the 141 IP Performance Metrics (IPPM) working group. They provide a method 142 and protocol for measuring delay and packet loss of IP flows, and are 143 designed for wide scale deployment in the network to provide 144 ubiquitous performance data. Both OWAMP and TWAMP use control 145 protocol and test protocol. The control protocol is used to 146 negotiate test session between test endpoints, start and stop the 147 test, and fetch the test result for OWAMP. The test protocol runs 148 over UDP and conducts the test. 150 OWAMP can be used to perform one-way packet loss measurement, and 151 requires synchronized time defined by GPS. The test results are 152 collected at the receiving endpoints and returned using the control 153 protocol. TWAMP is more simplified, and used for two-way packet loss 154 measurement. The opposite endpoint is regarded as a reflector, and 155 the test results are collected at the sender. 157 2.1.3. Proprietary Tools 159 There are some other proprietary performance measurement tools 160 incorporating embedded and external probes. The probes generate and 161 inject extra packets into the network to mimic the service flows that 162 are intended to be tested. The performance of the target service 163 flows can be evaluated by measuring the performance of the injected 164 packets. Compared with Ping, these proprietary tools normally 165 support more services, which include not only ICMP, but TCP, UDP, 166 HTTP, etc. 168 The embedded proprietary tools have been widely implemented by 169 routers to provide automatic detection of IP performance. Examples 170 of this kind of tools include RPM (Juniper), IPSLA (Cisco), NQA 171 (Huawei/H3C), SAA (ALU), etc. By necessary configurations on the 172 router, the embedded tools support multi-service testing of multiple 173 queues on an interface. Packet loss rate can be measured with ICMP 174 ping function of the tool. Routers send out ICMP packets 175 automatically according to the configured parameters, so the embedded 176 tool is working in a similar way as ping method described above. 178 2.2. Passive Approach 180 2.2.1. Interface Statistics Report 182 Forwarding devices maintain statistics report of every interface. 183 The report shows the detailed status of the interface as well as 184 traffic information, including inbound and outbound speed and packet 185 count. For a typical router, traffic statistics show number of 186 packets transmitted and discarded by an interface, and even on the 187 basis of QoS queue, so the entire packet loss rate of a link or 188 packet loss rates regarding different queues can be calculated. 189 Traffic data on the report can be displayed through CLI or obtained 190 using SNMP which allows automatic packet loss sampling. 192 2.2.2. Coloring Based Performance Measurement 194 The concept of coloring based performance measurement is introduced 195 in [I-D.tempia-opsawg-p3m], and [I-D.chen-coloring-based-ipfpm- 196 framework] defines a framework for coloring based IP Flow Performance 197 Measurement (IPFPM). By periodically setting/changing one or more 198 bits of the IP header of the packets that belong to an IP flow to 199 "color" the packets into different colors, the IP flow is split into 200 different consecutive blocks. Packets in the same block have the 201 same color and packets in consecutive blocks have different colors. 202 This method gives a way to a measurement node to count and calculate, 203 without inserting any extra auxiliary OAM packets, packet loss based 204 on each color block. Since the measurement is based on the real 205 traffic data, the measurement results will reflect the real 206 performance of the tested flow. 208 3. Test on Packet Loss Rate Measurement 210 This section describes test result on packet loss rate measurement 211 using different methods. Test equipment covers routers from several 212 vendors. Results show the diverse outcome of the methods used, and 213 the diverse responding mechanism of routers. 215 3.1. Basic Test Information 217 The basic topology of testing can be depicted as follows. 219 +--------+ +---------+ +---------+ +--------+ 220 | Probe1 |------| Router1 |-----------| Router2 |------| Probe2 | 221 +--------+ GE +---------+ 10G POS +---------+ GE +--------+ 222 | | | | 223 10GE | | 10GE 10GE | | 10GE 224 | | | | 225 Port1 | | Port2 Port3 | | Port4 226 +---------------------------+ 227 | Tester | 228 +---------------------------+ 230 Figure 1: Basic topology for packet loss rate test 232 Two routers are connected by a 10G POS link, and each router is 233 connected to the tester by two 10GE links. The tester generates 234 unidirectional/bidirectional traffic between port 1 and port 3, and 235 between port 2 and port 4, with frame length of 400 bytes. The total 236 volume of traffic injected into a router by the tester is more than 237 10G, leading to congestion when the traffic passes through the 10G 238 POS link between the two routers. Routers and probes generate ping 239 packets for testing, with frame length of 400 and DSCP field of 0. 241 We tested routers from 3 vendors, indicated as A, B, and C in the 242 following parts of discussion. The tester generated different levels 243 of congestion, and we tested packet loss rates on the 10G POS 244 interconnection link on those congestion levels with CLI, SNMP, and 245 interface statistics report. 247 3.2. Ping with CLI vs. SNMP 249 Some routing boxes by default treat ping packets generated with CLI 250 and SNMP in different ways. The following is a test on this issue. 252 to tester +---------+ +---------+ to tester 253 ---10G----| Router1 |-----------| Router2 |----10G--- 254 ---10G----| | 10G | |----10G--- 255 +---------+ +---------+ 257 ping with CLI ----------> 258 ping with SNMP----------> 259 test traffic 260 ----------------------------------------------------> 262 The tester generates test traffic at 20 Gbps, and sends the traffic 263 into a router of vendor A. The traffic goes through the 10G 264 interconnecting link and past the router of vendor B on the other 265 end. We use ping with CLI and SNMP on router A to test packet loss 266 rate on the interconnecting link. The DSCP fields of test traffic 267 and ping packets are all left to be 0.. 269 By default, router A forwards the test traffic with the basic 270 priority, like BE class. The ping packets with CLI are also treated 271 as of best effort class, but ping packets with SNMP are given a 272 higher priority, some class like network control. So the two kinds 273 of ping are actually testing packet loss of streams in different 274 classes. The test result verifies the issue. Ping with SNMP shows 275 no packet loss, and ping with CLI shows a packet loss rate of around 276 50%. 278 The forwarding class of ICMP packets can be configured on router A. 279 In the following tests we put all traffic in the same basic class. 281 3.3. Ping Behaviors of Routers 283 We considered the following test cases (TCs) when investigating 284 packet loss rate with ping on the link between two different routers. 286 TC 1: Router sends ICMP echo request packets with SNMP instruction 287 to the peering router. 289 +---------+ +---------+ 290 | Router1 |-----------| Router2 | 291 +---------+ +---------+ 293 ping with SNMP----------> 295 TC 2: Router sends ICMP echo request packets with CLI to the peering 296 router. 298 +---------+ +---------+ 299 | Router1 |-----------| Router2 | 300 +---------+ +---------+ 302 ping with CLI ----------> 304 TC 3: Router sends ICMP echo request packets with SNMP instruction 305 to the probe behind the peering router. 307 +---------+ +---------+ +--------+ 308 | Router1 |-----------| Router2 |------| Probe2 | 309 +---------+ +---------+ +--------+ 311 ping with SNMP---------------------------> 313 TC 4: Router sends ICMP echo request packets with CLI to the probe 314 behind the peering router. 316 +---------+ +---------+ +--------+ 317 | Router1 |-----------| Router2 |------| Probe2 | 318 +---------+ +---------+ +--------+ 320 ping with CLI ---------------------------> 322 TC 5: Probe behind router sends ICMP echo request packets to the 323 probe behind the peering router. 325 +--------+ +---------+ +---------+ +--------+ 326 | Probe1 |------| Router1 |-----------| Router2 |------| Probe2 | 327 +--------+ +---------+ +---------+ +--------+ 329 ping with CLI--------------------------------------------> 330 The link between the two routers is injected bidirectional or 331 unidirectional test traffic to cause congestion. The packet loss 332 rate of test traffic is calculated with the Rx and Tx rate on the 333 tester. We use router A, B and C in pairs and get the ICMP packet 334 loss rate in each test case. The comparison of the packet loss rate 335 of ICMP and test traffic shows diverse behaviors of ping process on 336 routers. The following tables show the test results 338 +------------------------------------------------------------------+ 339 | Pkt loss rate of | ICMP pkt loss rate | ICMP pkt loss rate | 340 | test traffic |(echo req drct: A->B)|(echo req drct: B->A)| 341 |----------------------|---------------------|---------------------| 342 | A->B B->A | TC1 TC2 TC3 TC4 TC5 | TC1 TC2 TC3 TC4 TC5 | 343 |----------------------|---------------------|---------------------| 344 | 48.60% 48.60% | 54% 56% 80% 76% 73% | 54% 54% 58% 58% 77% | 345 | 28% 28% | 27% 30% 61% 58% 47% | 32% 32% 27% 21% 53% | 346 | 7.60% 7.60% | 9% 12% 15% 18% 21% | 13% 15% 11% 11% 21% | 347 | 48.60% No traffic | 54% 56% 57% 54% 54% | 62% 56% 54% 48% 56% | 348 | 28% No traffic | 31% 33% 32% 33% 33% | 36% 34% 34% 35% 35% | 349 | 7.60% No traffic | 14% 13% 12% 9% 14% | 14% 13% 11% 12% 14% | 350 |No traffic 48.60% | 1% 0% 54% 50% 47% | 1% 1% 0% 1% 50% | 351 |No traffic 28% | 0% 0% 26% 31% 28% | 0% 0% 0% 0% 28% | 352 |No traffic 7.60% | 0% 0% 10% 9% 9% | 0% 0% 0% 0% 8% | 353 +------------------------------------------------------------------+ 355 Table 1: Test result when interconnecting router A and router B 357 +------------------------------------------------------------------+ 358 | Pkt loss rate of | ICMP pkt loss rate | ICMP pkt loss rate | 359 | test traffic |(echo req drct: A->B)|(echo req drct: C->A)| 360 |----------------------|---------------------|---------------------| 361 | A->C C->A | TC1 TC2 TC3 TC4 TC5 | TC1 TC2 TC3 TC4 TC5 | 362 |----------------------|---------------------|---------------------| 363 | 48.70% 44.70% | 58% 54% 57% 58% 53% | 57% 55% 48% 57% 56% | 364 | 28% 22.40% | 38% 31% 37% 33% 35% | 30% 33% 33% 37% 35% | 365 | 7.70% 7.30% | 14% 13% 13% 13% 12% | 16% 13% 15% 16% 14% | 366 | 48.80% No traffic | 50% 54% 51% 53% 55% | 54% 56% 55% 59% 57% | 367 | 28% No traffic | 27% 29% 32% 32% 33% | 35% 30% 35% 33% 33% | 368 | 7.60% No traffic | 11% 10% 15% 15% 13% | 11% 11% 15% 15% 13% | 369 |No traffic 44.50% | 0% 0% 0% 0% 0% | 0% 0% 0% 0% 0% | 370 |No traffic 22.60% | 0% 0% 0% 0% 0% | 0% 0% 0% 0% 0% | 371 |No traffic 7.74% | 0% 0% 0% 0% 0% | 0% 0% 0% 0% 0% | 372 +------------------------------------------------------------------+ 374 Table 2: Test result when interconnecting router A and router C 376 +------------------------------------------------------------------+ 377 | Pkt loss rate of | ICMP pkt loss rate | ICMP pkt loss rate | 378 | test traffic |(echo req drct: C->B)|(echo req drct: B->C)| 379 |----------------------|---------------------|---------------------| 380 | C->B B->C | TC1 TC2 TC5 | TC1 TC2 TC5 | 381 |----------------------|---------------------|---------------------| 382 | 48.76% 44.69% | 1% 0% 54% | 0% 1% 50%| 383 | 28.04% 22.29% | 0% 0% 40% | 0% 1% 29%| 384 | 7.62% 7.62% | 0% 0% 11% | 0% 0% 8%| 385 | 48.69% No traffic | 0% 0% 0% | 0% 0% 0%| 386 | 28.03% No traffic | 0% 0% 0% | 0% 0% 0%| 387 | 7.62% No traffic | 0% 0% 0% | 0% 0% 0%| 388 |No traffic 44.50% | 1% 0% 51% | 0% 1% 51%| 389 |No traffic 22.29% | 0% 0% 29% | 0% 0% 29%| 390 |No traffic 7.74% | 0% 0% 9% | 0% 0% 10%| 391 +------------------------------------------------------------------+ 393 Table 3: Test result when interconnecting router C and router B 395 The behaviors of the three vendors' routers are summarized here, and 396 we leave the discussion on reasons for the behaviors to the next 397 section. 399 Router A: Ping by router A with SNMP, CLI and by the probe behind 400 router A lead to similar usable results. However, all the methods 401 encounter larger errors when the test traffic is less congested. 403 Router B: Ping by router B with SNMP and CLI will not report 404 correctly the packet loss rate of test traffic. Ping by the probe 405 behind router B gives usable result of packet loss rate, but also 406 with certain errors. 408 Router C: Ping by router C with SNMP, CLI and by the probe behind 409 router C will not report correctly the packet loss rate of test 410 traffic. 412 We can further highlight the outcomes when testing the packet loss 413 rate on the interconnection link between each pair of routers. 415 Router A - router B: If one wants to get relatively accurate value 416 of packet loss rate in all congestion scenarios, he is advised to 417 use ping between probes (test case 5), or have A generate ping to 418 the probe behind B. 420 Router A - router C: All the test methods will only reflect the 421 outbound packet loss rate of A. 423 Router B - router C: Packet loss rate is difficult to measure with 424 this combination- only using ping between probes (test case 5) can 425 reflect the outbound packet loss rate of B. 427 3.4. Statistics Report of Routers 429 We also checked the interface statistics reports given with CLI on 430 the 3 routers, and we confirmed that the outbound packet loss rate of 431 an interface obtained from the statistics report was in accordance 432 with the actual packet loss rate of test traffic. The following 433 table shows the test result. 435 +----------------------------------------------------------------+ 436 | Router | Outbound pkt loss rate of | Outbound pkt loss rate | 437 | | test traffic |shown on statistics report | 438 |--------|---------------------------|---------------------------| 439 | A | 48.52% | 48.52% | 440 | B | 48.52% | 48.52% | 441 | C | 44.60% | 44.60% | 442 +----------------------------------------------------------------+ 444 Table 4: Test result when referring to the statistics report on 445 routers 447 4. Measurement Issues 449 This section describes issues encountered when measuring the packet 450 loss rate of a link using different testing methods. 452 4.1. Issues with Ping 454 Routers from every vendor have their unique processing procedure when 455 sending and receiving ICMP packets, thus resulting in diverse ping 456 packet loss rates, as described in the section above. Errors exist 457 using the ping method, and in some cases ping no longer reflects the 458 actual packet loss rate correctly. Relevant issues that have to be 459 taken into account include: 461 Forwarding class: When sending ping packets locally, routers are 462 likely to put the packets into a certain QoS queue/class although 463 the DSCP field of ICMP packets is kept zero. QoS queue of ping 464 may be different than that of the traffic to be measured, and even 465 ping packets sent by CLI commands and SNMP are in different queues 466 by default. Usually forwarding class can be adjusted by CLI or 467 SNMP commands. 469 Inner priority: For some routers, although ping traffic and service 470 traffic will not be treated differently by QoS, packets sent out 471 by the router itself, for example ping packets, are put into an 472 inner high priority while other forwarding service traffic into 473 low priority. These kinds of inner priority are valid within the 474 interior of routers and do not rewrite the packets. One of the 475 purposes of using the priorities is to get the protocol packets 476 (ping included) processed in prior. These priorities are set by 477 vendor and may not be able to adjust, so in this case ping will 478 not give the correct packet loss rate as ping packets are not 479 processed and discarded together with service traffic. 481 Ingress line card: If the ping testing is conducted on a probe which 482 is connected or IP reachable to the router, then the ping packets 483 will be treated by the router as forwarding traffic, eliminating 484 the queue and priority issues. However, the location of 485 interfaces through which ingress traffic is received matters when 486 using some types of routers. In this case, the router employs a 487 polling schedule which allows traffic from different line cards or 488 modules to get forwarding chance. For a card with small volume of 489 traffic, the chance will be little but not none. So if ping 490 packets come through a card different from the high-volume service 491 traffic, the packets would probably get enough forwarding 492 resources as ping traffic itself requires little bandwidth. As a 493 result, ping will suffer little from congestion and shows 494 disaccord in packet loss rate. 496 Internal rate limitation: Routers normally have rate limitation 497 towards CPU, which is considered a kind of protection to the 498 control plane of routers. So if a packet is sent to CPU for 499 processing rather than line card ASIC (e.g. in many routers, an 500 ICMP echo reply packet received in response to an earlier echo 501 request packet sent by the router will be sent to the CPU), it 502 might be influenced by the rate limiter. Typical rate limitation 503 of ICMP packets would be 1000 pps, though the value is highly 504 dependent on vendor implementation and can be configured. In 505 practical deployment, if there is a large number of ICMP packets 506 sending to a router, the ping test packets may be dropped, causing 507 test errors. This problem did not arise in our test in section 3 508 as the ICMP traffic is rather small. 510 4.2. Issues with OWAMP and TWAMP 512 OWAMP and TWAMP fall into the category of active measurement, so the 513 general issues of active measurement apply to them. When using the 514 two methods, one is advised to make sure that the measurement traffic 515 will have the same drop probability as non-measurement traffic. 516 However, it is usually difficult to guarantee this, as too many 517 factors effect the behavior of traffic. 519 4.3. Issues with Proprietary Tools 521 Since the proprietary tools are implemented by vendors independently, 522 interoperability is one of the major issues when using the tools, 523 especially for one-way measurement. Besides, these tools also share 524 the common issues of active measurement. The accuracy of results 525 depends on the rate, numbers and interval of the injected packets. 526 It also needs to guarantee that the injected packets follows the same 527 path as the tested packets, otherwise the results cannot reflect the 528 real performance. 530 Although these tools provide automatic testing method, the basic 531 principle is still to ping from the router itself. So it is believed 532 toolset method will experience the same issues about class and 533 priority as local ping from router does. However, we did not test 534 diagnosis toolsets, and the discussion is left to be further 535 continued. 537 4.4. Issues with Interface Statistics Report 539 Interface statistic is the most direct and accurate way to get 540 performance of an interface. Packet loss rate calculated from 541 traffic statistics is in accordance with the expected value. By 542 referring to statistics collected from the endpoint routers, 543 bidirectional packet loss rate can easily be obtained. 545 However, this approach requires access to routers, while in some 546 scenarios it is difficult to do that. For example, if we would like 547 to know the inbound packet loss rate of the interconnection link to 548 another service operator, we may have to rely on statistics provided 549 by the peering router. Normally, this information is not easily 550 shared by interworking operators. 552 4.5. Issues with Coloring Based Performance Measurement 554 The challenge for coloring based performance measurement is that 555 there are not so many bits in the IP header that can be used for IP 556 packet coloring. Operators have to carefully think of the color bits 557 selection to make sure that the setting and changing of the color 558 bits will not affect the normal packet forwarding and process. 560 5. Considerations and Recommendations 562 We summarize the above analysis here and come to the following 563 considerations: 565 a. The ping method to measure packet loss rate is easy to be 566 influenced by the diverse processing mechanism of ICMP packet 567 within routers. If this method is to be used on a router, one is 568 advised to make sure that the ICMP packets experience the same 569 forwarding and discarding courses as the service traffic (of 570 which the packet loss rate is to be measured) does, otherwise the 571 measurement will not make sense. When measuring with ping, the 572 following points are also worth reminding: 574 * Packet loss rate given by measurement with ping is a value 575 related to a certain forwarding class in which the ICMP 576 packets are forwarded. So it is not a scientific way to say 577 what the packet loss rate is on a link if traffic is 578 transmitted in more than one class on the link. 580 * Measurement with ping is enough if one only wants to get a 581 general, qualitative picture of packet loss. But if one is to 582 measure precisely and quantitatively, possible errors 583 (sometimes very large errors) should be taken into account. 585 * If configured in the right way on router, ping with CLI and 586 SNMP lead to similar results. 588 b. It is more likely to get good results if a probe is used to 589 perform ping measurement (though not 100% guaranteed), but 590 following issues also need to be considered. 592 * If the probe is directly connected to a router, then a router 593 port is occupied. This will be a problem for routers with 594 limited or expensive port resources, as the probing traffic is 595 usually extremely small. 597 * If the probe is more than one hop away from a router, load of 598 the path to the router is supposed to be under the congestion 599 level. 601 c. Interface statistics report gives us the most accurate value of 602 pack loss rate, and the value is irrelevant to router platforms. 603 From the report we can find numbers of packets being received, 604 transmitted, and discarded in different classes within a period 605 of time, thus we get packet loss rate. Actually this is indeed 606 how packet loss rate is defined. 608 * Referring to report requires access to routers, which may be 609 easier if routers are within a single administrative area. 610 However it gets annoying if more routers are evolved, for 611 instance measurement on a long path with a number of routers. 613 * Router interface report only gives the outbound packet loss 614 rate. If we want to see if traffic in the other direction is 615 congested, we'll have to check the upstream routers in that 616 direction. This will be difficult on certain links, say, 617 interconnection link to another provider. 619 6. Security Considerations 621 TBD. 623 7. IANA Considerations 625 This memo includes no request to IANA. 627 8. Acknowledgements 629 The authors would like to thank Brian Trammell for the kind comments. 631 9. References 633 9.1. Normative References 635 [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way 636 Packet Loss Metric for IPPM", RFC 2680, September 1999. 638 [RFC4443] Conta, A., Deering, S., and M. Gupta, "Internet Control 639 Message Protocol (ICMPv6) for the Internet Protocol 640 Version 6 (IPv6) Specification", RFC 4443, March 2006. 642 [RFC4656] Shalunov, S. and B. Teitelbaum, "A One-way Active 643 Measurement Protocol (OWAMP)", RFC 4656, September 2006. 645 [RFC5357] Hedayat, K. and R. Krzanowski, "A Two-Way Active 646 Measurement Protocol (TWAMP)", RFC 5357, October 2008. 648 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 649 August 2012. 651 [RFC792] Postel, J., "Internet Control Message Protocol", RFC 792, 652 September 1981. 654 9.2. Informative References 656 [I-D.chen-coloring-based-ipfpm-framework] 657 Chen, M., Liu, H., and Y. Yin, "Coloring based IP Flow 658 Performance Measurement Framework", draft-chen-coloring- 659 based-ipfpm-framework-01 (Work in Progress), February 660 2013. 662 [I-D.tempia-opsawg-p3m] 663 Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda, 664 "A packet based method for passive performance 665 monitoring", draft-tempia-opsawg-p3m-03 (Work in 666 Progress), February 2013. 668 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 669 "Framework for IP Performance Metrics", RFC 2330, May 670 1998. 672 Authors' Addresses 674 Peng Fan 675 China Mobile 676 32 Xuanwumen West Street, Xicheng District 677 Beijing 100053 678 P.R. China 680 Email: fanpeng@chinamobile.com 682 Lu Huang 683 China Mobile 684 32 Xuanwumen West Street, Xicheng District 685 Beijing 100053 686 P.R. China 688 Email: huanglu@chinamobile.com 690 Mach(Guoyi) Chen 691 Huawei Technologies 693 Email: mach.chen@huawei.com 695 Nagendra Kumar 696 Cisco Systems 698 Email: naikumar@cisco.com