idnits 2.17.1 draft-ietf-ippm-6man-pdm-option-13.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 26, 2017) is 2486 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT N. Elkins 3 Inside Products 4 R. Hamilton 5 Chemical Abstracts Service 6 M. Ackermann 7 Intended Status: Proposed Standard BCBS Michigan 8 Expires: December 28, 2017 June 26, 2017 10 IPv6 Performance and Diagnostic Metrics (PDM) Destination Option 11 draft-ietf-ippm-6man-pdm-option-13 13 Abstract 15 To assess performance problems, this document describes optional 16 headers embedded in each packet that provide sequence numbers and 17 timing information as a basis for measurements. Such measurements 18 may be interpreted in real-time or after the fact. This document 19 specifies the Performance and Diagnostic Metrics (PDM) destination 20 options extension header. The field limits, calculations, and usage 21 in measurement of PDM are included in this document. 23 Status of this Memo 25 This Internet-Draft is submitted to IETF in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as 31 Internet-Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/1id-abstracts.html 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html 44 Copyright and License Notice 46 Copyright (c) 2017 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 50 3: This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 5 63 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 5 64 1.2 Rationale for defined solution . . . . . . . . . . . . . . . 5 65 1.3 IPv6 Transition Technologies . . . . . . . . . . . . . . . . 6 66 2 Measurement Information Derived from PDM . . . . . . . . . . . . 6 67 2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . . 6 68 2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . . 7 69 3 Performance and Diagnostic Metrics Destination Option Layout . . 7 70 3.1 Destination Options Header . . . . . . . . . . . . . . . . . 7 71 3.2 Performance and Diagnostic Metrics Destination Option . . . 7 72 3.2.1 PDM Layout . . . . . . . . . . . . . . . . . . . . . . . 7 73 3.2.2 Base Unit for Time Measurement . . . . . . . . . . . . . 10 74 3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 11 75 3.4 Header Placement Using IPSec ESP Mode . . . . . . . . . . . 11 76 3.4.1 Using ESP Transport Mode . . . . . . . . . . . . . . . . 11 77 3.4.2 Using ESP Tunnel Mode . . . . . . . . . . . . . . . . . 12 78 3.5 Implementation Considerations . . . . . . . . . . . . . . . 12 79 3.5.1 PDM Activation . . . . . . . . . . . . . . . . . . . . . 12 80 3.5.2 PDM Timestamps . . . . . . . . . . . . . . . . . . . . . 12 81 3.6 Dynamic Configuration Options . . . . . . . . . . . . . . . 12 82 3.7 Information Access and Storage . . . . . . . . . . . . . . . 13 83 4 Security Considerations . . . . . . . . . . . . . . . . . . . . 13 84 4.1 Resource Consumption and Resource Consumption Attacks . . . 13 85 4.2 Pervasive monitoring . . . . . . . . . . . . . . . . . . . . 13 86 4.3 PDM as a Covert Channel . . . . . . . . . . . . . . . . . . 14 87 4.4 Timing Attacks . . . . . . . . . . . . . . . . . . . . . . . 14 88 5 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 15 89 6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 90 6.1 Normative References . . . . . . . . . . . . . . . . . . . . 15 91 6.2 Informative References . . . . . . . . . . . . . . . . . . . 16 92 Appendix A: Context for PDM . . . . . . . . . . . . . . . . . . . 16 93 A.1 End User Quality of Service (QoS) . . . . . . . . . . . . . 16 94 A.2 Need for a Packet Sequence Number (PSN) . . . . . . . . . . 17 95 A.3 Rationale for Defined Solution . . . . . . . . . . . . . . . 17 96 A.4 Use PDM with Other Headers . . . . . . . . . . . . . . . . . 17 97 Appendix B : Timing Considerations . . . . . . . . . . . . . . . . 19 98 B.1 Timing Differential Calculations . . . . . . . . . . . . . . 19 99 B.2 Considerations of this time-differential representation . . 20 100 B.2.1 Limitations with this encoding method . . . . . . . . . 20 101 B.2.2 Loss of precision induced by timer value truncation . . 21 102 Appendix C: Sample Packet Flows . . . . . . . . . . . . . . . . . 22 103 C.1 PDM Flow - Simple Client Server . . . . . . . . . . . . . . 22 104 C.1.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . 23 105 C.1.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . 23 106 C.1.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . 24 107 C.1.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . 25 108 C.1.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . 26 109 C.2 Other Flows . . . . . . . . . . . . . . . . . . . . . . . . 26 110 C.2.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . 26 111 C.2.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . 28 112 C.2.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . 29 113 Appendix D: Potential Overhead Considerations . . . . . . . . . . 30 114 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 31 115 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 32 117 1 Background 119 To assess performance problems, measurements based on optional 120 sequence numbers and timing may be embedded in each packet. Such 121 measurements may be interpreted in real-time or after the fact. 123 As defined in RFC2460 [RFC2460], destination options are carried by 124 the IPv6 Destination Options extension header. Destination options 125 include optional information that need be examined only by the IPv6 126 node given as the destination address in the IPv6 header, not by 127 routers or other "middle boxes". This document specifies the 128 Performance and Diagnostic Metrics (PDM) destination option. The 129 field limits, calculations, and usage in measurement of the PDM 130 destination options extension header are included in this document. 132 1.1 Terminology 134 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 135 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 136 document are to be interpreted as described in RFC 2119 [RFC2119]. 138 1.2 Rationale for defined solution 140 The current IPv6 specification does not provide timing nor a similar 141 field in the IPv6 main header or in any extension header. The IPv6 142 Performance and Diagnostic Metrics destination option (PDM) provides 143 such fields. 145 Advantages include: 147 1. Real measure of actual transactions. 149 2. Ability to span organizational boundaries with consistent 150 instrumentation. 152 3. No time synchronization needed between session partners 154 4. Ability to handle all transport protocols (TCP, UDP, SCTP, etc) in 155 a uniform way 157 The PDM provides the ability to determine quickly if the (latency) 158 problem is in the network or in the server (application). That is, 159 it is a fast way to do triage. For more information on background 160 and usage of PDM, see Appendix A. 162 1.3 IPv6 Transition Technologies 164 In the path to full implementation of IPv6, transition technologies 165 such as translation or tunneling may be employed. It is possible 166 that an IPv6 packet containing PDM may be dropped if using IPv6 167 transition technologies. For example, an implementation using a 168 translation technique (IPv6 to IPv4) which does not support or 169 recognize the IPv6 Destination Options extension header may simply 170 drop the packet rather than translating it without the extension 171 header. 173 It is also possible that some devices in the network may not 174 correctly handle multiple IPv6 Extension Headers, including the IPv6 175 Destination Option. For example, adding the PDM header to a packet 176 may push the layer 4 information to a point in the packet where it is 177 not visible to filtering logic, and may be dropped. This kind of 178 situation is expected to become rare over time. 180 2 Measurement Information Derived from PDM 182 Each packet contains information about the sender and receiver. In IP 183 protocol, the identifying information is called a "5-tuple". 185 The 5-tuple consists of: 187 SADDR : IP address of the sender 188 SPORT : Port for sender 189 DADDR : IP address of the destination 190 DPORT : Port for destination 191 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.) 193 The PDM contains the following base fields: 195 PSNTP : Packet Sequence Number This Packet 196 PSNLR : Packet Sequence Number Last Received 197 DELTATLR : Delta Time Last Received 198 DELTATLS : Delta Time Last Sent 200 Other fields for storing time scaling factors are also in the PDM and 201 will be described in section 3. 203 This information, combined with the 5-tuple, allows the measurement 204 of the following metrics: 206 1. Round-trip delay 207 2. Server delay 209 2.1 Round-Trip Delay 210 Round-trip *Network* delay is the delay for packet transfer from a 211 source host to a destination host and then back to the source host. 212 This measurement has been defined, and the advantages and 213 disadvantages discussed in "A Round-trip Delay Metric for IPPM" 214 [RFC2681]. 216 2.2 Server Delay 218 Server delay is the interval between when a packet is received by a 219 device and the first corresponding packet is sent back in response. 220 This may be "Server Processing Time". It may also be a delay caused 221 by acknowledgements. Server processing time includes the time taken 222 by the combination of the stack and application to return the 223 response. The stack delay may be related to network performance. If 224 this aggregate time is seen as a problem, and there is a need to make 225 a clear distinction between application processing time and stack 226 delay, including that caused by the network, then more client based 227 measurements are needed. 229 3 Performance and Diagnostic Metrics Destination Option Layout 231 3.1 Destination Options Header 233 The IPv6 Destination Options Header is used to carry optional 234 information that needs to be examined only by a packet's destination 235 node(s). The Destination Options Header is identified by a Next 236 Header value of 60 in the immediately preceding header and is defined 237 in RFC2460 [RFC2460]. The IPv6 Performance and Diagnostic Metrics 238 Destination Option (PDM) is implemented as an IPv6 Option carried in 239 the Destination Options Header. The PDM does not require time 240 synchronization. 242 3.2 Performance and Diagnostic Metrics Destination Option 244 3.2.1 PDM Layout 246 The IPv6 Performance and Diagnostic Metrics Destination Option (PDM) 247 contains the following fields: 249 SCALEDTLR: Scale for Delta Time Last Received 250 SCALEDTLS: Scale for Delta Time Last Sent 251 PSNTP : Packet Sequence Number This Packet 252 PSNLR : Packet Sequence Number Last Received 253 DELTATLR : Delta Time Last Received 254 DELTATLS : Delta Time Last Sent 256 PDM has alignment requirements. Following the convention in IPv6, 257 these options are aligned in a packet so that multi-octet values 258 within the Option Data field of each option fall on natural 259 boundaries (i.e., fields of width n octets are placed at an integer 260 multiple of n octets from the start of the header, for n = 1, 2, 4, 261 or 8) [RFC2460]. 263 The PDM destination option is encoded in type-length-value (TLV) 264 format as follows: 266 0 1 2 3 267 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 268 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 269 | Option Type | Option Length | ScaleDTLR | ScaleDTLS | 270 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 271 | PSN This Packet | PSN Last Received | 272 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 273 | Delta Time Last Received | Delta Time Last Sent | 274 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 276 Option Type 278 TBD = 0xXX (TBD) [To be assigned by IANA] [RFC2780] 280 In keeping with RFC2460[RFC2460], the two high order bits of the 281 Option Type field are encoded to indicate specific processing of the 282 option; for the PDM destination option, these two bits MUST be set to 283 00. 285 The third high order bit of the Option Type specifies whether or not 286 the Option Data of that option can change en-route to the packet's 287 final destination. 289 In the PDM, the value of the third high order bit MUST be 0. 291 Option Length 293 8-bit unsigned integer. Length of the option, in octets, excluding 294 the Option Type and Option Length fields. This field MUST be set to 295 10. 297 Scale Delta Time Last Received (SCALEDTLR) 299 8-bit unsigned integer. This is the scaling value for the Delta Time 300 Last Received (DELTATLR) field. The possible values are from 0-255. 301 See Section 4 for further discussion on Timing Considerations and 302 formatting of the scaling values. 304 Scale Delta Time Last Sent (SCALEDTLS) 306 8-bit signed integer. This is the scaling value for the Delta Time 307 Last Sent (DELTATLS) field. The possible values are from 0 to 255. 309 Packet Sequence Number This Packet (PSNTP) 311 16-bit unsigned integer. This field will wrap. It is intended for 312 use while analyzing packet traces. 314 Initialized at a random number and incremented monotonically for each 315 packet of the session flow of the 5-tuple. The random number 316 initialization is intended to make it harder to spoof and insert such 317 packets. 319 Operating systems MUST implement a separate packet sequence number 320 counter per 5-tuple. 322 Packet Sequence Number Last Received (PSNLR) 324 16-bit unsigned integer. This is the PSNTP of the packet last 325 received on the 5-tuple. 327 This field is initialized to 0. 329 Delta Time Last Received (DELTATLR) 331 A 16-bit unsigned integer field. The value is set according to the 332 scale in SCALEDTLR. 334 Delta Time Last Received = (Send time packet n - Receive time packet 335 n-1) 337 Delta Time Last Sent (DELTATLS) 339 A 16-bit unsigned integer field. The value is set according to the 340 scale in SCALEDTLS. 342 Delta Time Last Sent = (Receive time packet n - Send time packet n-1) 344 3.2.2 Base Unit for Time Measurement 346 A time differential is always a whole number in a CPU; it is the unit 347 specification -- hours, seconds, nanoseconds -- that determine what 348 the numeric value means. For PDM, the base time unit is 1 attosecond 349 (asec). This allows for a common unit and scaling of the time 350 differential among all IP stacks and hardware implementations. 352 Note that PDM provides the ability to measure both time differentials 353 that are extremely small, and time differentials in a 354 Delay/Disruption Tolerant Networking (DTN) environment where the 355 delays may be very great. To store a time differential in just 16 356 bits that must range in this way will require some scaling of the 357 time differential value. 359 One issue is the conversion from the native time base in the CPU 360 hardware of whatever device is in use to some number of attoseconds. 361 It might seem this will be an astronomical number, but the conversion 362 is straightforward. It involves multiplication by an appropriate 363 power of 10 to change the value into a number of attoseconds. Then, 364 to scale the value so that it fits into DELTATLR or DELTATLS, the 365 value is shifted by of a number of bits, retaining the 16 high-order 366 or most significant bits. The number of bits shifted becomes the 367 scaling factor, stored as SCALEDTLR or SCALEDTLS, respectively. For 368 additional information of this process, including examples, please 369 see Appendix A. 371 3.3 Header Placement 373 The PDM Destination Option is placed as defined in RFC2460 [RFC2460]. 374 There may be a choice of where to place the Destination Options 375 header. If using ESP mode, please see section 3.4 of this document 376 for placement of the PDM Destination Options header. 378 For each IPv6 packet header, the PDM MUST NOT appear more than once. 379 However, an encapsulated packet MAY contain a separate PDM associated 380 with each encapsulated IPv6 header. 382 3.4 Header Placement Using IPSec ESP Mode 384 IPSec Encapsulating Security Payload (ESP) is defined in [RFC4303] 385 and is widely used. Section 3.1.1 of [RFC4303] discusses placement 386 of Destination Options Headers. 388 The placement of PDM is different depending on if ESP is used in 389 tunnel or transport mode. 391 In ESP case, no 5-tuple is available, as there are no port numbers. 392 ESP flow should be identified only by using SADDR, DADDR and PROTOC. 393 The SPI numbers SHOULD be ignored when considering the flow over 394 which PDM information is measured. 396 3.4.1 Using ESP Transport Mode 398 Note that Destination Options may be placed before or after ESP or 399 both. If using PDM in ESP transport mode, PDM MUST be placed after 400 the ESP header so as not to leak information. 402 3.4.2 Using ESP Tunnel Mode 404 Note that Destination Options may be placed before or after ESP or 405 both in both the outer set of IP headers and the inner set of IP 406 headers. A tunnel endpoint that creates a new packet may decide to 407 use PDM independent of the use of PDM of the original packet to 408 enable delay measurements between the two tunnel endpoints. 410 3.5 Implementation Considerations 412 3.5.1 PDM Activation 414 An implementation should provide an interface to enable or disable 415 the use of PDM. This specification recommends having PDM off by 416 default. 418 PDM MUST NOT be turned on merely if a packet is received with a PDM 419 header. The received packet could be spoofed by another device. 421 3.5.2 PDM Timestamps 423 The PDM timestamps are intended to isolate wire time from server or 424 host time, but may necessarily attribute some host processing time to 425 network latency. 427 RFC2330 [RFC2330] "Framework for IP Performance Metrics" describes 428 two notions of wire time in section 10.2. These notions are only 429 defined in terms of an Internet host H observing an Internet link L 430 at a particular location: 432 + For a given IP packet P, the 'wire arrival time' of P at H on L 433 is the first time T at which any bit of P has appeared at H's 434 observational position on L. 436 + For a given IP packet P, the 'wire exit time' of P at H on L is 437 the first time T at which all the bits of P have appeared at H's 438 observational position on L. 440 This specification does not define the exact H's observing position 441 on L. That is left for the deployment setups to define. However, the 442 position where PDM timestamps are taken SHOULD be as close to the 443 physical network interface as possible. Not all implementations will 444 be able to achieve the ideal level of measurement. 446 3.6 Dynamic Configuration Options 447 If the PDM destination options extension header is used, then it MAY 448 be turned on for all packets flowing through the host, applied to an 449 upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP 450 address only. These are at the discretion of the implementation. 452 3.7 Information Access and Storage 454 Measurement information provided by PDM may be made accessible for 455 higher layers or the user itself. Similar to activating the use of 456 PDM, the implementation may also provide an interface to indicate if 457 received 459 PDM information may be stored, if desired. If a packet with PDM 460 information is received and the information should be stored, the 461 upper layers may be notified. Furthermore, the implementation should 462 define a configurable maximum lifetime after which the information 463 can be removed as well as a configurable maximum amount of memory 464 that should be allocated for PDM information. 466 4 Security Considerations 468 PDM may introduce some new security weaknesses. 470 4.1 Resource Consumption and Resource Consumption Attacks 472 PDM needs to calculate the deltas for time and keep track of the 473 sequence numbers. This means that control blocks which reside in 474 memory may be kept at the end hosts per 5-tuple. 476 A limit on how much memory is being used SHOULD be implemented. 478 Without a memory limit, any time a control block is kept in memory, 479 an attacker can try to misuse the control blocks to cause excessive 480 resource consumption. This could be used to compromise the end host. 482 PDM is used only at the end hosts and memory is used only at the end 483 host and not at routers or middle boxes. 485 4.2 Pervasive monitoring 487 Since PDM passes in the clear, a concern arises as to whether the 488 data can be used to fingerprint the system or somehow obtain 489 information about the contents of the payload. 491 Let us discuss fingerprinting of the end host first. It is possible 492 that seeing the pattern of deltas or the absolute values could give 493 some information as to the speed of the end host - that is, if it is 494 a very fast system or an older, slow device. This may be useful to 495 the attacker. However, if the attacker has access to PDM, the 496 attacker also has access to the entire packet and could make such a 497 deduction based merely on the time frames elapsed between packets 498 WITHOUT PDM. 500 As far as deducing the content of the payload, in terms of the 501 application level information such as web page, user name, user 502 password and so on, it appears to us that PDM is quite unhelpful in 503 this regard. Having said that, the ability to separate wire-time 504 from processing time may potentially provide an attacker with 505 additional information. It is conceivable that an attacker could 506 attempt to deduce the type of application in use by noting the server 507 time and payload length. Some encryption algorithms attempt to 508 obfuscate the packet length to avoid just such vulnerabilities. In 509 the future, encryption algorithms may wish to obfuscate the server 510 time as well. 512 4.3 PDM as a Covert Channel 514 PDM provides a set of fields in the packet which could be used to 515 leak data. But, there is no real reason to suspect that PDM would be 516 chosen rather than another part of the payload or another Extension 517 Header. 519 A firewall or another device could sanity check the fields within the 520 PDM but randomly assigned sequence numbers and delta times might be 521 expected to vary widely. The biggest problem though is how an 522 attacker would get access to PDM in the first place to leak data. 523 The attacker would have to either compromise the end host or have Man 524 in the Middle (MitM). It is possible that either one could change 525 the fields. But, then the other end host would get sequence numbers 526 and deltas that don't make any sense. 528 It is conceivable that someone could compromise an end host and make 529 it start sending packets with PDM without the knowledge of the host. 530 But, again, the bigger problem is the compromise of the end host. 531 Once that is done, the attacker probably has better ways to leak 532 data. 534 Having said that, if a PDM aware middle box or an implementation 535 (destination host) detects some number of "nonsensical" sequence 536 numbers or timing information, it could take action to block, 537 discard, or alert on this traffic. 539 4.4 Timing Attacks 541 The fact that PDM can help in the separation of node processing time 542 from network latency brings value to performance monitoring. Yet, it 543 is this very characteristic of PDM which may be misused to make 544 certain new type of timing attacks against protocols and 545 implementations possible. 547 Depending on the nature of the cryptographic protocol used, it may be 548 possible to leak the credentials of the device. For example, if an 549 attacker can see that PDM is being used, then the attacker might use 550 PDM to launch a timing attack against the keying material used by the 551 cryptographic protocol. 553 An implementation may want to be sure that PDM is enabled only for 554 certain ip addresses, or only for some ports. Additionally, the 555 implementation SHOULD require an explicit restart of monitoring after 556 a certain time period (for example for 1 hour), to make sure that PDM 557 is not accidentally left on after debugging has been done etc. 559 Even so, if using PDM, a user "Consent to be Measured" SHOULD be a 560 pre-requisite for using PDM. Consent is common in enterprises and 561 with some subscription services. The actual content of "Consent to 562 be Measured" will differ by site but it SHOULD make clear that the 563 traffic is being measured for quality of service and to assist in 564 diagnostics as well as to make clear that there may be potential 565 risks of certain vulnerabilities if the traffic is captured during a 566 diagnostic session. 568 5 IANA Considerations 570 This draft requests an Destination Option Type assignment with the 571 act bits set to 00 and the chg bit set to 0 from the Destination 572 Options and Hop-by-Hop Options sub-registry of Internet Protocol 573 Version 6 (IPv6) Parameters [ref to RFCs and URL below]. 575 http://www.iana.org/assignments/ipv6-parameters/ipv6- 576 parameters.xhtml#ipv6-parameters-2 578 Hex Value Binary Value Description Reference 579 act chg rest 580 ------------------------------------------------------------------- 581 TBD TBD Performance and [This draft] 582 Diagnostic Metrics 583 (PDM) 585 6 References 587 6.1 Normative References 589 [RFC1122] Braden, R., "Requirements for Internet Hosts -- 590 Communication Layers", RFC 1122, October 1989. 592 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 593 Requirement Levels", BCP 14, RFC 2119, March 1997. 595 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 596 (IPv6) Specification", RFC 2460, December 1998. 598 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 599 Delay Metric for IPPM", RFC 2681, September 1999. 601 [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines 602 For Values In the Internet Protocol and Related Headers", BCP 37, RFC 603 2780, March 2000. 605 [RFC4303] Kent, S, "IP Encapsulating Security Payload (ESP)", RFC 606 4303, December 2005. 608 6.2 Informative References 610 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 611 "Framework for IP Performance Metrics", RFC 2330, May 1998. 613 [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP 614 Timestamp Option-01", Internet Draft, July 2013. [Work in Progress] 616 Appendix A: Context for PDM 618 A.1 End User Quality of Service (QoS) 620 The timing values in the PDM embedded in the packet will be used to 621 estimate QoS as experienced by an end user device. 623 For many applications, the key user performance indicator is response 624 time. When the end user is an individual, he is generally 625 indifferent to what is happening along the network; what he really 626 cares about is how long it takes to get a response back. But this is 627 not just a matter of individuals' personal convenience. In many 628 cases, rapid response is critical to the business being conducted. 630 Low, reliable and acceptable response times are not just "nice to 631 have". On many networks, the impact can be financial hardship or can 632 endanger human life. In some cities, the emergency police contact 633 system operates over IP; law enforcement, at all levels, use IP 634 networks; transactions on our stock exchanges are settled using IP 635 networks. The critical nature of such activities to our daily lives 636 and financial well-being demand a simple solution to support response 637 time measurements. 639 A.2 Need for a Packet Sequence Number (PSN) 641 While performing network diagnostics of an end-to-end connection, it 642 often becomes necessary to isolate the factors along the network path 643 responsible for problems. Diagnostic data may be collected at 644 multiple places along the path (if possible), or at the source and 645 destination. Then, in post-collection processing, the diagnostic 646 data corresponding to each packet at different observation points 647 must be matched for proper measurements. A sequence number in each 648 packet provides sufficient basis for the matching process. If need 649 be, the timing fields may be used along with the sequence number to 650 ensure uniqueness. 652 This method of data collection along the path is of special use to 653 determine where packet loss or packet corruption is happening. 655 The packet sequence number needs to be unique in the context of the 656 session (5-tuple). 658 A.3 Rationale for Defined Solution 660 One of the important functions of PDM is to allow you to quickly 661 dispatch the right set of diagnosticians. Within network or server 662 latency, there may be many components. The job of the diagnostician 663 is to rule each one out until the culprit is found. 665 How PDM fits into this diagnostic picture is that PDM will quickly 666 tell you how to escalate. PDM will point to either the network area 667 or the server area. Within the server latency, PDM does not tell 668 you if the bottleneck is in the IP stack or the application or buffer 669 allocation. Within the network latency, PDM does not tell you which 670 of the network segments or middle boxes is at fault. 672 What PDM does tell you is whether the problem is in the network or 673 the server. 675 A.4 Use PDM with Other Headers 677 For diagnostics, one my want to use PDM with other headers (L2, L3, 678 etc). For example, if PDM is used is by a technician (or tool) 679 looking at a packet capture, within the packet capture, they would 680 have available to them the layer 2 header, IP header (v6 or v4), TCP, 681 UCP, ICMP, SCTP or other headers. All information would be looked 682 at together to make sense of the packet flow. The technician or 683 processing tool could analyze, report or ignore the data from PDM, as 684 necessary. 686 For an example of how PDM can help with TCP retransmit problems, 687 please look at Appendix C. 689 Appendix B : Timing Considerations 691 B.1 Timing Differential Calculations 693 The time counter in a CPU is a binary whole number, representing a 694 number of milliseconds (msec), microseconds (usec) or even 695 picoseconds (psec). Representing one of these values as attoseconds 696 (asec) means multiplying by 10 raised to some exponent. Refer to this 697 table of equalities: 699 Base value = # of sec = # of asec 1000s of asec 700 --------------- ------------- ------------- ------------- 701 1 second 1 sec 10**18 asec 1000**6 asec 702 1 millisecond 10**-3 sec 10**15 asec 1000**5 asec 703 1 microsecond 10**-6 sec 10**12 asec 1000**4 asec 704 1 nanosecond 10**-9 sec 10**9 asec 1000**3 asec 705 1 picosecond 10**-12 sec 10**6 asec 1000**2 asec 706 1 femtosecond 10**-15 sec 10**3 asec 1000**1 asec 708 For example, if you have a time differential expressed in 709 microseconds, since each microsecond is 10**12 asec, you would 710 multiply your time value by 10**12 to obtain the number of 711 attoseconds. If you time differential is expressed in nanoseconds, 712 you would multiply by 10**9 to get the number of attoseconds. 714 The result is a binary value that will need to be shortened by a 715 number of bits so it will fit into the 16-bit PDM DELTA field. 717 The next step is to divide by 2 until the value is contained in just 718 16 significant bits. The exponent of the value in the last column of 719 of the table is useful here; the initial scaling factor is that 720 exponent multiplied by 10. This is the minimum number of low-order 721 bits to be shifted-out or discarded. It represents dividing the time 722 value by 1024 raised to that exponent. 724 The resulting value may still be too large to fit into 16 bits, but 725 can be normalized by shifting out more bits (dividing by 2) until the 726 value fits into the 16-bit DELTA field. The number of extra bits 727 shifted out is then added to the scaling factor. The scaling factor, 728 the total number of low-order bits dropped, is the SCALEDTL value. 730 For example: say an application has these start and finish timer 731 values (hexadecimal values, in microseconds): 733 Finish: 27C849234 usec (02:57:58.997556) 734 -Start: 27C83F696 usec (02:57:58.957718) 735 ========== ========= =============== 736 Difference 9B9E usec 00.039838 sec or 39838 usec 738 To convert this differential value to binary attoseconds, multiply 739 the number of microseconds by 10**12. Divide by 1024**4, or simply 740 discard 40 bits from the right. The result is 36232, or 8D88 in hex, 741 with a scaling factor or SCALEDTL value of 40. 743 For another example, presume the time differential is larger, say 744 32.311072 seconds, which is 32311072 usec. Each microsecond is 10**12 745 asec, so multiply by 10**12, giving the hexadecimal value 746 1C067FCCAE8120000. Using the initial scaling factor of 40, drop the 747 last 10 characters (40 bits) from that string, giving 1C067FC. This 748 will not fit into a DELTA field, as it is 25 bits long. Shifting the 749 value to the right another 9 bits results in a DELTA value of E033, 750 with a resulting scaling factor of 49. 752 When the time differential value is a small number, regardless of the 753 time unit, the exponent trick given above is not useful in 754 determining the proper scaling value. For example, if the time 755 differential is 3 seconds and you want to convert that directly, you 756 would follow this path: 758 3 seconds = 3*10**18 asec (decimal) 759 = 29A2241AF62C0000 asec (hexadecimal) 761 If you just truncate the last 60 bits, you end up with a delta value 762 of 2 and a scaling factor of 60, when what you really wanted was a 763 delta value with more significant digits. The most precision with 764 which you can store this value in 16 bits is A688, with a scaling 765 factor of 46. 767 B.2 Considerations of this time-differential representation 769 There are a few considerations to be taken into account with this 770 representation of a time differential. The first is whether there are 771 any limitations on the maximum or minimum time differential that can 772 be expressed using method of a delta value and a scaling factor. The 773 second is the amount of imprecision introduced by this method. 775 B.2.1 Limitations with this encoding method 777 The DELTATLS and DELTATLR fields store only the 16 most-significant 778 bits of the time differential value. Thus the range, excluding the 779 scaling factor, is from 0 to 65535, or a maximum of 2**16-1. This 780 method is further described in [TRAM-TCPM]. 782 The actual magnitude of the time differential is determined by the 783 scaling factor. SCALEDTLR and SCALEDTLS are 8-bit unsigned integers, 784 so the scaling factor ranges from 0 to 255. The smallest number that 785 can be represented would have a value of 1 in the delta field and a 786 value of 0 in the associated scale field. This is the representation 787 for 1 attosecond. Clearly this allows PDM to measure extremely small 788 time differentials. 790 On the other end of the scale, the maximum delta value is 65535, or 791 FFFF in hexadecimal. If the maximum scale value of 255 is used, the 792 time differential represented is 65535*2**255, which is over 3*10**55 793 years, essentially, forever. So there appears to be no real 794 limitation to the time differential that can be represented. 796 B.2.2 Loss of precision induced by timer value truncation 798 As PDM specifies the DELTATLR and DELTATLS values as 16-bit unsigned 799 integers, any time the precision is greater than those 16 bits, there 800 will be truncation of the trailing bits, with an accompanying loss of 801 precision in the value. 803 Any time differential value smaller than 65536 asec can be stored 804 exactly in DELTATLR or DELTATLS, because the representation of this 805 value requires at most 16 bits. 807 Since the time differential values in PDM are measured in 808 attoseconds, the range of values that would be truncated to the same 809 encoded value is 2**(Scale)-1 asec. 811 For example, the smallest time differential that would be truncated 812 to fit into a delta field is 814 1 0000 0000 0000 0000 = 65536 asec 816 This value would be encoded as a delta value of 8000 (hexadecimal) 817 with a scaling factor of 1. The value 819 1 0000 0000 0000 0001 = 65537 asec 821 would also be encoded as a delta value of 8000 with a scaling factor 822 of 1. This actually is the largest value that would be truncated to 823 that same encoded value. When the scale value is 1, the value range 824 is calculated as 2**1 - 1, or 1 asec, which you can see is the 825 difference between these minimum and maximum values. 827 The scaling factor is defined as the number of low-order bits 828 truncated to reduce the size of the resulting value so it fits into a 829 16-bit delta field. If, for example, you had to truncate 12 bits, the 830 loss of precision would depend on the bits you truncated. The range 831 of these values would be 833 0000 0000 0000 = 0 asec 835 to 836 1111 1111 1111 = 4095 asec 838 So the minimum loss of precision would be 0 asec, where the delta 839 value exactly represents the time differential, and the maximum loss 840 of precision would be 4095 asec. As stated above, the scaling factor 841 of 12 means the maximum loss of precision is 2**12-1 asec, or 4095 842 asec. 844 Compare this loss of precision to the actual time differential. The 845 range of actual time differential values that would incur this loss 846 of precision is from 848 1000 0000 0000 0000 0000 0000 0000 = 2**27 asec or 134217728 asec 849 to 850 1111 1111 1111 1111 1111 1111 1111 = 2**28-1 asec or 268435455 asec 852 Granted, these are small values, but the point is, any value between 853 these two values will have a maximum loss of precision of 4095 asec, 854 or about 0.00305% for the first value, as encoded, and at most 855 0.001526% for the second. These maximum-loss percentages are 856 consistent for all scaling values. 858 Appendix C: Sample Packet Flows 860 C.1 PDM Flow - Simple Client Server 862 Following is a sample simple flow for the PDM with one packet sent 863 from Host A and one packet received by Host B. The PDM does not 864 require time synchronization between Host A and Host B. The 865 calculations to derive meaningful metrics for network diagnostics are 866 shown below each packet sent or received. 868 C.1.1 Step 1 870 Packet 1 is sent from Host A to Host B. The time for Host A is set 871 initially to 10:00AM. 873 The time and packet sequence number are saved by the sender 874 internally. The packet sequence number and delta times are sent in 875 the packet. 877 Packet 1 879 +----------+ +----------+ 880 | | | | 881 | Host | ----------> | Host | 882 | A | | B | 883 | | | | 884 +----------+ +----------+ 886 PDM Contents: 888 PSNTP : Packet Sequence Number This Packet: 25 889 PSNLR : Packet Sequence Number Last Received: - 890 DELTATLR : Delta Time Last Received: - 891 SCALEDTLR: Scale of Delta Time Last Received: 0 892 DELTATLS : Delta Time Last Sent: - 893 SCALEDTLS: Scale of Delta Time Last Sent: 0 895 Internally, within the sender, Host A, it must keep: 897 Packet Sequence Number of the last packet sent: 25 898 Time the last packet was sent: 10:00:00 900 Note, the initial PSNTP from Host A starts at a random number. In 901 this case, 25. The time in these examples is shown in seconds for 902 the sake of simplicity. 904 C.1.2 Step 2 906 Packet 1 is received at Host B. Its time is set to one hour later 907 than Host A. In this case, 11:00AM 909 Internally, within the receiver, Host B, it must note: 911 Packet Sequence Number of the last packet received: 25 912 Time the last packet was received : 11:00:03 914 Note, this timestamp is in Host B time. It has nothing whatsoever to 915 do with Host A time. The Packet Sequence Number of the last packet 916 received will become PSNLR which will be sent out in the packet sent 917 by Host B in the next step. The time last received will be used to 918 calculate the DELTALR value to be sent out in the packet sent by Host 919 B in the next step. 921 C.1.3 Step 3 923 Packet 2 is sent by Host B to Host A. Note, the initial packet 924 sequence number (PSNTP) from Host B starts at a random number. In 925 this case, 12. Before sending the packet, Host B does a calculation 926 of deltas. Since Host B knows when it is sending the packet, and it 927 knows when it received the previous packet, it can do the following 928 calculation: 930 Sending time : packet 2 - receive time : packet 1 932 The result of this calculation is called: Delta Time Last Received 933 (DELTATLR) 935 Note, both sending time and receive time are saved internally in Host 936 B. They do not travel in the packet. Only the Delta is in the 937 packet. 939 Assume that within Host B is the following: 941 Packet Sequence Number of the last packet received: 25 942 Time the last packet was received: 11:00:03 943 Packet Sequence Number of this packet: 12 944 Time this packet is being sent: 11:00:07 946 Now a delta value to be sent out in the packet can be calculated. 947 DELTATLR becomes: 949 4 seconds = 11:00:07 - 11:00:03 = 3782DACE9D900000 asec 951 This is the derived metric: Server Delay. The time and scaling 952 factor must be converted; in this case, the time differential is 953 DE0B, and the scaling factor is 2E, or 46 in decimal. Then, these 954 values, along with the packet sequence numbers will be sent to Host A 955 as follows: 957 Packet 2 959 +----------+ +----------+ 960 | | | | 961 | Host | <---------- | Host | 962 | A | | B | 963 | | | | 964 +----------+ +----------+ 966 PDM Contents: 968 PSNTP : Packet Sequence Number This Packet: 12 969 PSNLR : Packet Sequence Number Last Received: 25 970 DELTATLR : Delta Time Last Received: DE0B (4 seconds) 971 SCALEDTLR: Scale of Delta Time Last Received: 2E (46 decimal) 972 DELTATLS : Delta Time Last Sent: - 973 SCALEDTLS: Scale of Delta Time Last Sent: 0 975 The metric left to be calculated is the Round-Trip Delay. This will 976 be calculated by Host A when it receives Packet 2. 978 C.1.4 Step 4 980 Packet 2 is received at Host A. Remember, its time is set to one 981 hour earlier than Host B. Internally, it must note: 983 Packet Sequence Number of the last packet received: 12 984 Time the last packet was received : 10:00:12 986 Note, this timestamp is in Host A time. It has nothing whatsoever to 987 do with Host B time. 989 So, now, Host A can calculate total end-to-end time. That is: 991 End-to-End Time = Time Last Received - Time Last Sent 993 For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was 994 received by Host A at 10:00:12 so: 996 End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT 997 delay combined). This time may also be called total Overall Round- 998 Trip Time (RTT) which includes Network RTT and Host Response Time. 1000 This derived metric we will call Delta Time Last Sent (DELTATLS) 1002 Round trip delay can now be calculated. The formula is: 1004 Round trip delay = (Delta Time Last Sent - Delta Time Last Received) 1005 Or: 1007 Round trip delay = 12 - 4 or 8 1009 Now, the only problem is that at this point all metrics are in Host A 1010 only and not exposed in a packet. To do that, we need a third packet. 1012 Note: this simple example assumes one send and one receive. That 1013 is done only for purposes of explaining the function of the PDM. In 1014 cases where there are multiple packets returned, one would take the 1015 time in the last packet in the sequence. The calculations of such 1016 timings and intelligent processing is the function of post-processing 1017 of the data. 1019 C.1.5 Step 5 1021 Packet 3 is sent from Host A to Host B. 1023 +----------+ +----------+ 1024 | | | | 1025 | Host | ----------> | Host | 1026 | A | | B | 1027 | | | | 1028 +----------+ +----------+ 1030 PDM Contents: 1032 PSNTP : Packet Sequence Number This Packet: 26 1033 PSNLR : Packet Sequence Number Last Received: 12 1034 DELTATLR : Delta Time Last Received: 0 1035 SCALEDTLS: Scale of Delta Time Last Received 0 1036 DELTATLS : Delta Time Last Sent: A688 (scaled value) 1037 SCALEDTLR: Scale of Delta Time Last Received: 30 (48 decimal) 1039 To calculate Two-Way Delay, any packet capture device may look at 1040 these packets and do what is necessary. 1042 C.2 Other Flows 1044 What has been discussed so far is a simple flow with one packet sent 1045 and one returned. Let's look at how PDM may be useful in other 1046 types of flows. 1048 C.2.1 PDM Flow - One Way Traffic 1050 The flow on a particular session may not be a send-receive paradigm. 1051 Let us consider some other situations. In the case of a one-way 1052 flow, one might see the following: 1054 Note: The time is expressed in generic units for simplicity. That 1055 is, these values do not represent a number of attoseconds, 1056 microseconds or any other real units of time. 1058 Packet Sender PSN PSN Delta Time Delta Time 1059 This Packet Last Recvd Last Recvd Last Sent 1060 ===================================================================== 1061 1 Server 1 0 0 0 1062 2 Server 2 0 0 5 1063 3 Server 3 0 0 12 1064 4 Server 4 0 0 20 1066 What does this mean and how is it useful? 1068 In a one-way flow, only the Delta Time Last Sent will be seen as 1069 used. Recall, Delta Time Last Sent is the difference between the 1070 send of one packet from a device and the next. This is a measure of 1071 throughput for the sender - according to the sender's point of view. 1072 That is, it is a measure of how fast is the application itself (with 1073 stack time included) able to send packets. 1075 How might this be useful? If one is having a performance issue at 1076 the client and sees that packet 2, for example, is sent after 5 time 1077 units from the server but takes 10 times that long to arrive at the 1078 destination, then one may safely conclude that there are delays in 1079 the path other than at the server which may be causing the delivery 1080 issue of that packet. Such delays may include the network links, 1081 middle-boxes, etc. 1083 Now, true one-way traffic is quite rare. What people often mean by 1084 "one-way" traffic is an application such as FTP where a group of 1085 packets (for example, a TCP window size worth) is sent, then the 1086 sender waits for acknowledgment. This type of flow would actually 1087 fall into the "multiple-send" traffic model. 1089 C.2.2 PDM Flow - Multiple Send Traffic 1091 Assume that two packets are sent for each ACK from the server. For 1092 example, a TCP flow will do this, per RFC1122 [RFC1122] Section- 1093 4.2.3. 1095 Packet Sender PSN PSN Delta Time Delta Time 1096 This Packet Last Recvd Last Recvd Last Sent 1097 ===================================================================== 1098 1 Server 1 0 0 0 1099 2 Server 2 0 0 5 1100 3 Client 1 2 20 0 1101 4 Server 3 1 10 15 1103 How might this be used? 1105 Notice that in packet 3, the client has a value of Delta Time Last 1106 received of 20. Recall that Delta Time Last Received is the Send 1107 time of packet 3 - receive time of packet 2. So, what does one know 1108 now? In this case, Delta Time Last Received is the processing time 1109 for the Client to send the next packet. 1111 How to interpret this depends on what is actually being sent. 1112 Remember, PDM is not being used in isolation, but to supplement the 1113 fields found in other headers. Let's take some examples: 1115 1. Client is sending a standalone TCP ACK. One would find this by 1116 looking at the payload length in the IPv6 header and the TCP 1117 Acknowledgement field in the TCP header. So, in this case, the 1118 client is taking 20 units to send back the ACK. This may or may not 1119 be interesting. 1121 2. Client is sending data with the packet. Again, one would find 1122 this by looking at the payload length in the IPv6 header and the TCP 1123 Acknowledgement field in the TCP header. So, in this case, the 1124 client is taking 20 units to send back data. This may represent 1125 "User Think Time". Again, this may or may not be interesting, in 1126 isolation. But, if there is a performance problem receiving data at 1127 the server, then taken in conjunction with RTT or other packet timing 1128 information, this information may be quite interesting. 1130 Of course, one also needs to look at the PSN Last Received field to 1131 make sure of the interpretation of this data. That is, to make 1132 sure that the Delta Last Received corresponds to the packet of 1133 interest. 1135 The benefits of PDM are that such information is now available in a 1136 uniform manner for all applications and all protocols without 1137 extensive changes required to applications. 1139 C.2.3 PDM Flow - Multiple Send with Errors 1141 Let us now look at a case of how PDM may be able to help in a case of 1142 TCP retransmission and add to the information that is sent in the TCP 1143 header. 1145 Assume that three packets are sent with each send from the server. 1147 From the server, this is what is seen. 1149 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1150 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1151 ===================================================================== 1152 1 Server 1 0 0 0 123 100 1153 2 Server 2 0 0 5 223 100 1154 3 Server 3 0 0 5 333 100 1156 The client, however, does not receive all the packets. From the 1157 client, this is what is seen for the packets sent from the server. 1159 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1160 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1161 ===================================================================== 1162 1 Server 1 0 0 0 123 100 1163 2 Server 3 0 0 5 333 100 1165 Let's assume that the server now retransmits the packet. (Obviously, 1166 a duplicate acknowledgment sequence for fast retransmit or a 1167 retransmit timeout would occur. To illustrate the point, these 1168 packets are being left out.) 1170 So, then if a TCP retransmission is done, then from the client, this 1171 is what is seen for the packets sent from the server. 1173 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1174 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1175 ===================================================================== 1176 1 Server 4 0 0 30 223 100 1178 The server has resent the old packet 2 with TCP sequence number of 1179 223. The retransmitted packet now has a PSN This Packet value of 4. 1181 The Delta Last Sent is 30 - the time between sending the packet with 1182 PSN of 3 and this current packet. 1184 Let's say that packet 4 is lost again. Then, after some amount of 1185 time (RTO) then the packet with TCP sequence number of 223 is resent. 1187 From the client, this is what is seen for the packets sent from the 1188 server. 1190 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1191 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1192 ===================================================================== 1193 1 Server 5 0 0 60 223 100 1195 If now, this packet arrives at the destination, one has a very good 1196 idea that packets exist which are being sent from the server as 1197 retransmissions and not arriving at the client. This is because the 1198 PSN of the resent packet from the server is 5 rather than 4. If we 1199 had used TCP sequence number alone, we would never have seen this 1200 situation. The TCP sequence number in all situations is 223. 1202 This situation would be experienced by the user of the application 1203 (the human being actually sitting somewhere) as a "hangs" or long 1204 delay between packets. On large networks, to diagnose problems such 1205 as these where packets are lost somewhere on the network, one has to 1206 take multiple traces to find out exactly where. 1208 The first thing is to start with doing a trace at the client and the 1209 server. So, we can see if the server sent a particular packet and 1210 the client received it. If the client did not receive it, then we 1211 start tracking back to trace points at the router right after the 1212 server and the router right before the client. Did they get these 1213 packets which the server has sent? This is a time consuming 1214 activity. 1216 With PDM, we can speed up the diagnostic time because we may be able 1217 to use only the trace taken at the client to see what the server is 1218 sending. 1220 Appendix D: Potential Overhead Considerations 1222 One might wonder as to the potential overhead of PDM. First, PDM is 1223 entirely optional. That is, a site may choose to implement PDM or 1224 not as they wish. If they are happy with the costs of PDM vs. the 1225 benefits, then the choice should be theirs. 1227 Below is a table outlining the potential overhead in terms of 1228 additional time to deliver the response to the end user for various 1229 assumed RTTs. 1231 Bytes RTT Bytes Bytes New Overhead 1232 in Packet Per Millisec in PDM RTT 1233 ===================================================================== 1234 1000 1000 milli 1 16 1016.000 16.000 milli 1235 1000 100 milli 10 16 101.600 1.600 milli 1236 1000 10 milli 100 16 10.160 .160 milli 1237 1000 1 milli 1000 16 1.016 .016 milli 1239 Below are some examples of actual RTTs for packets traversing large 1240 enterprise networks. The first example is for packets going to 1241 multiple business partners. 1243 Bytes RTT Bytes Bytes New Overhead 1244 in Packet Per Millisec in PDM RTT 1245 ===================================================================== 1246 1000 17 milli 58 16 17.360 .360 milli 1248 The second example is for packets at a large enterprise customer 1249 within a data center. Notice that the scale is now in microseconds 1250 rather than milliseconds. 1252 Bytes RTT Bytes Bytes New Overhead 1253 in Packet Per Microsec in PDM RTT 1254 ===================================================================== 1255 1000 20 micro 50 16 20.320 .320 micro 1257 As with other diagnostic tools, such as packet traces, a certain 1258 amount of processing time will be required to create and process PDM. 1259 Since PDM is lightweight (has only a few variables), we expect the 1260 processing time to be minimal. 1262 Acknowledgments 1264 The authors would like to thank Keven Haining, Al Morton, Brian 1265 Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick 1266 Troth for their comments and assistance. We would also like to thank 1267 Tero Kivinen and Jouni Korhonen for their detailed and perceptive 1268 reviews. 1270 Authors' Addresses 1272 Nalini Elkins 1273 Inside Products, Inc. 1274 36A Upper Circle 1275 Carmel Valley, CA 93924 1276 United States 1277 Phone: +1 831 659 8360 1278 Email: nalini.elkins@insidethestack.com 1279 http://www.insidethestack.com 1281 Robert M. Hamilton 1282 Chemical Abstracts Service 1283 A Division of the American Chemical Society 1284 2540 Olentangy River Road 1285 Columbus, Ohio 43202 1286 United States 1287 Phone: +1 614 447 3600 x2517 1288 Email: rhamilton@cas.org 1289 http://www.cas.org 1291 Michael S. Ackermann 1292 Blue Cross Blue Shield of Michigan 1293 P.O. Box 2888 1294 Detroit, Michigan 48231 1295 United States 1296 Phone: +1 310 460 4080 1297 Email: mackermann@bcbsm.com 1298 http://www.bcbsm.com