idnits 2.17.1 draft-ietf-ippm-6man-pdm-option-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 9, 2015) is 3234 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT N. Elkins 3 Inside Products 4 R. Hamilton 5 Chemical Abstracts Service 6 M. Ackermann 7 Intended Status: Proposed Standard BCBS Michigan 8 Expires: December 11, 2015 June 9, 2015 10 IPv6 Performance and Diagnostic Metrics (PDM) Destination Option 11 draft-ietf-ippm-6man-pdm-option-00 13 Table of Contents 15 1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 4 16 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4 17 1.2 End User Quality of Service (QoS) . . . . . . . . . . . . . 4 18 1.3 Need for a Packet Sequence Number . . . . . . . . . . . . . 5 19 1.4 Rationale for proposed solution . . . . . . . . . . . . . . 5 20 1.5 PDM Works in Collaboration with Other Headers . . . . . . . 6 21 2 Measurement Information Derived from PDM . . . . . . . . . . . . 6 22 2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . . 7 23 2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . . 7 24 3 Performance and Diagnostic Metrics Destination Option Layout . . 7 25 3.1 Destination Options Header . . . . . . . . . . . . . . . . . 7 26 3.2 Performance and Diagnostic Metrics Destination Option . . . 7 27 3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 10 28 3.4 Implementation Considerations . . . . . . . . . . . . . . . 11 29 3.5 Dynamic Configuration Options . . . . . . . . . . . . . . . 11 30 3.6 5-tuple Aging . . . . . . . . . . . . . . . . . . . . . . . 12 31 4 Considerations of Timing Representation . . . . . . . . . . . . 12 32 4.1 Encoding the Delta-Time Values . . . . . . . . . . . . . . . 12 33 4.2 Timer registers are different on different hardware . . . . 12 34 4.3 Timer Units on Other Systems . . . . . . . . . . . . . . . . 13 35 4.4 Time Base . . . . . . . . . . . . . . . . . . . . . . . . . 13 36 4.5 Timer-value scaling . . . . . . . . . . . . . . . . . . . . 14 37 4.6 Limitations with this encoding method . . . . . . . . . . . 15 38 4.7 Lack of precision induced by timer value truncation . . . . 16 39 5 PDM Flow - Simple Client Server . . . . . . . . . . . . . . . . 17 40 5.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 41 5.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 42 5.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 43 5.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 44 5.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 45 6 Other Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 21 46 6.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . . . 21 47 6.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . . . 22 48 6.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . . . 23 49 7 Potential Overhead Considerations . . . . . . . . . . . . . . . 25 50 8 Security Considerations . . . . . . . . . . . . . . . . . . . . 26 51 9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 26 52 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 26 53 10.1 Normative References . . . . . . . . . . . . . . . . . . . 26 54 10.2 Informative References . . . . . . . . . . . . . . . . . . 27 55 11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 27 56 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 27 58 Abstract 60 To assess performance problems, measurements based on optional 61 sequence numbers and timing may be embedded in each packet. Such 62 measurements may be interpreted in real-time or after the fact. An 63 implementation of the existing IPv6 Destination Options extension 64 header, the Performance and Diagnostic Metrics (PDM) Destination 65 Options extension header as well as the field limits, calculations, 66 and usage of the PDM in measurement are included in this document. 68 Status of this Memo 70 This Internet-Draft is submitted to IETF in full conformance with the 71 provisions of BCP 78 and BCP 79. 73 Internet-Drafts are working documents of the Internet Engineering 74 Task Force (IETF), its areas, and its working groups. Note that 75 other groups may also distribute working documents as 76 Internet-Drafts. 78 Internet-Drafts are draft documents valid for a maximum of six months 79 and may be updated, replaced, or obsoleted by other documents at any 80 time. It is inappropriate to use Internet-Drafts as reference 81 material or to cite them other than as "work in progress." 83 The list of current Internet-Drafts can be accessed at 84 http://www.ietf.org/1id-abstracts.html 86 The list of Internet-Draft Shadow Directories can be accessed at 87 http://www.ietf.org/shadow.html 89 Copyright and License Notice 91 Copyright (c) 2015 IETF Trust and the persons identified as the 92 document authors. All rights reserved. 94 IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 95 3: This document is subject to BCP 78 and the IETF Trust's Legal 96 Provisions Relating to IETF Documents 97 (http://trustee.ietf.org/license-info) in effect on the date of 98 publication of this document. Please review these documents 99 carefully, as they describe your rights and restrictions with respect 100 to this document. Code Components extracted from this document must 101 include Simplified BSD License text as described in Section 4.e of 102 the Trust Legal Provisions and are provided without warranty as 103 described in the Simplified BSD License. 105 1 Background 107 To assess performance problems, measurements based on optional 108 sequence numbers and timing may be embedded in each packet. Such 109 measurements may be interpreted in real-time or after the fact. An 110 implementation of the existing IPv6 Destination Options extension 111 header, the Performance and Diagnostic Metrics (PDM) Destination 112 Options extension header has been proposed in a companion document. 113 This document specifies the layout, field limits, calculations, and 114 usage of the PDM in measurement. 116 As defined in RFC2460 [RFC2460], destination options are carried by 117 the IPv6 Destination Options extension header. Destination options 118 include optional information that need be examined only by the IPv6 119 node given as the destination address in the IPv6 header, not by 120 routers or other "middle boxes". This document specifies a new 121 destination option, the Performance and Diagnostic Metrics (PDM) 122 destination option. 124 1.1 Terminology 126 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 127 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 128 document are to be interpreted as described in RFC 2119 [RFC2119]. 130 1.2 End User Quality of Service (QoS) 132 The difference between timing values in the PDM traveling along with 133 the packet will be used to estimate QoS as experienced by an end user 134 device. 136 For many applications, the key user performance indicator is response 137 time. When the end user is an individual, he is generally 138 indifferent to what is happening along the network; what he really 139 cares about is how long it takes to get a response back. But this is 140 not just a matter of individuals' personal convenience. In many 141 cases, rapid response is critical to the business being conducted. 143 When the end user is a device (e.g. with the Internet of Things), 144 what matters is the speed with which requested data can be 145 transferred -- specifically, whether the requested data can be 146 transferred in time to accomplish the desired actions. This can be 147 important when the relevant external conditions are subject to rapid 148 change. 150 Response time and consistency are not just "nice to have". On many 151 networks, the impact can be financial hardship or endanger human 152 life. In some cities, the emergency police contact system operates 153 over IP, law enforcement uses TCP/IP networks, transactions on our 154 stock exchanges are settled using IP networks. The critical nature 155 of such activities to our daily lives and financial well-being demand 156 a simple solution to support measurements. 158 1.3 Need for a Packet Sequence Number 160 While performing network diagnostics of an end-to-end connection, it 161 often becomes necessary to find the device along the network path 162 creating problems. Diagnostic data may be collected at multiple 163 places along the path (if possible), or at the source and 164 destination. Then, in post-collection processing, the diagnostic 165 data corresponding to each packet at different observation points 166 must be matched for proper measurements. A sequence number in each 167 packet provides sufficient basis for the matching process. If need 168 be, the timing fields may be used along with the sequence number to 169 ensure uniqueness. 171 This method of data collection along the path is of special use to 172 determine where packet loss or packet corruption is happening. 174 The packet sequence number needs to be unique in the context of the 175 session (5-tuple). See section 2 for a definition of 5-tuple. 177 1.4 Rationale for proposed solution 179 The current IPv6 specification does not provide timing nor a similar 180 field in the IPv6 main header or in any extension header. So, we 181 propose the IPv6 Performance and Diagnostic Metrics destination 182 option (PDM). 184 Advantages include: 186 1. Real measure of actual transactions. 187 2. Independence from transport layer protocols. 188 3. Ability to span organizational boundaries with consistent 189 instrumentation 190 4. No time synchronization needed between session partners 192 The PDM provides the ability to quickly determine if the (latency) 193 problem is in the network or in the server (application). More 194 intermediate measurements may be needed if the host or network 195 discrimination is not sufficient. At the client, TCP/IP stack time 196 vs. applications time may still need to be broken out by client 197 software. 199 1.5 PDM Works in Collaboration with Other Headers 201 The purpose of the PDM is not to supplant all the variables present 202 in all other headers but to provide data which is not available or 203 very difficult to get. The way PDM would be used is by a technician 204 (or tool) looking at a packet capture. Within the packet capture, 205 they would have available to them the layer 2 header, IP header (v6 206 or v4), TCP, UCP, ICMP, SCTP or other headers. All information 207 would be looked at together to make sense of the packet flow. The 208 technician or processing tool could analyze, report or ignore the 209 data from PDM, as necessary. 211 For an example of how PDM can help with TCP retransmit problems, 212 please look at section 8. 214 2 Measurement Information Derived from PDM 216 Each packet contains information about the sender and receiver. In IP 217 protocol, the identifying information is called a "5-tuple". 219 The 5-tuple consists of: 221 SADDR : IP address of the sender 222 SPORT : Port for sender 223 DADDR : IP address of the destination 224 DPORT : Port for destination 225 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.) 227 The PDM contains the following base fields: 229 PSNTP : Packet Sequence Number This Packet 230 PSNLR : Packet Sequence Number Last Received 231 DELTATLR : Delta Time Last Received 232 DELTATLS : Delta Time Last Sent 234 Other fields for scaling and time base are also in the PDM and will 235 be described in section 3. 237 This information, combined with the 5-tuple, allows the measurement 238 of the following metrics: 240 1. Round-trip delay 241 2. Server delay 243 2.1 Round-Trip Delay 245 Round-trip *Network* delay is the delay for packet transfer from a 246 source host to a destination host and then back to the source host. 247 This measurement has been defined, and the advantages and 248 disadvantages discussed in "A Round-trip Delay Metric for IPPM" 249 [RFC2681]. 251 2.2 Server Delay 253 Server delay is the interval between when a packet is received by a 254 device and the first corresponding packet is sent back in response. 255 This may be "Server Processing Time". It may also be a delay caused 256 by acknowledgements. Server processing time includes the time taken 257 by the combination of the stack and application to return the 258 response. The stack delay may be related to network performance. If 259 this aggregate time is seen as a problem, and there is a need to make 260 a clear distinction between application processing time and stack 261 delay, including that caused by the network, then more client based 262 measurements are needed. 264 3 Performance and Diagnostic Metrics Destination Option Layout 266 3.1 Destination Options Header 268 The IPv6 Destination Options Header is used to carry optional 269 information that need be examined only by a packet's destination 270 node(s). The Destination Options Header is identified by a Next 271 Header value of 60 in the immediately preceding header and is defined 272 in RFC2460 [RFC2460]. The IPv6 Performance and Diagnostic Metrics 273 Destination Option (PDM) is an implementation of the Destination 274 Options Header (Next Header value = 60). The PDM does not require 275 time synchronization. 277 3.2 Performance and Diagnostic Metrics Destination Option 279 The IPv6 Performance and Diagnostic Metrics Destination Option (PDM) 280 contains the following fields: 282 TIMEBASE : Base timer unit 283 SCALEDTLR: Scale for Delta Time Last Received 284 SCALEDTLS: Scale for Delta Time Last Sent 285 PSNTP : Packet Sequence Number This Packet 286 PSNLR : Packet Sequence Number Last Received 287 DELTATLR : Delta Time Last Received 288 DELTATLS : Delta Time Last Sent 290 The PDM destination option is encoded in type-length-value (TLV) 291 format as follows: 293 0 1 2 3 294 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 295 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 296 | Option Type | Option Length |TB |ScaleDTLR | ScaleDTLS | 297 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 298 | PSN This Packet | PSN Last Received | 299 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 300 | Delta Time Last Received | Delta Time Last Sent | 301 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 303 Option Type 305 TBD = 0xXX (TBD) [To be assigned by IANA] [RFC2780] 307 Option Length 309 8-bit unsigned integer. Length of the option, in octets, excluding 310 the Option Type and Option Length fields. This field MUST be set to 311 16. 313 Time Base 315 2-bit unsigned integer. It will indicate the lowest granularity 316 possible for this device. That is, for a value of 00 in the Time 317 Base field, a value of 1 in the DELTA fields indicates 1 318 microsecond. 320 This field is being included so that a device may choose the 321 granularity which most suits its timer ticks. That is, so that it 322 does not have to do more work than needed to convert values required 323 for the PDM. 325 The possible values of Time Base are as follows: 327 00 - milliseconds 328 01 - microseconds 329 10 - nanoseconds 330 11 - picoseconds 332 Scale Delta Time Last Received (SCALEDTLR) 334 7-bit signed integer. This is the scaling value for the Delta Time 335 Last Received (DELTATLR) field. The possible values are from -128 to 336 +127. See Section 4 for further discussion on Timing Considerations 337 and formatting of the scaling values. 339 Scale Delta Time Last Sent (SCALEDTLS) 341 7-bit signed integer. This is the scaling value for the Delta Time 342 Last Sent (DELTATLS) field. The possible values are from -128 to 343 +127. 345 Packet Sequence Number This Packet (PSNTP) 347 16-bit unsigned integer. This field will wrap. It is intended for 348 human use. That is, while to be used while analyzing packet traces. 350 Initialized at a random number and monotonically incremented for each 351 packet on the 5-tuple. The 5-tuple consists of the source and 352 destination IP addresses, the source and destination ports, and the 353 upper layer protocol (ex. TCP, ICMP, etc). The random number 354 initialization is to make it harder to spoof and insert such packets. 356 Operating systems MUST implement a separate packet sequence number 357 counter per 5-tuple. Operating systems MUST NOT implement a single 358 counter for all connections. 360 Packet Sequence Number Last Received (PSNLR) 362 16-bit unsigned integer. This is the PSN of the packet last received 363 on the 5-tuple. 365 Delta Time Last Received (DELTATLR) 367 A 16-bit unsigned integer field. The value is according to the scale 368 in SCALEDTLR. 370 DELTATLR = Send time packet 2 - Receive time packet 1 372 Delta TimeLast Sent (DELTATLS) 374 A 16-bit unsigned integer field. The value is according to the 375 scale in SCALEDTLS. 377 Delta Time Last Sent = Receive time packet 2 - Send time packet 1 379 Option Type 381 The two highest-order bits of the Option Type field are encoded to 382 indicate specific processing of the option; for the PDM destination 383 option, these two bits MUST be set to 00. This indicates the 384 following processing requirements: 386 00 - skip over this option and continue processing the header. 388 RFC2460 [RFC2460] defines other values for the Option Type field. 389 These MUST NOT be used in the PDM. The other values are as follows: 391 01 - discard the packet. 393 10 - discard the packet and, regardless of whether or not the 394 packet's Destination Address was a multicast address, send an ICMP 395 Parameter Problem, Code 2, message to the packet's Source Address, 396 pointing to the unrecognized Option Type. 398 11 - discard the packet and, only if the packet's Destination Address 399 was not a multicast address, send an ICMP Parameter Problem, Code 2, 400 message to the packet's Source Address, pointing to the unrecognized 401 Option Type. 403 In keeping with RFC2460 [RFC2460], the third-highest-order bit of the 404 Option Type specifies whether or not the Option Data of that option 405 can change en-route to the packet's final destination. 407 In the PDM, the value of the third-highest-order bit MUST be 0. The 408 possible values are as follows: 410 0 - Option Data does not change en-route 412 1 - Option Data may change en-route 414 The three high-order bits described above are to be treated as part 415 of the Option Type, not independent of the Option Type. That is, a 416 particular option is identified by a full 8-bit Option Type, not just 417 the low-order 5 bits of an Option Type. 419 3.3 Header Placement 421 The PDM destination option MUST be placed as follows: 423 - Before the upper-layer header. That is, this is the last 424 extension header. 426 This follows the order defined in RFC2460 [RFC2460] 428 IPv6 header 430 Hop-by-Hop Options header 432 Destination Options header 434 Routing header 436 Fragment header 438 Authentication header 440 Encapsulating Security Payload header 442 Destination Options header 444 upper-layer header 446 For each IPv6 packet header, the PDM MUST NOT appear more than once. 447 However, an encapsulated packet MAY contain a separate PDM associated 448 with each encapsulated IPv6 header. 450 3.4 Implementation Considerations 452 The PDM destination options extension header SHOULD be turned on by 453 each stack on a host node. It MAY also be turned on only in case of 454 diagnostics needed for problem resolution. 456 3.5 Dynamic Configuration Options 458 If implemented, each operating system MUST have a default 459 configuration parameter, e.g. diag_header_sys_default_value=yes/no. 460 The operating system MAY also have a dynamic configuration option to 461 change the configuration setting as needed. 463 If the PDM destination options extension header is used, then it MAY 464 be turned on for all packets flowing through the host, applied to an 465 upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP 466 address only. These are at the discretion of the implementation. 468 The PDM MUST NOT be changed dynamically via packet flow as this may 469 create potential security violation or DoS attack by numerous packets 470 turning the header on and off. 472 As with all other destination options extension headers, the PDM is 473 for destination nodes only. As specified above, intermediate devices 474 MUST neither set nor modify this field. 476 3.6 5-tuple Aging 478 Within the operating system, metrics must be kept on a 5-tuple basis. 480 The 5-tuple is: 482 SADDR : IP address of the sender SPORT : Port for sender DADDR : IP 483 address of the destination DPORT : Port for destination PROTC : 484 Protocol for upper layer (ex. TCP, UDP, ICMP) 486 The question comes of when to stop keeping data or restarting the 487 numbering for a 5-tuple. For example, in the case of TCP, at some 488 point, the connection will terminate. Keeping data in control blocks 489 forever, will have unfortunate consequences for the operating system. 491 So, the recommendation is to use a known aging parameter such as Max 492 Segment Lifetime (MSL) as defined in Transmission Control Protocol 493 [RFC0793] to reuse or drop the control block. The choice of aging 494 parameter is left up to the implementation. 496 4 Considerations of Timing Representation 498 4.1 Encoding the Delta-Time Values 500 This section makes reference to and expands on the document "Encoding 501 of Time Intervals for the TCP Timestamp Option" [TRAM-TCPM]. 503 4.2 Timer registers are different on different hardware 505 One of the problems with timestamp recording is the variety of 506 hardware that generates the time value to be used. Different CPUs 507 track the time in registers of different sizes, and the most- 508 frequently-iterated bit could be the first on the left or the first 509 on the right. In order to generate some examples here it is necessary 510 to indicate the type of timer register being used. 512 As described in the "IBM z/Architecture Principles of Operation" 513 [IBM-POPS], the Time-Of-Day clock in a zSeries CPU is a 104-bit 514 register, where bit 51 is incremented approximately every 515 microsecond: 517 1 518 0 1 2 3 4 5 6 0 519 +--------+---------+---------+---------+---------+---------+--+...+ 520 | | | | | |* | | 521 +--------+---------+---------+---------+---------+---------+--+...+ 522 ^ ^ ^ 523 0 51 = 1 usec 103 525 To represent these values concisely a hexadecimal representation will 526 be used, where each digit represents 4 binary bits. Thus: 528 0000 0000 0000 0001 = 1 timer unit (2**-12 usec, or about 244 psec) 529 0000 0000 0000 1000 = 1 microsecond 530 0000 0000 003E 8000 = 1 millisecond 531 0000 0000 F424 0000 = 1 second 532 0000 0039 3870 0000 = 1 minute 533 0000 0D69 3A40 0000 = 1 hour 534 0001 41DD 7600 0000 = 1 day 536 Note that only the first 64 bits of the register are commonly 537 represented, as that represents a count of timer units on this 538 hardware. Commonly the first 52 bits are all that are displayed, as 539 that represents a count of microseconds. 541 4.3 Timer Units on Other Systems 543 This encoding method works the same with other hardware clock 544 formats. The method uses a microsecond as the basic value and allows 545 for large time differentials. 547 4.4 Time Base 549 This specification allows for the fact that different CPU TOD clocks 550 use different binary points. For some clocks, a value of 1 could 551 indicate 1 microsecond, whereas other clocks could use the value 1 to 552 indicate 1 millisecond. In the former case, the binary digits to the 553 right of that binary point measure 2**(-n) microseconds, and in the 554 latter case, 2**(-n) milliseconds. 556 The Time Base allows us to ensure we have a common reference, at the 557 very least, common knowledge of what the binary point is for the 558 transmitted values. 560 We propose a base unit for the time. This is a 2-bit integer 561 indicating the lowest granularity possible for this device. That is, 562 for a value of 00 in the Time Base field, a value of 1 in the DELTA 563 fields indicates 1 picosecond. 565 The possible values of Time Base are as follows: 567 00 - milliseconds 568 01 - microseconds 569 10 - nanoseconds 570 11 - picoseconds 572 Time base is not necessarily equivalent to length of one timer tick. 573 That is, on many, if not all, systems, the timer tick value will not 574 be in complete units of nanoseconds, milliseconds, etc. For example, 575 on an IBM zSeries machine, one timer tick (or clock unit) is 2 to the 576 -12th microseconds. 578 Therefore, some amount of conversion may be needed to approximate 579 Time Base units. 581 4.5 Timer-value scaling 583 As discussed in [TRAM-TCPM] we propose storing not an entire time- 584 interval value, but just the most significant bits of that value, 585 along with a scaling factor to indicate the magnitude of the time- 586 interval value. In our case, we will use the high-order 16 bits. The 587 scaling value will be the number of bits in the timer register to the 588 right of the 16th significant bit. That is, if the timer register 589 contains this binary value: 591 1110100011010100101001010001000000000000 592 <-16 bits -><-24 bits -> 594 then, the values stored would be 1110 1000 1101 0100 in binary (E8D4 595 hexadecimal) for the time value and 24 for the scaling value. Note 596 that the displayed value is the binary equivalent of 1 second 597 expressed in picoseconds. 599 The below table represents a device which has a TimeBase of 600 picosecond (or 00). The smallest and simplest value to represent is 601 1 picosecond; the time value stored is 1, and the scaling value is 0. 602 Using values from the table below, we have: 604 Time value in Encoded Scaling 605 Delta time picoseconds value decimal 606 -------------------------------------------------------- 607 1 picosecond 1 1 0 608 1 nanosecond 3E8 3E8 0 609 1 microsecond F4240 F424 4 610 1 millisecond 3B9ACA00 3B9A 16 611 1 second E8D4A51000 E8D4 24 612 1 minute 3691D6AFC000 3691 32 613 1 hour cca2e51310000 CCA2 36 614 1 day 132f4579c980000 132F 44 615 365 days 1b5a660ea44b80000 1B5A 52 617 Sample binary values (high order 16 bits taken) 619 1 psec 1 0001 620 1 nsec 3E8 0011 1110 1000 621 1 usec F4240 1111 0100 0010 0100 0000 622 1 msec 3B9ACA00 0011 1011 1001 1010 1100 1010 0000 0000 623 1 sec E8D4A51000 1110 1000 1101 0100 1010 0101 0001 0000 0000 0000 625 4.6 Limitations with this encoding method 627 If we follow the specification in [TRAM-TCPM], the size of one of 628 these time-interval fields is limited to this 11-bit value and five- 629 bit scale, so that they fit into a 16-bit space. With that 630 limitation, the maximum value that could be stored in 16 bits is: 632 11-bit value Scale 633 ============= ====== 634 1111 1111 111 1 1111 636 or an encoded value of 3FF and a scale value of 31. This value 637 corresponds to any time differential between: 639 || 640 11 1111 1111 1000 0000 0000 0000 0000 0000 0000 0000 (binary) 641 3 F F 8 0 0 0 0 0 0 0 (hexadecimal) 643 and 645 11 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 (binary) 646 3 F F F F F F F F F F (hexadecimal) 648 This time value, 3FFFFFFFFFF, converts to 50 days, 21 hours, 40 649 minutes and 46.511103 seconds. A time differential 1 microsecond 650 longer won't fit into 16 bits using this encoding method. 652 4.7 Lack of precision induced by timer value truncation 654 When the bit values following the first 11 significant bits are 655 truncated, obviously loss of precision in the value. The range of 656 values that will be truncated to the same encoded value is 657 2**(Scale)-1 microseconds. 659 The smallest time differential value that will be truncated is 661 1000 0000 0000 = 2.048 msec 663 The value 665 1000 0000 0001 = 2.049 msec 667 will be truncated to the same encoded value, which is 400 in hex, 668 with a scale value of 1. With the scale value of 1, the value range 669 is calculated as 2**1 - 1, or 1 usec, which you can see is the 670 difference between these minimum and maximum values. 672 With that in mind, let's look at that table of delta time values 673 again, where the Precision is the range from the smallest value 674 corresponding to this encoded value to the largest: 676 Time value in Encoded 677 Delta time microseconds value Scale Precision 678 1 microsecond 1 1 0 0:00.000000 679 1 millisecond 38E 38E 0 0:00.000000 680 1 second F4240 7A1 9 0:00.000511 681 1 minute 3938700 727 15 0:00.032767 682 1 hour D693A400 6B4 21 0:02.097151 683 1 day 141DD76000 507 26 1:07.108863 684 Maximum value 3FFFFFFFFFF 7FF 31 35:47.483647 686 So, when measuring the delay between transmission of two packets, or 687 between the reception of two packets, any delay shorter than 50 days 688 21 hours and change can be stored in this encoded fashion within 16 689 bits. When you encode, for example, a DTN response time delay of 50 690 days, 21 hours and 40 minutes, you can be assured of accuracy within 691 35 minutes. 693 5 PDM Flow - Simple Client Server 695 Following is a sample simple flow for the PDM with one packet sent 696 from Host A and one packet received by Host B. The PDM does not 697 require time synchronization between Host A and Host B. The 698 calculations to derive meaningful metrics for network diagnostics are 699 shown below each packet sent or received. 701 Each packet, in addition to the PDM contains information on the 702 sender and receiver. As discussed before, a 5-tuple consists of: 704 SADDR : IP address of the sender 705 SPORT : Port for sender 706 DADDR : IP address of the destination 707 DPORT : Port for destination 708 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP) 710 It should be understood that the packet identification information is 711 in each packet. We will not repeat that in each of the following 712 steps. 714 5.1 Step 1 716 Packet 1 is sent from Host A to Host B. The time for Host A is set 717 initially to 10:00AM. 719 The time and packet sequence number are saved by the sender 720 internally. The packet sequence number and delta times are sent in 721 the packet. 723 Packet 1 725 +----------+ +----------+ 726 | | | | 727 | Host | ----------> | Host | 728 | A | | B | 729 | | | | 730 +----------+ +----------+ 732 PDM Contents: 734 PSNTP : Packet Sequence Number This Packet: 25 735 PSNLR : Packet Sequence Number Last Received: - 736 DELTATLR : Delta Time Last Received: - 737 SCALEDTLR: Scale of Delta Time Last Received: 0 738 DELTATLS : Delta Time Last Sent: - 739 SCALEDTLS: Scale of Delta Time Last Sent: 0 740 TIMEBASE : Granularity of Time: 00 (Milliseconds) 742 Internally, within the sender, Host A, it must keep: 744 Packet Sequence Number of the last packet sent: 25 745 Time the last packet was sent: 10:00:00 747 Note, the initial PSNTP from Host A starts at a random number. In 748 this case, 25. The time in these examples is shown in seconds for 749 the sake of simplicity. 751 5.2 Step 2 753 Packet 1 is received at Host B. Its time is set to one hour later 754 than Host A. In this case, 11:00AM 756 Internally, within the receiver, Host B, it must note: 758 Packet Sequence Number of the last packet received: 25 759 Time the last packet was received : 11:00:03 761 Note, this timestamp is in Host B time. It has nothing whatsoever to 762 do with Host A time. The Packet Sequence Number of the last packet 763 received will become PSNLR which will be sent out in the packet sent 764 by Host B in the next step. The time last received will be used to 765 calculate the DELTALR value to be sent out in the packet sent by Host 766 B in the next step. 768 5.3 Step 3 770 Packet 2 is sent by Host B to Host A. Note, the initial packet 771 sequence number (PSNTP) from Host B starts at a random number. In 772 this case, 12. Before sending the packet, Host B does a calculation 773 of deltas. Since Host B knows when it is sending the packet, and it 774 knows when it received the previous packet, it can do the following 775 calculation: 777 Sending time (packet 2) - receive time (packet 1) 779 We will call the result of this calculation: Delta Time Last 780 Received 782 That is: 784 DELTATLR = Sending time (packet 2) - receive time (packet 1) 786 Note, both sending time and receive time are saved internally in Host 787 B. They do not travel in the packet. Only the Delta is in the 788 packet. 790 Assume that within Host B is the following: 792 Packet Sequence Number of the last packet received: 25 793 Time the last packet was received: 11:00:03 794 Packet Sequence Number of this packet: 12 795 Time this packet is being sent: 11:00:07 797 We can now calculate a delta value to be sent out in the packet. 798 DELTATLR becomes: 800 4 seconds = 11:00:07 - 11:00:03 802 This is the derived metric: Server Delay. The time and scaling 803 factor must be calculated. Then, this value, along with the packet 804 sequence numbers will be sent to Host A as follows: 806 Packet 2 808 +----------+ +----------+ 809 | | | | 810 | Host | <---------- | Host | 811 | A | | B | 812 | | | | 813 +----------+ +----------+ 815 PDM Contents: 817 PSNTP : Packet Sequence Number This Packet: 12 818 PSNLR : Packet Sequence Number Last Received: 25 819 DELTATLR : Delta Time Last Received: 3A35 (4 seconds) 820 SCALEDTLR: Scale of Delta Time Last Received: 25 821 DELTATLS : Delta Time Last Sent: - 822 SCALEDTLS: Scale of Delta Time Last Sent: 0 823 TIMEBASE : Granularity of Time: 00 (Milliseconds) 825 The metric left to be calculated is the Round-Trip Delay. This will 826 be calculated by Host A when it receives Packet 2. 828 5.4 Step 4 830 Packet 2 is received at Host A. Remember, its time is set to one 831 hour earlier than Host B. Internally, it must note: 833 Packet Sequence Number of the last packet received: 12 834 Time the last packet was received : 10:00:12 836 Note, this timestamp is in Host A time. It has nothing whatsoever to 837 do with Host B time. 839 So, now, Host A can calculate total end-to-end time. That is: 841 End-to-End Time = Time Last Received - Time Last Sent 843 For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was 844 received by Host A at 10:00:12 so: 846 End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT 847 delay combined). This time may also be called total Overall Round- 848 trip time (which includes Network RTT and Host Response Time). 850 This derived metric we will call DELTATLS or Delta Time Last Sent. 852 We can now also calculate round trip delay. The formula is: 854 Round trip delay = DELTATLS - DELTATLR 856 Or: 858 Round trip delay = 12 - 4 or 8 860 Now, the only problem is that at this point all metrics are in Host A 861 only and not exposed in a packet. To do that, we need a third packet. 863 Note: this simple example assumes one send and one receive. That 864 is done only for purposes of explaining the function of the PDM. In 865 cases where there are multiple packets returned, one would take the 866 time in the last packet in the sequence. The calculations of such 867 timings and intelligent processing is the function of post-processing 868 of the data. 870 5.5 Step 5 872 Packet 3 is sent from Host A to Host B. 874 +----------+ +----------+ 875 | | | | 876 | Host | ----------> | Host | 877 | A | | B | 878 | | | | 879 +----------+ +----------+ 881 PDM Contents: 883 PSNTP : Packet Sequence Number This Packet: 26 884 PSNLR : Packet Sequence Number Last Received: 12 885 DELTATLR : Delta Time Last Received: 0 886 SCALEDTLS: Scale of Delta Time Last Received 0 887 DELTATLS : Delta Time Last Sent: 105e (12 seconds) 888 SCALEDTLR: Scale of Delta Time Last Received: 26 889 TIMEBASE : Granularity of Time: 00 (Milliseconds) 891 To calculate Two-Way Delay, any packet capture device may look at 892 these packets and do what is necessary. 894 6 Other Flows 896 What we have discussed so far is a simple flow with one packet sent 897 and one returned. Let's look at how PDM may be useful in other 898 types of flows. 900 6.1 PDM Flow - One Way Traffic 902 The flow on a particular session may not be a send-receive paradigm. 903 Let us consider some other situations. In the case of a one-way 904 flow, one might see the following: 906 Packet Sender PSN PSN Delta Time Delta Time 907 This Packet Last Recvd Last Recvd Last Sent 908 ===================================================================== 909 1 Server 1 0 0 0 910 2 Server 2 0 0 5 911 3 Server 3 0 0 12 912 4 Server 4 0 0 20 914 What does this mean and how is it useful? 916 In a one-way flow, only the Delta Time Last Sent will be seen as 917 used. Recall, Delta Time Last Sent is the difference between the 918 send of one packet from a device and the next. This is a measure of 919 throughput for the sender - according to the sender's point of view. 920 That is, it is a measure of how fast is the application itself (with 921 stack time included) able to send packets. 923 How might this be useful? If one is having a performance issue at 924 the client and sees that packet 2, for example, is sent after 5 925 microseconds from the server but takes 3 minutes to arrive at the 926 destination, then one may safely conclude that there are delays in 927 the path other than at the server which may be causing the delivery 928 issue of that packet. Such delays may include the network links, 929 middle-boxes, etc. 931 Now, true one-way traffic is quite rare. What people often mean by 932 "one-way" traffic is an application such as FTP where a group of 933 packets (for example, a TCP window size worth) is sent, then the 934 sender waits for acknowledgment. This type of flow would actually 935 fall into the "multiple-send" traffic model. 937 6.2 PDM Flow - Multiple Send Traffic 939 Assume that two packets are sent for each ACK from the server. 941 Packet Sender PSN PSN Delta Time Delta Time 942 This Packet Last Recvd Last Recvd Last Sent 943 ===================================================================== 944 1 Server 1 0 0 0 945 2 Server 2 0 0 5 946 3 Client 1 2 20 0 947 4 Server 3 1 10 15 949 How might this be used? 951 Notice that in packet 3, the client has a value of Delta Time Last 952 received of 20. Recall that Delta Time Last Received is the Send 953 time of packet 3 - receive time of packet 2. So, what does one know 954 now? In this case, Delta Time Last Received is the processing time 955 for the Client to send the next packet. 957 How to interpret this depends on what is actually being sent. 958 Remember, PDM is not being used in isolation, but to supplement the 959 fields found in other headers. Let's take some examples: 961 1. Client is sending a standalone TCP ACK. One would find this by 962 looking at the payload length in the IPv6 header and the TCP 963 Acknowledgement field in the TCP header. So, in this case, the 964 client is taking 20 units to send back the ACK. This may or may not 965 be interesting. 967 2. Client is sending data with the packet. Again, one would find 968 this by looking at the payload length in the IPv6 header and the TCP 969 Acknowledgement field in the TCP header. So, in this case, the 970 client is taking 20 units to send back data. This may represent 971 "User Think Time". Again, this may or may not be interesting, in 972 isolation. But, if there is a performance problem receiving data at 973 the server, then taken in conjunction with RTT or other packet timing 974 information, this information may be quite interesting. 976 Of course, one also needs to look at the PSN Last Received field to 977 make sure of the interpretation of this data. That is, to make 978 sure that the Delta Last Received corresponds to the packet of 979 interest. 981 The benefits of PDM are that we have such information available in a 982 uniform manner for all applications and all protocols without 983 extensive changes required to applications. 985 6.3 PDM Flow - Multiple Send with Errors 987 One might wonder if all of the functions of PDM might be better 988 suited to TCP or a TCP option. Let us take the case of how PDM may 989 help in a case of TCP retransmissions in a way that TCP options or 990 TCP ACK / SEQ would not. 992 Assume that three packets are sent with each send from the server. 994 From the server, this is what is seen. 996 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 997 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 998 ===================================================================== 999 1 Server 1 0 0 0 123 100 1000 2 Server 2 0 0 5 223 100 1001 3 Server 3 0 0 5 333 100 1003 The client however, does not get all the packets. From the client, 1004 this is what is seen for the packets sent from the server. 1006 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1007 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1008 ===================================================================== 1009 1 Server 1 0 0 0 123 100 1010 2 Server 3 0 0 5 333 100 1012 Let's assume that the server now retransmits the packet. 1013 (Obviously, a duplicate acknowledgment sequence for fast retransmit 1014 or a retransmit timeout would occur. To illustrate the point, these 1015 packets are being left out.) 1017 So, then if a TCP retransmission is done, then from the client, this 1018 is what is seen for the packets sent from the server. 1020 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1021 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1022 ===================================================================== 1023 1 Server 4 0 0 30 223 100 1025 The server has resent the old packet 2 with TCP sequence number of 1026 223. The retransmitted packet now has a PSN This Packet value of 4. 1027 The Delta Last Sent is 30 - the time between sending the packet with 1028 PSN of 3 and this current packet. 1030 Let's say that packet 4 STILL does not make it. Then, after some 1031 amount of time (RTO) then the packet with TCP sequence number of 223 1032 is resent. 1034 From the client, this is what is seen for the packets sent from the 1035 server. 1037 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1038 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1039 ===================================================================== 1040 1 Server 5 0 0 60 223 100 1042 If now, this packet makes it, one has a very good idea that packets 1043 exist which are being sent from the server as retransmissions and not 1044 making it to the client. This is because the PSN of the resent 1045 packet from the server is 5 rather than 4. If we had used TCP 1046 sequence number alone, we would never have seen this situation. 1047 Because the TCP sequence number in all situations is 223. 1049 This situation would be experienced by the user of the application 1050 (the human being actually sitting somewhere) as a "hangs" or long 1051 delay between packets. On large networks, to diagnose problems such 1052 as these where packets are lost somewhere on the network, one has to 1053 take multiple traces to find out exactly where. 1055 The first thing is to start with doing a trace at the client and the 1056 server. So, we can see if the server sent a particular packet and 1057 the client received it. If the client did not receive it, then we 1058 start tracking back to trace points at the router right after the 1059 server and the router right before the client. Did they get these 1060 packets which the server has sent? This is a time consuming 1061 activity. 1063 With PDM, we can speed up the diagnostic time because we may be able 1064 to use only the trace taken at the client to see what the server is 1065 sending. 1067 7 Potential Overhead Considerations 1069 Questions have been posed as to the potential overhead of PDM. 1070 First, PDM is entirely optional. That is, a site may choose to 1071 implement PDM or not as they wish. If they are happy with the costs 1072 of PDM vs. the benefits, then the choice should be theirs. 1074 Below is a table outlining the potential overhead in terms of 1075 additional time to deliver the response to the end user for various 1076 assumed RTTs. 1078 Bytes RTT Bytes Bytes New Overhead 1079 in Packet Per Milli in PDM RTT 1080 ===================================================================== 1081 1000 1000 milli 1 16 1016.000 16.000 milli 1082 1000 100 milli 10 16 101.600 1.600 milli 1083 1000 10 milli 100 16 10.160 .160 milli 1084 1000 1 milli 1000 16 1.016 .016 milli 1085 Below are some examples of actual RTTs for packets traversing large 1086 enterprise networks. The first example is for packets going to 1087 multiple business partners. 1089 Bytes RTT Bytes Bytes New Overhead 1090 in Packet Per Milli in PDM RTT 1091 ===================================================================== 1092 1000 17 milli 58 16 17.360 .360 milli 1094 The second example is for packets at a large enterprise customer 1095 within a data center. Notice that the scale is now in microseconds 1096 rather than milliseconds. 1098 Bytes RTT Bytes Bytes New Overhead 1099 in Packet Per Micro in PDM RTT 1100 ===================================================================== 1101 1000 20 micro 50 16 20.320 .320 micro 1103 8 Security Considerations 1105 The PDM MUST NOT be changed dynamically via packet flow as this 1106 creates a possibility for potential security violations or DoS 1107 attacks by numerous packets turning the header on and off. 1109 Attackers may also send many packets from multiple ports, for example 1110 by doing a port scan. This will cause the stack to create many 1111 control blocks. This is the same problem as seen for SYN flood 1112 attacks. Similar protections should be implemented by the stack to 1113 preserve the integrity of memory. 1115 9 IANA Considerations 1117 Option Type to be assigned by IANA [RFC2780]. 1119 10 References 1121 10.1 Normative References 1123 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1124 793, September 1981. 1126 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1127 Requirement Levels", BCP 14, RFC 2119, March 1997. 1129 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1130 (IPv6) Specification", RFC 2460, December 1998. 1132 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1133 Delay Metric for IPPM", RFC 2681, September 1999. 1135 [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For 1136 Values In the Internet Protocol and Related Headers", BCP 37, RFC 1137 2780, March 2000. 1139 10.2 Informative References 1141 [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP 1142 Timestamp Option-01", Internet Draft, July 2013. [Work in Progress] 1144 [IBM-POPS] IBM Corporation, "IBM z/Architecture Principles of 1145 Operation", SA22-7832, 1990-2012 1147 11 Acknowledgments 1149 The authors would like to thank Keven Haining, Al Morton, Brian 1150 Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick 1151 Troth for their comments and assistance. 1153 Authors' Addresses 1155 Nalini Elkins 1156 Inside Products, Inc. 1157 36A Upper Circle 1158 Carmel Valley, CA 93924 1159 United States 1160 Phone: +1 831 659 8360 1161 Email: nalini.elkins@insidethestack.com 1162 http://www.insidethestack.com 1164 Robert Hamilton 1165 Chemical Abstracts Service 1166 A Division of the American Chemical Society 1167 2540 Olentangy River Road 1168 Columbus, Ohio 43202 1169 United States 1170 Phone: +1 614 447 3600 x2517 1171 Email: rhamilton@cas.org 1172 http://www.cas.org 1174 Michael S. Ackermann 1175 Blue Cross Blue Shield of Michigan 1176 P.O. Box 2888 1177 Detroit, Michigan 48231 1178 United States 1179 Phone: +1 310 460 4080 1180 Email: mackermann@bcbsmi.com 1181 http://www.bcbsmi.com