idnits 2.17.1 draft-ietf-ippm-6man-pdm-option-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 7, 2016) is 2880 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT N. Elkins 3 Inside Products 4 R. Hamilton 5 Chemical Abstracts Service 6 M. Ackermann 7 Intended Status: Proposed Standard BCBS Michigan 8 Expires: December 9, 2016 June 7, 2016 10 IPv6 Performance and Diagnostic Metrics (PDM) Destination Option 11 draft-ietf-ippm-6man-pdm-option-03 13 Table of Contents 15 1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 4 16 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4 17 1.2 End User Quality of Service (QoS) . . . . . . . . . . . . . 4 18 1.3 Need for a Packet Sequence Number . . . . . . . . . . . . . 5 19 1.4 Rationale for proposed solution . . . . . . . . . . . . . . 5 20 1.5 PDM Works in Collaboration with Other Headers . . . . . . . 6 21 1.6 IPv6 Transition Technologies . . . . . . . . . . . . . . . . 6 22 2 Measurement Information Derived from PDM . . . . . . . . . . . . 6 23 2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . . 7 24 2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . . 7 25 3 Performance and Diagnostic Metrics Destination Option Layout . . 7 26 3.1 Destination Options Header . . . . . . . . . . . . . . . . . 7 27 3.2 Performance and Diagnostic Metrics Destination Option . . . 7 28 3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 10 29 3.4 Header Placement Using IPSec ESP Mode . . . . . . . . . . . 11 30 3.5 Implementation Considerations . . . . . . . . . . . . . . . 11 31 3.6 Dynamic Configuration Options . . . . . . . . . . . . . . . 12 32 3.6 5-tuple Aging . . . . . . . . . . . . . . . . . . . . . . . 12 33 4 Considerations of Timing Representation . . . . . . . . . . . . 12 34 4.1 Encoding the Delta-Time Values . . . . . . . . . . . . . . . 12 35 4.2 Timer registers are different on different hardware . . . . 13 36 4.3 Timer Units on Other Systems . . . . . . . . . . . . . . . . 13 37 4.4 Time Base . . . . . . . . . . . . . . . . . . . . . . . . . 14 38 4.5 Timer-value scaling . . . . . . . . . . . . . . . . . . . . 14 39 4.6 Limitations with this encoding method . . . . . . . . . . . 15 40 4.7 Lack of precision induced by timer value truncation . . . . 16 41 5 PDM Flow - Simple Client Server . . . . . . . . . . . . . . . . 17 42 5.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 43 5.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 44 5.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 45 5.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 46 5.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 48 6 Other Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 21 49 6.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . . . 21 50 6.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . . . 22 51 6.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . . . 23 52 7 Potential Overhead Considerations . . . . . . . . . . . . . . . 25 53 8 Security Considerations . . . . . . . . . . . . . . . . . . . . 26 54 9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 26 55 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 27 56 10.1 Normative References . . . . . . . . . . . . . . . . . . . 27 57 10.2 Informative References . . . . . . . . . . . . . . . . . . 27 58 11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 27 59 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 28 61 Abstract 63 To assess performance problems, measurements based on optional 64 sequence numbers and timing may be embedded in each packet. Such 65 measurements may be interpreted in real-time or after the fact. An 66 implementation of the existing IPv6 Destination Options extension 67 header, the Performance and Diagnostic Metrics (PDM) Destination 68 Options extension header as well as the field limits, calculations, 69 and usage of the PDM in measurement are included in this document. 71 Status of this Memo 73 This Internet-Draft is submitted to IETF in full conformance with the 74 provisions of BCP 78 and BCP 79. 76 Internet-Drafts are working documents of the Internet Engineering 77 Task Force (IETF), its areas, and its working groups. Note that 78 other groups may also distribute working documents as 79 Internet-Drafts. 81 Internet-Drafts are draft documents valid for a maximum of six months 82 and may be updated, replaced, or obsoleted by other documents at any 83 time. It is inappropriate to use Internet-Drafts as reference 84 material or to cite them other than as "work in progress." 86 The list of current Internet-Drafts can be accessed at 87 http://www.ietf.org/1id-abstracts.html 89 The list of Internet-Draft Shadow Directories can be accessed at 90 http://www.ietf.org/shadow.html 92 Copyright and License Notice 94 Copyright (c) 2016 IETF Trust and the persons identified as the 95 document authors. All rights reserved. 97 IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 98 3: This document is subject to BCP 78 and the IETF Trust's Legal 99 Provisions Relating to IETF Documents 100 (http://trustee.ietf.org/license-info) in effect on the date of 101 publication of this document. Please review these documents 102 carefully, as they describe your rights and restrictions with respect 103 to this document. Code Components extracted from this document must 104 include Simplified BSD License text as described in Section 4.e of 105 the Trust Legal Provisions and are provided without warranty as 106 described in the Simplified BSD License. 108 1 Background 110 To assess performance problems, measurements based on optional 111 sequence numbers and timing may be embedded in each packet. Such 112 measurements may be interpreted in real-time or after the fact. An 113 implementation of the existing IPv6 Destination Options extension 114 header, the Performance and Diagnostic Metrics (PDM) Destination 115 Options extension header has been proposed in a companion document. 116 This document specifies the layout, field limits, calculations, and 117 usage of the PDM in measurement. 119 As defined in RFC2460 [RFC2460], destination options are carried by 120 the IPv6 Destination Options extension header. Destination options 121 include optional information that need be examined only by the IPv6 122 node given as the destination address in the IPv6 header, not by 123 routers or other "middle boxes". This document specifies a new 124 destination option, the Performance and Diagnostic Metrics (PDM) 125 destination option. 127 1.1 Terminology 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in RFC 2119 [RFC2119]. 133 1.2 End User Quality of Service (QoS) 135 The timing values in the PDM embedded in the packet will be used to 136 estimate QoS as experienced by an end user device. 138 For many applications, the key user performance indicator is response 139 time. When the end user is an individual, he is generally 140 indifferent to what is happening along the network; what he really 141 cares about is how long it takes to get a response back. But this is 142 not just a matter of individuals' personal convenience. In many 143 cases, rapid response is critical to the business being conducted. 145 When the end user is a device (e.g. with the Internet of Things), 146 what matters is the speed with which requested data can be 147 transferred -- specifically, whether the requested data can be 148 transferred in time to accomplish the desired actions. This can be 149 important when the relevant external conditions are subject to rapid 150 change. 152 Low, reliable and acceptable responses times are not just "nice to 153 have". On many networks, the impact can be financial hardship or 154 endanger human life. In some cities, the emergency police contact 155 system operates over IP, law enforcement uses TCP/IP networks, 156 transactions on our stock exchanges are settled using IP networks. 157 The critical nature of such activities to our daily lives and 158 financial well-being demand a simple solution to support response 159 time measurements. 161 1.3 Need for a Packet Sequence Number 163 While performing network diagnostics of an end-to-end connection, it 164 often becomes necessary to isolate the factors along the network path 165 responsible for problems. Diagnostic data may be collected at 166 multiple places along the path (if possible), or at the source and 167 destination. Then, in post-collection processing, the diagnostic 168 data corresponding to each packet at different observation points 169 must be matched for proper measurements. A sequence number in each 170 packet provides sufficient basis for the matching process. If need 171 be, the timing fields may be used along with the sequence number to 172 ensure uniqueness. 174 This method of data collection along the path is of special use to 175 determine where packet loss or packet corruption is happening. 177 The packet sequence number needs to be unique in the context of the 178 session (5-tuple). See section 2 for a definition of 5-tuple. 180 1.4 Rationale for proposed solution 182 The current IPv6 specification does not provide timing nor a similar 183 field in the IPv6 main header or in any extension header. So, we 184 propose the IPv6 Performance and Diagnostic Metrics destination 185 option (PDM). 187 Advantages include: 189 1. Real measure of actual transactions. 190 2. Independence from transport layer protocols. 191 3. Ability to span organizational boundaries with consistent 192 instrumentation 193 4. No time synchronization needed between session partners 195 The PDM provides the ability to quickly determine if the (latency) 196 problem is in the network or in the server (application). More 197 intermediate measurements may be needed if the host or network 198 discrimination is not sufficient. At the client, TCP/IP stack time 199 vs. applications time may still need to be broken out by client 200 software. 202 1.5 PDM Works in Collaboration with Other Headers 204 The purpose of the PDM is not to supplant all the variables present 205 in all other headers but to provide data which is not available or 206 very difficult to get. The way PDM would be used is by a technician 207 (or tool) looking at a packet capture. Within the packet capture, 208 they would have available to them the layer 2 header, IP header (v6 209 or v4), TCP, UCP, ICMP, SCTP or other headers. All information 210 would be looked at together to make sense of the packet flow. The 211 technician or processing tool could analyze, report or ignore the 212 data from PDM, as necessary. 214 For an example of how PDM can help with TCP retransmit problems, 215 please look at section 8. 217 1.6 IPv6 Transition Technologies 219 In the path to full implementation of IPv6, transition technologies 220 such as translation or tunneling may be employed. The PDM header is 221 not expected to work in such scenarios. It is likely that an IPv6 222 packet containing PDM will be dropped if using IPv6 transition 223 technologies. 225 2 Measurement Information Derived from PDM 227 Each packet contains information about the sender and receiver. In IP 228 protocol, the identifying information is called a "5-tuple". 230 The 5-tuple consists of: 232 SADDR : IP address of the sender 233 SPORT : Port for sender 234 DADDR : IP address of the destination 235 DPORT : Port for destination 236 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.) 238 The PDM contains the following base fields: 240 PSNTP : Packet Sequence Number This Packet 241 PSNLR : Packet Sequence Number Last Received 242 DELTATLR : Delta Time Last Received 243 DELTATLS : Delta Time Last Sent 245 Other fields for scaling and time base are also in the PDM and will 246 be described in section 3. 248 This information, combined with the 5-tuple, allows the measurement 249 of the following metrics: 251 1. Round-trip delay 252 2. Server delay 254 2.1 Round-Trip Delay 256 Round-trip *Network* delay is the delay for packet transfer from a 257 source host to a destination host and then back to the source host. 258 This measurement has been defined, and the advantages and 259 disadvantages discussed in "A Round-trip Delay Metric for IPPM" 260 [RFC2681]. 262 2.2 Server Delay 264 Server delay is the interval between when a packet is received by a 265 device and the first corresponding packet is sent back in response. 266 This may be "Server Processing Time". It may also be a delay caused 267 by acknowledgements. Server processing time includes the time taken 268 by the combination of the stack and application to return the 269 response. The stack delay may be related to network performance. If 270 this aggregate time is seen as a problem, and there is a need to make 271 a clear distinction between application processing time and stack 272 delay, including that caused by the network, then more client based 273 measurements are needed. 275 3 Performance and Diagnostic Metrics Destination Option Layout 277 3.1 Destination Options Header 279 The IPv6 Destination Options Header is used to carry optional 280 information that needs to be examined only by a packet's destination 281 node(s). The Destination Options Header is identified by a Next 282 Header value of 60 in the immediately preceding header and is defined 283 in RFC2460 [RFC2460]. The IPv6 Performance and Diagnostic Metrics 284 Destination Option (PDM) is an implementation of the Destination 285 Options Header. The PDM does not require time synchronization. 287 3.2 Performance and Diagnostic Metrics Destination Option 289 The IPv6 Performance and Diagnostic Metrics Destination Option (PDM) 290 contains the following fields: 292 TIMEBASE : Base timer unit 293 SCALEDTLR: Scale for Delta Time Last Received 294 SCALEDTLS: Scale for Delta Time Last Sent 295 PSNTP : Packet Sequence Number This Packet 296 PSNLR : Packet Sequence Number Last Received 297 DELTATLR : Delta Time Last Received 298 DELTATLS : Delta Time Last Sent 300 The PDM destination option is encoded in type-length-value (TLV) 301 format as follows: 303 0 1 2 3 304 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 305 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 306 | Option Type | Option Length |TB |ScaleDTLR | ScaleDTLS | 307 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 308 | PSN This Packet | PSN Last Received | 309 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 310 | Delta Time Last Received | Delta Time Last Sent | 311 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 313 Option Type 315 TBD = 0xXX (TBD) [To be assigned by IANA] [RFC2780] 317 Option Length 319 8-bit unsigned integer. Length of the option, in octets, excluding 320 the Option Type and Option Length fields. This field MUST be set to 321 16. 323 Time Base 325 2-bit unsigned integer. It will indicate the lowest granularity 326 possible for this device. That is, for a value of 00 in the Time 327 Base field, a value of 1 in the DELTA fields indicates 1 328 microsecond. 330 This field is being included so that a device may choose the 331 granularity which most suits its timer ticks. That is, so that it 332 does not have to do more work than needed to convert values required 333 for the PDM. 335 The possible values of Time Base are as follows: 337 00 - milliseconds 338 01 - microseconds 339 10 - nanoseconds 340 11 - picoseconds 342 Scale Delta Time Last Received (SCALEDTLR) 344 7-bit signed integer. This is the scaling value for the Delta Time 345 Last Received (DELTATLR) field. The possible values are from -128 to 346 +127. See Section 4 for further discussion on Timing Considerations 347 and formatting of the scaling values. 349 Scale Delta Time Last Sent (SCALEDTLS) 351 7-bit signed integer. This is the scaling value for the Delta Time 352 Last Sent (DELTATLS) field. The possible values are from -128 to 353 +127. 355 Packet Sequence Number This Packet (PSNTP) 357 16-bit unsigned integer. This field will wrap. It is intended for 358 use while analyzing packet traces. 360 Initialized at a random number and incremented monotonically for each 361 packet of the session flow of the 5-tuple. The 5-tuple consists of 362 the source and destination IP addresses, the source and destination 363 ports, and the upper layer protocol (ex. TCP, ICMP, etc). The 364 random number initialization is intended to make it harder to spoof 365 and insert such packets. 367 Operating systems MUST implement a separate packet sequence number 368 counter per 5-tuple. Operating systems MUST NOT implement a single 369 counter for all connections. 371 Packet Sequence Number Last Received (PSNLR) 373 16-bit unsigned integer. This is the PSNTP of the packet last 374 received on the 5-tuple. 376 Delta Time Last Received (DELTATLR) 378 A 16-bit unsigned integer field. The value is set according to the 379 scale in SCALEDTLR. 381 DELTATLR = Send time packet 2 - Receive time packet 1 383 Delta TimeLast Sent (DELTATLS) 385 A 16-bit unsigned integer field. The value is set according to the 386 scale in SCALEDTLS. 388 Delta Time Last Sent = Receive time packet 2 - Send time packet 1 390 Option Type 392 The two highest-order bits of the Option Type field are encoded to 393 indicate specific processing of the option; for the PDM destination 394 option, these two bits MUST be set to 00. This indicates the 395 following processing requirements: 397 00 - skip over this option and continue processing the header. 399 RFC2460 [RFC2460] defines other values for the Option Type field. 400 These MUST NOT be used in the PDM. 402 In keeping with RFC2460 [RFC2460], the third-highest-order bit of the 403 Option Type specifies whether or not the Option Data of that option 404 can change en-route to the packet's final destination. 406 In the PDM, the value of the third-highest-order bit MUST be 0. The 407 possible values are as follows: 409 0 - Option Data does not change en-route 411 1 - Option Data may change en-route 413 The three high-order bits described above are to be treated as part 414 of the Option Type, not independent of the Option Type. That is, a 415 particular option is identified by a full 8-bit Option Type, not just 416 the low-order 5 bits of an Option Type. 418 3.3 Header Placement 420 The PDM destination option MUST be placed as follows: 422 - Before the upper-layer header or the ESP header. 424 This follows the order defined in RFC2460 [RFC2460] 426 IPv6 header 428 Hop-by-Hop Options header 430 Destination Options header <-------- 432 Routing header 434 Fragment header 435 Authentication header 437 Encapsulating Security Payload header 439 Destination Options header <------------ 441 upper-layer header 443 Note that there is a choice of where to place the Destination Options 444 header. If using ESP mode, please see section 3.4 of this document 445 for placement of the PDM Destination Options header. 447 For each IPv6 packet header, the PDM MUST NOT appear more than once. 448 However, an encapsulated packet MAY contain a separate PDM associated 449 with each encapsulated IPv6 header. 451 3.4 Header Placement Using IPSec ESP Mode 453 IP Encapsulating Security Payload (ESP) is defined in [RFC4303] and 454 is widely used. Section 3.1.1 of [RFC4303] discusses placement of 455 Destination Options Headers. Below is the diagram from [RFC4303] 456 discussing placement. PDM MUST be placed before the ESP header in 457 order to work. If placed before the ESP header, the PDM header will 458 flow in the clear over the network thus allowing gathering of 459 performance and diagnostic data without sacrificing security. 461 BEFORE APPLYING ESP 463 --------------------------------------- 464 IPv6 | | ext hdrs | | | 465 | orig IP hdr |if present| TCP | Data | 466 --------------------------------------- 468 AFTER APPLYING ESP 469 --------------------------------------------------------- 470 IPv6 | orig |hop-by-hop,dest*,| |dest| | | ESP | ESP| 471 |IP hdr|routing,fragment.|ESP|opt*|TCP|Data|Trailer| ICV| 472 --------------------------------------------------------- 473 |<--- encryption ---->| 474 |<------ integrity ------>| 476 * = if present, could be before ESP, after ESP, or both 478 3.5 Implementation Considerations 480 The PDM destination options extension header SHOULD be turned on by 481 each stack on a host node. It MAY also be turned on only in case of 482 diagnostics needed for problem resolution. 484 3.6 Dynamic Configuration Options 486 If implemented, each operating system MUST have a default 487 configuration parameter, e.g. diag_header_sys_default_value=yes/no. 488 The operating system MAY also have a dynamic configuration option to 489 change the configuration setting as needed. 491 If the PDM destination options extension header is used, then it MAY 492 be turned on for all packets flowing through the host, applied to an 493 upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP 494 address only. These are at the discretion of the implementation. 496 The PDM MUST NOT be changed dynamically via packet flow as this may 497 create potential security violation or DoS attack by numerous packets 498 turning the header on and off. 500 As with all other destination options extension headers, the PDM is 501 for destination nodes only. As specified above, intermediate devices 502 MUST neither set nor modify this field. 504 3.6 5-tuple Aging 506 Within the operating system, metrics must be kept on a 5-tuple basis. 508 The 5-tuple is: 510 SADDR : IP address of the sender SPORT : Port for sender DADDR : IP 511 address of the destination DPORT : Port for destination PROTC : 512 Protocol for upper layer (ex. TCP, UDP, ICMP) 514 The question comes of when to stop keeping data or restarting the 515 numbering for a 5-tuple. For example, in the case of TCP, at some 516 point, the connection will terminate. Keeping data in control blocks 517 forever, will have unfortunate consequences for the operating system. 519 So, the recommendation is to use a known aging parameter such as Max 520 Segment Lifetime (MSL) as defined in Transmission Control Protocol 521 [RFC0793] to reuse or drop the control block. The choice of aging 522 parameter is left up to the implementation. 524 4 Considerations of Timing Representation 526 4.1 Encoding the Delta-Time Values 528 This section makes reference to and expands on the document "Encoding 529 of Time Intervals for the TCP Timestamp Option" [TRAM-TCPM]. 531 4.2 Timer registers are different on different hardware 533 One of the problems with timestamp recording is the variety of 534 hardware that generates the time value to be used. Different CPUs 535 track the time in registers of different sizes, and the most- 536 frequently-iterated bit could be the first on the left or the first 537 on the right. In order to generate some examples here it is necessary 538 to indicate the type of timer register being used. 540 As described in the "IBM z/Architecture Principles of Operation" 541 [IBM-POPS], the Time-Of-Day clock in a zSeries CPU is a 104-bit 542 register, where bit 51 is incremented approximately every 543 microsecond: 545 1 546 0 1 2 3 4 5 6 0 547 +--------+---------+---------+---------+---------+---------+--+...+ 548 | | | | | |* | | 549 +--------+---------+---------+---------+---------+---------+--+...+ 550 ^ ^ ^ 551 0 51 = 1 usec 103 553 To represent these values concisely a hexadecimal representation will 554 be used, where each digit represents 4 binary bits. Thus: 556 0000 0000 0000 0001 = 1 timer unit (2**-12 usec, or about 244 psec) 557 0000 0000 0000 1000 = 1 microsecond 558 0000 0000 003E 8000 = 1 millisecond 559 0000 0000 F424 0000 = 1 second 560 0000 0039 3870 0000 = 1 minute 561 0000 0D69 3A40 0000 = 1 hour 562 0001 41DD 7600 0000 = 1 day 564 Note that only the first 64 bits of the register are commonly 565 represented, as that represents a count of timer units on this 566 hardware. Commonly the first 52 bits are all that are displayed, as 567 that represents a count of microseconds. 569 4.3 Timer Units on Other Systems 571 This encoding method works the same with other hardware clock 572 formats. The method uses a microsecond as the basic value and allows 573 for large time differentials. 575 4.4 Time Base 577 This specification allows for the fact that different CPU TOD clocks 578 use different binary points. For some clocks, a value of 1 could 579 indicate 1 microsecond, whereas other clocks could use the value 1 to 580 indicate 1 millisecond. In the former case, the binary digits to the 581 right of that binary point measure 2**(-n) microseconds, and in the 582 latter case, 2**(-n) milliseconds. 584 The Time Base allows us to ensure we have a common reference, at the 585 very least, common knowledge of what the binary point is for the 586 transmitted values. 588 We propose a base unit for the time. This is a 2-bit integer 589 indicating the lowest granularity possible for this device. That is, 590 for a value of 00 in the Time Base field, a value of 1 in the DELTA 591 fields indicates 1 picosecond. 593 The possible values of Time Base are as follows: 595 00 - milliseconds 596 01 - microseconds 597 10 - nanoseconds 598 11 - picoseconds 600 Time base is not necessarily equivalent to length of one timer tick. 601 That is, on many, if not all, systems, the timer tick value will not 602 be in complete units of nanoseconds, milliseconds, etc. For example, 603 on an IBM zSeries machine, one timer tick (or clock unit) is 2 to the 604 -12th microseconds. 606 Therefore, some amount of conversion may be needed to approximate 607 Time Base units. 609 4.5 Timer-value scaling 611 As discussed in [TRAM-TCPM] we propose storing not an entire time- 612 interval value, but just the most significant bits of that value, 613 along with a scaling factor to indicate the magnitude of the time- 614 interval value. In our case, we will use the high-order 16 bits. The 615 scaling value will be the number of bits in the timer register to the 616 right of the 16th significant bit. That is, if the timer register 617 contains this binary value: 619 1110100011010100101001010001000000000000 620 <-16 bits -><-24 bits -> 622 then, the values stored would be 1110 1000 1101 0100 in binary (E8D4 623 hexadecimal) for the time value and 24 for the scaling value. Note 624 that the displayed value is the binary equivalent of 1 second 625 expressed in picoseconds. 627 The below table represents a device which has a TimeBase of 628 picosecond (or 00). The smallest and simplest value to represent is 629 1 picosecond; the time value stored is 1, and the scaling value is 0. 630 Using values from the table below, we have: 632 Time value in Encoded Scaling 633 Delta time picoseconds value decimal 634 -------------------------------------------------------- 635 1 picosecond 1 1 0 636 1 nanosecond 3E8 3E8 0 637 1 microsecond F4240 F424 4 638 1 millisecond 3B9ACA00 3B9A 16 639 1 second E8D4A51000 E8D4 24 640 1 minute 3691D6AFC000 3691 32 641 1 hour cca2e51310000 CCA2 36 642 1 day 132f4579c980000 132F 44 643 365 days 1b5a660ea44b80000 1B5A 52 645 Sample binary values (high order 16 bits taken) 647 1 psec 1 0001 648 1 nsec 3E8 0011 1110 1000 649 1 usec F4240 1111 0100 0010 0100 0000 650 1 msec 3B9ACA00 0011 1011 1001 1010 1100 1010 0000 0000 651 1 sec E8D4A51000 1110 1000 1101 0100 1010 0101 0001 0000 0000 0000 653 4.6 Limitations with this encoding method 655 If we follow the specification in [TRAM-TCPM], the size of one of 656 these time-interval fields is limited to this 11-bit value and five- 657 bit scale, so that they fit into a 16-bit space. With that 658 limitation, the maximum value that could be stored in 16 bits is: 660 11-bit value Scale 661 ============= ====== 662 1111 1111 111 1 1111 664 or an encoded value of 3FF and a scale value of 31. This value 665 corresponds to any time differential between: 667 || 668 11 1111 1111 1000 0000 0000 0000 0000 0000 0000 0000 (binary) 669 3 F F 8 0 0 0 0 0 0 0 (hexadecimal) 671 and 673 11 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 (binary) 674 3 F F F F F F F F F F (hexadecimal) 676 This time value, 3FFFFFFFFFF, converts to 50 days, 21 hours, 40 677 minutes and 46.511103 seconds. A time differential 1 microsecond 678 longer won't fit into 16 bits using this encoding method. 680 4.7 Lack of precision induced by timer value truncation 682 When the bit values following the first 11 significant bits are 683 truncated, obviously loss of precision in the value. The range of 684 values that will be truncated to the same encoded value is 685 2**(Scale)-1 microseconds. 687 The smallest time differential value that will be truncated is 689 1000 0000 0000 = 2.048 msec 691 The value 693 1000 0000 0001 = 2.049 msec 695 will be truncated to the same encoded value, which is 400 in hex, 696 with a scale value of 1. With the scale value of 1, the value range 697 is calculated as 2**1 - 1, or 1 usec, which you can see is the 698 difference between these minimum and maximum values. 700 With that in mind, let's look at that table of delta time values 701 again, where the Precision is the range from the smallest value 702 corresponding to this encoded value to the largest: 704 Time value in Encoded 705 Delta time microseconds value Scale Precision 706 1 microsecond 1 1 0 0:00.000000 707 1 millisecond 38E 38E 0 0:00.000000 708 1 second F4240 7A1 9 0:00.000511 709 1 minute 3938700 727 15 0:00.032767 710 1 hour D693A400 6B4 21 0:02.097151 711 1 day 141DD76000 507 26 1:07.108863 712 Maximum value 3FFFFFFFFFF 7FF 31 35:47.483647 713 So, when measuring the delay between transmission of two packets, or 714 between the reception of two packets, any delay shorter than 50 days 715 21 hours and change can be stored in this encoded fashion within 16 716 bits. When you encode, for example, a DTN response time delay of 50 717 days, 21 hours and 40 minutes, you can be assured of accuracy within 718 35 minutes. 720 5 PDM Flow - Simple Client Server 722 Following is a sample simple flow for the PDM with one packet sent 723 from Host A and one packet received by Host B. The PDM does not 724 require time synchronization between Host A and Host B. The 725 calculations to derive meaningful metrics for network diagnostics are 726 shown below each packet sent or received. 728 Each packet, in addition to the PDM contains information on the 729 sender and receiver. As discussed before, a 5-tuple consists of: 731 SADDR : IP address of the sender 732 SPORT : Port for sender 733 DADDR : IP address of the destination 734 DPORT : Port for destination 735 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP) 737 It should be understood that the packet identification information is 738 in each packet. We will not repeat that in each of the following 739 steps. 741 5.1 Step 1 743 Packet 1 is sent from Host A to Host B. The time for Host A is set 744 initially to 10:00AM. 746 The time and packet sequence number are saved by the sender 747 internally. The packet sequence number and delta times are sent in 748 the packet. 750 Packet 1 752 +----------+ +----------+ 753 | | | | 754 | Host | ----------> | Host | 755 | A | | B | 756 | | | | 757 +----------+ +----------+ 759 PDM Contents: 761 PSNTP : Packet Sequence Number This Packet: 25 762 PSNLR : Packet Sequence Number Last Received: - 763 DELTATLR : Delta Time Last Received: - 764 SCALEDTLR: Scale of Delta Time Last Received: 0 765 DELTATLS : Delta Time Last Sent: - 766 SCALEDTLS: Scale of Delta Time Last Sent: 0 767 TIMEBASE : Granularity of Time: 00 (Milliseconds) 769 Internally, within the sender, Host A, it must keep: 771 Packet Sequence Number of the last packet sent: 25 772 Time the last packet was sent: 10:00:00 774 Note, the initial PSNTP from Host A starts at a random number. In 775 this case, 25. The time in these examples is shown in seconds for 776 the sake of simplicity. 778 5.2 Step 2 780 Packet 1 is received at Host B. Its time is set to one hour later 781 than Host A. In this case, 11:00AM 783 Internally, within the receiver, Host B, it must note: 785 Packet Sequence Number of the last packet received: 25 786 Time the last packet was received : 11:00:03 788 Note, this timestamp is in Host B time. It has nothing whatsoever to 789 do with Host A time. The Packet Sequence Number of the last packet 790 received will become PSNLR which will be sent out in the packet sent 791 by Host B in the next step. The time last received will be used to 792 calculate the DELTALR value to be sent out in the packet sent by Host 793 B in the next step. 795 5.3 Step 3 797 Packet 2 is sent by Host B to Host A. Note, the initial packet 798 sequence number (PSNTP) from Host B starts at a random number. In 799 this case, 12. Before sending the packet, Host B does a calculation 800 of deltas. Since Host B knows when it is sending the packet, and it 801 knows when it received the previous packet, it can do the following 802 calculation: 804 Sending time (packet 2) - receive time (packet 1) 805 We will call the result of this calculation: Delta Time Last 806 Received 808 That is: 810 DELTATLR = Sending time (packet 2) - receive time (packet 1) 812 Note, both sending time and receive time are saved internally in Host 813 B. They do not travel in the packet. Only the Delta is in the 814 packet. 816 Assume that within Host B is the following: 818 Packet Sequence Number of the last packet received: 25 819 Time the last packet was received: 11:00:03 820 Packet Sequence Number of this packet: 12 821 Time this packet is being sent: 11:00:07 823 We can now calculate a delta value to be sent out in the packet. 824 DELTATLR becomes: 826 4 seconds = 11:00:07 - 11:00:03 828 This is the derived metric: Server Delay. The time and scaling 829 factor must be calculated. Then, this value, along with the packet 830 sequence numbers will be sent to Host A as follows: 832 Packet 2 834 +----------+ +----------+ 835 | | | | 836 | Host | <---------- | Host | 837 | A | | B | 838 | | | | 839 +----------+ +----------+ 841 PDM Contents: 843 PSNTP : Packet Sequence Number This Packet: 12 844 PSNLR : Packet Sequence Number Last Received: 25 845 DELTATLR : Delta Time Last Received: 3A35 (4 seconds) 846 SCALEDTLR: Scale of Delta Time Last Received: 25 847 DELTATLS : Delta Time Last Sent: - 848 SCALEDTLS: Scale of Delta Time Last Sent: 0 849 TIMEBASE : Granularity of Time: 00 (Milliseconds) 851 The metric left to be calculated is the Round-Trip Delay. This will 852 be calculated by Host A when it receives Packet 2. 854 5.4 Step 4 856 Packet 2 is received at Host A. Remember, its time is set to one 857 hour earlier than Host B. Internally, it must note: 859 Packet Sequence Number of the last packet received: 12 860 Time the last packet was received : 10:00:12 862 Note, this timestamp is in Host A time. It has nothing whatsoever to 863 do with Host B time. 865 So, now, Host A can calculate total end-to-end time. That is: 867 End-to-End Time = Time Last Received - Time Last Sent 869 For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was 870 received by Host A at 10:00:12 so: 872 End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT 873 delay combined). This time may also be called total Overall Round- 874 trip time (which includes Network RTT and Host Response Time). 876 This derived metric we will call DELTATLS or Delta Time Last Sent. 878 We can now also calculate round trip delay. The formula is: 880 Round trip delay = DELTATLS - DELTATLR 882 Or: 884 Round trip delay = 12 - 4 or 8 886 Now, the only problem is that at this point all metrics are in Host A 887 only and not exposed in a packet. To do that, we need a third packet. 889 Note: this simple example assumes one send and one receive. That 890 is done only for purposes of explaining the function of the PDM. In 891 cases where there are multiple packets returned, one would take the 892 time in the last packet in the sequence. The calculations of such 893 timings and intelligent processing is the function of post-processing 894 of the data. 896 5.5 Step 5 898 Packet 3 is sent from Host A to Host B. 900 +----------+ +----------+ 901 | | | | 902 | Host | ----------> | Host | 903 | A | | B | 904 | | | | 905 +----------+ +----------+ 907 PDM Contents: 909 PSNTP : Packet Sequence Number This Packet: 26 910 PSNLR : Packet Sequence Number Last Received: 12 911 DELTATLR : Delta Time Last Received: 0 912 SCALEDTLS: Scale of Delta Time Last Received 0 913 DELTATLS : Delta Time Last Sent: 105e (12 seconds) 914 SCALEDTLR: Scale of Delta Time Last Received: 26 915 TIMEBASE : Granularity of Time: 00 (Milliseconds) 917 To calculate Two-Way Delay, any packet capture device may look at 918 these packets and do what is necessary. 920 6 Other Flows 922 What we have discussed so far is a simple flow with one packet sent 923 and one returned. Let's look at how PDM may be useful in other 924 types of flows. 926 6.1 PDM Flow - One Way Traffic 928 The flow on a particular session may not be a send-receive paradigm. 929 Let us consider some other situations. In the case of a one-way 930 flow, one might see the following: 932 Packet Sender PSN PSN Delta Time Delta Time 933 This Packet Last Recvd Last Recvd Last Sent 934 ===================================================================== 935 1 Server 1 0 0 0 936 2 Server 2 0 0 5 937 3 Server 3 0 0 12 938 4 Server 4 0 0 20 940 What does this mean and how is it useful? 941 In a one-way flow, only the Delta Time Last Sent will be seen as 942 used. Recall, Delta Time Last Sent is the difference between the 943 send of one packet from a device and the next. This is a measure of 944 throughput for the sender - according to the sender's point of view. 945 That is, it is a measure of how fast is the application itself (with 946 stack time included) able to send packets. 948 How might this be useful? If one is having a performance issue at 949 the client and sees that packet 2, for example, is sent after 5 950 microseconds from the server but takes 3 minutes to arrive at the 951 destination, then one may safely conclude that there are delays in 952 the path other than at the server which may be causing the delivery 953 issue of that packet. Such delays may include the network links, 954 middle-boxes, etc. 956 Now, true one-way traffic is quite rare. What people often mean by 957 "one-way" traffic is an application such as FTP where a group of 958 packets (for example, a TCP window size worth) is sent, then the 959 sender waits for acknowledgment. This type of flow would actually 960 fall into the "multiple-send" traffic model. 962 6.2 PDM Flow - Multiple Send Traffic 964 Assume that two packets are sent for each ACK from the server. 966 Packet Sender PSN PSN Delta Time Delta Time 967 This Packet Last Recvd Last Recvd Last Sent 968 ===================================================================== 969 1 Server 1 0 0 0 970 2 Server 2 0 0 5 971 3 Client 1 2 20 0 972 4 Server 3 1 10 15 974 How might this be used? 976 Notice that in packet 3, the client has a value of Delta Time Last 977 received of 20. Recall that Delta Time Last Received is the Send 978 time of packet 3 - receive time of packet 2. So, what does one know 979 now? In this case, Delta Time Last Received is the processing time 980 for the Client to send the next packet. 982 How to interpret this depends on what is actually being sent. 983 Remember, PDM is not being used in isolation, but to supplement the 984 fields found in other headers. Let's take some examples: 986 1. Client is sending a standalone TCP ACK. One would find this by 987 looking at the payload length in the IPv6 header and the TCP 988 Acknowledgement field in the TCP header. So, in this case, the 989 client is taking 20 units to send back the ACK. This may or may not 990 be interesting. 992 2. Client is sending data with the packet. Again, one would find 993 this by looking at the payload length in the IPv6 header and the TCP 994 Acknowledgement field in the TCP header. So, in this case, the 995 client is taking 20 units to send back data. This may represent 996 "User Think Time". Again, this may or may not be interesting, in 997 isolation. But, if there is a performance problem receiving data at 998 the server, then taken in conjunction with RTT or other packet timing 999 information, this information may be quite interesting. 1001 Of course, one also needs to look at the PSN Last Received field to 1002 make sure of the interpretation of this data. That is, to make 1003 sure that the Delta Last Received corresponds to the packet of 1004 interest. 1006 The benefits of PDM are that we have such information available in a 1007 uniform manner for all applications and all protocols without 1008 extensive changes required to applications. 1010 6.3 PDM Flow - Multiple Send with Errors 1012 One might wonder if all of the functions of PDM might be better 1013 suited to TCP or a TCP option. Let us take the case of how PDM may 1014 help in a case of TCP retransmissions in a way that TCP options or 1015 TCP ACK / SEQ would not. 1017 Assume that three packets are sent with each send from the server. 1019 From the server, this is what is seen. 1021 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1022 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1023 ===================================================================== 1024 1 Server 1 0 0 0 123 100 1025 2 Server 2 0 0 5 223 100 1026 3 Server 3 0 0 5 333 100 1028 The client however, does not get all the packets. From the client, 1029 this is what is seen for the packets sent from the server. 1031 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1032 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1033 ===================================================================== 1034 1 Server 1 0 0 0 123 100 1035 2 Server 3 0 0 5 333 100 1037 Let's assume that the server now retransmits the packet. 1038 (Obviously, a duplicate acknowledgment sequence for fast retransmit 1039 or a retransmit timeout would occur. To illustrate the point, these 1040 packets are being left out.) 1042 So, then if a TCP retransmission is done, then from the client, this 1043 is what is seen for the packets sent from the server. 1045 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1046 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1047 ===================================================================== 1048 1 Server 4 0 0 30 223 100 1050 The server has resent the old packet 2 with TCP sequence number of 1051 223. The retransmitted packet now has a PSN This Packet value of 4. 1052 The Delta Last Sent is 30 - the time between sending the packet with 1053 PSN of 3 and this current packet. 1055 Let's say that packet 4 STILL does not make it. Then, after some 1056 amount of time (RTO) then the packet with TCP sequence number of 223 1057 is resent. 1059 From the client, this is what is seen for the packets sent from the 1060 server. 1062 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1063 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1064 ===================================================================== 1065 1 Server 5 0 0 60 223 100 1067 If now, this packet makes it, one has a very good idea that packets 1068 exist which are being sent from the server as retransmissions and not 1069 making it to the client. This is because the PSN of the resent 1070 packet from the server is 5 rather than 4. If we had used TCP 1071 sequence number alone, we would never have seen this situation. 1072 Because the TCP sequence number in all situations is 223. 1074 This situation would be experienced by the user of the application 1075 (the human being actually sitting somewhere) as a "hangs" or long 1076 delay between packets. On large networks, to diagnose problems such 1077 as these where packets are lost somewhere on the network, one has to 1078 take multiple traces to find out exactly where. 1080 The first thing is to start with doing a trace at the client and the 1081 server. So, we can see if the server sent a particular packet and 1082 the client received it. If the client did not receive it, then we 1083 start tracking back to trace points at the router right after the 1084 server and the router right before the client. Did they get these 1085 packets which the server has sent? This is a time consuming 1086 activity. 1088 With PDM, we can speed up the diagnostic time because we may be able 1089 to use only the trace taken at the client to see what the server is 1090 sending. 1092 7 Potential Overhead Considerations 1094 Questions have been posed as to the potential overhead of PDM. 1095 First, PDM is entirely optional. That is, a site may choose to 1096 implement PDM or not as they wish. If they are happy with the costs 1097 of PDM vs. the benefits, then the choice should be theirs. 1099 Below is a table outlining the potential overhead in terms of 1100 additional time to deliver the response to the end user for various 1101 assumed RTTs. 1103 Bytes RTT Bytes Bytes New Overhead 1104 in Packet Per Milli in PDM RTT 1105 ===================================================================== 1106 1000 1000 milli 1 16 1016.000 16.000 milli 1107 1000 100 milli 10 16 101.600 1.600 milli 1108 1000 10 milli 100 16 10.160 .160 milli 1109 1000 1 milli 1000 16 1.016 .016 milli 1111 Below are some examples of actual RTTs for packets traversing large 1112 enterprise networks. The first example is for packets going to 1113 multiple business partners. 1115 Bytes RTT Bytes Bytes New Overhead 1116 in Packet Per Milli in PDM RTT 1117 ===================================================================== 1118 1000 17 milli 58 16 17.360 .360 milli 1120 The second example is for packets at a large enterprise customer 1121 within a data center. Notice that the scale is now in microseconds 1122 rather than milliseconds. 1124 Bytes RTT Bytes Bytes New Overhead 1125 in Packet Per Micro in PDM RTT 1126 ===================================================================== 1127 1000 20 micro 50 16 20.320 .320 micro 1129 8 Security Considerations 1131 The PDM MUST NOT be changed dynamically via packet flow as this 1132 creates a possibility for potential security violations or DoS 1133 attacks by numerous packets turning the header on and off. 1135 Attackers may also send many packets from multiple ports, for example 1136 by doing a port scan. This will cause the stack to create many 1137 control blocks. This is the same problem as seen for SYN flood 1138 attacks. Similar protections should be implemented by the stack to 1139 preserve the integrity of memory. 1141 9 IANA Considerations 1143 This draft requests an Option Type assignment in the Destination 1144 Options and Hop-by-Hop Options sub-registry of Internet Protocol 1145 Version 6 (IPv6) Parameters [ref to RFCs and URL below]. 1147 http://www.iana.org/assignments/ipv6-parameters/ipv6- 1148 parameters.xhtml#ipv6-parameters-2 1150 Hex Value Binary Value Description Reference 1151 act chg rest 1152 ------------------------------------------------------------------- 1153 TBD TBD Performance and [This draft] 1154 Diagnostic Metrics 1155 (PDM) 1157 10 References 1159 10.1 Normative References 1161 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1162 793, September 1981. 1164 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1165 Requirement Levels", BCP 14, RFC 2119, March 1997. 1167 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1168 (IPv6) Specification", RFC 2460, December 1998. 1170 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1171 Delay Metric for IPPM", RFC 2681, September 1999. 1173 [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For 1174 Values In the Internet Protocol and Related Headers", BCP 37, RFC 1175 2780, March 2000. 1177 [RFC4303] Kent, S, "IP Encapsulating Security Payload (ESP)", RFC 1178 4303, December 2005. 1180 10.2 Informative References 1182 [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP 1183 Timestamp Option-01", Internet Draft, July 2013. [Work in Progress] 1185 [IBM-POPS] IBM Corporation, "IBM z/Architecture Principles of 1186 Operation", SA22-7832, 1990-2012 1188 11 Acknowledgments 1190 The authors would like to thank Keven Haining, Al Morton, Brian 1191 Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick 1192 Troth for their comments and assistance. 1194 Authors' Addresses 1196 Nalini Elkins 1197 Inside Products, Inc. 1198 36A Upper Circle 1199 Carmel Valley, CA 93924 1200 United States 1201 Phone: +1 831 659 8360 1202 Email: nalini.elkins@insidethestack.com 1203 http://www.insidethestack.com 1205 Robert Hamilton 1206 Chemical Abstracts Service 1207 A Division of the American Chemical Society 1208 2540 Olentangy River Road 1209 Columbus, Ohio 43202 1210 United States 1211 Phone: +1 614 447 3600 x2517 1212 Email: rhamilton@cas.org 1213 http://www.cas.org 1215 Michael S. Ackermann 1216 Blue Cross Blue Shield of Michigan 1217 P.O. Box 2888 1218 Detroit, Michigan 48231 1219 United States 1220 Phone: +1 310 460 4080 1221 Email: mackermann@bcbsm.com 1222 http://www.bcbsm.com