idnits 2.17.1 draft-ietf-ippm-6man-pdm-option-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 23, 2016) is 2772 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 793 (Obsoleted by RFC 9293) ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET-DRAFT N. Elkins 3 Inside Products 4 R. Hamilton 5 Chemical Abstracts Service 6 M. Ackermann 7 Intended Status: Proposed Standard BCBS Michigan 8 Expires: March 27, 2017 September 23, 2016 10 IPv6 Performance and Diagnostic Metrics (PDM) Destination Option 11 draft-ietf-ippm-6man-pdm-option-06 13 Abstract 15 To assess performance problems, measurements based on optional 16 sequence numbers and timing may be embedded in each packet. Such 17 measurements may be interpreted in real-time or after the fact. An 18 implementation of the existing IPv6 Destination Options extension 19 header, the Performance and Diagnostic Metrics (PDM) Destination 20 Options extension header as well as the field limits, calculations, 21 and usage of the PDM in measurement are included in this document. 23 Status of this Memo 25 This Internet-Draft is submitted to IETF in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF), its areas, and its working groups. Note that 30 other groups may also distribute working documents as 31 Internet-Drafts. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 The list of current Internet-Drafts can be accessed at 39 http://www.ietf.org/1id-abstracts.html 41 The list of Internet-Draft Shadow Directories can be accessed at 42 http://www.ietf.org/shadow.html 44 Copyright and License Notice 46 Copyright (c) 2016 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 50 3: This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . 4 63 1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . 4 64 1.2 End User Quality of Service (QoS) . . . . . . . . . . . . . 4 65 1.3 Need for a Packet Sequence Number . . . . . . . . . . . . . 5 66 1.4 Rationale for defined solution . . . . . . . . . . . . . . . 5 67 1.5 PDM Works in Collaboration with Other Headers . . . . . . . 6 68 1.6 IPv6 Transition Technologies . . . . . . . . . . . . . . . . 6 69 2 Measurement Information Derived from PDM . . . . . . . . . . . . 6 70 2.1 Round-Trip Delay . . . . . . . . . . . . . . . . . . . . . . 7 71 2.2 Server Delay . . . . . . . . . . . . . . . . . . . . . . . . 7 72 3 Performance and Diagnostic Metrics Destination Option Layout . . 7 73 3.1 Destination Options Header . . . . . . . . . . . . . . . . . 7 74 3.2 Performance and Diagnostic Metrics Destination Option . . . 7 75 3.3 Header Placement . . . . . . . . . . . . . . . . . . . . . . 10 76 3.4 Header Placement Using IPSec ESP Mode . . . . . . . . . . . 11 77 3.5 Implementation Considerations . . . . . . . . . . . . . . . 12 78 3.6 Dynamic Configuration Options . . . . . . . . . . . . . . . 12 79 3.6 5-tuple Aging . . . . . . . . . . . . . . . . . . . . . . . 12 80 4 Considerations of Timing Representation . . . . . . . . . . . . 12 81 4.1 Encoding the Delta-Time Values . . . . . . . . . . . . . . . 12 82 4.2 Timer registers are different on different hardware . . . . 13 83 4.3 Timer Units on Other Systems . . . . . . . . . . . . . . . . 13 84 4.4 Time Base . . . . . . . . . . . . . . . . . . . . . . . . . 14 85 4.5 Timer-value scaling . . . . . . . . . . . . . . . . . . . . 14 86 4.6 Limitations with this encoding method . . . . . . . . . . . 15 87 4.7 Lack of precision induced by timer value truncation . . . . 16 88 5 PDM Flow - Simple Client Server . . . . . . . . . . . . . . . . 17 89 5.1 Step 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 90 5.2 Step 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 91 5.3 Step 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 92 5.4 Step 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 93 5.5 Step 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 94 6 Other Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 22 95 6.1 PDM Flow - One Way Traffic . . . . . . . . . . . . . . . . . 22 96 6.2 PDM Flow - Multiple Send Traffic . . . . . . . . . . . . . . 23 97 6.3 PDM Flow - Multiple Send with Errors . . . . . . . . . . . . 24 98 7 Potential Overhead Considerations . . . . . . . . . . . . . . . 26 99 8 Security Considerations . . . . . . . . . . . . . . . . . . . . 27 100 8.1. SYN Flood and Resource Consumption Attacks . . . . . . . . 27 101 8.2 Pervasive monitoring . . . . . . . . . . . . . . . . . . . 27 102 8.3 PDM as a Covert Channel . . . . . . . . . . . . . . . . . . 28 103 9 IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 28 104 10 References . . . . . . . . . . . . . . . . . . . . . . . . . . 29 105 10.1 Normative References . . . . . . . . . . . . . . . . . . . 29 106 10.2 Informative References . . . . . . . . . . . . . . . . . . 29 107 11 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 29 108 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 30 110 1 Background 112 To assess performance problems, measurements based on optional 113 sequence numbers and timing may be embedded in each packet. Such 114 measurements may be interpreted in real-time or after the fact. 116 As defined in RFC2460 [RFC2460], destination options are carried by 117 the IPv6 Destination Options extension header. Destination options 118 include optional information that need be examined only by the IPv6 119 node given as the destination address in the IPv6 header, not by 120 routers or other "middle boxes". This document specifies a new 121 destination option, the Performance and Diagnostic Metrics (PDM) 122 destination option. This document specifies the layout, field 123 limits, calculations, and usage of the PDM in measurement. 125 1.1 Terminology 127 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 128 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 129 document are to be interpreted as described in RFC 2119 [RFC2119]. 131 1.2 End User Quality of Service (QoS) 133 The timing values in the PDM embedded in the packet will be used to 134 estimate QoS as experienced by an end user device. 136 For many applications, the key user performance indicator is response 137 time. When the end user is an individual, he is generally 138 indifferent to what is happening along the network; what he really 139 cares about is how long it takes to get a response back. But this is 140 not just a matter of individuals' personal convenience. In many 141 cases, rapid response is critical to the business being conducted. 143 When the end user is a device (e.g. with the Internet of Things), 144 what matters is the speed with which requested data can be 145 transferred -- specifically, whether the requested data can be 146 transferred in time to accomplish the desired actions. This can be 147 important when the relevant external conditions are subject to rapid 148 change. 150 Low, reliable and acceptable responses times are not just "nice to 151 have". On many networks, the impact can be financial hardship or 152 endanger human life. In some cities, the emergency police contact 153 system operates over IP, law enforcement uses TCP/IP networks, 154 transactions on our stock exchanges are settled using IP networks. 156 The critical nature of such activities to our daily lives and 157 financial well-being demand a simple solution to support response 158 time measurements. 160 1.3 Need for a Packet Sequence Number 162 While performing network diagnostics of an end-to-end connection, it 163 often becomes necessary to isolate the factors along the network path 164 responsible for problems. Diagnostic data may be collected at 165 multiple places along the path (if possible), or at the source and 166 destination. Then, in post-collection processing, the diagnostic 167 data corresponding to each packet at different observation points 168 must be matched for proper measurements. A sequence number in each 169 packet provides sufficient basis for the matching process. If need 170 be, the timing fields may be used along with the sequence number to 171 ensure uniqueness. 173 This method of data collection along the path is of special use to 174 determine where packet loss or packet corruption is happening. 176 The packet sequence number needs to be unique in the context of the 177 session (5-tuple). See section 2 for a definition of 5-tuple. 179 1.4 Rationale for defined solution 181 The current IPv6 specification does not provide timing nor a similar 182 field in the IPv6 main header or in any extension header. So, we 183 define the IPv6 Performance and Diagnostic Metrics destination option 184 (PDM). 186 Advantages include: 188 1. Real measure of actual transactions. 189 2. Independence from transport layer protocols. 190 3. Ability to span organizational boundaries with consistent 191 instrumentation 192 4. No time synchronization needed between session partners 193 5. Ability to handle all transport protocols (TCP, UDP, SCTP, etc) 194 in a uniform way 196 The PDM provides the ability to quickly determine if the (latency) 197 problem is in the network or in the server (application). More 198 intermediate measurements may be needed if the host or network 199 discrimination is not sufficient. At the client, TCP/IP stack time 200 vs. applications time may still need to be broken out by client 201 software. 203 1.5 PDM Works in Collaboration with Other Headers 205 The purpose of the PDM is not to supplant all the variables present 206 in all other headers but to provide data which is not available or 207 very difficult to get. The way PDM would be used is by a technician 208 (or tool) looking at a packet capture. Within the packet capture, 209 they would have available to them the layer 2 header, IP header (v6 210 or v4), TCP, UCP, ICMP, SCTP or other headers. All information 211 would be looked at together to make sense of the packet flow. The 212 technician or processing tool could analyze, report or ignore the 213 data from PDM, as necessary. 215 For an example of how PDM can help with TCP retransmit problems, 216 please look at section 8. 218 1.6 IPv6 Transition Technologies 220 In the path to full implementation of IPv6, transition technologies 221 such as translation or tunneling may be employed. The PDM header is 222 not expected to work in such scenarios. It is likely that an IPv6 223 packet containing PDM will be dropped if using IPv6 transition 224 technologies. 226 2 Measurement Information Derived from PDM 228 Each packet contains information about the sender and receiver. In IP 229 protocol, the identifying information is called a "5-tuple". 231 The 5-tuple consists of: 233 SADDR : IP address of the sender 234 SPORT : Port for sender 235 DADDR : IP address of the destination 236 DPORT : Port for destination 237 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP, etc.) 239 The PDM contains the following base fields: 241 PSNTP : Packet Sequence Number This Packet 242 PSNLR : Packet Sequence Number Last Received 243 DELTATLR : Delta Time Last Received 244 DELTATLS : Delta Time Last Sent 246 Other fields for scaling and time base are also in the PDM and will 247 be described in section 3. 249 This information, combined with the 5-tuple, allows the measurement 250 of the following metrics: 252 1. Round-trip delay 253 2. Server delay 255 2.1 Round-Trip Delay 257 Round-trip *Network* delay is the delay for packet transfer from a 258 source host to a destination host and then back to the source host. 259 This measurement has been defined, and the advantages and 260 disadvantages discussed in "A Round-trip Delay Metric for IPPM" 261 [RFC2681]. 263 2.2 Server Delay 265 Server delay is the interval between when a packet is received by a 266 device and the first corresponding packet is sent back in response. 267 This may be "Server Processing Time". It may also be a delay caused 268 by acknowledgements. Server processing time includes the time taken 269 by the combination of the stack and application to return the 270 response. The stack delay may be related to network performance. If 271 this aggregate time is seen as a problem, and there is a need to make 272 a clear distinction between application processing time and stack 273 delay, including that caused by the network, then more client based 274 measurements are needed. 276 3 Performance and Diagnostic Metrics Destination Option Layout 278 3.1 Destination Options Header 280 The IPv6 Destination Options Header is used to carry optional 281 information that needs to be examined only by a packet's destination 282 node(s). The Destination Options Header is identified by a Next 283 Header value of 60 in the immediately preceding header and is defined 284 in RFC2460 [RFC2460]. The IPv6 Performance and Diagnostic Metrics 285 Destination Option (PDM) is an implementation of the Destination 286 Options Header. The PDM does not require time synchronization. 288 3.2 Performance and Diagnostic Metrics Destination Option 290 The IPv6 Performance and Diagnostic Metrics Destination Option (PDM) 291 contains the following fields: 293 TIMEBASE : Base timer unit 294 SCALEDTLR: Scale for Delta Time Last Received 295 SCALEDTLS: Scale for Delta Time Last Sent 296 PSNTP : Packet Sequence Number This Packet 297 PSNLR : Packet Sequence Number Last Received 298 DELTATLR : Delta Time Last Received 299 DELTATLS : Delta Time Last Sent 301 The PDM destination option is encoded in type-length-value (TLV) 302 format as follows: 304 0 1 2 3 305 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 306 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 307 | Option Type | Option Length |TB |ScaleDTLR | ScaleDTLS | 308 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 309 | PSN This Packet | PSN Last Received | 310 |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 311 | Delta Time Last Received | Delta Time Last Sent | 312 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 314 Option Type 316 TBD = 0xXX (TBD) [To be assigned by IANA] [RFC2780] 318 Option Length 320 8-bit unsigned integer. Length of the option, in octets, excluding 321 the Option Type and Option Length fields. This field MUST be set to 322 16. 324 Time Base 326 2-bit binary value. It will indicate the lowest granularity possible 327 for this device. That is, for a value of 00 in the Time Base field, 328 a value of 1 in the DELTA fields indicates 1 millisecond. 330 This field is being included so that a device may choose the 331 granularity which most suits its timer ticks. That is, so that it 332 does not have to do more work than needed to convert values required 333 for the PDM. 335 The possible values of Time Base are as follows: 337 00 - milliseconds 338 01 - microseconds 339 10 - nanoseconds 340 11 - picoseconds 342 Scale Delta Time Last Received (SCALEDTLR) 344 7-bit signed integer. This is the scaling value for the Delta Time 345 Last Received (DELTATLR) field. The possible values are from -128 to 346 +127. See Section 4 for further discussion on Timing Considerations 347 and formatting of the scaling values. 349 Scale Delta Time Last Sent (SCALEDTLS) 351 7-bit signed integer. This is the scaling value for the Delta Time 352 Last Sent (DELTATLS) field. The possible values are from -128 to 353 +127. 355 Packet Sequence Number This Packet (PSNTP) 357 16-bit unsigned integer. This field will wrap. It is intended for 358 use while analyzing packet traces. 360 Initialized at a random number and incremented monotonically for each 361 packet of the session flow of the 5-tuple. The 5-tuple consists of 362 the source and destination IP addresses, the source and destination 363 ports, and the upper layer protocol (ex. TCP, ICMP, etc). The 364 random number initialization is intended to make it harder to spoof 365 and insert such packets. 367 Operating systems MUST implement a separate packet sequence number 368 counter per 5-tuple. Operating systems MUST NOT implement a single 369 counter for all connections. 371 Packet Sequence Number Last Received (PSNLR) 373 16-bit unsigned integer. This is the PSNTP of the packet last 374 received on the 5-tuple. 376 Delta Time Last Received (DELTATLR) 378 A 16-bit unsigned integer field. The value is set according to the 379 scale in SCALEDTLR. 381 Delta Time Last Received = (Send time packet 2 - Receive time packet 382 1) 383 Delta Time Last Sent (DELTATLS) 385 A 16-bit unsigned integer field. The value is set according to the 386 scale in SCALEDTLS. 388 Delta Time Last Sent = (Receive time packet 2 - Send time packet 1) 390 Option Type 392 The two highest-order bits of the Option Type field are encoded to 393 indicate specific processing of the option; for the PDM destination 394 option, these two bits MUST be set to 00. This indicates the 395 following processing requirements: 397 00 - skip over this option and continue processing the header. 399 RFC2460 [RFC2460] defines other values for the Option Type field. 400 These MUST NOT be used in the PDM. 402 In keeping with RFC2460 [RFC2460], the third-highest-order bit of the 403 Option Type specifies whether or not the Option Data of that option 404 can change en-route to the packet's final destination. 406 In the PDM, the value of the third-highest-order bit MUST be 0. The 407 possible values are as follows: 409 0 - Option Data does not change en-route 411 1 - Option Data may change en-route 413 The three high-order bits described above are to be treated as part 414 of the Option Type, not independent of the Option Type. That is, a 415 particular option is identified by a full 8-bit Option Type, not just 416 the low-order 5 bits of an Option Type. 418 3.3 Header Placement 420 The PDM destination option MUST be placed as follows: 422 - Before the upper-layer header or the ESP header. 424 This follows the order defined in RFC2460 [RFC2460] 426 IPv6 header 428 Hop-by-Hop Options header 429 Destination Options header <-------- 431 Routing header 433 Fragment header 435 Authentication header 437 Encapsulating Security Payload header 439 Destination Options header <------------ 441 upper-layer header 443 Note that there is a choice of where to place the Destination Options 444 header. If using ESP mode, please see section 3.4 of this document 445 for placement of the PDM Destination Options header. 447 For each IPv6 packet header, the PDM MUST NOT appear more than once. 448 However, an encapsulated packet MAY contain a separate PDM associated 449 with each encapsulated IPv6 header. 451 3.4 Header Placement Using IPSec ESP Mode 453 IP Encapsulating Security Payload (ESP) is defined in [RFC4303] and 454 is widely used. Section 3.1.1 of [RFC4303] discusses placement of 455 Destination Options Headers. Below is the diagram from [RFC4303] 456 discussing placement. PDM MUST be placed before the ESP header in 457 order to work. If placed before the ESP header, the PDM header will 458 flow in the clear over the network thus allowing gathering of 459 performance and diagnostic data without sacrificing security. 461 BEFORE APPLYING ESP 463 --------------------------------------- 464 IPv6 | | ext hdrs | | | 465 | orig IP hdr |if present| TCP | Data | 466 --------------------------------------- 468 AFTER APPLYING ESP 469 --------------------------------------------------------- 470 IPv6 | orig |hop-by-hop,dest*,| |dest| | | ESP | ESP| 471 |IP hdr|routing,fragment.|ESP|opt*|TCP|Data|Trailer| ICV| 472 --------------------------------------------------------- 473 |<--- encryption ---->| 474 |<------ integrity ------>| 476 * = if present, could be before ESP, after ESP, or both 478 3.5 Implementation Considerations 480 The PDM destination options extension header SHOULD be turned on by 481 each stack on a host node. It MAY also be turned on only in case of 482 diagnostics needed for problem resolution. 484 3.6 Dynamic Configuration Options 486 If implemented, each operating system MUST have a default 487 configuration parameter, e.g. diag_header_sys_default_value=yes/no. 488 The operating system MAY also have a dynamic configuration option to 489 change the configuration setting as needed. 491 If the PDM destination options extension header is used, then it MAY 492 be turned on for all packets flowing through the host, applied to an 493 upper-layer protocol (TCP, UDP, SCTP, etc), a local port, or IP 494 address only. These are at the discretion of the implementation. 496 As with all other destination options extension headers, the PDM is 497 for destination nodes only. As specified above, intermediate devices 498 MUST neither set nor modify this field. 500 3.6 5-tuple Aging 502 Within the operating system, metrics must be kept on a 5-tuple basis. 504 The 5-tuple is: 506 SADDR : IP address of the sender SPORT : Port for sender DADDR : IP 507 address of the destination DPORT : Port for destination PROTC : 508 Protocol for upper layer (ex. TCP, UDP, ICMP) 510 The question comes of when to stop keeping data or restarting the 511 numbering for a 5-tuple. For example, in the case of TCP, at some 512 point, the connection will terminate. Keeping data in control blocks 513 forever, will have unfortunate consequences for the operating system. 515 So, the recommendation is to use a known aging parameter such as Max 516 Segment Lifetime (MSL) as defined in Transmission Control Protocol 517 [RFC0793] to reuse or drop the control block. The choice of aging 518 parameter is left up to the implementation. 520 4 Considerations of Timing Representation 522 4.1 Encoding the Delta-Time Values 524 This section makes reference to and expands on the document "Encoding 525 of Time Intervals for the TCP Timestamp Option" [TRAM-TCPM]. 527 4.2 Timer registers are different on different hardware 529 One of the problems with timestamp recording is the variety of 530 hardware that generates the time value to be used. Different CPUs 531 track the time in registers of different sizes, and the most- 532 frequently-iterated bit could be the first on the left or the first 533 on the right. In order to generate some examples here it is necessary 534 to indicate the type of timer register being used. 536 As described in the "IBM z/Architecture Principles of Operation" 537 [IBM-POPS], the Time-Of-Day clock in a zSeries CPU is a 104-bit 538 register, where bit 51 is incremented approximately every 539 microsecond: 541 1 542 0 1 2 3 4 5 6 0 543 +--------+---------+---------+---------+---------+---------+--+...+ 544 | | | | | |* | | 545 +--------+---------+---------+---------+---------+---------+--+...+ 546 ^ ^ ^ 547 0 51 = 1 usec 103 549 To represent these values concisely a hexadecimal representation will 550 be used, where each digit represents 4 binary bits. Thus: 552 0000 0000 0000 0001 = 1 timer unit (2**-12 usec, or about 244 psec) 553 0000 0000 0000 1000 = 1 microsecond 554 0000 0000 003E 8000 = 1 millisecond 555 0000 0000 F424 0000 = 1 second 556 0000 0039 3870 0000 = 1 minute 557 0000 0D69 3A40 0000 = 1 hour 558 0001 41DD 7600 0000 = 1 day 560 Note that only the first 64 bits of the register are commonly 561 represented, as that represents a count of timer units on this 562 hardware. Commonly the first 52 bits are all that are displayed, as 563 that represents a count of microseconds. 565 4.3 Timer Units on Other Systems 567 This encoding method works the same with other hardware clock 568 formats. The method uses a microsecond as the basic value and allows 569 for large time differentials. 571 4.4 Time Base 573 This specification allows for the fact that different CPU TOD clocks 574 use different binary points. For some clocks, a value of 1 could 575 indicate 1 microsecond, whereas other clocks could use the value 1 to 576 indicate 1 millisecond. In the former case, the binary digits to the 577 right of that binary point measure 2**(-n) microseconds, and in the 578 latter case, 2**(-n) milliseconds. 580 The Time Base allows us to ensure we have a common reference, at the 581 very least, common knowledge of what the binary point is for the 582 transmitted values. 584 We define a base unit for the time. This is a 2-bit binary value 585 indicating the lowest granularity possible for this device. That is, 586 for a value of 00 in the Time Base field, a value of 1 in the DELTA 587 fields indicates 1 millisecond. 589 The possible values of Time Base are as follows: 591 00 - milliseconds 592 01 - microseconds 593 10 - nanoseconds 594 11 - picoseconds 596 Time base is not necessarily equivalent to length of one timer tick. 597 That is, on many, if not all, systems, the timer tick value will not 598 be in complete units of nanoseconds, milliseconds, etc. For example, 599 on an IBM zSeries machine, one timer tick (or clock unit) is 2 to the 600 -12th microseconds. 602 Therefore, some amount of conversion may be needed to approximate 603 Time Base units. 605 4.5 Timer-value scaling 607 As discussed in [TRAM-TCPM] we define storing not an entire time- 608 interval value, but just the most significant bits of that value, 609 along with a scaling factor to indicate the magnitude of the time- 610 interval value. In our case, we will use the high-order 16 bits. The 611 scaling value will be the number of bits in the timer register to the 612 right of the 16th significant bit. That is, if the timer register 613 contains this binary value: 615 1110100011010100101001010001000000000000 616 <-16 bits -><-24 bits -> 618 then, the values stored would be 1110 1000 1101 0100 in binary (E8D4 619 hexadecimal) for the time value and 24 for the scaling value. Note 620 that the displayed value is the binary equivalent of 1 second 621 expressed in picoseconds. 623 The below table represents a device which has a TimeBase of 624 picoseconds (or 11). The smallest and simplest value to represent is 625 1 picosecond; the time value stored is 1, and the scaling value is 0. 626 Using values from the table below, we have: 628 Time value in Encoded Scaling 629 Delta time picoseconds value decimal 630 -------------------------------------------------------- 631 1 picosecond 1 1 0 632 1 nanosecond 3E8 3E8 0 633 1 microsecond F4240 F424 4 634 1 millisecond 3B9ACA00 3B9A 16 635 1 second E8D4A51000 E8D4 24 636 1 minute 3691D6AFC000 3691 32 637 1 hour cca2e51310000 CCA2 36 638 1 day 132f4579c980000 132F 44 639 365 days 1b5a660ea44b80000 1B5A 52 641 Sample binary values (high order 16 bits taken) 643 1 psec 1 0001 644 1 nsec 3E8 0011 1110 1000 645 1 usec F4240 1111 0100 0010 0100 0000 646 1 msec 3B9ACA00 0011 1011 1001 1010 1100 1010 0000 0000 647 1 sec E8D4A51000 1110 1000 1101 0100 1010 0101 0001 0000 0000 0000 649 4.6 Limitations with this encoding method 651 According to the specification in [TRAM-TCPM] the size of one such 652 time-interval field is limited to this 11-bit value and 5-bit 653 unsigned scale so that they fit into a 16 bit space. With that 654 limitation, the maximum value that could be stored in 16 bits is: 656 11-bit value Scale 657 ============= ====== 658 1111 1111 111 1 1111 660 or an encoded value of 3FF and a scale value of 31. This value, in 661 microseconds, corresponds to any time differential between: 663 || 664 11 1111 1111 1000 0000 0000 0000 0000 0000 0000 0000 (binary) 665 3 F F 8 0 0 0 0 0 0 0 (hexadecimal) 667 and 669 11 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 (binary) 670 3 F F F F F F F F F F (hexadecimal) 672 The latter time value, 3FFFFFFFFFF, converts to 50 days, 21 hours 40 673 minutes and 46.511103 seconds. A time differential 1 microsecond 674 longer won't fit into 16 bits using this encoding scheme. 676 Thus, PDM separates the scaling value from the interval value, giving 677 a full 16 bits for the interval (DELTA) values and introducing a 678 signed scaling value that can range from -128 to +127. 680 4.7 Lack of precision induced by timer value truncation 682 As PDM specifies the DELTA values as 16-bit unsigned values, any time 683 the precision is greater than those 16 bits, there will be truncation 684 of the trailing bits, with an obvious loss precision in the value. 685 Using microseconds as the Time Base, the range of values that would 686 be truncated to the same encoded value is 2**(Scale)-1 microseconds. 688 The smallest time differential that would be truncated is 690 1 0000 0000 0000 0000 = 65536 usec 692 The value 694 1 0000 0000 0000 0001 = 65537 usec 696 would be truncated to the same encoded value, which is 8000 in hex 697 with a Scale value of 1. When the Scale value is 1, the value range 698 is calculated as 2**1 - 1, or 1 microsecond, which you can see is the 699 difference between these minimum and maximum values. 701 With that in mind, let's look at that table of DELTA time values 702 again, where the Precision is the range from the smallest value 703 corresponding to this encoded value to the largest: 705 Time value in Encoded 706 Delta time microseconds value Scale Precision 707 1 microsecond 1 1 0 0:00.000000 708 1 millisecond 38E 38E 0 0:00.000000 709 1 second F4240 F424 4 0:00.000015 710 1 minute 3938700 3938 12 0:00.004095 711 1 hour D693A400 D693 16 0:00.065535 712 1 day 141DD76000 141D 24 0:16.777215 713 Maximum value 3FFFFFFFFFF FFFF 36 19:05:19.476735 715 This maximum DELTA value is nearly 52,125 days. At the greatest 716 value, you can be assured of accuracy within about 19 hours. More 717 simply, response times in the range of a few seconds can be encoded 718 with a loss of precision no greater than a tenth of a millisecond. 719 Most importantly, response times shorter than 65.5 milliseconds can 720 be encoded with no loss of precision. 722 5 PDM Flow - Simple Client Server 724 Following is a sample simple flow for the PDM with one packet sent 725 from Host A and one packet received by Host B. The PDM does not 726 require time synchronization between Host A and Host B. The 727 calculations to derive meaningful metrics for network diagnostics are 728 shown below each packet sent or received. 730 Each packet, in addition to the PDM contains information on the 731 sender and receiver. As discussed before, a 5-tuple consists of: 733 SADDR : IP address of the sender 734 SPORT : Port for sender 735 DADDR : IP address of the destination 736 DPORT : Port for destination 737 PROTC : Protocol for upper layer (ex. TCP, UDP, ICMP) 739 It should be understood that the packet identification information is 740 in each packet. We will not repeat that in each of the following 741 steps. 743 5.1 Step 1 745 Packet 1 is sent from Host A to Host B. The time for Host A is set 746 initially to 10:00AM. 748 The time and packet sequence number are saved by the sender 749 internally. The packet sequence number and delta times are sent in 750 the packet. 752 Packet 1 754 +----------+ +----------+ 755 | | | | 756 | Host | ----------> | Host | 757 | A | | B | 758 | | | | 759 +----------+ +----------+ 761 PDM Contents: 763 PSNTP : Packet Sequence Number This Packet: 25 764 PSNLR : Packet Sequence Number Last Received: - 765 DELTATLR : Delta Time Last Received: - 766 SCALEDTLR: Scale of Delta Time Last Received: 0 767 DELTATLS : Delta Time Last Sent: - 768 SCALEDTLS: Scale of Delta Time Last Sent: 0 769 TIMEBASE : Granularity of Time: 00 (Milliseconds) 771 Internally, within the sender, Host A, it must keep: 773 Packet Sequence Number of the last packet sent: 25 774 Time the last packet was sent: 10:00:00 776 Note, the initial PSNTP from Host A starts at a random number. In 777 this case, 25. The time in these examples is shown in seconds for 778 the sake of simplicity. 780 5.2 Step 2 782 Packet 1 is received at Host B. Its time is set to one hour later 783 than Host A. In this case, 11:00AM 785 Internally, within the receiver, Host B, it must note: 787 Packet Sequence Number of the last packet received: 25 788 Time the last packet was received : 11:00:03 790 Note, this timestamp is in Host B time. It has nothing whatsoever to 791 do with Host A time. The Packet Sequence Number of the last packet 792 received will become PSNLR which will be sent out in the packet sent 793 by Host B in the next step. The time last received will be used to 794 calculate the DELTALR value to be sent out in the packet sent by Host 795 B in the next step. 797 5.3 Step 3 799 Packet 2 is sent by Host B to Host A. Note, the initial packet 800 sequence number (PSNTP) from Host B starts at a random number. In 801 this case, 12. Before sending the packet, Host B does a calculation 802 of deltas. Since Host B knows when it is sending the packet, and it 803 knows when it received the previous packet, it can do the following 804 calculation: 806 Sending time : packet 2 - receive time : packet 1 807 We will call the result of this calculation: Delta Time Last Received 808 (DELTATLR) 810 That is: 812 Delta Time Last Received = (Sending time: packet 2 - receive time: 813 packet 1) 815 Note, both sending time and receive time are saved internally in Host 816 B. They do not travel in the packet. Only the Delta is in the 817 packet. 819 Assume that within Host B is the following: 821 Packet Sequence Number of the last packet received: 25 822 Time the last packet was received: 11:00:03 823 Packet Sequence Number of this packet: 12 824 Time this packet is being sent: 11:00:07 826 We can now calculate a delta value to be sent out in the packet. 827 DELTATLR becomes: 829 4 seconds = 11:00:07 - 11:00:03 831 This is the derived metric: Server Delay. The time and scaling 832 factor must be calculated. Then, this value, along with the packet 833 sequence numbers will be sent to Host A as follows: 835 Packet 2 837 +----------+ +----------+ 838 | | | | 839 | Host | <---------- | Host | 840 | A | | B | 841 | | | | 842 +----------+ +----------+ 844 PDM Contents: 846 PSNTP : Packet Sequence Number This Packet: 12 847 PSNLR : Packet Sequence Number Last Received: 25 848 DELTATLR : Delta Time Last Received: FA0 (4 seconds) 849 SCALEDTLR: Scale of Delta Time Last Received: 00 850 DELTATLS : Delta Time Last Sent: - 851 SCALEDTLS: Scale of Delta Time Last Sent: 0 852 TIMEBASE : Granularity of Time: 00 (Milliseconds) 854 The metric left to be calculated is the Round-Trip Delay. This will 855 be calculated by Host A when it receives Packet 2. 857 5.4 Step 4 859 Packet 2 is received at Host A. Remember, its time is set to one 860 hour earlier than Host B. Internally, it must note: 862 Packet Sequence Number of the last packet received: 12 863 Time the last packet was received : 10:00:12 865 Note, this timestamp is in Host A time. It has nothing whatsoever to 866 do with Host B time. 868 So, now, Host A can calculate total end-to-end time. That is: 870 End-to-End Time = Time Last Received - Time Last Sent 872 For example, packet 25 was sent by Host A at 10:00:00. Packet 12 was 873 received by Host A at 10:00:12 so: 875 End-to-End time = 10:00:12 - 10:00:00 or 12 (Server and Network RT 876 delay combined). This time may also be called total Overall Round- 877 trip time (which includes Network RTT and Host Response Time). 879 This derived metric we will call Delta Time Last Sent (DELTATLS) 881 We can now also calculate round trip delay. The formula is: 883 Round trip delay = (Delta Time Last Sent - Delta Time Last Received) 885 Or: 887 Round trip delay = 12 - 4 or 8 889 Now, the only problem is that at this point all metrics are in Host A 890 only and not exposed in a packet. To do that, we need a third packet. 892 Note: this simple example assumes one send and one receive. That 893 is done only for purposes of explaining the function of the PDM. In 894 cases where there are multiple packets returned, one would take the 895 time in the last packet in the sequence. The calculations of such 896 timings and intelligent processing is the function of post-processing 897 of the data. 899 5.5 Step 5 901 Packet 3 is sent from Host A to Host B. 903 +----------+ +----------+ 904 | | | | 905 | Host | ----------> | Host | 906 | A | | B | 907 | | | | 908 +----------+ +----------+ 910 PDM Contents: 912 PSNTP : Packet Sequence Number This Packet: 26 913 PSNLR : Packet Sequence Number Last Received: 12 914 DELTATLR : Delta Time Last Received: 0 915 SCALEDTLS: Scale of Delta Time Last Received 0 916 DELTATLS : Delta Time Last Sent: 2EE0 (12 seconds) 917 SCALEDTLR: Scale of Delta Time Last Received: 00 918 TIMEBASE : Granularity of Time: 00 (Milliseconds) 920 To calculate Two-Way Delay, any packet capture device may look at 921 these packets and do what is necessary. 923 6 Other Flows 925 What we have discussed so far is a simple flow with one packet sent 926 and one returned. Let's look at how PDM may be useful in other 927 types of flows. 929 6.1 PDM Flow - One Way Traffic 931 The flow on a particular session may not be a send-receive paradigm. 932 Let us consider some other situations. In the case of a one-way 933 flow, one might see the following: 935 Packet Sender PSN PSN Delta Time Delta Time 936 This Packet Last Recvd Last Recvd Last Sent 937 ===================================================================== 938 1 Server 1 0 0 0 939 2 Server 2 0 0 5 940 3 Server 3 0 0 12 941 4 Server 4 0 0 20 943 What does this mean and how is it useful? 944 In a one-way flow, only the Delta Time Last Sent will be seen as 945 used. Recall, Delta Time Last Sent is the difference between the 946 send of one packet from a device and the next. This is a measure of 947 throughput for the sender - according to the sender's point of view. 948 That is, it is a measure of how fast is the application itself (with 949 stack time included) able to send packets. 951 How might this be useful? If one is having a performance issue at 952 the client and sees that packet 2, for example, is sent after 5 953 microseconds from the server but takes 3 minutes to arrive at the 954 destination, then one may safely conclude that there are delays in 955 the path other than at the server which may be causing the delivery 956 issue of that packet. Such delays may include the network links, 957 middle-boxes, etc. 959 Now, true one-way traffic is quite rare. What people often mean by 960 "one-way" traffic is an application such as FTP where a group of 961 packets (for example, a TCP window size worth) is sent, then the 962 sender waits for acknowledgment. This type of flow would actually 963 fall into the "multiple-send" traffic model. 965 6.2 PDM Flow - Multiple Send Traffic 967 Assume that two packets are sent for each ACK from the server. For 968 example, a TCP flow will do this, per RFC1122 [RFC1122] Section- 969 4.2.3. 971 Packet Sender PSN PSN Delta Time Delta Time 972 This Packet Last Recvd Last Recvd Last Sent 973 ===================================================================== 974 1 Server 1 0 0 0 975 2 Server 2 0 0 5 976 3 Client 1 2 20 0 977 4 Server 3 1 10 15 979 How might this be used? 981 Notice that in packet 3, the client has a value of Delta Time Last 982 received of 20. Recall that Delta Time Last Received is the Send 983 time of packet 3 - receive time of packet 2. So, what does one know 984 now? In this case, Delta Time Last Received is the processing time 985 for the Client to send the next packet. 987 How to interpret this depends on what is actually being sent. 988 Remember, PDM is not being used in isolation, but to supplement the 989 fields found in other headers. Let's take some examples: 991 1. Client is sending a standalone TCP ACK. One would find this by 992 looking at the payload length in the IPv6 header and the TCP 993 Acknowledgement field in the TCP header. So, in this case, the 994 client is taking 20 units to send back the ACK. This may or may not 995 be interesting. 997 2. Client is sending data with the packet. Again, one would find 998 this by looking at the payload length in the IPv6 header and the TCP 999 Acknowledgement field in the TCP header. So, in this case, the 1000 client is taking 20 units to send back data. This may represent 1001 "User Think Time". Again, this may or may not be interesting, in 1002 isolation. But, if there is a performance problem receiving data at 1003 the server, then taken in conjunction with RTT or other packet timing 1004 information, this information may be quite interesting. 1006 Of course, one also needs to look at the PSN Last Received field to 1007 make sure of the interpretation of this data. That is, to make 1008 sure that the Delta Last Received corresponds to the packet of 1009 interest. 1011 The benefits of PDM are that we have such information available in a 1012 uniform manner for all applications and all protocols without 1013 extensive changes required to applications. 1015 6.3 PDM Flow - Multiple Send with Errors 1017 Let us now look at a case of how PDM may be able to help in a case of 1018 TCP retransmission and add to the information that is sent in the TCP 1019 header. 1021 Assume that three packets are sent with each send from the server. 1023 From the server, this is what is seen. 1025 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1026 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1027 ===================================================================== 1028 1 Server 1 0 0 0 123 100 1029 2 Server 2 0 0 5 223 100 1030 3 Server 3 0 0 5 333 100 1032 The client, however, does not receive all the packets. From the 1033 client, this is what is seen for the packets sent from the server. 1035 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1036 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1037 ===================================================================== 1038 1 Server 1 0 0 0 123 100 1039 2 Server 3 0 0 5 333 100 1041 Let's assume that the server now retransmits the packet. 1042 (Obviously, a duplicate acknowledgment sequence for fast retransmit 1043 or a retransmit timeout would occur. To illustrate the point, these 1044 packets are being left out.) 1046 So, then if a TCP retransmission is done, then from the client, this 1047 is what is seen for the packets sent from the server. 1049 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1050 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1051 ===================================================================== 1052 1 Server 4 0 0 30 223 100 1054 The server has resent the old packet 2 with TCP sequence number of 1055 223. The retransmitted packet now has a PSN This Packet value of 4. 1056 The Delta Last Sent is 30 - the time between sending the packet with 1057 PSN of 3 and this current packet. 1059 Let's say that packet 4 is lost again. Then, after some amount of 1060 time (RTO) then the packet with TCP sequence number of 223 is resent. 1062 From the client, this is what is seen for the packets sent from the 1063 server. 1065 Pkt Sender PSN PSN Delta Time Delta Time TCP Data 1066 This Pkt LastRecvd LastRecvd LastSent SEQ Bytes 1067 ===================================================================== 1068 1 Server 5 0 0 60 223 100 1070 If now, this packet arrives at the destination, one has a very good 1071 idea that packets exist which are being sent from the server as 1072 retransmissions and not arriving at the client. This is because the 1073 PSN of the resent packet from the server is 5 rather than 4. If we 1074 had used TCP sequence number alone, we would never have seen this 1075 situation. The TCP sequence number in all situations is 223. 1077 This situation would be experienced by the user of the application 1078 (the human being actually sitting somewhere) as a "hangs" or long 1079 delay between packets. On large networks, to diagnose problems such 1080 as these where packets are lost somewhere on the network, one has to 1081 take multiple traces to find out exactly where. 1083 The first thing is to start with doing a trace at the client and the 1084 server. So, we can see if the server sent a particular packet and 1085 the client received it. If the client did not receive it, then we 1086 start tracking back to trace points at the router right after the 1087 server and the router right before the client. Did they get these 1088 packets which the server has sent? This is a time consuming 1089 activity. 1091 With PDM, we can speed up the diagnostic time because we may be able 1092 to use only the trace taken at the client to see what the server is 1093 sending. 1095 7 Potential Overhead Considerations 1097 Questions have been posed as to the potential overhead of PDM. 1098 First, PDM is entirely optional. That is, a site may choose to 1099 implement PDM or not as they wish. If they are happy with the costs 1100 of PDM vs. the benefits, then the choice should be theirs. 1102 Below is a table outlining the potential overhead in terms of 1103 additional time to deliver the response to the end user for various 1104 assumed RTTs. 1106 Bytes RTT Bytes Bytes New Overhead 1107 in Packet Per Milli in PDM RTT 1108 ===================================================================== 1109 1000 1000 milli 1 16 1016.000 16.000 milli 1110 1000 100 milli 10 16 101.600 1.600 milli 1111 1000 10 milli 100 16 10.160 .160 milli 1112 1000 1 milli 1000 16 1.016 .016 milli 1114 Below are some examples of actual RTTs for packets traversing large 1115 enterprise networks. The first example is for packets going to 1116 multiple business partners. 1118 Bytes RTT Bytes Bytes New Overhead 1119 in Packet Per Milli in PDM RTT 1120 ===================================================================== 1121 1000 17 milli 58 16 17.360 .360 milli 1123 The second example is for packets at a large enterprise customer 1124 within a data center. Notice that the scale is now in microseconds 1125 rather than milliseconds. 1127 Bytes RTT Bytes Bytes New Overhead 1128 in Packet Per Micro in PDM RTT 1129 ===================================================================== 1130 1000 20 micro 50 16 20.320 .320 micro 1132 8 Security Considerations 1134 PDM does not introduce any new security weakness. 1136 8.1. SYN Flood and Resource Consumption Attacks 1138 PDM needs to calculate the deltas for time and keep track of the 1139 sequence numbers. This means that control blocks must be kept at the 1140 end hosts per 5-tuple. Any time a control block is kept, an 1141 attacker can try to mis-use the control blocks such that there is a 1142 compromise of the end host. 1144 PDM is used only at the end hosts and the control blocks are only 1145 kept at the end host and not at routers or middle boxes. Remember, 1146 PDM is an implementation of the Destination Option extension header. 1148 A "SYN flood" type of attack succeeds because a TCP SYN packet is 1149 small but it causes the end host to start creating a place holder for 1150 the session such that quite a bit of control block and other storage 1151 is used. This is an asynchronous type of attack in that a small 1152 amount of work by the attacker creates a large amount of work by the 1153 resource attacked. 1155 For PDM, the amount of data to be kept is quite small. That is, the 1156 control block is quite lightweight. Concerns about SYN Flood and 1157 other type of resource consumption attacks (memory, processing power, 1158 etc) can be alleviated by having a limit on the number of control 1159 block entries. 1161 We recommend that implementation of PDM SHOULD have a limit on the 1162 number of control block entries. 1164 8.2 Pervasive monitoring 1166 Since PDM passes in the clear, a concern arises as to whether the 1167 data can be used to fingerprint the system or somehow obtain 1168 information about the contents of the payload. 1170 Let us discuss fingerprinting of the end host first. It is possible 1171 that seeing the pattern of deltas or the absolute values could give 1172 some information as to the speed of the end host - that is, if it is 1173 a very fast system or an older, slow device. This may be useful to 1174 the attacker. However, if the attacker has access to PDM, the 1175 attacker also has access to the entire packet and could make such a 1176 deduction based merely on the time frames elapsed between packets 1177 WITHOUT PDM. 1179 As far as deducing the content of the payload, it appears to us that 1180 PDM is quite unhelpful in this regard. 1182 8.3 PDM as a Covert Channel 1184 PDM provides a set of fields in the packet which could be used to 1185 leak data. But, there is no real reason to suspect that PDM would 1186 be chosen rather than another part of the payload or another 1187 Extension Header. 1189 A firewall or another device could sanity check the fields within the 1190 PDM but randomly assigned sequence numbers and delta times might be 1191 expected to vary widely. The biggest problem though is how an 1192 attacker would get access to PDM in the first place to leak data. 1193 The attacker would have to either compromise the end host or have Man 1194 in the Middle (MitM). It is possible that either one could change 1195 the fields. But, then the other end host would get sequence numbers 1196 and deltas that don't make any sense. Presumably, one is using PDM 1197 and doing packet tracing for diagnostic purposes, so the changes 1198 would be obvious. It is conceivable that someone could compromise 1199 an end host and make it start sending packets with PDM without the 1200 knowledge of the host. But, again, the bigger problem is the 1201 compromise of the end host. Once that is done, the attacker 1202 probably has better ways to leak data. 1204 Having said that, an implementation SHOULD stop using PDM if it gets 1205 some number of "nonsensical" sequence numbers. 1207 9 IANA Considerations 1209 This draft requests an Option Type assignment in the Destination 1210 Options and Hop-by-Hop Options sub-registry of Internet Protocol 1211 Version 6 (IPv6) Parameters [ref to RFCs and URL below]. 1213 http://www.iana.org/assignments/ipv6-parameters/ipv6- 1214 parameters.xhtml#ipv6-parameters-2 1216 Hex Value Binary Value Description Reference 1217 act chg rest 1218 ------------------------------------------------------------------- 1219 TBD TBD Performance and [This draft] 1220 Diagnostic Metrics 1221 (PDM) 1223 10 References 1225 10.1 Normative References 1227 [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 1228 793, September 1981. 1230 [RFC1122] Braden, R., "Requirements for Internet Hosts -- 1231 Communication Layers", RFC 1122, October 1989. 1233 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1234 Requirement Levels", BCP 14, RFC 2119, March 1997. 1236 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1237 (IPv6) Specification", RFC 2460, December 1998. 1239 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1240 Delay Metric for IPPM", RFC 2681, September 1999. 1242 [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For 1243 Values In the Internet Protocol and Related Headers", BCP 37, RFC 1244 2780, March 2000. 1246 [RFC4303] Kent, S, "IP Encapsulating Security Payload (ESP)", RFC 1247 4303, December 2005. 1249 10.2 Informative References 1251 [TRAM-TCPM] Trammel, B., "Encoding of Time Intervals for the TCP 1252 Timestamp Option-01", Internet Draft, July 2013. [Work in Progress] 1254 [IBM-POPS] IBM Corporation, "IBM z/Architecture Principles of 1255 Operation", SA22-7832, 1990-2012 1257 11 Acknowledgments 1259 The authors would like to thank Keven Haining, Al Morton, Brian 1260 Trammel, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick 1261 Troth for their comments and assistance. 1263 Authors' Addresses 1265 Nalini Elkins 1266 Inside Products, Inc. 1267 36A Upper Circle 1268 Carmel Valley, CA 93924 1269 United States 1270 Phone: +1 831 659 8360 1271 Email: nalini.elkins@insidethestack.com 1272 http://www.insidethestack.com 1274 Robert Hamilton 1275 Chemical Abstracts Service 1276 A Division of the American Chemical Society 1277 2540 Olentangy River Road 1278 Columbus, Ohio 43202 1279 United States 1280 Phone: +1 614 447 3600 x2517 1281 Email: rhamilton@cas.org 1282 http://www.cas.org 1284 Michael S. Ackermann 1285 Blue Cross Blue Shield of Michigan 1286 P.O. Box 2888 1287 Detroit, Michigan 48231 1288 United States 1289 Phone: +1 310 460 4080 1290 Email: mackermann@bcbsm.com 1291 http://www.bcbsm.com