idnits 2.17.1 draft-amf-ippm-route-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC2330, updated by this document, for RFC5378 checks: 1998-05-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 26, 2017) is 2345 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2460' is defined on line 832, but no explicit reference was found in the text == Unused Reference: 'RFC2675' is defined on line 836, but no explicit reference was found in the text == Unused Reference: 'RFC4494' is defined on line 849, but no explicit reference was found in the text == Unused Reference: 'RFC5644' is defined on line 864, but no explicit reference was found in the text == Unused Reference: 'RFC6282' is defined on line 873, but no explicit reference was found in the text == Unused Reference: 'RFC6437' is defined on line 878, but no explicit reference was found in the text == Unused Reference: 'RFC6564' is defined on line 883, but no explicit reference was found in the text == Unused Reference: 'RFC7045' is defined on line 892, but no explicit reference was found in the text == Unused Reference: 'I-D.brockners-inband-oam-data' is defined on line 925, but no explicit reference was found in the text == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-00 ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) ** Downref: Normative reference to an Informational RFC: RFC 2991 ** Downref: Normative reference to an Informational RFC: RFC 5835 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 7820 Summary: 7 errors (**), 0 flaws (~~), 11 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Alvarez-Hamelin 3 Internet-Draft Universidad de Buenos Aires 4 Updates: 2330 (if approved) A. Morton 5 Intended status: Standards Track AT&T Labs 6 Expires: April 29, 2018 J. Fabini 7 TU Wien 8 October 26, 2017 10 Advanced Unidirectional Route Assessment 11 draft-amf-ippm-route-01 13 Abstract 15 This memo introduces an advanced unidirectional route assessment 16 metric and associated measurement methodology, based on the IP 17 Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC 18 2330 in the areas of path-related terminology and path description, 19 primarily to include the possibility of parallel subpaths between a 20 given Source and Destination pair, owing to the presence of multi- 21 path technologies. 23 Requirements Language 25 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 26 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 27 document are to be interpreted as described in RFC 2119 [RFC2119]. 29 Status of This Memo 31 This Internet-Draft is submitted in full conformance with the 32 provisions of BCP 78 and BCP 79. 34 Internet-Drafts are working documents of the Internet Engineering 35 Task Force (IETF). Note that other groups may also distribute 36 working documents as Internet-Drafts. The list of current Internet- 37 Drafts is at https://datatracker.ietf.org/drafts/current/. 39 Internet-Drafts are draft documents valid for a maximum of six months 40 and may be updated, replaced, or obsoleted by other documents at any 41 time. It is inappropriate to use Internet-Drafts as reference 42 material or to cite them other than as "work in progress." 44 This Internet-Draft will expire on April 29, 2018. 46 Copyright Notice 48 Copyright (c) 2017 IETF Trust and the persons identified as the 49 document authors. All rights reserved. 51 This document is subject to BCP 78 and the IETF Trust's Legal 52 Provisions Relating to IETF Documents 53 (https://trustee.ietf.org/license-info) in effect on the date of 54 publication of this document. Please review these documents 55 carefully, as they describe your rights and restrictions with respect 56 to this document. Code Components extracted from this document must 57 include Simplified BSD License text as described in Section 4.e of 58 the Trust Legal Provisions and are provided without warranty as 59 described in the Simplified BSD License. 61 Table of Contents 63 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 64 1.1. Issues with Earlier Work to define Route . . . . . . . . 3 65 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 66 3. Route Metric Terms and Definitions . . . . . . . . . . . . . 5 67 3.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 5 68 3.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 69 3.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 6 70 3.4. Related Round-Trip Delay and Loss Definitions . . . . . . 8 71 3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 8 72 3.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 9 73 4. Route Assessment Methodologies . . . . . . . . . . . . . . . 9 74 4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 10 75 4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 11 76 4.3. Combining Different Methods . . . . . . . . . . . . . . . 12 77 5. Background on Round-Trip Delay Measurement Goals . . . . . . 13 78 6. Tools to Measure Delays in the Internet . . . . . . . . . . . 14 79 7. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 15 80 8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 16 81 9. Security Considerations . . . . . . . . . . . . . . . . . . . 17 82 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 83 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 84 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 17 85 12.1. Normative References . . . . . . . . . . . . . . . . . . 17 86 12.2. Informative References . . . . . . . . . . . . . . . . . 20 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 89 1. Introduction 91 The IETF IP Performance Metrics (IPPM) working group first created a 92 framework for metric development in [RFC2330]. This framework has 93 stood the test of time and enabled development of many fundamental 94 metrics. It has been updated in the area of metric composition 95 [RFC5835], and in several areas related to active stream measurement 96 of modern networks with reactive properties [RFC7312]. 98 The [RFC2330] framework motivated the development of "performance and 99 reliability metrics for paths through the Internet," and Section 5 of 100 [RFC2330] defines terms that support description of a path under 101 test. However, metrics for assessment of path components and related 102 performance aspects had not been attempted in IPPM when the [RFC2330] 103 framework was written. 105 This memo takes-up the route measurement challenge and specifies a 106 new route metric, two practical frameworks for methods of measurement 107 (using either active or hybrid active-passive methods [RFC7799]), and 108 round-trip delay and link information discovery using the results of 109 measurements. 111 1.1. Issues with Earlier Work to define Route 113 Section 7 of [RFC2330] presented a simple example of a "route" metric 114 along with several other examples. The example is reproduced below 115 (where the reference is to Section 5 of [RFC2330]): 117 "route: The path, as defined in Section 5, from A to B at a given 118 time." 120 This example provides a starting point to develop a more complete 121 definition of route. Areas needing clarification include: 123 Time: In practice, the route will be assessed over a time interval, 124 because active path detection methods like [PT] rely on TTL limits 125 for their operation and cannot accomplish discovery of all hosts 126 using a single packet. 128 Type-P: The legacy route definition lacks the option to cater for 129 packet-dependent routing. In this memo, we assess the route for a 130 specific packet of Type-P, and reflect this in the metric 131 definition. The methods of measurement determine the specific 132 Type-P used. 134 Parallel Paths: This a reality of Internet paths and a strength of 135 advanced route assessment methods, so the metric must acknowledge 136 this possibility. Use of Equal Cost Multi-Path (ECMP) and Unequal 137 Cost Multi-Path (UCMP) technologies are common sources of parallel 138 subpaths. 140 Cloud Subpath: May contain hosts that do not decrement TTL or Hop 141 Limit, but may have two or more exchange links connecting 142 "discoverable" hosts or routers. Parallel subpaths contained 143 within clouds cannot be discovered. The assessment methods only 144 discover hosts or routers on the path that decrement TTL or Hop 145 Count, or cooperate with interrogation protocols. The presence of 146 tunnels and nested tunnels further complicate assessment by hiding 147 hops. 149 Hop: Although the [RFC2330] definition was a link-host pair, only 150 hosts are discoverable or have the capability to cooperate with 151 interrogation protocols where link information may be exposed. 153 The refined definition of Route metrics begins in the sections that 154 follow. 156 2. Scope 158 The purpose of this memo is to add new route metrics and methods of 159 measurement to the existing set of IPPM metrics. 161 The scope is to define route metrics that can identify the path taken 162 by a packet or a flow traversing the Internet between any two hosts. 164 <@@@@ or only hosts communicating at the IP layer? We would have to 165 re-define the Src and Dst Parameters and Host Identity if we 166 generalize beyond IP. Should we include MPLS and the capabilities of 167 [RFC8029], with explicit multipath identification (section 6.2.6)? > 169 Also, to specify a framework for active methods of measurement which 170 use the techniques described in [PT] at a minimum, and a framework 171 for hybrid active-passive methods of measurement, such as the Hybrid 172 Type I method [RFC7799] described in 173 [I-D.ietf-ippm-ioam-data](intended only for single administrative 174 domains), which do not rely on ICMP and provide a protocol for 175 explicit interrogation of nodes on a path. Combinations of active 176 methods and hybrid active-passive methods are also in-scope. 178 Further, this memo provides additional analysis of the round-trip 179 delay measurements made possible by the methods, in an effort to 180 discover more details about the path, such as the link technology in 181 use. 183 This memo updates Section 5 of [RFC2330] in the areas of path-related 184 terminology and path description, primarily to include the 185 possibility of parallel subpaths between a given Source and 186 Destination address pair (possibly resulting from Equal Cost Multi- 187 Path (ECMP) and Unequal Cost Multi-Path (UCMP) technologies). 189 There are several simple non-goals of this memo. There is no attempt 190 to assess the reverse path from any host on the path to the host 191 attempting the path measurement. The reverse path contribution to 192 delay will be that experienced by ICMP packets (in active methods), 193 and may be different from UDP or TCP packets. Also, the round trip 194 delay will include an unknown contribution of processing time at the 195 host that generates the ICMP response. Therefore, the ICMP-based 196 active methods are not supposed to yield accurate, reproducible 197 estimations of the round-trip delay that UDP or TCP packets will 198 experience. 200 3. Route Metric Terms and Definitions 202 This section sets requirements for the following components to 203 support the Route Metric: 205 Note: the definitions concentrate on the IP-layer, but can be 206 extended to other layers, and follow agreements on the scope. 208 Host Identity For hosts communicating at the IP-layer, the globally 209 routable IP address(es) which the host uses when communicating 210 with other hosts under normal or error conditions. The Host 211 Identity revealed (and its connection to a Host Name through 212 reverse DNS) determines whether interfaces to parallel links can 213 be associated with a single host, or appear to be unique hosts. 215 Discoverable Host For hosts communicating at the IP-layer, 216 compliance with Section 3.2.2.4 of [RFC1122] when discarding a 217 packet due to TTL or Hop Limit Exceeded condition, MUST result in 218 sending the corresponding Time Exceeded message (containing a form 219 of host identity) to the source. This requirement is also 220 consistent with section 5.3.1 of [RFC1812] for routers. 222 Cooperating Host Hosts MUST respond to direct queries for their host 223 identity as part of a previously agreed and established 224 interrogation protocol. Hosts SHOULD also provide information 225 such as arrival/departure interface identification, arrival 226 timestamp, and any relevant information about the host or specific 227 link which delivered the query to the host. 229 Hop A Hop MUST contain a Host Identity, and MAY contain arrival and/ 230 or departure interface identification. 232 3.1. Formal Name 234 Type-P-Route-Ensemble-Method-Variant, abbreviated as Route Ensemble. 236 Note that Type-P depends heavily on the chosen method and variant. 238 3.2. Parameters 240 This section lists the REQUIRED input factors to specify a Route 241 metric. 243 o Src, the IP address of a host 245 o Dst, the IP address of a host 247 o i, the TTL or Hop Limit of a packet sent from the host at Src to 248 the host at Dst. 250 o MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops). 252 o T0, a time (start of measurement interval) 254 o Tf, a time (end of measurement interval) 256 o T, the host time of a packet as measured at MP(Src), meaning 257 Measurement Point at the Source. 259 o Ta, the host time of a reply packet's *arrival* as measured at 260 MP(Src), assigned to packets that arrive within a "reasonable" 261 time (see parameter below). 263 o Tmax, a maximum waiting time for reply packets to return to the 264 source, set sufficiently long to disambiguate packets with long 265 delays from packets that are discarded (lost), thus the 266 distribution of delay is not truncated. 268 o F, the number of different flows simulated by the method and 269 variant. 271 o flow, the stream of packets with the same n-tuple of designated 272 header fields that (when held constant) results in identical 273 treatment in a multi-path decision (such as that taken in load 274 balancing). 276 o Type-P, the complete description of the packets for which this 277 assessment applies (including the flow-defining fields). 279 3.3. Metric Definitions 281 This section defines the REQUIRED measurement components of the Route 282 metrics (unless otherwise indicated): 284 M, the total number of packets sent between T0 and Tf. 286 N, the smallest value of i needed for a packet to be received at Dst 287 (sent between T0 and Tf). 289 Nmax, the largest value of i needed for a packet to be received at 290 Dst (sent between T0 and Tf). Nmax may be equal to N. 292 Next, define a *singleton* definition for a Hop on the path, with 293 sufficient indexes to identify all Hops identified in a measurement 294 interval. 296 A Hop, designated h(i,j), the IP address and/or identity of one of j 297 Discoverable Hosts (or Cooperating Hosts) that are i hops away from 298 the host with IP address = Src during the measurement interval, T0 to 299 Tf. As defined above, a Hop singleton measurement MUST contain a 300 Host Identity, hid(i,j), and MAY contain one or more of the following 301 attributes: 303 o a(i,j) Arrival Interface ID 305 o d(i,j) Departure Interface ID 307 o t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the 308 hop, or approximated from the sending time of the packet that 309 revealed the hop) 311 o Measurements of Round Trip Delay (for each packet that reveals the 312 same Host Identity and attributes, but not timestamp of course, 313 see next section) 315 Now that Host Identities and related information can be positioned 316 according to their distance from the host with address Src in hops, 317 we introduce two forms of Routes: 319 A Route Ensemble is defined as the combination of all routes 320 traversed by different flows from the host at Src address to the host 321 at Dst address. The route traversed by each flow (with addresses Src 322 and Dst, and other fields which constitute flow criteria) is a member 323 of the ensemble and called a Member Route. 325 Using h(i,j) and components and parameters, further define: 327 A Member Route is an ordered graph {h(1,j), ... h(Nj, j)} in the 328 context of a single flow, where h(i-1, j) and h(i, j) are by 1 hop 329 away from each other and Nj=Dst is the minimum TTL value needed by 330 the packet on Member Route j to reach Dst. Member Routes must be 331 unique. This uniqueness requires that any two Member routes j and k 332 that are part of the same Route Ensemble differ either in terms of 333 minimum hop count Nj and Nk to reach the destination Dst, or, in the 334 case of identical hop count Nj=Nk, they have at least one distinct 335 hop: h(i,j) != h(i, k) for at least one i (i=1..Nj). 337 The Route Ensemble from Src to Dst, during the measurement interval 338 T0 to Tf, is the aggregate of all m distinct Member Routes discovered 339 between the two hosts with Src and Dst addresses. More formally, 340 with the host having address Src omitted: 342 Route Ensemble = { 343 {h(1,1), h(2,1), h(3,1), ... h(N1,1)=Dst}, 344 {h(1,2), h(2,2), h(3,2),..., h(N2,2)=Dst}, 345 ... 346 {h(1,m), h(2,m), h(3,m), ....h(Nm,m)=Dst} 347 } 349 where the following conditions apply: i <= Nj <= Nmax (j=1..m) 351 Note that some h(i,j) may be empty (null) in the case that systems do 352 not reply (not discoverable, or not cooperating). 354 h(i-1,j) and h(i,j) are the Hops on the same Member Route one hop 355 away from each other. 357 Hop h(i,j) may be identical with h(k,l) for i!=k and j!=l ; which 358 means there may be portions shared among different Member Routes 359 (parts of various routes may overlap). 361 3.4. Related Round-Trip Delay and Loss Definitions 363 RTD(i,j,T) is defined as a singleton of the [RFC2681] Round-trip 364 Delay between the host with IP address = Src and the host at Hop 365 h(i,j) at time T. 367 RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss 368 between the host with IP address = Src and the host at Hop h(i,j) at 369 time T. 371 3.5. Discussion 373 Depending on the way that Host Identity is revealed, it may be 374 difficult to determine parallel subpaths between the same pair of 375 hosts (i.e. multiple parallel links). It is easier to detect 376 parallel subpaths involving different hosts. 378 o If a pair of discovered hosts identify two different IP addresses, 379 then they will appear to be different hosts. 381 o If a pair of discovered hosts identify two different IP addresses, 382 and the IP addresses resolve to the same host name (in the DNS), 383 then they will appear to be the same hosts. 385 o If a discovered host always replies using the same IP address, 386 regardless of the interface a packet arrives on, then multiple 387 parallel links cannot be detected at the IP layer. 389 o If parallel links between routers are aggregated below the IP 390 layer, In other words, all links share the same pair of IP 391 addresses, then the existence of these parallel links can't be 392 detected at IP layer. 394 Section 9.2 of [RFC2330] describes Temporal Composition of metrics, 395 and introduces the possibility of a relationship between earlier 396 measurement results and the results for measurement at the current 397 time (for a given metric). If this topic is investigated further, 398 there may be some value in establishing a Temporal Composition 399 relationship for Route Metrics. However, this relationship does not 400 represent a forecast of future route conditions in any way. 402 When a route assessment employs packets at the IP layer (for 403 example), the reality of flow assignment to parallel subpaths 404 involves layers above IP. Thus, the measured Route Ensemble is 405 applicable to IP and higher layers (as described in the methodology's 406 packet of Type-P and flow parameters). 408 @@@@ Editor's Note: There is an opportunity to investigate and 409 discuss the RFC 2330 notion of equal treatment for a class of 410 packets, "...very useful to know if a given Internet component treats 411 equally a class C of different types of packets", as it applies to 412 Route measurements. Knowledge of "class C" parameters on a path 413 potentially reduces the number of flows required for a given method. 415 3.6. Reporting the Metric 417 @@@@ to be provided 419 4. Route Assessment Methodologies 421 There are two classes of methods described in this section, active 422 methods relying on the reaction to TTL or Hop Limit Exceeded 423 condition to discover hosts on a path, and Hybrid active-passive 424 methods that involve direct interrogation of cooperating hosts 425 (usually within a single domain). Description of these methods 426 follow. 428 @@@@ Editor's Note: We need to incorporate description of Type-P 429 packets (with the flow parameters) used in each method below. 431 4.1. Active Methodologies 433 We have chosen to describe the method based on that employed in 434 current open source tools, thereby providing a practical framework 435 for further advanced techniques to be included as method variants. 436 This method is applicable to use across multiple administrative 437 domains. 439 Paris-traceroute [PT] provides some measure of protection from path 440 variation generated by ECMP load balancing, and it ensures traceroute 441 packets will follow the same path in 98% of cases according to 442 [SCAMPER]. If it is necessary to find every path possible between 443 two hosts, Paris-traceroute provides "exhaustive" mode while scamper 444 provides "tracelb" (stands for traceroute load balance). 446 The Type-P of packets used could be ICMP (as ones in the original 447 traceroute), UDP and TCP. The later are used when a particular 448 characteristic is needed to verify, such as filtering or traffic 449 shaping on specific ports (i.e., services). 451 The advanced route assessment methods used in Paris-traceroute [PT] 452 keep the critical fields constant for every packet to maintain the 453 appearance of the same flow. Since route assessment can be conducted 454 using TCP, UDP or ICMP packets, this method REQUIRES the Diffserv 455 field, the protocol number, IP source and destination addresses, and 456 the port settings for TCP or UDP kept constant. For ICMP probes, the 457 method additionally REQUIRES the type, code, and ICMP checksum 458 constant; which take the same position in the header of an IP packet, 459 e.g., bytes 20 to 23 when the header IP has no options. 461 Maintaining a constant checksum in ICMP is most challenging because 462 the ICMP Sequence Number is part of the calculation. The advanced 463 traceroute method requires calculations using the IP Sequence Number 464 Field and the Identifier Field, yielding a constant ICMP checksum in 465 successive packets. For an example of calculations to maintain a 466 constant checksum, see Appendix A of [RFC7820], where revision of a 467 timestamp field is complemented by modifying the 2 octet checksum 468 complement field (these fields take the roles of the ICMP Sequence 469 Number Identifier Fields, respectively). 471 For TCP and UDP packets, the checksum must also be kept constant. 472 Therefore, the first four bytes of UDP (or TCP) data field are 473 modified to compensate for fields that change from packet to packet. 475 Note: other variants of advanced traceroute are planned be described. 477 Finally, the return path is also important to check. Taking into 478 account that it is an ICMP time exceeded (during transit) packet, the 479 source and destination IP are constant for every reply. Then, we 480 should consider the fields in the first 32 bits of the protocol on 481 the top of IP: the type and code of ICMP packet, and its checksum. 482 Again, to maintain the ICMP checksum constant for the returning 483 packets, we need to consider the whole ICMP message. It contains the 484 IP header of the discarded packet plus the first 8 bytes of the IP 485 payload; that is some of the fields of TCP header, the UDP header 486 plus four data bytes, the ICMP header plus four bytes. Therefore, 487 for UDP case the data field is used to maintain the ICMP checksum 488 constant in the returning packet. For the ICMP case, the identifier 489 and sequence fields of the sent ICMP probe are manipulated to be 490 constant. The TCP case presents no problem because its first eight 491 bytes will be the same for every packet probe. 493 Formally, to maintain the same flow in the measurements to a certain 494 hop, the Type-P-Route-Ensemble-Method-Variant packets should be[PT]: 496 o TCP case: Fields Src, Dst, port-Src, port_Dst, and Diffserv Field 497 should be the same. 499 o UDP case: Fields Src, Dst, port-Src, port-Dst, and Diffserv Field 500 should be the same, the UDP-checksum should change to maintain 501 constant the IP checksum of the ICMP time exceeded reply. Then, 502 the data length should be fixed, and the data field is used to 503 fixing it (consider that ICMP checksum uses its data field, which 504 contains the original IP header plus 8 bytes of UDP, where TTL, IP 505 identification, IP checksum, and UDP checksum changes). 507 o ICMP case: The Data field should compensate variations on TTL, IP 508 identification, and IP checksum for every packet. 510 Then, the way to identify different hops and attempts of the same 511 flow is: 513 o TCP case: The IP identification field. 515 o UDP case: The IP identification field. 517 o ICMP case: The IP identification field, and ICMP Sequence number. 519 4.2. Hybrid Methodologies 521 The Hybrid Type I methods provide an alternative method for Route 522 Member assessment. As mentioned in the Scope section, 523 [I-D.ietf-ippm-ioam-data] provides a possible set of data fields that 524 would support route identification. 526 In general, nodes in the measured domain would be equipped with 527 specific abilities: 529 1. The ingress node adds one or more fields to the measurement 530 packets, and identifies to other nodes in the domain that a route 531 assessment will be conducted using one or more specific packets. 532 The packets typically originate from a host outside the domain, 533 and constitute normal traffic on the domain. 535 2. Each node visited by the specific packet within in the domain 536 identifies itself in a data field of the packet (the field has 537 been added for this purpose). 539 3. When a measurement packet reaches the edge node of the domain, 540 the edge node adds its identity to the list, removes all the 541 identities from the packet, forwards the packet onward, and 542 communicates the ordered list of node identities to the intended 543 receiver. 545 In addition to node identity, nodes may also identify the ingress and 546 egress interfaces utilized by the tracing packet, the time of day 547 when the packet was processed, and other generic data (as described 548 in section 4 of [I-D.ietf-ippm-ioam-data]). 550 4.3. Combining Different Methods 552 In principle, there are advantages if the entity conducting Route 553 measurements can utilize both forms of advanced methods (active and 554 hybrid), and combine the results. For example, if there are hosts 555 involved in the path that qualify as Cooperating Hosts, but not as 556 Discoverable Hosts, then a more complete view of hops on the path is 557 possible when a hybrid method (or interrogation protocol) is applied 558 and the results are combined with the active method results collected 559 across all other domains. 561 In order to combine the results of active and hybrid/interrogation 562 methods, the network hosts that are part of a domain supporting an 563 interrogation protocol have the following attributes: 565 1. Hosts at the ingress to the domain SHOULD be both Discoverable 566 and Cooperating, and SHOULD reveal the same Host Identity in 567 response to both active and hybrid methods. 569 2. Any Hosts within the domain that are both Discoverable and 570 Cooperating SHOULD reveal the same Host Identity in response to 571 both active and hybrid methods. 573 3. Hosts at the egress to the domain SHOULD be both Discoverable and 574 Cooperating, and SHOULD reveal the same Host Identity in response 575 to both active and hybrid methods. 577 When Hosts follow these requirements, it becomes a simple matter to 578 match single domain measurements with the overlapping results from a 579 multidomain measurement. 581 In practice, Internet users do not typically have the ability to 582 utilize the OAM capabilities of networks that their packets traverse, 583 so the results from a remote domain supporting an interrogation 584 protocol would not normally be accessible. However, a network 585 operator could combine interrogation results from their access domain 586 with other measurements revealing the path outside their domain. 588 5. Background on Round-Trip Delay Measurement Goals 590 The aim of this method is to use packet probes to unveil the paths 591 between any two end-hosts of the network. Moreover, information 592 derived from RTD measurements might be meaningful to identify: 594 1. Intercontinental submarine links 596 2. Satellite communications 598 3. Congestion 600 4. Inter-domain paths 602 This categorization is widely accepted in the literature and among 603 operators alike, and it can be trusted with empirical data and 604 several sources as ground of truth (e.g., [RTTSub] [bdrmap][IDCong]). 606 The first two categories correspond to the physical distance 607 dependency on Round Trip Delay (RTD) while the last one binds RTD 608 with queueing delay on routers. Due to the significant contribution 609 of propagation delay in long distance hops, RTD will be at least 610 100ms on transatlantic hops, depending on the geolocation of the 611 vantage points. Moreover, RTD is typically greater than 480ms when 612 two hops are connected using geostationary satellite technology 613 (i.e., their orbit is at 36000km). Detecting congestion with latency 614 implies deeper mathematical understanding since network traffic load 615 is not stationary. Nonetheless, as the first approach, a link seems 616 to be congested if after sending several traceroute probes, it is 617 possible to detect congestion observing different statistics 618 parameters (e.g., see [IDCong]). 620 6. Tools to Measure Delays in the Internet 622 Internet routing is complex because it depends on the policies of 623 thousands Autonomous Systems (AS). While most of the routers perform 624 load balancing on flows using Equal Cost Multiple Path (ECMP), a few 625 still divide the workload through packet-based techniques. The 626 former scenario is defined according to [RFC2991] while the latter 627 generates a round-robin scheme to deliver every new outgoing packet. 628 ECMP keeps flow state in the router to ensure every packet of a flow 629 is delivered by the same path, and this avoids increasing the packet 630 delay variation and possibly producing overwhelming packet reordering 631 in TCP flows. 633 Taking into account that Internet protocol was designed under the 634 "end-to-end" principle, the IP payload and its header do not provide 635 any information about the routes or path necessary to reach some 636 destination. For this reason, the well-known tool traceroute was 637 developed to gather the IP addresses of each hop along a path using 638 the ICMP protocol [RFC0792]. Besides, traceroute adds the measured 639 RTD from each hop. However, the growing complexity of the Internet 640 makes it more challenging to develop accurate traceroute 641 implementation. For instance, the early traceroute tools would be 642 inaccurate in the current network, mainly because they were not 643 designed to retain flow state. However, evolved traceroute tools, 644 such as Paris-traceroute [PT] [MLB] and Scamper [SCAMPER], expect to 645 encounter ECMP and achieve more accurate results when they do. 647 Paris-traceroute-like tools operate in the following way: every 648 packet should follow the same path because the sensitive fields of 649 the header are controlled to appear as the same flow. This means 650 that source and destination IP addresses, source and destination port 651 numbers are the same in every packet. Additionally, Differentiated 652 Services Code Point (DSCP), checksum and ICMP code should remain 653 constant since they may affect the path selection. 655 Today's traceroute tools can send either UDP, TCP or ICMP packet 656 probes. Since ICMP header does not include transport layer 657 information, there are no fields for source and destination port 658 numbers. For this reason, these tools keep constant ICMP type, code, 659 and checksum fields to generate a kind of flow. However, the 660 checksum may vary in every packet, therefore when probes use ICMP 661 packets, ICMP Identifier and Sequence Number are manipulated to 662 maintain constant checksum in every packet. On the other hand, when 663 UDP probes are generated, the expected variation in the checksum of 664 each packet is again compensated by manipulating the payload. 666 Paris-traceroute allows its users to measure RTD in every hop of the 667 path for a particular flow. Furthermore, either Paris-traceroute or 668 Scamper is capable of unveiling the many available paths between a 669 source and destination (which are visible to this method). This task 670 is accomplished by repeating complete traceroute measurements with 671 different flow parameters for each measurement. The Framework for IP 672 Performance Metrics (IPPM) ([RFC2330] updated by[RFC7312]) has the 673 flexibility to require that the round-trip delay measurement 674 [RFC2681] uses packets with the constraints to assure that all 675 packets in a single measurement appear as the same flow. This 676 flexibility covers ICMP, UDP, and TCP. The accompanying methodology 677 of [RFC2681] needs to be expanded to report the sequential hop 678 identifiers along with RTD measurements, but no new metric definition 679 is needed. 681 7. RTD Measurements Statistics 683 Several articles have shown that network traffic presents a self- 684 similar nature [SSNT] [MLRM] which is accountable for filling the 685 queues of the routers. Moreover, router queues are designed to 686 handle traffic bursts, which is one of the most remarkable features 687 of self-similarity. Naturally, while queue length increases, the 688 delay to traverse the queue increases as well and leads to an 689 increase on RTD. Due to traffic bursts generate short-term overflow 690 on buffers (spiky patterns), every RTD only depicts the queueing 691 status on the instant when that packet probe was in transit. For 692 this reason, several RTD measurements during a time window could 693 begin to describe the random behavior of latency. Loss must also be 694 accounted for in the methodology. 696 To understand the ongoing process, examining the quartiles provides a 697 non-parametric way of analysis. Quartiles are defined by five 698 values: minimum RTD (m), RTD value of the 25% of the Empirical 699 Cumulative Distribution Function (ECDF) (Q1), the median value (Q2), 700 the RTD value of the 75% of the ECDF (Q3) and the maximum RTD (M). 701 Congestion can be inferred when RTD measurements are spread apart, 702 and consequently, the Inter-Quartile Range (IQR), the distance 703 between Q3 and Q1, increases its value. 705 This procedure requires to compute quartile values "on the fly" using 706 the algorithm presented in [P2]. 708 This procedure allow us to update the quartiles value whenever a new 709 measurement arrives, which is radically different from classic 710 methods of computing quartiles because they need to use the whole 711 dataset to compute the values. This way of calculus provides savings 712 in memory and computing time. 714 To sum up, the proposed measurement procedure consists in performing 715 traceroutes several times to obtain samples of the RTD in every hop 716 from a path, during a time window (W) and compute the quantiles for 717 every hop. This could be done for a single path flow or for every 718 detected path flow. 720 Even though a particular hop may be understood as the amount of hops 721 away from the source, a more detailed classification could be used. 722 For example, a possible classification may be identify ICMP Time 723 Exceeded packets coming from the same routers to those who have the 724 same hop distance, IP address of the router which is replying and TTL 725 value of the received ICMP packet. 727 Thus, the proposed methodology is based on this algorithm: 729 ================================================================ 730 1 input: W (window time of the measurement) 731 2 i_t (time between two measurements) 732 3 E (True: exhaustive, False: a single path) 733 4 Dst (destination IP address) 734 5 output: Qs (quartiles for every hop and alt in the path(s) to Dst) 735 ---------------------------------------------------------------- 736 6 T . 813 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 814 Communication Layers", STD 3, RFC 1122, 815 DOI 10.17487/RFC1122, October 1989, 816 . 818 [RFC1812] Baker, F., Ed., "Requirements for IP Version 4 Routers", 819 RFC 1812, DOI 10.17487/RFC1812, June 1995, 820 . 822 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 823 Requirement Levels", BCP 14, RFC 2119, 824 DOI 10.17487/RFC2119, March 1997, 825 . 827 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 828 "Framework for IP Performance Metrics", RFC 2330, 829 DOI 10.17487/RFC2330, May 1998, 830 . 832 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 833 (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460, 834 December 1998, . 836 [RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms", 837 RFC 2675, DOI 10.17487/RFC2675, August 1999, 838 . 840 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 841 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 842 September 1999, . 844 [RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and 845 Multicast Next-Hop Selection", RFC 2991, 846 DOI 10.17487/RFC2991, November 2000, 847 . 849 [RFC4494] Song, JH., Poovendran, R., and J. Lee, "The AES-CMAC-96 850 Algorithm and Its Use with IPsec", RFC 4494, 851 DOI 10.17487/RFC4494, June 2006, 852 . 854 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 855 Zekauskas, "A One-way Active Measurement Protocol 856 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 857 . 859 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 860 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 861 RFC 5357, DOI 10.17487/RFC5357, October 2008, 862 . 864 [RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance 865 Metrics (IPPM): Spatial and Multicast", RFC 5644, 866 DOI 10.17487/RFC5644, October 2009, 867 . 869 [RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for 870 Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April 871 2010, . 873 [RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6 874 Datagrams over IEEE 802.15.4-Based Networks", RFC 6282, 875 DOI 10.17487/RFC6282, September 2011, 876 . 878 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 879 "IPv6 Flow Label Specification", RFC 6437, 880 DOI 10.17487/RFC6437, November 2011, 881 . 883 [RFC6564] Krishnan, S., Woodyatt, J., Kline, E., Hoagland, J., and 884 M. Bhatia, "A Uniform Format for IPv6 Extension Headers", 885 RFC 6564, DOI 10.17487/RFC6564, April 2012, 886 . 888 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 889 DOI 10.17487/RFC6673, August 2012, 890 . 892 [RFC7045] Carpenter, B. and S. Jiang, "Transmission and Processing 893 of IPv6 Extension Headers", RFC 7045, 894 DOI 10.17487/RFC7045, December 2013, 895 . 897 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 898 Framework for IP Performance Metrics (IPPM)", RFC 7312, 899 DOI 10.17487/RFC7312, August 2014, 900 . 902 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 903 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 904 May 2016, . 906 [RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way 907 Active Measurement Protocol (OWAMP) and Two-Way Active 908 Measurement Protocol (TWAMP)", RFC 7820, 909 DOI 10.17487/RFC7820, March 2016, 910 . 912 [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., 913 Aldrin, S., and M. Chen, "Detecting Multiprotocol Label 914 Switched (MPLS) Data-Plane Failures", RFC 8029, 915 DOI 10.17487/RFC8029, March 2017, 916 . 918 12.2. Informative References 920 [bdrmap] Luckie, M., Dhamdhere, A., Huffaker, B., Clark, D., and 921 KC. Claffy, "bdrmap: Inference of Borders Between IP 922 Networks", In Proceedings of the 2016 ACM on Internet 923 Measurement Conference, pp. 381-396. ACM, 2016. 925 [I-D.brockners-inband-oam-data] 926 Brockners, F., Bhandari, S., Pignataro, C., Gredler, H., 927 Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov, 928 P., Chang, R., and d. daniel.bernier@bell.ca, "Data Fields 929 for In-situ OAM", draft-brockners-inband-oam-data-07 (work 930 in progress), July 2017. 932 [IDCong] Luckie, M., Dhamdhere, A., Clark, D., and B. Huffaker, 933 "Challenges in inferring Internet interdomain congestion", 934 In Proceedings of the 2014 Conference on Internet 935 Measurement Conference, pp. 15-22. ACM, 2014. 937 [MLB] Augustin, B., Friedman, T., and R. Teixeira, "Measuring 938 load-balanced paths in the Internet", Proceedings of the 939 7th ACM SIGCOMM conference on Internet measurement, pp. 940 149-160. ACM, 2007., 2007. 942 [MLRM] Fontugne, R., Mazel, J., and K. Fukuda, "An empirical 943 mixture model for large-scale RTT measurements", 2015 944 IEEE Conference on Computer Communications (INFOCOM), pp. 945 2470-2478. IEEE, 2015., 2015. 947 [P2] Jain, R. and I. Chlamtac, "The P 2 algorithm for dynamic 948 calculation of quantiles and histograms without storing 949 observations", Communications of the ACM 28.10 (1985): 950 1076-1085, 2015. 952 [PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F., 953 Friedman, T., Latapy, M., Magnien, C., and R. Teixeira, 954 "Avoiding traceroute anomalies with Paris traceroute", 955 Proceedings of the 6th ACM SIGCOMM conference on Internet 956 measurement, pp. 153-158. ACM, 2006., 2006. 958 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 959 Aitken, P., and A. Akhter, "A Framework for Large-Scale 960 Measurement of Broadband Performance (LMAP)", RFC 7594, 961 DOI 10.17487/RFC7594, September 2015, 962 . 964 [RTTSub] Bischof, Z., Rula, J., and F. Bustamante, "In and out of 965 Cuba: Characterizing Cuba's connectivity", In Proceedings 966 of the 2015 ACM Conference on Internet Measurement 967 Conference, pp. 487-493. ACM, 2015. 969 [SCAMPER] Matthew Luckie, M., "Scamper: a scalable and extensible 970 packet prober for active measurement of the Internet", 971 Proceedings of the 10th ACM SIGCOMM conference on 972 Internet measurement, pp. 239-245. ACM, 2010., 2010. 974 [SSNT] Park, K. and W. Willinger, "Self-Similar Network Traffic 975 and Performance Evaluation (1st ed.)", John Wiley & Sons, 976 Inc., New York, NY, USA, 2000. 978 Authors' Addresses 980 Jose Ignacio Alvarez-Hamelin 981 Universidad de Buenos Aires 982 Av. Paseo Colon 850 983 Buenos Aires C1063ACV 984 Argentine 986 Phone: +54 11 5285-0716 987 Email: ihameli@cnet.fi.uba.ar 988 URI: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ 989 Al Morton 990 AT&T Labs 991 200 Laurel Avenue South 992 Middletown, NJ 07748 993 USA 995 Phone: +1 732 420 1571 996 Fax: +1 732 368 1192 997 Email: acmorton@att.com 998 URI: http://home.comcast.net/~acmacm/ 1000 Joachim Fabini 1001 TU Wien 1002 Gusshausstrasse 25/E389 1003 Vienna 1040 1004 Austria 1006 Phone: +43 1 58801 38813 1007 Fax: +43 1 58801 38898 1008 Email: Joachim.Fabini@tuwien.ac.at 1009 URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/