idnits 2.17.1 draft-ietf-ippm-route-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document seems to lack the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords -- however, there's a paragraph with a matching beginning. Boilerplate error? (The document does seem to have the reference to RFC 2119 which the ID-Checklist requires). (Using the creation date from RFC2330, updated by this document, for RFC5378 checks: 1998-05-01) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 3, 2019) is 1608 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: 'RFC2460' is defined on line 1027, but no explicit reference was found in the text == Unused Reference: 'RFC2675' is defined on line 1031, but no explicit reference was found in the text == Unused Reference: 'RFC4494' is defined on line 1044, but no explicit reference was found in the text == Unused Reference: 'RFC5644' is defined on line 1064, but no explicit reference was found in the text == Unused Reference: 'RFC6282' is defined on line 1078, but no explicit reference was found in the text == Unused Reference: 'RFC6437' is defined on line 1083, but no explicit reference was found in the text == Unused Reference: 'RFC6564' is defined on line 1093, but no explicit reference was found in the text == Unused Reference: 'RFC7045' is defined on line 1102, but no explicit reference was found in the text == Outdated reference: A later version (-17) exists of draft-ietf-ippm-ioam-data-08 ** Downref: Normative reference to an Informational RFC: RFC 2330 ** Obsolete normative reference: RFC 2460 (Obsoleted by RFC 8200) ** Downref: Normative reference to an Informational RFC: RFC 2991 ** Downref: Normative reference to an Informational RFC: RFC 5835 ** Downref: Normative reference to an Informational RFC: RFC 7312 ** Downref: Normative reference to an Informational RFC: RFC 7799 ** Downref: Normative reference to an Experimental RFC: RFC 7820 Summary: 7 errors (**), 0 flaws (~~), 11 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group J. Alvarez-Hamelin 3 Internet-Draft Universidad de Buenos Aires 4 Updates: 2330 (if approved) A. Morton 5 Intended status: Standards Track AT&T Labs 6 Expires: May 6, 2020 J. Fabini 7 TU Wien 8 C. Pignataro 9 Cisco Systems, Inc. 10 R. Geib 11 Deutsche Telekom 12 November 3, 2019 14 Advanced Unidirectional Route Assessment (AURA) 15 draft-ietf-ippm-route-06 17 Abstract 19 This memo introduces an advanced unidirectional route assessment 20 (AURA) metric and associated measurement methodology, based on the IP 21 Performance Metrics (IPPM) Framework RFC 2330. This memo updates RFC 22 2330 in the areas of path-related terminology and path description, 23 primarily to include the possibility of parallel subpaths between a 24 given Source and Destination pair, owing to the presence of multi- 25 path technologies. 27 Requirements Language 29 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 30 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 31 "OPTIONAL" in this document are to be interpreted as described in BCP 32 14[RFC2119] [RFC8174] when, and only when, they appear in all 33 capitals, as shown here. 35 Status of This Memo 37 This Internet-Draft is submitted in full conformance with the 38 provisions of BCP 78 and BCP 79. 40 Internet-Drafts are working documents of the Internet Engineering 41 Task Force (IETF). Note that other groups may also distribute 42 working documents as Internet-Drafts. The list of current Internet- 43 Drafts is at https://datatracker.ietf.org/drafts/current/. 45 Internet-Drafts are draft documents valid for a maximum of six months 46 and may be updated, replaced, or obsoleted by other documents at any 47 time. It is inappropriate to use Internet-Drafts as reference 48 material or to cite them other than as "work in progress." 49 This Internet-Draft will expire on May 6, 2020. 51 Copyright Notice 53 Copyright (c) 2019 IETF Trust and the persons identified as the 54 document authors. All rights reserved. 56 This document is subject to BCP 78 and the IETF Trust's Legal 57 Provisions Relating to IETF Documents 58 (https://trustee.ietf.org/license-info) in effect on the date of 59 publication of this document. Please review these documents 60 carefully, as they describe your rights and restrictions with respect 61 to this document. Code Components extracted from this document must 62 include Simplified BSD License text as described in Section 4.e of 63 the Trust Legal Provisions and are provided without warranty as 64 described in the Simplified BSD License. 66 Table of Contents 68 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 69 1.1. Issues with Earlier Work to define Route . . . . . . . . 3 70 2. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 71 3. Route Metric Terms and Definitions . . . . . . . . . . . . . 5 72 3.1. Formal Name . . . . . . . . . . . . . . . . . . . . . . . 6 73 3.2. Parameters . . . . . . . . . . . . . . . . . . . . . . . 6 74 3.3. Metric Definitions . . . . . . . . . . . . . . . . . . . 7 75 3.4. Related Round-Trip Delay and Loss Definitions . . . . . . 9 76 3.5. Discussion . . . . . . . . . . . . . . . . . . . . . . . 9 77 3.6. Reporting the Metric . . . . . . . . . . . . . . . . . . 10 78 4. Route Assessment Methodologies . . . . . . . . . . . . . . . 10 79 4.1. Active Methodologies . . . . . . . . . . . . . . . . . . 10 80 4.1.1. Temporal Composition for Route Metrics . . . . . . . 12 81 4.1.2. Routing Class C Identification . . . . . . . . . . . 13 82 4.1.3. Intermediate Observation Point Route Measurement . . 14 83 4.2. Hybrid Methodologies . . . . . . . . . . . . . . . . . . 15 84 4.3. Combining Different Methods . . . . . . . . . . . . . . . 15 85 5. Background on Round-Trip Delay Measurement Goals . . . . . . 16 86 6. Tools to Measure Delays in the Internet . . . . . . . . . . . 17 87 7. RTD Measurements Statistics . . . . . . . . . . . . . . . . . 18 88 8. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 20 89 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 90 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 91 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 92 12. Appendix I MPLS Methods for Route Assessment . . . . . . . . 21 93 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 94 13.1. Normative References . . . . . . . . . . . . . . . . . . 21 95 13.2. Informative References . . . . . . . . . . . . . . . . . 24 96 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 98 1. Introduction 100 The IETF IP Performance Metrics (IPPM) working group first created a 101 framework for metric development in [RFC2330]. This framework has 102 stood the test of time and enabled development of many fundamental 103 metrics. It has been updated in the area of metric composition 104 [RFC5835], and in several areas related to active stream measurement 105 of modern networks with reactive properties [RFC7312]. 107 The [RFC2330] framework motivated the development of "performance and 108 reliability metrics for paths through the Internet," and Section 5 of 109 [RFC2330] defines terms that support description of a path under 110 test. However, metrics for assessment of path components and related 111 performance aspects had not been attempted in IPPM when the [RFC2330] 112 framework was written. 114 This memo takes-up the route measurement challenge and specifies a 115 new route metric, two practical frameworks for methods of measurement 116 (using either active or hybrid active-passive methods [RFC7799]), and 117 Round-Trip Delay and link information discovery using the results of 118 measurements. All route measurements are limited by the willingness 119 of hosts along the path to be discovered, to cooperate with the 120 methods used, or to recognize that the measurement operation is 121 taking place (such as when tunnels are present). 123 1.1. Issues with Earlier Work to define Route 125 Section 7 of [RFC2330] presented a simple example of a "route" metric 126 along with several other examples. The example is reproduced below 127 (where the reference is to Section 5 of [RFC2330]): 129 "route: The path, as defined in Section 5, from A to B at a given 130 time." 132 This example provides a starting point to develop a more complete 133 definition of route. Areas needing clarification include: 135 Time: In practice, the route will be assessed over a time interval, 136 because active path detection methods like [PT] rely on TTL limits 137 for their operation and cannot accomplish discovery of all hosts 138 using a single packet. 140 Type-P: The legacy route definition lacks the option to cater for 141 packet-dependent routing. In this memo, we assess the route for a 142 specific packet of Type-P, and reflect this in the metric 143 definition. The methods of measurement determine the specific 144 Type-P used. 146 Parallel Paths: Parallel paths are a reality of the Internet and a 147 strength of advanced route assessment methods, so the metric must 148 acknowledge this possibility. Use of Equal Cost Multi-Path (ECMP) 149 and Unequal Cost Multi-Path (UCMP) technologies are common sources 150 of parallel subpaths. 152 Cloud Subpath: May contain hosts that do not decrement TTL or Hop 153 Limit, but may have two or more exchange links connecting 154 "discoverable" hosts or routers. Parallel subpaths contained 155 within clouds cannot be discovered. The assessment methods only 156 discover hosts or routers on the path that decrement TTL or Hop 157 Count, or cooperate with interrogation protocols. The presence of 158 tunnels and nested tunnels further complicate assessment by hiding 159 hops. 161 Hop: Although the [RFC2330] definition of a hop was a link-host 162 pair, only hosts are discoverable or have the capability to 163 cooperate with interrogation protocols where link information may 164 be exposed. 166 The refined definition of Route metrics begins in the sections that 167 follow. 169 2. Scope 171 The purpose of this memo is to add new route metrics and methods of 172 measurement to the existing set of IPPM metrics. 174 The scope is to define route metrics that can identify the path taken 175 by a packet or a flow traversing the Internet between two hosts. 176 Although primarily intended for hosts communicating on the Internet 177 with IP, the definitions and metrics are constructed to be applicable 178 to other network domains, if desired. The methods of measurement to 179 assess the path may not be able to discover all hosts comprising the 180 path, but such omissions are often deterministic and explainable 181 sources of error. 183 Also, to specify a framework for active methods of measurement which 184 use the techniques described in [PT] at a minimum, and a framework 185 for hybrid active-passive methods of measurement, such as the Hybrid 186 Type I method [RFC7799] described in 187 [I-D.ietf-ippm-ioam-data](intended only for single administrative 188 domains), which do not rely on ICMP and provide a protocol for 189 explicit interrogation of nodes on a path. Combinations of active 190 methods and hybrid active-passive methods are also in-scope. 192 Further, this memo provides additional analysis of the round-trip 193 delay measurements made possible by the methods, in an effort to 194 discover more details about the path, such as the link technology in 195 use. 197 This memo updates Section 5 of [RFC2330] in the areas of path-related 198 terminology and path description, primarily to include the 199 possibility of parallel subpaths between a given Source and 200 Destination address pair (possibly resulting from Equal Cost Multi- 201 Path (ECMP) and Unequal Cost Multi-Path (UCMP) technologies). 203 There are several simple non-goals of this memo. There is no attempt 204 to assess the reverse path from any host on the path to the host 205 attempting the path measurement. The reverse path contribution to 206 delay will be that experienced by ICMP packets (in active methods), 207 and may be different from delays experienced by UDP or TCP packets. 208 Also, the round trip delay will include an unknown contribution of 209 processing time at the host that generates the ICMP response. 210 Therefore, the ICMP-based active methods are not supposed to yield 211 accurate, reproducible estimations of the Round-Trip Delay that UDP 212 or TCP packets will experience. 214 3. Route Metric Terms and Definitions 216 This section sets requirements for the following components to 217 support the Route Metric: 219 Host A Host as defined in [RFC2330] (a computer capable of IP 220 communication, includes routers), a.k.a. RFC 2330 Host. 222 Node A Node is any network function on the path capable of IP-layer 223 Communication, includes RFC 2330 Hosts. 225 Node Identity The unique address for Nodes communicating within the 226 network domain. For Nodes communicating on the Internet with IP, 227 it is the globally routable IP address(es) which the Node uses 228 when communicating with other Nodes under normal or error 229 conditions. The Node Identity revealed (and its connection to a 230 Node Name through reverse DNS) determines whether interfaces to 231 parallel links can be associated with a single Node, or appear to 232 identify unique Nodes. 234 Discoverable Node Nodes that convey their Node Identity according to 235 the requirements of their network domain, such as when error 236 conditions are detected by that Node. For Nodes communicating 237 with IP packets, compliance with Section 3.2.2.4 of [RFC1122] when 238 discarding a packet due to TTL or Hop Limit Exceeded condition, 239 MUST result in sending the corresponding Time Exceeded message 240 (containing a form of Node identity) to the source. This 241 requirement is also consistent with section 5.3.1 of [RFC1812] for 242 routers. 244 Cooperating Node Nodes that MUST respond to direct queries for their 245 Node identity as part of a previously agreed and established 246 interrogation protocol. Nodes SHOULD also provide information 247 such as arrival/departure interface identification, arrival 248 timestamp, and any relevant information about the Node or specific 249 link which delivered the query to the Node. 251 Hop A Hop MUST contain a Node Identity, and MAY contain arrival and/ 252 or departure interface identification, round trip delay, and an 253 arrival timestamp. 255 3.1. Formal Name 257 Type-P-Route-Ensemble-Method-Variant, abbreviated as Route Ensemble. 259 Note that Type-P depends heavily on the chosen method and variant. 261 3.2. Parameters 263 This section lists the REQUIRED input factors to specify a Route 264 metric. 266 o Src, the address of a Node (such as the globally routable IP 267 address). 269 o Dst, the address of a Node (such as the globally routable IP 270 address). 272 o i, the limit on the number of Hops a specific packet may visit as 273 it traverses from the Node at Src to the Node at Dst (such as the 274 TTL or Hop Limit). 276 o MaxHops, the maximum value of i used, (i=1,2,3,...MaxHops). 278 o T0, a time (start of measurement interval) 280 o Tf, a time (end of measurement interval) 282 o MP(address), Measurement Point at address, such as Src or Dst, 283 usually at the same node stack layer as "address". 285 o T, the Node time of a packet as measured at MP(Src), meaning 286 Measurement Point at the Source. 288 o Ta, the Node time of a reply packet's *arrival* as measured at 289 MP(Src), assigned to packets that arrive within a "reasonable" 290 time (see parameter below). 292 o Tmax, a maximum waiting time for reply packets to return to the 293 source, set sufficiently long to disambiguate packets with long 294 delays from packets that are discarded (lost), such that the 295 distribution of Round-Trip Delay is not truncated. 297 o F, the number of different flows simulated by the method and 298 variant. 300 o flow, the stream of packets with the same n-tuple of designated 301 header fields that (when held constant) result in identical 302 treatment in a multi-path decision (such as the decision taken in 303 load balancing). Note: The IPv6 flow label MAY be included in the 304 flow definition if the MP(Src) is a Tunnel End Point (TEP) 305 complying with [RFC6438] guidelines. 307 o Type-P, the complete description of the packets for which this 308 assessment applies (including the flow-defining fields). 310 3.3. Metric Definitions 312 This section defines the REQUIRED measurement components of the Route 313 metrics (unless otherwise indicated): 315 M, the total number of packets sent between T0 and Tf. 317 N, the smallest value of i needed for a packet to be received at Dst 318 (sent between T0 and Tf). 320 Nmax, the largest value of i needed for a packet to be received at 321 Dst (sent between T0 and Tf). Nmax may be equal to N. 323 Next, define a *singleton* definition for a Hop on the path, with 324 sufficient indexes to identify all Hops identified in a measurement 325 interval. 327 A Hop, designated h(i,j), the IP address and/or identity of one of j 328 Discoverable Nodes (or Cooperating Nodes) that are i hops away from 329 the Node with address = Src during the measurement interval, T0 to 330 Tf. As defined above, a Hop singleton measurement MUST contain a 331 Node Identity, hid(i,j), and MAY contain one or more of the following 332 attributes: 334 o a(i,j) Arrival Interface ID (e.g., when [RFC5837] is supported) 335 o d(i,j) Departure Interface ID (e.g., when [RFC5837] is supported) 337 o t(i,j) Arrival Timestamp (where t(i,j) is ideally supplied by the 338 Hop, or approximated from the sending time of the packet that 339 revealed the Hop) 341 o Measurements of Round-Trip Delay (for each packet that reveals the 342 same Node Identity and flow attributes, but not timestamp of 343 course, see next section) 345 Node Identities and related information can be ordered by their 346 distance from the Node with address Src in Hops h(i,j). Based on 347 this, two forms of Routes are distinguished: 349 A Route Ensemble is defined as the combination of all routes 350 traversed by different flows from the Node at Src address to the Node 351 at Dst address. A single Route traversed by a single flow 352 (determined by an unambiguous tuple of addresses Src and Dst, and 353 other identical flow criteria) is a member of the Route Ensemble and 354 called a Member Route. 356 Using h(i,j) and components and parameters, further define: 358 When considering the set of Hops in the context of a single flow, a 359 Member Route j is an ordered list {h(1,j), ... h(Nj, j)} where h(i-1, 360 j) and h(i, j) are by 1 hop away from each other and Nj satisfying 361 h(Nj,j)=Dst is the minimum count of Hops needed by the packet on 362 Member Route j to reach Dst. Member Routes must be unique. The 363 uniqueness property requires that any two Member routes j and k that 364 are part of the same Route Ensemble differ either in terms of minimum 365 hop count Nj and Nk to reach the destination Dst, or, in the case of 366 identical hop count Nj=Nk, they have at least one distinct Hop: 367 h(i,j) != h(i, k) for at least one i (i=1..Nj). 369 All the optional information collected to describe a Member Route, 370 such as the arrival interface, departure interface, and Round Trip 371 Delay at each Hop, turns each list item into a rich structure. There 372 may be information on the links between Hops, possibly information on 373 the routing (arrival interface and departure interface), an estimate 374 of distance between Hops based on Round-Trip Delay measurements and 375 calculations, and a time stamp indicating when all these additional 376 details were valid. 378 The Route Ensemble from Src to Dst, during the measurement interval 379 T0 to Tf, is the aggregate of all m distinct Member Routes discovered 380 between the two Nodes with Src and Dst addresses. More formally, 381 with the Node having address Src omitted: 383 Route Ensemble = { 384 {h(1,1), h(2,1), h(3,1), ... h(N1,1)=Dst}, 385 {h(1,2), h(2,2), h(3,2),..., h(N2,2)=Dst}, 386 ... 387 {h(1,m), h(2,m), h(3,m), ....h(Nm,m)=Dst} 388 } 390 where the following conditions apply: i <= Nj <= Nmax (j=1..m) 392 Note that some h(i,j) may be empty (null) in the case that systems do 393 not reply (not discoverable, or not cooperating). 395 h(i-1,j) and h(i,j) are the Hops on the same Member Route one hop 396 away from each other. 398 Hop h(i,j) may be identical with h(k,l) for i!=k and j!=l ; which 399 means there may be portions shared among different Member Routes 400 (parts of Member Routes may overlap). 402 3.4. Related Round-Trip Delay and Loss Definitions 404 RTD(i,j,T) is defined as a singleton of the [RFC2681] Round-Trip 405 Delay between the Node with address = Src and the Node at Hop h(i,j) 406 at time T. 408 RTL(i,j,T) is defined as a singleton of the [RFC6673] Round-trip Loss 409 between the Node with address = Src and the Node at Hop h(i,j) at 410 time T. 412 3.5. Discussion 414 Depending on the way that Node Identity is revealed, it may be 415 difficult to determine parallel subpaths between the same pair of 416 Nodes (i.e. multiple parallel links). It is easier to detect 417 parallel subpaths involving different Nodes. 419 o If a pair of discovered Nodes identify two different addresses, 420 then they will appear to be different Nodes. 422 o If a pair of discovered Nodes identify two different IP addresses, 423 and the IP addresses resolve to the same Node name (in the DNS), 424 then they will appear to be the same Nodes. 426 o If a discovered Node always replies using the same network 427 address, regardless of the interface a packet arrives on, then 428 multiple parallel links cannot be detected in that network domain. 429 This condition may apply to traceroute-style methods, but may not 430 apply to other hybrid methods based on In-situ Operations, 431 Administration, and Maintenance (IOAM). 433 o If parallel links between routers are aggregated below the IP 434 layer, then from Node point of view, all these links share the 435 same pair of IP addresses. The existence of these parallel links 436 can't be detected at IP layer. This applies to other network 437 domains with layers below them, as well. This condition may apply 438 to traceroute-style methods, but may not apply to other hybrid 439 methods based on IOAM. 441 When a route assessment employs IP packets (for example), the reality 442 of flow assignment to parallel subpaths involves layers above IP. 443 Thus, the measured Route Ensemble is applicable to IP and higher 444 layers (as described in the methodology's packet of Type-P and flow 445 parameters). 447 3.6. Reporting the Metric 449 An Information Model and an XML Data Model for Storing Traceroute 450 Measurements is available in [RFC5388]. The measured information at 451 each hop includes four pieces of information: a one-dimensional hop 452 index, Node symbolic address, Node IP address, and RTD for each 453 response. 455 The description of Hop information that may be collected according to 456 this memo covers more dimensions, as defined in Section 3.3 above. 457 For example, the Hop index is two-dimensional to capture the 458 complexity of a Route Ensemble, and it contains corresponding Node 459 identities at a minimum. The models need to be expanded to include 460 these features, as well as Arrival Interface ID, Departure Interface 461 ID, and Arrival Timestamp, when available. The original sending 462 Timestamp from the Src Node anchors a particular measurement in time. 464 4. Route Assessment Methodologies 466 There are two classes of methods described in this section, active 467 methods relying on the reaction to TTL or Hop Limit Exceeded 468 condition to discover Nodes on a path, and Hybrid active-passive 469 methods that involve direct interrogation of cooperating Nodes 470 (usually within a single domain). Description of these methods 471 follow. 473 4.1. Active Methodologies 475 This section describes the method employed by current open source 476 tools, thereby providing a practical framework for further advanced 477 techniques to be included as method variants. This method is 478 applicable for use across multiple administrative domains. 480 Paris-traceroute [PT] provides some measure of protection from path 481 variation generated by ECMP load balancing, and it ensures traceroute 482 packets will follow the same path in 98% of cases according to 483 [SCAMPER]. If it is necessary to find every usable path between two 484 Nodes, Paris-traceroute provides "exhaustive" mode while scamper 485 provides "tracelb" (stands for traceroute load balance). 487 The Type-P of packets used could be ICMP (as in the original 488 traceroute), UDP or TCP. The latter are used when a particular 489 characteristic needs to be to verified, such as filtering or traffic 490 shaping on specific ports (i.e., services). [SCAMPER] supports IPv6 491 traceroute measurements, keeping the FlowLabel constant in all 492 packets. 494 The advanced route assessment methods used in Paris-traceroute [PT] 495 keep the critical fields constant for every packet to maintain the 496 appearance of the same flow. Since route assessment can be conducted 497 using TCP, UDP or ICMP packets, this method REQUIRES the Diffserv 498 field, the protocol number, IP source and destination addresses, and 499 the port settings for TCP or UDP to be kept constant. For ICMP 500 probes, the method additionally REQUIRES keeping the type, code, and 501 ICMP checksum constant (the latter occupy the corresponding positions 502 in the header of an IP packet, e.g., bytes 20 to 23 when the header 503 IP has no options). 505 Maintaining a constant checksum in ICMP is most challenging because 506 the ICMP Sequence Number is part of the calculation. The advanced 507 traceroute method requires calculations using the IP Sequence Number 508 Field and the Identifier Field, yielding a constant ICMP checksum in 509 successive packets. For an example of calculations to maintain a 510 constant checksum, see Appendix A of [RFC7820], where revision of a 511 timestamp field is complemented by modifying the 2 octet checksum 512 complement field (these fields take the roles of the ICMP Sequence 513 Number and Identifier Fields, respectively). 515 For TCP and UDP packets, the checksum must also be kept constant. 516 Therefore, the first four bytes of UDP (or TCP) data field are 517 modified to compensate for fields that change from packet to packet. 519 Finally, the return path is also important to check. Taking into 520 account that it is an ICMP time exceeded (during transit) packet, the 521 source and destination IP are constant for every reply. Then, we 522 should consider the fields in the first 32 bits of the protocol on 523 the top of IP: the type and code of ICMP packet, and its checksum. 524 Again, to keep the ICMP checksum constant for the returning packets, 525 we need to consider the whole ICMP message. It contains the IP 526 header of the discarded packet plus the first 8 bytes of the IP 527 payload; that is some of the fields of the TCP header, the UDP header 528 plus four data bytes, the ICMP header plus four bytes. Therefore, 529 for the UDP case the data field is used to maintain the ICMP checksum 530 constant in the returning packet. For the ICMP case, the identifier 531 and sequence fields of the sent ICMP probe are manipulated to be 532 constant. The TCP case presents no problem because its first eight 533 bytes will be the same for every packet probe. 535 Formally, to maintain the same flow in the measurements to a certain 536 hop, the Type-P-Route-Ensemble-Method-Variant packets should be[PT]: 538 o TCP case: Fields Src, Dst, port-Src, port_Dst, and Diffserv Field 539 should be the same. 541 o UDP case: Fields Src, Dst, port-Src, port-Dst, and Diffserv Field 542 should be the same, the UDP-checksum should change to keep the IP 543 checksum of the ICMP time exceeded reply constant. Then, the data 544 length should be fixed, and the data field is used to fixing it 545 (consider that ICMP checksum uses its data field, which contains 546 the original IP header plus 8 bytes of UDP, where TTL or Hop 547 Limit, IP identification, IP checksum, and UDP checksum changes). 549 o ICMP case: The Data field should compensate variations on TTL or 550 Hop Limit, IP identification, and IP checksum for every packet. 552 Then, the way to identify different hops and attempts of the same 553 flow is: 555 o TCP case: The IP identification field. 557 o UDP case: The IP identification field. 559 o ICMP case: The IP identification field, and ICMP Sequence number. 561 4.1.1. Temporal Composition for Route Metrics 563 The Active Route Assessment Methods described above have the ability 564 to discover portions of a path where ECMP load balancing is present, 565 observed as two or more unique Member Routes having one or more 566 distinct Hops which are part of the Route Ensemble. Likewise, 567 attempts to deliberately vary the flow characteristics to discover 568 all Member Routes will reveal portions of the path which are flow- 569 invariant. 571 Section 9.2 of [RFC2330] describes Temporal Composition of metrics, 572 and introduces the possibility of a relationship between earlier 573 measurement results and the results for measurement at the current 574 time (for a given metric). There is value in establishing a Temporal 575 Composition relationship for Route Metrics. However, this 576 relationship does not represent a forecast of future route conditions 577 in any way. 579 For Route Metric measurements, the value of Temporal Composition is 580 to reduce the measurement iterations required with repeated 581 measurements. Reduced iterations are possible by inferring that 582 current measurements using fixed and previously measured flow 583 characteristics: 585 o will have many common hops with previous measurements. 587 o will have relatively time-stable results at the ingress and egress 588 portions of the path when measured from user locations, as opposed 589 to measurements of backbone networks and across inter-domain 590 gateways. 592 o may have greater potential for time-variation in path portions 593 where ECMP load balancing is observed (because increasing or 594 decreasing the pool of links changes the hash calculations). 596 Optionally, measurement systems may take advantage of the inferences 597 above when seeking to reduce measurement iterations, after exhaustive 598 measurements indicate that the time-stable properties are present. 599 Repetitive Active Route measurement systems: 601 1. SHOULD occasionally check path portions which have exhibited 602 stable results over time, particularly ingress and egress 603 portions of the path. 605 2. SHOULD continue testing portions of the path that have previously 606 exhibited ECMP load balancing. 608 3. SHALL trigger re-assessment of the complete path and Route 609 Ensemble, if any change in hops is observed for a specific (and 610 previously tested) flow. 612 4.1.2. Routing Class C Identification 614 There is an opportunity to apply the [RFC2330] notion of equal 615 treatment for a class of packets, "...very useful to know if a given 616 Internet component treats equally a class C of different types of 617 packets", as it applies to Route measurements. Knowledge of "class 618 C" parameters (unrelated to address classes of the past) on a path 619 potentially reduces the number of flows required for a given method 620 to assess a Route Ensemble over time. 622 First, recognize that each Member Route of a Route Ensemble will have 623 a corresponding Routing Class C. Class C can be discovered by 624 testing with multiple flows, all of which traverse the unique set of 625 hops that comprise a specific Member Route. 627 Second, recognize that the different Routing Classes depend primarily 628 on the hash functions used at each instance of ECMP load balancing on 629 the path. 631 Third, recognize the synergy with Temporal Composition methods 632 (described above) where evaluation intends to discover time-stable 633 portions of each Member Route so that more emphasis can be placed on 634 ECMP portions that also determine Class C. 636 The methods to assess the various Routing Class C characteristics 637 benefit from the following measurement capabilities: 639 o flows designed to determine which n-tuple header fields are 640 considered by a given hash function and ECMP hop on the path, and 641 which are not. This operation immediately narrows the search 642 space, where possible, and partially defines a Routing Class C. 644 o a priori knowledge of the possible types of hash functions in use 645 also helps to design the flows for testing (major router vendors 646 publish information about these hash functions, examples are here 647 https://www.researchgate.net/ 648 publication/281571413_COMPARISON_OF_HASH_STRATEGIES_FOR_FLOW- 649 BASED_LOAD_BALANCING ). 651 o ability to direct the emphasis of current measurements on ECMP 652 portions of the path, based on recent past measurement results 653 (the Routing Class C of some portions of the path is essentially 654 "all packets"). 656 4.1.3. Intermediate Observation Point Route Measurement 658 There are many examples where passive monitoring of a flow at an 659 Observation Point within the network can detect unexpected Round Trip 660 Delay or Delay Variation. But how can the cause of the anomalous 661 delay be investigated further --from the Observation Point -- 662 possibly located at an intermediate point on the path? 664 In this case, knowledge that the flow of interest belongs to a 665 specific Routing Class C will enable measurement of the route where 666 anomalous delay has been observed. Specifically, Round-Trip Delay 667 assessment to each Hop on the path between the Observation Point and 668 the Destination for the flow of interest may discover high or 669 variable delay on a specific link and Hop combination. 671 The determination of a Routing Class C which includes the flow of 672 interest is as described in the section above, aided by computation 673 of the relevant hash function output as the target. 675 4.2. Hybrid Methodologies 677 The Hybrid Type I methods provide an alternative method for Route 678 Member assessment. As mentioned in the Scope section, 679 [I-D.ietf-ippm-ioam-data] provides a possible set of data fields that 680 would support route identification. 682 In general, nodes in the measured domain would be equipped with 683 specific abilities: 685 o Support of the "Loopback" Flag (L-bit), where a copy of the packet 686 is returned to the encapsulating node, and the packet is processed 687 like any other IOAM packet on the return transfer. 689 In addition to node identity, nodes may also identify the ingress and 690 egress interfaces utilized by the tracing packet, the time of day 691 when the packet was processed, and other generic data (as described 692 in section 4 of [I-D.ietf-ippm-ioam-data]). Interface identification 693 isn't necessarily limited to IP, i.e. different links in a bundle 694 (LACP) could be identified. Equally well, links without explicit IP 695 addresses can be identified (like with unnumbered interfaces in an 696 IGP deployment). 698 Note that Type-P packet specification for this method will likely be 699 a partial specification, because most of the packet fields are 700 determined by the user traffic. The packet header(s) added by the 701 Hybrid method can certainly be specified in Type-P. 703 4.3. Combining Different Methods 705 In principle, there are advantages if the entity conducting Route 706 measurements can utilize both forms of advanced methods (active and 707 hybrid), and combine the results. For example, if there are Nodes 708 involved in the path that qualify as Cooperating Nodes, but not as 709 Discoverable Nodes, then a more complete view of Hops on the path is 710 possible when a hybrid method (or interrogation protocol) is applied 711 and the results are combined with the active method results collected 712 across all other domains. 714 In order to combine the results of active and hybrid/interrogation 715 methods, the network Nodes that are part of a domain supporting an 716 interrogation protocol have the following attributes: 718 1. Nodes at the ingress to the domain SHOULD be both Discoverable 719 and Cooperating, and SHOULD reveal the same Node Identity in 720 response to both active and hybrid methods. 722 2. Any Nodes within the domain that are both Discoverable and 723 Cooperating SHOULD reveal the same Node Identity in response to 724 both active and hybrid methods. 726 3. Nodes at the egress to the domain SHOULD be both Discoverable and 727 Cooperating, and SHOULD reveal the same Node Identity in response 728 to both active and hybrid methods. 730 When Nodes follow these requirements, it becomes a simple matter to 731 match single domain measurements with the overlapping results from a 732 multidomain measurement. 734 In practice, Internet users do not typically have the ability to 735 utilize the OAM capabilities of networks that their packets traverse, 736 so the results from a remote domain supporting an interrogation 737 protocol would not normally be accessible. However, a network 738 operator could combine interrogation results from their access domain 739 with other measurements revealing the path outside their domain. 741 5. Background on Round-Trip Delay Measurement Goals 743 The aim of this method is to use packet probes to unveil the paths 744 between any two end-Nodes of the network. Moreover, information 745 derived from RTD measurements might be meaningful to identify: 747 1. Intercontinental submarine links 749 2. Satellite communications 751 3. Congestion 753 4. Inter-domain paths 755 This categorization is widely accepted in the literature and among 756 operators alike, and it can be trusted with empirical data and 757 several sources as ground of truth (e.g., [RTTSub] ) but it is an 758 inference measurement nonetheless [bdrmap][IDCong]. 760 The first two categories correspond to the physical distance 761 dependency on Round-Trip Delay (RTD) while the last one binds RTD 762 with queueing delay on routers. Due to the significant contribution 763 of propagation delay in long distance hops, RTD will be on the order 764 of 100ms on transatlantic hops, depending on the geolocation of the 765 vantage points. Moreover, RTD is typically greater than 480ms when 766 two hops are connected using geostationary satellite technology 767 (i.e., their orbit is at 36000km). Detecting congestion with latency 768 implies deeper mathematical understanding since network traffic load 769 is not stationary. Nonetheless, as the first approach, a link seems 770 to be congested if after sending several traceroute probes, it is 771 possible to detect congestion observing different statistics 772 parameters (e.g., see [IDCong]). 774 6. Tools to Measure Delays in the Internet 776 Internet routing is complex because it depends on the policies of 777 thousands Autonomous Systems (AS). While most of the routers perform 778 load balancing on flows using Equal Cost Multiple Path (ECMP), a few 779 still divide the workload through packet-based techniques. The 780 former scenario is defined according to [RFC2991] while the latter 781 generates a round-robin scheme to deliver every new outgoing packet. 782 ECMP uses a hashing function to ensure that every packet of a flow is 783 delivered by the same path, and this avoids increasing the packet 784 delay variation and possibly producing overwhelming packet reordering 785 in TCP flows. 787 Taking into account that Internet protocol was designed under the 788 "end-to-end" principle, the IP payload and its header do not provide 789 any information about the routes or path necessary to reach some 790 destination. For this reason, the well-known tool traceroute was 791 developed to gather the IP addresses of each hop along a path using 792 the ICMP protocol [RFC0792]. Besides, traceroute adds the measured 793 RTD from each hop. However, the growing complexity of the Internet 794 makes it more challenging to develop an accurate traceroute 795 implementation. For instance, the early traceroute tools would be 796 inaccurate in the current network, mainly because they were not 797 designed to retain flow state. However, evolved traceroute tools, 798 such as Paris-traceroute [PT] [MLB] and Scamper [SCAMPER], expect to 799 encounter ECMP and achieve more accurate results when they do. 801 Paris-traceroute-like tools operate in the following way: every 802 packet should follow the same path because the sensitive fields of 803 the header are controlled to appear as the same flow. This means 804 that source and destination IP addresses, source and destination port 805 numbers are the same in every packet. Additionally, Differentiated 806 Services Code Point (DSCP), checksum and ICMP code should remain 807 constant since they may affect the path selection. 809 Today's traceroute tools can send either UDP, TCP or ICMP packet 810 probes. Since ICMP header does not include transport layer 811 information, there are no fields for source and destination port 812 numbers. For this reason, these tools keep constant ICMP type, code, 813 and checksum fields to generate a kind of flow. However, the 814 checksum may vary in every packet, therefore when probes use ICMP 815 packets, ICMP Identifier and sequence number are manipulated to 816 maintain constant checksum in every packet. On the other hand, when 817 UDP probes are generated, the expected variation in the checksum of 818 each packet is again compensated by manipulating the payload. 820 Paris-traceroute allows its users to measure RTD in every hop of the 821 path for a particular flow. Furthermore, either Paris-traceroute or 822 Scamper is capable of unveiling the many available paths between a 823 source and destination (which are visible to this method). This task 824 is accomplished by repeating complete traceroute measurements with 825 different flow parameters for each measurement. The Framework for IP 826 Performance Metrics (IPPM) ([RFC2330] updated by[RFC7312]) has the 827 flexibility to require that the Round-Trip Delay measurement 828 [RFC2681] uses packets with the constraints to assure that all 829 packets in a single measurement appear as the same flow. This 830 flexibility covers ICMP, UDP, and TCP. The accompanying methodology 831 of [RFC2681] needs to be expanded to report the sequential hop 832 identifiers along with RTD measurements, but no new metric definition 833 is needed. 835 7. RTD Measurements Statistics 837 Several articles have shown that network traffic presents a self- 838 similar nature [SSNT] [MLRM] which is accountable for filling the 839 queues of the routers. Moreover, router queues are designed to 840 handle traffic bursts, which is one of the most remarkable features 841 of self-similarity. Naturally, while queue length increases, the 842 delay to traverse the queue increases as well and leads to an 843 increase on RTD. Due to traffic bursts generate short-term overflow 844 on buffers (spiky patterns), every RTD only depicts the queueing 845 status on the instant when that packet probe was in transit. For 846 this reason, several RTD measurements during a time window could 847 begin to describe the random behavior of latency. Loss must also be 848 accounted for in the methodology. 850 To understand the ongoing process, examining the quartiles provides a 851 non-parametric way of analysis. Quartiles are defined by five 852 values: minimum RTD (m), RTD value of the 25% of the Empirical 853 Cumulative Distribution Function (ECDF) (Q1), the median value (Q2), 854 the RTD value of the 75% of the ECDF (Q3) and the maximum RTD (M). 855 Congestion can be inferred when RTD measurements are spread apart, 856 and consequently, the Inter-Quartile Range (IQR), the distance 857 between Q3 and Q1, increases its value. 859 This procedure requires to compute quartile values "on the fly" using 860 the algorithm presented in [P2]. 862 This procedure allows us to update the quartiles value whenever a new 863 measurement arrives, which is radically different from classic 864 methods of computing quartiles because they need to use the whole 865 dataset to compute the values. This way of calculus provides savings 866 in memory and computing time. 868 To sum up, the proposed measurement procedure consists in performing 869 traceroutes several times to obtain samples of the RTD in every hop 870 from a path, during a time window (W) and compute the quantiles for 871 every hop. This could be done for a single Member Route flow or for 872 every detected Route Ensemble flow. 874 Even though a particular hop may be understood as the amount of hops 875 away from the source, a more detailed classification could be used. 876 For example, a possible classification may be identify ICMP Time 877 Exceeded packets coming from the same routers to those who have the 878 same hop distance, IP address of the router which is replying and TTL 879 or Hop Limit value of the received ICMP packet. 881 Thus, the proposed methodology is based on this algorithm: 883 ================================================================ 884 1 input: W (window time of the measurement) 885 2 i_t (time between two measurements) 886 3 E (True: exhaustive, False: a single path) 887 4 Dst (destination IP address) 888 5 output: Qs (quartiles for every hop and alt in the path(s) to Dst) 889 ---------------------------------------------------------------- 890 6 T . 1008 [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - 1009 Communication Layers", STD 3, RFC 1122, 1010 DOI 10.17487/RFC1122, October 1989, 1011 . 1013 [RFC1812] Baker, F., Ed., "Requirements for IP Version 4 Routers", 1014 RFC 1812, DOI 10.17487/RFC1812, June 1995, 1015 . 1017 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1018 Requirement Levels", BCP 14, RFC 2119, 1019 DOI 10.17487/RFC2119, March 1997, 1020 . 1022 [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, 1023 "Framework for IP Performance Metrics", RFC 2330, 1024 DOI 10.17487/RFC2330, May 1998, 1025 . 1027 [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 1028 (IPv6) Specification", RFC 2460, DOI 10.17487/RFC2460, 1029 December 1998, . 1031 [RFC2675] Borman, D., Deering, S., and R. Hinden, "IPv6 Jumbograms", 1032 RFC 2675, DOI 10.17487/RFC2675, August 1999, 1033 . 1035 [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip 1036 Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, 1037 September 1999, . 1039 [RFC2991] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and 1040 Multicast Next-Hop Selection", RFC 2991, 1041 DOI 10.17487/RFC2991, November 2000, 1042 . 1044 [RFC4494] Song, JH., Poovendran, R., and J. Lee, "The AES-CMAC-96 1045 Algorithm and Its Use with IPsec", RFC 4494, 1046 DOI 10.17487/RFC4494, June 2006, 1047 . 1049 [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. 1050 Zekauskas, "A One-way Active Measurement Protocol 1051 (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, 1052 . 1054 [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. 1055 Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", 1056 RFC 5357, DOI 10.17487/RFC5357, October 2008, 1057 . 1059 [RFC5388] Niccolini, S., Tartarelli, S., Quittek, J., Dietz, T., and 1060 M. Swany, "Information Model and XML Data Model for 1061 Traceroute Measurements", RFC 5388, DOI 10.17487/RFC5388, 1062 December 2008, . 1064 [RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance 1065 Metrics (IPPM): Spatial and Multicast", RFC 5644, 1066 DOI 10.17487/RFC5644, October 2009, 1067 . 1069 [RFC5835] Morton, A., Ed. and S. Van den Berghe, Ed., "Framework for 1070 Metric Composition", RFC 5835, DOI 10.17487/RFC5835, April 1071 2010, . 1073 [RFC5837] Atlas, A., Ed., Bonica, R., Ed., Pignataro, C., Ed., Shen, 1074 N., and JR. Rivers, "Extending ICMP for Interface and 1075 Next-Hop Identification", RFC 5837, DOI 10.17487/RFC5837, 1076 April 2010, . 1078 [RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6 1079 Datagrams over IEEE 802.15.4-Based Networks", RFC 6282, 1080 DOI 10.17487/RFC6282, September 2011, 1081 . 1083 [RFC6437] Amante, S., Carpenter, B., Jiang, S., and J. Rajahalme, 1084 "IPv6 Flow Label Specification", RFC 6437, 1085 DOI 10.17487/RFC6437, November 2011, 1086 . 1088 [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label 1089 for Equal Cost Multipath Routing and Link Aggregation in 1090 Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, 1091 . 1093 [RFC6564] Krishnan, S., Woodyatt, J., Kline, E., Hoagland, J., and 1094 M. Bhatia, "A Uniform Format for IPv6 Extension Headers", 1095 RFC 6564, DOI 10.17487/RFC6564, April 2012, 1096 . 1098 [RFC6673] Morton, A., "Round-Trip Packet Loss Metrics", RFC 6673, 1099 DOI 10.17487/RFC6673, August 2012, 1100 . 1102 [RFC7045] Carpenter, B. and S. Jiang, "Transmission and Processing 1103 of IPv6 Extension Headers", RFC 7045, 1104 DOI 10.17487/RFC7045, December 2013, 1105 . 1107 [RFC7312] Fabini, J. and A. Morton, "Advanced Stream and Sampling 1108 Framework for IP Performance Metrics (IPPM)", RFC 7312, 1109 DOI 10.17487/RFC7312, August 2014, 1110 . 1112 [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with 1113 Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, 1114 May 2016, . 1116 [RFC7820] Mizrahi, T., "UDP Checksum Complement in the One-Way 1117 Active Measurement Protocol (OWAMP) and Two-Way Active 1118 Measurement Protocol (TWAMP)", RFC 7820, 1119 DOI 10.17487/RFC7820, March 2016, 1120 . 1122 [RFC8029] Kompella, K., Swallow, G., Pignataro, C., Ed., Kumar, N., 1123 Aldrin, S., and M. Chen, "Detecting Multiprotocol Label 1124 Switched (MPLS) Data-Plane Failures", RFC 8029, 1125 DOI 10.17487/RFC8029, March 2017, 1126 . 1128 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 1129 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 1130 May 2017, . 1132 13.2. Informative References 1134 [bdrmap] Luckie, M., Dhamdhere, A., Huffaker, B., Clark, D., and 1135 KC. Claffy, "bdrmap: Inference of Borders Between IP 1136 Networks", In Proceedings of the 2016 ACM on Internet 1137 Measurement Conference, pp. 381-396. ACM, 2016. 1139 [IDCong] Luckie, M., Dhamdhere, A., Clark, D., and B. Huffaker, 1140 "Challenges in inferring Internet interdomain 1141 congestion", In Proceedings of the 2014 Conference on 1142 Internet Measurement Conference, pp. 15-22. ACM, 2014. 1144 [MLB] Augustin, B., Friedman, T., and R. Teixeira, "Measuring 1145 load-balanced paths in the Internet", Proceedings of the 1146 7th ACM SIGCOMM conference on Internet measurement, pp. 1147 149-160. ACM, 2007., 2007. 1149 [MLRM] Fontugne, R., Mazel, J., and K. Fukuda, "An empirical 1150 mixture model for large-scale RTT measurements", 2015 1151 IEEE Conference on Computer Communications (INFOCOM), pp. 1152 2470-2478. IEEE, 2015., 2015. 1154 [P2] Jain, R. and I. Chlamtac, "The P 2 algorithm for dynamic 1155 calculation of quantiles and histograms without storing 1156 observations", Communications of the ACM 28.10 (1985): 1157 1076-1085, 2015. 1159 [PT] Augustin, B., Cuvellier, X., Orgogozo, B., Viger, F., 1160 Friedman, T., Latapy, M., Magnien, C., and R. Teixeira, 1161 "Avoiding traceroute anomalies with Paris traceroute", 1162 Proceedings of the 6th ACM SIGCOMM conference on Internet 1163 measurement, pp. 153-158. ACM, 2006., 2006. 1165 [RFC7325] Villamizar, C., Ed., Kompella, K., Amante, S., Malis, A., 1166 and C. Pignataro, "MPLS Forwarding Compliance and 1167 Performance Requirements", RFC 7325, DOI 10.17487/RFC7325, 1168 August 2014, . 1170 [RFC7594] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., 1171 Aitken, P., and A. Akhter, "A Framework for Large-Scale 1172 Measurement of Broadband Performance (LMAP)", RFC 7594, 1173 DOI 10.17487/RFC7594, September 2015, 1174 . 1176 [RFC8403] Geib, R., Ed., Filsfils, C., Pignataro, C., Ed., and N. 1177 Kumar, "A Scalable and Topology-Aware MPLS Data-Plane 1178 Monitoring System", RFC 8403, DOI 10.17487/RFC8403, July 1179 2018, . 1181 [RTTSub] Bischof, Z., Rula, J., and F. Bustamante, "In and out of 1182 Cuba: Characterizing Cuba's connectivity", In Proceedings 1183 of the 2015 ACM Conference on Internet Measurement 1184 Conference, pp. 487-493. ACM, 2015. 1186 [SCAMPER] Matthew Luckie, M., "Scamper: a scalable and extensible 1187 packet prober for active measurement of the 1188 Internet", Proceedings of the 10th ACM SIGCOMM conference 1189 on Internet measurement, pp. 239-245. ACM, 2010., 2010. 1191 [SSNT] Park, K. and W. Willinger, "Self-Similar Network Traffic 1192 and Performance Evaluation (1st ed.)", John Wiley & Sons, 1193 Inc., New York, NY, USA, 2000. 1195 Authors' Addresses 1197 J. Ignacio Alvarez-Hamelin 1198 Universidad de Buenos Aires 1199 Av. Paseo Colon 850 1200 Buenos Aires C1063ACV 1201 Argentine 1203 Phone: +54 11 5285-0716 1204 Email: ihameli@cnet.fi.uba.ar 1205 URI: http://cnet.fi.uba.ar/ignacio.alvarez-hamelin/ 1207 Al Morton 1208 AT&T Labs 1209 200 Laurel Avenue South 1210 Middletown, NJ 07748 1211 USA 1213 Phone: +1 732 420 1571 1214 Fax: +1 732 368 1192 1215 Email: acm@research.att.com 1217 Joachim Fabini 1218 TU Wien 1219 Gusshausstrasse 25/E389 1220 Vienna 1040 1221 Austria 1223 Phone: +43 1 58801 38813 1224 Fax: +43 1 58801 38898 1225 Email: Joachim.Fabini@tuwien.ac.at 1226 URI: http://www.tc.tuwien.ac.at/about-us/staff/joachim-fabini/ 1227 Carlos Pignataro 1228 Cisco Systems, Inc. 1229 7200-11 Kit Creek Road 1230 Research Triangle Park, NC 27709 1231 USA 1233 Email: cpignata@cisco.com 1235 Ruediger Geib 1236 Deutsche Telekom 1237 Heinrich Hertz Str. 3-7 1238 Darmstadt 64295 1239 Germany 1241 Phone: +49 6151 5812747 1242 Email: Ruediger.Geib@telekom.de