idnits 2.17.1 draft-ietf-alto-path-vector-25.txt: -(2748): Line appears to be too long, but this could be caused by non-ascii characters in UTF-8 encoding Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == There are 2 instances of lines with non-ascii characters in the document. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (20 March 2022) is 767 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Outdated reference: A later version (-28) exists of draft-ietf-alto-performance-metrics-26 Summary: 0 errors (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ALTO K. Gao 3 Internet-Draft Sichuan University 4 Intended status: Experimental Y. Lee 5 Expires: 21 September 2022 Samsung 6 S. Randriamasy 7 Nokia Bell Labs 8 Y.R. Yang 9 Yale University 10 J. Zhang 11 Tongji University 12 20 March 2022 14 An ALTO Extension: Path Vector 15 draft-ietf-alto-path-vector-25 17 Abstract 19 This document is an extension to the base Application-Layer Traffic 20 Optimization (ALTO) protocol. It extends the ALTO Cost Map and ALTO 21 Property Map services so that an application can decide which 22 endpoint(s) to connect based on not only numerical/ordinal cost 23 values but also fine-grained abstract information of the paths. This 24 is useful for applications whose performance is impacted by specified 25 components of a network on the end-to-end paths, e.g., they may infer 26 that several paths share common links and prevent traffic bottlenecks 27 by avoiding such paths. This extension introduces a new abstraction 28 called Abstract Network Element (ANE) to represent these components 29 and encodes a network path as a vector of ANEs. Thus, it provides a 30 more complete but still abstract graph representation of the 31 underlying network(s) for informed traffic optimization among 32 endpoints. 34 Status of This Memo 36 This Internet-Draft is submitted in full conformance with the 37 provisions of BCP 78 and BCP 79. 39 Internet-Drafts are working documents of the Internet Engineering 40 Task Force (IETF). Note that other groups may also distribute 41 working documents as Internet-Drafts. The list of current Internet- 42 Drafts is at https://datatracker.ietf.org/drafts/current/. 44 Internet-Drafts are draft documents valid for a maximum of six months 45 and may be updated, replaced, or obsoleted by other documents at any 46 time. It is inappropriate to use Internet-Drafts as reference 47 material or to cite them other than as "work in progress." 48 This Internet-Draft will expire on 21 September 2022. 50 Copyright Notice 52 Copyright (c) 2022 IETF Trust and the persons identified as the 53 document authors. All rights reserved. 55 This document is subject to BCP 78 and the IETF Trust's Legal 56 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 57 license-info) in effect on the date of publication of this document. 58 Please review these documents carefully, as they describe your rights 59 and restrictions with respect to this document. Code Components 60 extracted from this document must include Revised BSD License text as 61 described in Section 4.e of the Trust Legal Provisions and are 62 provided without warranty as described in the Revised BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 67 2. Requirements Languages . . . . . . . . . . . . . . . . . . . 6 68 3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 6 69 4. Requirements and Use Cases . . . . . . . . . . . . . . . . . 7 70 4.1. Design Requirements . . . . . . . . . . . . . . . . . . . 7 71 4.2. Sample Use Cases . . . . . . . . . . . . . . . . . . . . 10 72 4.2.1. Exposing Network Bottlenecks . . . . . . . . . . . . 11 73 4.2.2. Resource Exposure for CDN and Service Edge . . . . . 15 74 5. Path Vector Extension: Overview . . . . . . . . . . . . . . . 17 75 5.1. Abstract Network Element (ANE) . . . . . . . . . . . . . 18 76 5.1.1. ANE Entity Domain . . . . . . . . . . . . . . . . . . 19 77 5.1.2. Ephemeral and Persistent ANEs . . . . . . . . . . . . 19 78 5.1.3. Property Filtering . . . . . . . . . . . . . . . . . 20 79 5.2. Path Vector Cost Type . . . . . . . . . . . . . . . . . . 20 80 5.3. Multipart Path Vector Response . . . . . . . . . . . . . 21 81 5.3.1. Identifying the Media Type of the Root Object . . . . 22 82 5.3.2. References to Part Messages . . . . . . . . . . . . . 22 83 6. Specification: Basic Data Types . . . . . . . . . . . . . . . 23 84 6.1. ANE Name . . . . . . . . . . . . . . . . . . . . . . . . 23 85 6.2. ANE Entity Domain . . . . . . . . . . . . . . . . . . . . 23 86 6.2.1. Entity Domain Type . . . . . . . . . . . . . . . . . 23 87 6.2.2. Domain-Specific Entity Identifier . . . . . . . . . . 23 88 6.2.3. Hierarchy and Inheritance . . . . . . . . . . . . . . 23 89 6.2.4. Media Type of Defining Resource . . . . . . . . . . . 23 90 6.3. ANE Property Name . . . . . . . . . . . . . . . . . . . . 24 91 6.4. Initial ANE Property Types . . . . . . . . . . . . . . . 24 92 6.4.1. Maximum Reservable Bandwidth . . . . . . . . . . . . 24 93 6.4.2. Persistent Entity ID . . . . . . . . . . . . . . . . 25 94 6.4.3. Examples . . . . . . . . . . . . . . . . . . . . . . 25 95 6.5. Path Vector Cost Type . . . . . . . . . . . . . . . . . . 26 96 6.5.1. Cost Metric: ane-path . . . . . . . . . . . . . . . . 26 97 6.5.2. Cost Mode: array . . . . . . . . . . . . . . . . . . 27 98 6.6. Part Resource ID and Part Content ID . . . . . . . . . . 27 99 7. Specification: Service Extensions . . . . . . . . . . . . . . 27 100 7.1. Notations . . . . . . . . . . . . . . . . . . . . . . . . 27 101 7.2. Multipart Filtered Cost Map for Path Vector . . . . . . . 28 102 7.2.1. Media Type . . . . . . . . . . . . . . . . . . . . . 28 103 7.2.2. HTTP Method . . . . . . . . . . . . . . . . . . . . . 28 104 7.2.3. Accept Input Parameters . . . . . . . . . . . . . . . 28 105 7.2.4. Capabilities . . . . . . . . . . . . . . . . . . . . 29 106 7.2.5. Uses . . . . . . . . . . . . . . . . . . . . . . . . 30 107 7.2.6. Response . . . . . . . . . . . . . . . . . . . . . . 30 108 7.3. Multipart Endpoint Cost Service for Path Vector . . . . . 34 109 7.3.1. Media Type . . . . . . . . . . . . . . . . . . . . . 34 110 7.3.2. HTTP Method . . . . . . . . . . . . . . . . . . . . . 34 111 7.3.3. Accept Input Parameters . . . . . . . . . . . . . . . 34 112 7.3.4. Capabilities . . . . . . . . . . . . . . . . . . . . 35 113 7.3.5. Uses . . . . . . . . . . . . . . . . . . . . . . . . 35 114 7.3.6. Response . . . . . . . . . . . . . . . . . . . . . . 35 115 8. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 39 116 8.1. Sample Setup . . . . . . . . . . . . . . . . . . . . . . 39 117 8.2. Information Resource Directory . . . . . . . . . . . . . 39 118 8.3. Multipart Filtered Cost Map . . . . . . . . . . . . . . . 42 119 8.4. Multipart Endpoint Cost Service Resource . . . . . . . . 43 120 8.5. Incremental Updates . . . . . . . . . . . . . . . . . . . 48 121 8.6. Multi-cost . . . . . . . . . . . . . . . . . . . . . . . 50 122 9. Compatibility with Other ALTO Extensions . . . . . . . . . . 52 123 9.1. Compatibility with Legacy ALTO Clients/Servers . . . . . 53 124 9.2. Compatibility with Multi-Cost Extension . . . . . . . . . 53 125 9.3. Compatibility with Incremental Update . . . . . . . . . . 53 126 9.4. Compatibility with Cost Calendar . . . . . . . . . . . . 53 127 10. General Discussions . . . . . . . . . . . . . . . . . . . . . 54 128 10.1. Constraint Tests for General Cost Types . . . . . . . . 54 129 10.2. General Multi-Resource Query . . . . . . . . . . . . . . 54 130 11. Security Considerations . . . . . . . . . . . . . . . . . . . 55 131 12. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 57 132 12.1. ALTO Cost Metric Registry . . . . . . . . . . . . . . . 57 133 12.2. ALTO Cost Mode Registry . . . . . . . . . . . . . . . . 58 134 12.3. ALTO Entity Domain Type Registry . . . . . . . . . . . . 58 135 12.4. ALTO Entity Property Type Registry . . . . . . . . . . . 59 136 12.4.1. New ANE Property Type: Maximum Reservable 137 Bandwidth . . . . . . . . . . . . . . . . . . . . . . 59 138 12.4.2. New ANE Property Type: Persistent Entity ID . . . . 60 139 13. References . . . . . . . . . . . . . . . . . . . . . . . . . 60 140 13.1. Normative References . . . . . . . . . . . . . . . . . . 60 141 13.2. Informative References . . . . . . . . . . . . . . . . . 61 142 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 64 143 Appendix B. Revision Logs (To be removed before publication) . . 64 144 B.1. Changes since -20 . . . . . . . . . . . . . . . . . . . . 64 145 B.2. Changes since -19 . . . . . . . . . . . . . . . . . . . . 65 146 B.3. Changes since -18 . . . . . . . . . . . . . . . . . . . . 65 147 B.4. Changes since -17 . . . . . . . . . . . . . . . . . . . . 65 148 B.5. Changes since -16 . . . . . . . . . . . . . . . . . . . . 65 149 B.6. Changes since -15 . . . . . . . . . . . . . . . . . . . . 65 150 B.7. Changes since -14 . . . . . . . . . . . . . . . . . . . . 65 151 B.8. Changes since -13 . . . . . . . . . . . . . . . . . . . . 66 152 B.9. Changes since -12 . . . . . . . . . . . . . . . . . . . . 66 153 B.10. Changes since -11 . . . . . . . . . . . . . . . . . . . . 66 154 B.11. Changes since -10 . . . . . . . . . . . . . . . . . . . . 66 155 B.12. Changes since -09 . . . . . . . . . . . . . . . . . . . . 67 156 B.13. Changes since -08 . . . . . . . . . . . . . . . . . . . . 67 157 B.14. Changes Since Version -06 . . . . . . . . . . . . . . . . 67 158 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 68 160 1. Introduction 162 Network performance metrics are crucial to assess the Quality of 163 Experience (QoE) of applications. The ALTO protocol allows Internet 164 Service Providers (ISPs) to provide guidance, such as topological 165 distance between different end hosts, to overlay applications. Thus, 166 the overlay applications can potentially improve the perceived QoE by 167 better orchestrating their traffic to utilize the resources in the 168 underlying network infrastructure. 170 Existing ALTO Cost Map (Section 11.2.3 of [RFC7285]) and Endpoint 171 Cost Service (Section 11.5 of [RFC7285]) provide only cost 172 information on an end-to-end path defined by its endpoints: The base protocol [RFC7285] allows the 174 services to expose the topological distances of end-to-end paths, 175 while various extensions have been proposed to extend the capability 176 of these services, e.g., to express other performance metrics 177 [I-D.ietf-alto-performance-metrics], to query multiple costs 178 simultaneously [RFC8189], and to obtain the time-varying values 179 [RFC8896]. 181 While the existing extensions are sufficient for many overlay 182 applications, the QoE of some overlay applications depends not only 183 on the cost information of end-to-end paths, but also on particular 184 components of a network on the paths and their properties. For 185 example, job completion time, which is an important QoE metric for a 186 large-scale data analytics application, is impacted by shared 187 bottleneck links inside the carrier network as link capacity may 188 impact the rate of data input/output to the job. We refer to such 189 components of a network as Abstract Network Elements (ANE). 191 Predicting such information can be very complex without the help of 192 ISPs, for example, [BOXOPT] has shown that finding the optimal 193 bandwidth reservation for multiple flows can be NP-hard without 194 further information than whether a reservation succeeds. With proper 195 guidance from the ISP, an overlay application may be able to schedule 196 its traffic for better QoE. In the meantime, it may be helpful as 197 well for ISPs if applications could avoid using bottlenecks or 198 challenging the network with poorly scheduled traffic. 200 Despite the claimed benefits, ISPs are not likely to expose raw 201 details on their network paths: first for the sake of topology hiding 202 requirement, second because it may increase volume and computation 203 overhead, and last because applications do not necessarily need all 204 the network path details and are likely not able to understand them. 206 Therefore, it is beneficial for both ISPs and applications if an ALTO 207 server provides ALTO clients with an "abstract network state" that 208 provides the necessary information to applications, while hiding the 209 network complexity and confidential information. An "abstract 210 network state" is a selected set of abstract representations of 211 Abstract Network Elements traversed by the paths between pairs combined with properties of these Abstract Network 213 Elements that are relevant to the overlay applications' QoE. Both an 214 application via its ALTO client and the ISP via the ALTO server can 215 achieve better confidentiality and resource utilization by 216 appropriately abstracting relevant Abstract Network Elements. Server 217 scalability can also be improved by combining Abstract Network 218 Elements and their properties in a single response. 220 This document extends [RFC7285] to allow an ALTO server to convey 221 "abstract network state", for paths defined by their pairs. To this end, it introduces a new cost type 223 called "Path Vector" following the cost metric registration specified 224 in [RFC7285] and the updated cost mode registration specified in 225 [I-D.bw-alto-cost-mode]. A Path Vector is an array of identifiers 226 that identifies an Abstract Network Element, which can be associated 227 with various properties. The associations between ANEs and their 228 properties are encoded in an ALTO information resource called Unified 229 Property Map, which is specified in 230 [I-D.ietf-alto-unified-props-new]. 232 For better confidentiality, this document aims to minimize 233 information exposure of an ALTO server when providing Path Vector 234 service. In particular, this document enables and recommends that 235 first ANEs are constructed on demand, and second an ANE is only 236 associated with properties that are requested by an ALTO client. A 237 Path Vector response involves two ALTO Maps: the Cost Map that 238 contains the Path Vector results and the up-to-date Unified Property 239 Map that contains the properties requested for these ANEs. To 240 enforce consistency and improve server scalability, this document 241 uses the "multipart/related" content type defined in [RFC2387] to 242 return the two maps in a single response. 244 As a single ISP may not have the knowledge of the full Internet paths 245 between arbitrary endpoints, this document is mainly applicable 1) 246 when there is a single ISP between the requested source and 247 destination PIDs or endpoints, for example, ISP-hosted CDN/edge, 248 tenant interconnection in a single public cloud platform, etc.; or 2) 249 when the Path Vectors are generated from end-to-end measurement data. 251 2. Requirements Languages 253 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 254 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 255 "OPTIONAL" in this document are to be interpreted as described in BCP 256 14 [RFC2119] [RFC8174] when, and only when, they appear in all 257 capitals, as shown here. 259 When the words appear in lower case, they are to be interpreted with 260 their natural language meanings. 262 3. Terminology 264 This document extends the ALTO base protocol [RFC7285] and the 265 Unified Property Map extension [I-D.ietf-alto-unified-props-new]. In 266 addition to the terms defined in these documents, this document also 267 uses the following additional terms: 269 Abstract Network Element (ANE): An abstract representation for a 270 component in a network that handles data packets and whose 271 properties can potentially have an impact on the end-to-end 272 performance of traffic. An ANE can be a physical device such as a 273 router, a link or an interface, or an aggregation of devices such 274 as a subnetwork or a data center. 276 The definition of Abstract Network Element is similar to Network 277 Element defined in [RFC2216] in the sense that they both provide 278 an abstract representation of specific components of a network. 279 However, they have different criteria on how these particular 280 components are selected. Specifically, a Network Element requires 281 the components to be capable of exercising QoS control, while 282 Abstract Network Element only requires the components to have an 283 impact on the end-to-end performance. 285 ANE Name: A string that uniquely identifies an ANE in a specific 286 scope. An ANE can be constructed either statically in advance or 287 on demand based on the requested information. Thus, different 288 ANEs may only be valid within a particular scope, either ephemeral 289 or persistent. Within each scope, an ANE is uniquely identified 290 by an ANE Name, as defined in Section 6.1. Note that an ALTO 291 client must not assume ANEs in different scopes but with the same 292 ANE Name refer to the same component(s) of the network. 294 Path Vector: Path Vector, or ANE Path Vector, refers to a JSON array 295 of ANE Names. It is a generalization of BGP path vector. While 296 standard BGP path vector (Section 5.1.2 of [RFC4271]) specifies a 297 sequence of autonomous systems for a destination IP prefix, the 298 Path Vector defined in this extension specifies a sequence of ANEs 299 either for a source Provider-Defined Identifier (PID) and a 300 destination PID as in the CostMapData (11.2.3.6 in [RFC7285]), or 301 for a source endpoint and a destination endpoint as in the 302 EndpointCostMapData object (Section 11.5.1.6 of [RFC7285]). 304 Path Vector resource: An ALTO information resource (Section 8.1 of 305 [RFC7285]) which supports the extension defined in this document. 307 Path Vector cost type: A special cost type, which is specified in 308 Section 6.5. When this cost type is present in an IRD entry, it 309 indicates that the information resource is a Path Vector resource. 310 When this cost type is present in a Filtered Cost Map request or 311 an Endpoint Cost Service request, it indicates each cost value 312 must be interpreted as a Path Vector. 314 Path Vector request: The POST message sent to an ALTO Path Vector 315 resource. 317 Path Vector response: A Path Vector response refers to the 318 multipart/related message returned by a Path Vector resource. 320 4. Requirements and Use Cases 322 4.1. Design Requirements 324 This section gives an illustrative example of how an overlay 325 application can benefit from the extension defined in this document. 327 Assume that an application has control over a set of flows, which may 328 go through shared links/nodes and share bottlenecks. The application 329 seeks to schedule the traffic among multiple flows to get better 330 performance. The constraints of feasible rate allocations of those 331 flows will benefit the scheduling. However, Cost Maps as defined in 332 [RFC7285] can not reveal such information. 334 Specifically, consider a network as shown in Figure 1. The network 335 has 7 switches (sw1 to sw7) forming a dumb-bell topology. Switches 336 "sw1", "sw2", "sw3" and "sw4" are access switches, and sw5-sw7 form 337 the backbone. End hosts eh1 to eh4 are connected to access switches 338 sw1 to sw4 respectively. Assume that the bandwidth of link eh1 -> 339 sw1 and link sw1 -> sw5 is 150 Mbps, and the bandwidth of the other 340 links is 100 Mbps. 342 +-----+ 343 | | 344 --+ sw6 +-- 345 / | | \ 346 PID1 +-----+ / +-----+ \ +-----+ PID2 347 eh1__| |_ / \ ____| |__eh2 348 192.0.2.2 | sw1 | \ +--|--+ +--|--+ / | sw2 | 192.0.2.3 349 +-----+ \ | | | |/ +-----+ 350 \_| sw5 +---------+ sw7 | 351 PID3 +-----+ / | | | |\ +-----+ PID4 352 eh3__| |__/ +-----+ +-----+ \____| |__eh4 353 192.0.2.4 | sw3 | | sw4 | 192.0.2.5 354 +-----+ +-----+ 356 bw(eh1--sw1) = bw(sw1--sw5) = 150 Mbps 357 bw(eh2--sw2) = bw(eh3--sw3) = bw(eh4--sw4) = 100 Mbps 358 bw(sw1--sw5) = bw(sw3--sw5) = bw(sw2--sw7) = bw(sw4--sw7) = 100 Mbps 359 bw(sw5--sw6) = bw(sw5--sw7) = bw(sw6--sw7) = 100 Mbps 361 Figure 1: Raw Network Topology 363 The base ALTO topology abstraction of the network is shown in 364 Figure 2. Assume the cost map returns an hypothetical cost type 365 representing the available bandwidth between a source and a 366 destination. 368 +----------------------+ 369 {eh1} | | {eh2} 370 PID1 | | PID2 371 +------+ +------+ 372 | | 373 | | 374 {eh3} | | {eh4} 375 PID3 | | PID4 376 +------+ +------+ 377 | | 378 +----------------------+ 380 Figure 2: Base Topology Abstraction 382 Now assume the application wants to maximize the total rate of the 383 traffic among a set of pairs, say "eh1 -> eh2" 384 and "eh1 -> eh4". Let "x" denote the transmission rate of "eh1 -> 385 eh2" and "y" denote the rate of "eh1 -> eh4". The objective function 386 is 388 max(x + y). 390 With the ALTO Cost Map, the cost between PID1 and PID2 and between 391 PID1 and PID4 will both be 100 Mbps. The client can get a capacity 392 region of 394 x <= 100 Mbps, 395 y <= 100 Mbps. 397 With this information, the client may mistakenly think it can achieve 398 a maximum total rate of 200 Mbps. However, this rate is infeasible, 399 as there are only two potential cases: 401 * Case 1: "eh1 -> eh2" and "eh1 -> eh4" take different path segments 402 from "sw5" to "sw7". For example, if "eh1 -> eh2" uses path "eh1 403 -> sw1 -> sw5 -> sw6 -> sw7 -> sw2 -> eh2" and "eh1 -> eh4" uses 404 path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then the shared 405 bottleneck links are "eh1 -> sw1" and "sw1 -> sw5". In this case, 406 the capacity region is: 408 x <= 100 Mbps 409 y <= 100 Mbps 410 x + y <= 150 Mbps 412 and the real optimal total rate is 150 Mbps. 414 * Case 2: "eh1 -> eh2" and "eh1 -> eh4" take the same path segment 415 from "sw5" to "sw7". For example, if "eh1 -> eh2" uses path "eh1 416 -> sw1 -> sw5 -> sw7 -> sw2 -> eh2" and "eh1 -> eh4" also uses 417 path "eh1 -> sw1 -> sw5 -> sw7 -> sw4 -> eh4", then the shared 418 bottleneck link is "sw5 -> sw7". In this case, the capacity 419 region is: 421 x <= 100 Mbps 422 y <= 100 Mbps 423 x + y <= 100 Mbps 425 and the real optimal total rate is 100 Mbps. 427 Clearly, with more accurate and fine-grained information, the 428 application can gain a better prediction of its traffic and may 429 orchestrate its resources accordingly. However, to provide such 430 information, the network needs to expose abstract information beyond 431 the simple cost map abstraction. In particular: 433 * The ALTO server must expose abstract information about the network 434 paths that are traversed by the traffic between a source and a 435 destination beyond a simple numerical value, which allows the 436 overlay application to distinguish between Cases 1 and 2 and to 437 compute the optimal total rate accordingly. 439 * The ALTO server must allow the client to distinguish the common 440 ANE shared by "eh1 -> eh2" and "eh1 -> eh4", e.g., "eh1 - sw1" and 441 "sw1 - sw5" in Case 1. 443 * The ALTO server must expose abstract information on the properties 444 of the ANEs used by "eh1 -> eh2" and "eh1 -> eh4". For example, 445 an ALTO server can either expose the available bandwidth between 446 "eh1 - sw1", "sw1 - sw5", "sw5 - sw7", "sw5 - sw6", "sw6 - sw7", 447 "sw7 - sw2", "sw7 - sw4", "sw2 - eh2", "sw4 - eh4" in Case 1, or 448 expose 3 abstract elements "A", "B" and "C", which represent the 449 linear constraints that define the same capacity region in Case 1. 451 In general, we can conclude that to support the multiple flow 452 scheduling use case, the ALTO framework must be extended to satisfy 453 the following additional requirements: 455 AR1: An ALTO server must provide the ANEs that are important to 456 assess the QoE of the overlay application on the path of a 457 pair. 459 AR2: An ALTO server must provide information to identify how ANEs 460 are shared on the paths of different pairs. 462 AR3: An ALTO server must provide information on the properties that 463 are important to assess the QoE of the application for ANEs. 465 The extension defined in this document specifies a solution to expose 466 such abstract information. 468 4.2. Sample Use Cases 470 While the multiple flow scheduling problem is used to help identify 471 the additional requirements, the extension defined in this document 472 can be applied to a wide range of applications. This section 473 highlights some use cases that are reported. 475 4.2.1. Exposing Network Bottlenecks 477 An important use case of the Path Vector extension is to expose 478 network bottlenecks. Applications which need to perform large scale 479 data transfers can benefit from being aware of the resource 480 constraints exposed by this extension even if they have different 481 objectives. One such example is the Worldwide LHC Computing Grid 482 (WLCG), the largest example of a distributed computation 483 collaboration in the research and education world. 485 Figure 3 illustrates an example of using ALTO Path Vector as an 486 interface between the job optimizer for a data analytics system and 487 the network manager. In particular, we assume the objective of the 488 job optimizer is to minimize the job completion time. 490 In such a setting, the network-aware job optimizer (e.g., [CLARINET]) 491 takes a query and generates multiple query execution plans (QEP). It 492 can encode the QEPs as Path Vector requests that are send to an ALTO 493 server. The ALTO server obtains the routing information for the 494 flows in a QEP and finds links, routers, or middleboxes (e.g., a 495 stateful firewall) that can potentially become bottlenecks of the QEP 496 (e.g., see [NOVA] and [G2] for mechanisms to identify bottleneck 497 links under different settings). The resource constraint information 498 is encoded in a Path Vector response and returned to the ALTO client. 500 With the network resource constraints, the job optimizer may choose 501 the QEP with the optimal job completion time to be executed. It must 502 be noted that the ALTO framework itself does not offer the capability 503 to control the traffic. However, certain network managers may offer 504 ways to enforce resource guarantees, such as on-demand tunnels (e.g., 505 [SWAN]), demand vector (e.g., [HUG], [UNICORN]), etc. The traffic 506 control interfaces and mechanisms are out of the scope of this 507 document. 509 Data schema Queries 510 | | 511 \ / 512 +-------------+ +-----------------+ 513 | ALTO Client | <===============> | Job Optimizer | 514 +-------------+ +-----------------+ 515 PV | ^ PV | 516 Request | | Response | 517 | | On-demand resource | 518 (Data | | (Network allocation, demand | 519 Transfer | | Resource vector, etc. | 520 Intents) | | Constraints) (Non-ALTO interfaces)| 521 v | v 522 +-------------+ +-----------------+ 523 | ALTO Server | <===============> | Network Manager | 524 +-------------+ +-----------------+ 525 / | \ 526 | | | 527 WAN DC1 DC2 529 Figure 3: Example Use Case for Data Analytics 531 Another example is as illustrated in Figure 4. Consider a network 532 consisting of multiple sites and a non-blocking core network, i.e., 533 the links in the core network have sufficient bandwidth that they 534 will not become the bottleneck of the data transfers. 536 On-going transfers New transfer requests 537 \----\ | 538 | | 539 v v 540 +-------------+ +---------------+ 541 | ALTO Client | <===========> | Data Transfer | 542 +-------------+ | Scheduler | 543 ^ | ^ | PV request +---------------+ 544 | | | \--------------\ 545 | | \--------------\ | 546 | v PV response | v 547 +-------------+ +-------------+ 548 | ALTO Server | | ALTO Server | 549 +-------------+ +-------------+ 550 || || 551 +---------+ +---------+ 552 | Network | | Network | 553 | Manager | | Manager | 554 +---------+ +---------+ 555 . . 556 . _~_ __ . . . 557 . ( )( ) .___ 558 ~v~v~ /--( )------------( ) 559 ( )-----/ ( ) ( ) 560 ~w~w~ ~^~^~^~ ~v~v~ 561 Site 1 Non-blocking Core Site 2 563 Figure 4: Example Use Case for Cross-site Bottleneck Discovery 565 Site 1: 567 [c] 568 . 569 ........................................> [d] 570 +---+ 10 Gbps +---+ 10 Gbps +----+ 50 Gbps 571 | A |---------| B |---------| GW |--------- Core 572 +---+ +---+ +----+ 573 ................... 574 . . 575 . v 576 [a] [b] 578 Site 2: 580 [d] <........................................ [c] 581 +---+ 5 Gbps +---+ 10 Gbps +----+ 20 Gbps 582 | X |--------| Y |---------| GW |--------- Core 583 +---+ +---+ +----+ 584 .................... 585 . . 586 . v 587 [e] [f] 589 Figure 5: Example: Three Flows in Two Sites 591 With the Path Vector extension, a site can reveal the bottlenecks 592 inside its own network with necessary information (such as link 593 capacities) to the ALTO client, instead of providing the full 594 topology and routing information, or no bottleneck information at 595 all. The bottleneck information can be used to analyze the impact of 596 adding/removing data transfer flows, e.g., using the [G2] framework. 597 For example, assume hosts "a", "b", "c" are in site 1 and hosts "d", 598 "e", "f" are in site 2, and there are 3 flows in two sites: "a -> b", 599 "c -> d", "e -> f". For these flows, site 1 returns: 601 a: { b: [ane1] }, 602 c: { d: [ane1, ane2, ane3] } 604 ane1: bw = 10 Gbps (link: A->B) 605 ane2: bw = 10 Gbps (link: B->GW) 606 ane3: bw = 50 Gbps (link: GW->Core) 608 and site 2 returns: 610 c: { d: [anei, aneii, aneiii] } 611 e: { f: [aneiv] } 613 anei: bw = 5 Gbps (link Y->X) 614 aneii: bw = 10 Gbps (link GW->Y) 615 aneiii: bw = 20 Gbps (link Core->GW) 616 aneiv: bw = 10 Gbps (link Y->GW) 618 With the information, the data transfer scheduler can use algorithms 619 such as the theory on bottleneck structure [G2] to predict the 620 potential throughput of the flows. 622 4.2.2. Resource Exposure for CDN and Service Edge 624 A growing trend in today's applications (2021) is to bring storage 625 and computation closer to the end users for better QoE, such as 626 Content Delivery Network (CDN), AR/VR, and cloud gaming, as reported 627 in various documents (e.g., [SEREDGE] and [MOWIE]). Internet Service 628 Providers may deploy multiple layers of CDN caches, or more generally 629 service edges, with different latency and available resources 630 including number of CPU cores, memory, and storage. 632 For example, Figure 6 illustrates a typical edge-cloud scenario where 633 memory is measured in Gigabytes (G) and storage is measured in 634 Terabytes (T). The "on-premise" edge nodes are closest to the end 635 hosts and have the smallest latency, and the site-radio edge node and 636 access central office (CO) have larger latency but more available 637 resources. 639 +-------------+ +----------------------+ 640 | ALTO Client | <==========> | Application Provider | 641 +-------------+ +----------------------+ 642 PV | ^ PV | 643 Request | | Response | Resource allocation, 644 | | | service establishment, 645 (End hosts | | (Edge nodes | etc. 646 and cloud | | and metrics) | 647 servers) | | | 648 v | v 649 +-------------+ +---------------------+ 650 | ALTO Server | <=========> | Cloud-Edge Provider | 651 +-------------+ +---------------------+ 652 ____________________________________/\___________ 653 / \ 654 | (((o | 655 | 656 /_\ _~_ __ __ 657 a (/\_/\) ( ) ( )~( )_ 658 \ /------( )---------( )----\\---( ) 659 _|_ / (______) (___) ( ) 660 |_| -/ Site-radio Access CO (__________) 661 /---\ Edge Node 1 | Cloud DC 662 On premise | 663 /---------/ 664 (((o / 665 | / 666 Site-radio /_\ / 667 Edge Node 2(/\_/\)-----/ 668 /(_____)\ 669 ___ / \ --- 670 b--|_| -/ \--|_|--c 671 /---\ /---\ 672 On premise On premise 674 Figure 6: Example Use Case for Service Edge Exposure 676 a: { b: [ane1, ane2, ane3, ane4, ane5], 677 c: [ane1, ane2, ane3, ane4, ane6], 678 DC: [ane1, ane2, ane3] } 679 b: { c: [ane5, ane4, ane6], DC: [ane5, ane4, ane3] } 681 ane1: latency=5ms cpu=2 memory=8G storage=10T 682 (on premise, a) 684 ane2: latency=20ms cpu=4 memory=8G storage=10T 685 (Site-radio Edge Node 1) 687 ane3: latency=100ms cpu=8 memory=128G storage=100T 688 (Access CO) 690 ane4: latency=20ms cpu=4 memory=8G storage=10T 691 (Site-radio Edge Node 2) 693 ane5: latency=5ms cpu=2 memory=8G storage=10T 694 (on premise, b) 696 ane6: latency=5ms cpu=2 memory=8G storage=10T 697 (on premise, c) 699 Figure 7: Example Service Edge Query Results 701 With the extension defined in this document, an ALTO server can 702 selectively reveal the CDNs and service edges that reside along the 703 paths between different end hosts and/or the cloud servers, together 704 with their properties such as capabilities (e.g., storage, GPU) and 705 available Service Level Agreement (SLA) plans. See Figure 7 for an 706 example where the query is made for sources [a, b] and destinations 707 [b, c, DC]. Here each ANE represents a service edge and the 708 properties include access latency, available resources, etc. Note 709 the properties here are only used for illustration purposes and are 710 not part of this extension. 712 With the service edge information, an ALTO client may better conduct 713 CDN request routing or offload functionalities from the user 714 equipment to the service edge, with considerations on customized 715 quality of experience. 717 5. Path Vector Extension: Overview 719 This section provides a non-normative overview of the Path Vector 720 extension defined in this document. It is assumed that the readers 721 are familiar with both the base protocol [RFC7285] and the Unified 722 Property Map extension [I-D.ietf-alto-unified-props-new]. 724 To satisfy the additional requirements listed in Section 4.1, this 725 extension: 727 1. introduces the concept of Abstract Network Element (ANE) as the 728 abstraction of components in a network whose properties may have 729 an impact on the end-to-end performance of the traffic handled by 730 those components, 732 2. extends the Cost Map and Endpoint Cost Service to convey the ANEs 733 traversed by the path of a pair as Path 734 Vectors, and 736 3. uses the Unified Property Map to convey the association between 737 the ANEs and their properties. 739 Thus, an ALTO client can learn about the ANEs that are important to 740 assess the QoE of different pairs by 741 investigating the corresponding Path Vector value (AR1), identify 742 common ANEs if an ANE appears in the Path Vectors of multiple 743 pairs (AR2), and retrieve the properties of the 744 ANEs by searching the Unified Property Map (AR3). 746 5.1. Abstract Network Element (ANE) 748 This extension introduces ANE as an indirect and network-agnostic way 749 to specify a component or an aggregation of components of a network 750 whose properties have an impact on the end-to-end performance for 751 application traffic between endpoints. 753 ANEs allow ALTO servers to focus on common properties of different 754 types of network components. For example, the throughput of a flow 755 can be constrained by different components in a network: the capacity 756 of a physical link, the maximum throughput of a firewall, the 757 reserved bandwidth of an MPLS tunnel, etc. See the example below, 758 assume the throughput of the firewall is 100 Mbps and the capacity 759 for link (A, B) is also 100 Mbps, they result in the same constraint 760 on the total throughput of f1 and f2. Thus, they are identical when 761 treated as an ANE. 763 f1 | ^ f1 764 | | -----------------> 765 +----------+ +---+ +---+ 766 | Firewall | | A |-----| B | 767 +----------+ +---+ +---+ 768 | | -----------------> 769 v | f2 f2 771 When an ANE is defined by an ALTO server, it is assigned an 772 identifier by the ALTO server, i.e., a string of type ANEName as 773 specified in Section 6.1, and a set of associated properties. 775 5.1.1. ANE Entity Domain 777 In this extension, the associations between ANE and the properties 778 are conveyed in a Unified Property Map. Thus, ANEs must constitute an 779 entity domain (Section 5.1 of [I-D.ietf-alto-unified-props-new]), and 780 each ANE property must be an entity property (Section 5.2 of 781 [I-D.ietf-alto-unified-props-new]). 783 Specifically, this document defines a new entity domain called "ane" 784 as specified in Section 6.2 and defines two initial properties for 785 the ANE entity domain. 787 5.1.2. Ephemeral and Persistent ANEs 789 By design, ANEs are ephemeral and not to be used in further requests 790 to other ALTO resources. More precisely, the corresponding ANE names 791 are no longer valid beyond the scope of a Path Vector response or the 792 incremental update stream for a Path Vector request. Compared with 793 globally unique ANE names, ephemeral ANE has several benefits 794 including better privacy of the ISP's internal structure and more 795 flexible ANE computation. 797 For example, an ALTO server may define an ANE for each aggregated 798 bottleneck link between the sources and destinations specified in the 799 request. For requests with different sources and destinations, the 800 bottlenecks may be different but can safely reuse the same ANE names. 801 The client can still adjust its traffic based on the information but 802 is difficult to infer the underlying topology with multiple queries. 804 However, sometimes an ISP may intend to selectively reveal some 805 "persistent" network components which, opposite to being ephemeral, 806 have a longer life cycle. For example, an ALTO server may define an 807 ANE for each service edge cluster. Once a client chooses to use a 808 service edge, e.g., by deploying some user-defined functions, it may 809 want to stick to the service edge to avoid the complexity of state 810 transition or synchronization, and continuously query the properties 811 of the edge cluster. 813 This document provides a mechanism to expose such network components 814 as persistent ANEs. A persistent ANE has a persistent ID that is 815 registered in a Property Map, together with their properties. See 816 Section 6.2.4 and Section 6.4.2 for more detailed instructions on how 817 to identify ephemeral ANEs and persistent ANEs. 819 5.1.3. Property Filtering 821 Resource-constrained ALTO clients (see Section 4.1.2 of [RFC7285]) 822 may benefit from the filtering of Path Vector query results at the 823 ALTO server, as an ALTO client may only require a subset of the 824 available properties. 826 Specifically, the available properties for a given resource are 827 announced in the Information Resource Directory as a new capability 828 called "ane-property-names". The properties selected by a client as 829 being of interest are specified in the subsequent Path Vector queries 830 using the filter called 'ane-property-names'. The response includes 831 and only includes the selected properties for the ANEs in the 832 response. 834 The "ane-property-names" capability for Cost Map and for Endpoint 835 Cost Service is specified in Section 7.2.4 and Section 7.3.4 836 respectively. The "ane-property-names" filter for Cost Map and 837 Endpoint Cost Service is specified in Section 7.2.3 and Section 7.3.3 838 accordingly. 840 5.2. Path Vector Cost Type 842 For an ALTO client to correctly interpret the Path Vector, this 843 extension specifies a new cost type called the Path Vector cost type. 845 The Path Vector cost type must convey both the interpretation and 846 semantics in the "cost-mode" and "cost-metric" respectively. 847 Unfortunately, a single "cost-mode" value cannot fully specify the 848 interpretation of a Path Vector, which is a compound data type. For 849 example, in programming languages such as C++ where there existed a 850 JSON array type named JSONArray, a Path Vector will have the type of 851 JSONArray. 853 Instead of extending the "type system" of ALTO, this document takes a 854 simple and backward compatible approach. Specifically, the "cost- 855 mode" of the Path Vector cost type is "array", which indicates the 856 value is a JSON array. Then, an ALTO client must check the value of 857 the "cost-metric". If the value is "ane-path", it means that the 858 JSON array should be further interpreted as a path of ANENames. 860 The Path Vector cost type is specified in Section 6.5. 862 5.3. Multipart Path Vector Response 864 For a basic ALTO information resource, a response contains only one 865 type of ALTO resources, e.g., Network Map, Cost Map, or Property Map. 866 Thus, only one round of communication is required: An ALTO client 867 sends a request to an ALTO server, and the ALTO server returns a 868 response, as shown in Figure 8. 870 ALTO client ALTO server 871 |-------------- Request ---------------->| 872 |<------------- Response ----------------| 874 Figure 8: A Typical ALTO Request and Response 876 The extension defined in this document, on the other hand, involves 877 two types of information resources: Path Vectors conveyed in an 878 InfoResourceCostMap (defined in Section 11.2.3.6 of [RFC7285]) or an 879 InfoResourceEndpointCostMap (defined in Section 11.5.1.6 of 880 [RFC7285]), and ANE properties conveyed in an InfoResourceProperties 881 (defined in Section 7.6 of [I-D.ietf-alto-unified-props-new]). 883 Instead of two consecutive message exchanges, the extension defined 884 in this document enforces one round of communication. Specifically, 885 the ALTO client must include the source and destination pairs and the 886 requested ANE properties in a single request, and the ALTO server 887 must return a single response containing both the Path Vectors and 888 properties associated with the ANEs in the Path Vectors, as shown in 889 Figure 9. Since the two parts are bundled together in one response 890 message, their orders are interchangeable. See Section 7.2.6 and 891 Section 7.3.6 for details. 893 ALTO client ALTO server 894 |------------- PV Request -------------->| 895 |<----- PV Response (Cost Map Part) -----| 896 |<--- PV Response (Property Map Part) ---| 898 Figure 9: The Path Vector Extension Request and Response 900 This design is based on the following considerations: 902 1. ANEs may be constructed on demand, and potentially based on the 903 requested properties (See Section 5.1 for more details). If 904 sources and destinations are not in the same request as the 905 properties, an ALTO server either cannot construct ANEs on- 906 demand, or must wait until both requests are received. 908 2. As ANEs may be constructed on demand, mappings of each ANE to its 909 underlying network devices and resources can be specific to the 910 request. In order to respond to the Property Map request 911 correctly, an ALTO server must store the mapping of each Path 912 Vector request until the client fully retrieves the property 913 information. The "stateful" behavior may substantially harm the 914 server scalability and potentially lead to Denial-of-Service 915 attacks. 917 One approach to realize the one-round communication is to define a 918 new media type to contain both objects, but this violates modular 919 design. This document follows the standard-conforming usage of 920 "multipart/related" media type defined in [RFC2387] to elegantly 921 combine the objects. Path Vectors are encoded in an 922 InfoResourceCostMap or an InfoResourceEndpointCostMap, and the 923 Property Map is encoded in an InfoResourceProperties. They are 924 encapsulated as parts of a multipart message. The modular 925 composition allows ALTO servers and clients to reuse the data models 926 of the existing information resources. Specifically, this document 927 addresses the following practical issues using "multipart/related". 929 5.3.1. Identifying the Media Type of the Root Object 931 ALTO uses media type to indicate the type of an entry in the 932 Information Resource Directory (IRD) (e.g., "application/alto- 933 costmap+json" for Cost Map and "application/alto-endpointcost+json" 934 for Endpoint Cost Service). Simply putting "multipart/related" as 935 the media type, however, makes it impossible for an ALTO client to 936 identify the type of service provided by related entries. 938 To address this issue, this document uses the "type" parameter to 939 indicate the root object of a multipart/related message. For a Cost 940 Map resource, the "media-type" field in the IRD entry is "multipart/ 941 related" with the parameter "type=application/alto-costmap+json"; for 942 an Endpoint Cost Service, the parameter is "type=application/alto- 943 endpointcost+json". 945 5.3.2. References to Part Messages 947 As the response of a Path Vector resource is a multipart message with 948 two different parts, it is important that each part can be uniquely 949 identified. Following the designs of [RFC8895], this extension 950 requires that an ALTO server assigns a unique identifier to each part 951 of the multipart response message. This identifier, referred to as a 952 Part Resource ID (See Section 6.6 for details), is present in the 953 part message's "Content-ID" header. By concatenating the Part 954 Resource ID to the identifier of the Path Vector request, an ALTO 955 server/client can uniquely identify the Path Vector Part or the 956 Property Map part. 958 6. Specification: Basic Data Types 960 6.1. ANE Name 962 An ANE Name is encoded as a JSON string with the same format as that 963 of the type PIDName (Section 10.1 of [RFC7285]). 965 The type ANEName is used in this document to indicate a string of 966 this format. 968 6.2. ANE Entity Domain 970 The ANE entity domain associates property values with the Abstract 971 Network Elements in a Property Map. Accordingly, the ANE entity 972 domain always depends on a Property Map. 974 It must be noted that the term "domain" here does not refer to a 975 network domain. Rather, it is inherited from the "entity domain" 976 defined in Sec 3.2 in [I-D.ietf-alto-unified-props-new] that 977 represents the set of valid entities defined by an ALTO information 978 resource (called the defining information resource). 980 6.2.1. Entity Domain Type 982 The Entity Domain Type is "ane". 984 6.2.2. Domain-Specific Entity Identifier 986 The entity identifiers are the ANE Names in the associated Property 987 Map. 989 6.2.3. Hierarchy and Inheritance 991 There is no hierarchy or inheritance for properties associated with 992 ANEs. 994 6.2.4. Media Type of Defining Resource 996 The defining resource for entity domain type "ane" MUST be a Property 997 Map, i.e., the media type of defining resources is: 999 application/alto-propmap+json 1001 Specifically, for ephemeral ANEs that appear in a Path Vector 1002 response, their entity domain names MUST be exactly ".ane" and the 1003 defining resource of these ANEs is the Property Map part of the 1004 multipart response. Meanwhile, for any persistent ANE whose defining 1005 resource is a Property Map resource, its entity domain name MUST have 1006 the format of "PROPMAP.ane" where PROPMAP is the resource ID of the 1007 defining resource. Persistent entities are "persistent" because 1008 standalone queries can be made by an ALTO client to their defining 1009 resource(s) when the connection to the Path Vector service is closed. 1011 For example, the defining resource of an ephemeral ANE whose entity 1012 identifier is ".ane:NET1" is the Property Map part that contains this 1013 identifier. The defining resource of a persistent ANE whose entity 1014 identifier is "dc-props.ane:DC1" is the Property Map with the 1015 resource ID "dc-props". 1017 6.3. ANE Property Name 1019 An ANE Property Name is encoded as a JSON string with the same format 1020 as that of Entity Property Name (Section 5.2.2 of 1021 [I-D.ietf-alto-unified-props-new]). 1023 6.4. Initial ANE Property Types 1025 Two initial ANE property types are specified, "max-reservable- 1026 bandwidth" and "persistent-entity-id". 1028 Note that these property types do not depend on any information 1029 resource. As such, the EntityPropertyName MUST only have the 1030 EntityPropertyType part. 1032 6.4.1. Maximum Reservable Bandwidth 1034 The maximum reservable bandwidth property ("max-reservable- 1035 bandwidth") stands for the maximum bandwidth that can be reserved for 1036 all the traffic that traverses an ANE. The value MUST be encoded as 1037 a non-negative numerical cost value as defined in Section 6.1.2.1 of 1038 [RFC7285] and the unit is bit per second (bps). If this property is 1039 requested by the ALTO client but not present for an ANE in the server 1040 response, it MUST be interpreted as that the property is not defined 1041 for the ANE. 1043 This property can be offered in a setting where the ALTO server is 1044 part of a network system that provides on-demand resource allocation 1045 and the ALTO client is part of a user application. One existing 1046 example is [NOVA]: the ALTO server is part of an SDN controller and 1047 exposes a list of traversed network elements and associated link 1048 bandwidth to the client. The encoding in [NOVA] differs from the 1049 Path Vector response defined in this document that the Path Vector 1050 part and Property Map part are put in the same JSON object. 1052 In such a framework, the ALTO server exposes resource (e.g., 1053 reservable bandwidth) availability information to the ALTO client. 1054 How the client makes resource requests based on the information and 1055 how the resource allocation is achieved respectively depend on 1056 interfaces between the management system and the users or a higher- 1057 layer protocol (e.g., SDN network intents or MPLS tunnels), which are 1058 out of the scope of this document. 1060 6.4.2. Persistent Entity ID 1062 The persistent entity ID property is the entity identifier of the 1063 persistent ANE which an ephemeral ANE presents (See Section 5.1.2 for 1064 details). The value of this property is encoded with the format 1065 EntityID defined in Section 5.1.3 of 1066 [I-D.ietf-alto-unified-props-new]. 1068 In this format, the entity ID combines: 1070 * a defining information resource for the ANE on which a 1071 "persistent-entity-id" is queried, which is the Property Map 1072 resource defining the ANE as a persistent entity, together with 1073 the properties; 1075 * the persistent name of the ANE in that Property Map. 1077 With this format, the client has all the needed information for 1078 further standalone query properties on the persistent ANE. 1080 6.4.3. Examples 1082 To illustrate the use of "max-reservable-bandwidth", consider the 1083 following network with 5 nodes. Assume the client wants to query the 1084 maximum reservable bandwidth from H1 to H2. An ALTO server may split 1085 the network into two ANEs: "ane1" that represents the subnetwork with 1086 routers A, B, and C, and "ane2" that represents the subnetwork with 1087 routers B, D and E. The maximum reservable bandwidth for "ane1" is 1088 15 Mbps (using path A->C->B) and the maximum reservable bandwidth for 1089 "ane2" is 20 Mbps (using path B->D->E). 1091 20 Mbps 20 Mbps 1092 10 Mbps +---+ +---+ +---+ 1093 /----| B |---| D |----| E |---- H2 1094 +---+/ +---+ +---+ +---+ 1095 H1 ----| A | 15 Mbps| 1096 +---+\ +---+ 1097 \----| C | 1098 15 Mbps +---+ 1100 To illustrate the use of "persistent-entity-id", consider the 1101 scenario in Figure 6. As the life cycle of service edges are 1102 typically long, they may contain information that is not specific to 1103 the query. Such information can be stored in an individual unified 1104 property map and later be accessed by an ALTO client. 1106 For example, "ane1" in Figure 7 represents the on-premise service 1107 edge closest to host a. Assume the properties of the service edges 1108 are provided in a unified property map called "se-props" and the ID 1109 of the on-premise service edge is "9a0b55f7-7442-4d56-8a2c- 1110 b4cc6a8e3aa1", the "persistent-entity-id" of "ane1" will be "se- 1111 props.ane:9a0b55f7-7442-4d56-8a2c-b4cc6a8e3aa1". With this 1112 persistent entity ID, an ALTO client may send queries to the "se- 1113 props" resource with the entity ID ".ane:9a0b55f7-7442-4d56-8a2c- 1114 b4cc6a8e3aa1". 1116 6.5. Path Vector Cost Type 1118 This document defines a new cost type, which is referred to as the 1119 Path Vector cost type. An ALTO server MUST offer this cost type if 1120 it supports the extension defined in this document. 1122 6.5.1. Cost Metric: ane-path 1124 The cost metric "ane-path" indicates the value of such a cost type 1125 conveys an array of ANE names, where each ANE name uniquely 1126 represents an ANE traversed by traffic from a source to a 1127 destination. 1129 An ALTO client MUST interpret the Path Vector as if the traffic 1130 between a source and a destination logically traverses the ANEs in 1131 the same order as they appear in the Path Vector. 1133 When the Path Vector procedures defined in this document are in use, 1134 an ALTO server using the "ane-path" cost metric and the "array" cost 1135 mode (see Section 6.5.2) MUST return as the cost value a JSON array 1136 of ANEName and the client MUST also check that each element contained 1137 in the array is an ANEName (Section 6.1). Otherwise, the client MUST 1138 discard the response and SHOULD follow the instructions in 1139 Section 8.3.4.3 of [RFC7285] to handle the error. 1141 6.5.2. Cost Mode: array 1143 The cost mode "array" indicates that every cost value in the response 1144 body of a (Filtered) Cost Map or an Endpoint Cost Service MUST be 1145 interpreted as a JSON array object. While this cost mode can be 1146 applied to all cost metrics, additional specifications will be needed 1147 to clarify the semantics of the array cost mode when combined with 1148 cost metrics other than 'ane-path'. 1150 6.6. Part Resource ID and Part Content ID 1152 A Part Resource ID is encoded as a JSON string with the same format 1153 as that of the type ResourceID (Section 10.2 of [RFC7285]). 1155 Even though the client-id assigned to a Path Vector request and the 1156 Part Resource ID MAY contain up to 64 characters by their own 1157 definition, their concatenation (see Section 5.3.2) MUST also conform 1158 to the same length constraint. The same requirement applies to the 1159 resource ID of the Path Vector resource, too. Thus, it is 1160 RECOMMENDED to limit the length of resource ID and client ID related 1161 to a Path Vector resource to 31 characters. 1163 A Part Content ID conforms to the format of msg-id as specified in 1164 [RFC2387] and [RFC5322]. Specifically, it has the following format: 1166 "<" PART-RESOURCE-ID "@" DOMAIN-NAME ">" 1168 PART-RESOURCE-ID: PART-RESOURCE-ID has the same format as the Part 1169 Resource ID. It is used to identify whether a part message is a 1170 Path Vector or a Property Map. 1172 DOMAIN-NAME: DOMAIN-NAME has the same format as dot-atom-text 1173 specified in Section 3.2.3 of [RFC5322]. It must be the domain 1174 name of the ALTO server. 1176 7. Specification: Service Extensions 1178 7.1. Notations 1180 This document uses the same syntax and notations as introduced in 1181 Section 8.2 of RFC 7285 [RFC7285] to specify the extensions to 1182 existing ALTO resources and services. 1184 7.2. Multipart Filtered Cost Map for Path Vector 1186 This document introduces a new ALTO resource called multipart 1187 Filtered Cost Map resource, which allows an ALTO server to provide 1188 other ALTO resources associated with the Cost Map resource in the 1189 same response. 1191 7.2.1. Media Type 1193 The media type of the multipart Filtered Cost Map resource is 1194 "multipart/related" and the required "type" parameter MUST have a 1195 value of "application/alto-costmap+json". 1197 7.2.2. HTTP Method 1199 The multipart Filtered Cost Map is requested using the HTTP POST 1200 method. 1202 7.2.3. Accept Input Parameters 1204 The input parameters of the multipart Filtered Cost Map are supplied 1205 in the body of an HTTP POST request. This document extends the input 1206 parameters to a Filtered Cost Map, which is defined as a JSON object 1207 of type ReqFilteredCostMap in Section 4.1.2 of RFC 8189 [RFC8189], 1208 with a data format indicated by the media type "application/alto- 1209 costmapfilter+json", which is a JSON object of type 1210 PVReqFilteredCostMap: 1212 object { 1213 [EntityPropertyName ane-property-names<0..*>;] 1214 } PVReqFilteredCostMap : ReqFilteredCostMap; 1216 with fields: 1218 ane-property-names: A list of selected ANE properties to be included 1219 in the response. Each property in this list MUST match one of the 1220 supported ANE properties indicated in the resource's "ane- 1221 property-names" capability (Section 7.2.4). If the field is not 1222 present, it MUST be interpreted as an empty list. 1224 Example: Consider the network in Figure 1. If an ALTO client wants 1225 to query the "max-reservable-bandwidth" between PID1 and PID2, it can 1226 submit the following request. 1228 POST /costmap/pv HTTP/1.1 1229 Host: alto.example.com 1230 Accept: multipart/related;type=application/alto-costmap+json, 1231 application/alto-error+json 1232 Content-Length: 201 1233 Content-Type: application/alto-costmapfilter+json 1235 { 1236 "cost-type": { 1237 "cost-mode": "array", 1238 "cost-metric": "ane-path" 1239 }, 1240 "pids": { 1241 "srcs": [ "PID1" ], 1242 "dsts": [ "PID2" ] 1243 }, 1244 "ane-property-names": [ "max-reservable-bandwidth" ] 1245 } 1247 7.2.4. Capabilities 1249 The multipart Filtered Cost Map resource extends the capabilities 1250 defined in Section 4.1.1 of [RFC8189]. The capabilities are defined 1251 by a JSON object of type PVFilteredCostMapCapabilities: 1253 object { 1254 [EntityPropertyName ane-property-names<0..*>;] 1255 } PVFilteredCostMapCapabilities : FilteredCostMapCapabilities; 1257 with fields: 1259 ane-property-names: Defines a list of ANE properties that can be 1260 returned. If the field is not present, it MUST be interpreted as 1261 an empty list, indicating the ALTO server cannot provide any ANE 1262 property. 1264 This extension also introduces additional restrictions for the 1265 following fields: 1267 cost-type-names: The "cost-type-names" field MUST include the Path 1268 Vector cost type, unless explicitly documented by a future 1269 extension. This also implies that the Path Vector cost type MUST 1270 be defined in the "cost-types" of the Information Resource 1271 Directory's "meta" field. 1273 cost-constraints: If the "cost-type-names" field includes the Path 1274 Vector cost type, "cost-constraints" field MUST be "false" or not 1275 present unless specifically instructed by a future document. 1277 testable-cost-type-names (Section 4.1.1 of [RFC8189]): If the "cost- 1278 type-names" field includes the Path Vector cost type and the 1279 "testable-cost-type-names" field is present, the Path Vector cost 1280 type MUST NOT be included in the "testable-cost-type-names" field 1281 unless specifically instructed by a future document. 1283 7.2.5. Uses 1285 This member MUST include the resource ID of the network map based on 1286 which the PIDs are defined. If this resource supports "persistent- 1287 entity-id", it MUST also include the defining resources of persistent 1288 ANEs that may appear in the response. 1290 7.2.6. Response 1292 The response MUST indicate an error, using ALTO protocol error 1293 handling, as defined in Section 8.5 of [RFC7285], if the request is 1294 invalid. 1296 The "Content-Type" header of the response MUST be "multipart/related" 1297 as defined by [RFC2387] with the following parameters: 1299 type: The type parameter is mandatory and MUST be "application/alto- 1300 costmap+json". Note that [RFC2387] permits both parameters with 1301 and without the double quotes. 1303 start: The start parameter is as defined in [RFC2387] and is 1304 optional. If present, it MUST have the same value as the 1305 "Content-ID" header of the Path Vector part. 1307 boundary: The boundary parameter is as defined in Section 5.1.1 of 1308 [RFC2046] and is mandatory. 1310 The body of the response MUST consist of two parts: 1312 * The Path Vector part MUST include "Content-ID" and "Content-Type" 1313 in its header. The "Content-Type" MUST be "application/alto- 1314 costmap+json". The value of "Content-ID" MUST have the same 1315 format as the Part Content ID as specified in Section 6.6. 1317 The body of the Path Vector part MUST be a JSON object with the 1318 same format as defined in Section 11.2.3.6 of [RFC7285] when the 1319 "cost-type" field is present in the input parameters and MUST be a 1320 JSON object with the same format as defined in Section 4.1.3 of 1321 [RFC8189] if the "multi-cost-types" field is present. The JSON 1322 object MUST include the "vtag" field in the "meta" field, which 1323 provides the version tag of the returned CostMapData. The 1324 resource ID of the version tag MUST follow the format of 1325 resource-id '.' part-resource-id 1327 where "resource-id" is the resource Id of the Path Vector 1328 resource, and "part-resource-id" has the same value as the PART- 1329 RESOURCE-ID in the "Content-ID" of the Path Vector part. The 1330 "meta" field MUST also include the "dependent-vtags" field, whose 1331 value is a single-element array to indicate the version tag of the 1332 network map used, where the network map is specified in the "uses" 1333 attribute of the multipart Filtered Cost Map resource in IRD. 1335 * The Unified Property Map part MUST also include "Content-ID" and 1336 "Content-Type" in its header. The "Content-Type" MUST be 1337 "application/alto-propmap+json". The value of "Content-ID" MUST 1338 have the same format as the Part Content ID as specified in 1339 Section 6.6. 1341 The body of the Unified Property Map part is a JSON object with 1342 the same format as defined in Section 7.6 of 1343 [I-D.ietf-alto-unified-props-new]. The JSON object MUST include 1344 the "dependent-vtags" field in the "meta" field. The value of the 1345 "dependent-vtags" field MUST be an array of VersionTag objects as 1346 defined by Section 10.3 of [RFC7285]. The "vtag" of the Path 1347 Vector part MUST be included in the "dependent-vtags". If 1348 "persistent-entity-id" is requested, the version tags of the 1349 dependent resources that may expose the entities in the response 1350 MUST also be included. 1352 The PropertyMapData has one member for each ANEName that appears 1353 in the Path Vector part, which is an entity identifier belonging 1354 to the self-defined entity domain as defined in Section 5.1.2.3 of 1355 [I-D.ietf-alto-unified-props-new]. The EntityProps for each ANE 1356 has one member for each property that is both 1) associated with 1357 the ANE, and 2) specified in the "ane-property-names" in the 1358 request. If the Path Vector cost type is not included in the 1359 "cost-type" field or the "multi-cost-type" field, the "property- 1360 map" field MUST be present and the value MUST be an empty object 1361 ({}). 1363 A complete and valid response MUST include both the Path Vector part 1364 and the Property Map part in the multipart message. If any part is 1365 NOT present, the client MUST discard the received information and 1366 send another request if necessary. 1368 According to [RFC2387], the Path Vector part, whose media type is the 1369 same as the "type" parameter of the multipart response message, is 1370 the root object. Thus, it is the element the application processes 1371 first. Even though the "start" parameter allows it to be placed 1372 anywhere in the part sequence, it is RECOMMENDED that the parts 1373 arrive in the same order as they are processed, i.e., the Path Vector 1374 part is always put as the first part, followed by the Property Map 1375 part. When doing so, an ALTO server MAY choose not to set the 1376 "start" parameter, which implies the first part is the root object. 1378 Example: Consider the network in Figure 1. The response of the 1379 example request in Section 7.2.3 is as follows, where "ANE1" 1380 represents the aggregation of all the switches in the network. 1382 HTTP/1.1 200 OK 1383 Content-Length: 859 1384 Content-Type: multipart/related; boundary=example-1; 1385 type=application/alto-costmap+json 1387 --example-1 1388 Content-ID: 1389 Content-Type: application/alto-costmap+json 1391 { 1392 "meta": { 1393 "vtag": { 1394 "resource-id": "filtered-cost-map-pv.costmap", 1395 "tag": "fb20b76204814e9db37a51151faaaef2" 1396 }, 1397 "dependent-vtags": [ 1398 { 1399 "resource-id": "my-default-networkmap", 1400 "tag": "75ed013b3cb58f896e839582504f6228" 1401 } 1402 ], 1403 "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } 1404 }, 1405 "cost-map": { 1406 "PID1": { "PID2": ["ANE1"] } 1407 } 1408 } 1409 --example-1 1410 Content-ID: 1411 Content-Type: application/alto-propmap+json 1413 { 1414 "meta": { 1415 "dependent-vtags": [ 1416 { 1417 "resource-id": "filtered-cost-map-pv.costmap", 1418 "tag": "fb20b76204814e9db37a51151faaaef2" 1419 } 1420 ] 1421 }, 1422 "property-map": { 1423 ".ane:ANE1": { "max-reservable-bandwidth": 100000000 } 1424 } 1425 } 1427 7.3. Multipart Endpoint Cost Service for Path Vector 1429 This document introduces a new ALTO resource called multipart 1430 Endpoint Cost Service, which allows an ALTO server to provide other 1431 ALTO resources associated with the Endpoint Cost Service resource in 1432 the same response. 1434 7.3.1. Media Type 1436 The media type of the multipart Endpoint Cost Service resource is 1437 "multipart/related" and the required "type" parameter MUST have a 1438 value of "application/alto-endpointcost+json". 1440 7.3.2. HTTP Method 1442 The multipart Endpoint Cost Service resource is requested using the 1443 HTTP POST method. 1445 7.3.3. Accept Input Parameters 1447 The input parameters of the multipart Endpoint Cost Service resource 1448 are supplied in the body of an HTTP POST request. This document 1449 extends the input parameters to an Endpoint Cost Service, which is 1450 defined as a JSON object of type ReqEndpointCost in Section 4.2.2 of 1451 [RFC8189], with a data format indicated by the media type 1452 "application/alto-endpointcostparams+json", which is a JSON object of 1453 type PVReqEndpointCost: 1455 object { 1456 [EntityPropertyName ane-property-names<0..*>;] 1457 } PVReqEndpointcost : ReqEndpointcostMap; 1459 with fields: 1461 ane-property-names: This document defines the "ane-property-names" 1462 in PVReqEndpointcost as the same as in PVReqFilteredCostMap. See 1463 Section 7.2.3. 1465 Example: Consider the network in Figure 1. If an ALTO client wants 1466 to query the "max-reservable-bandwidth" between eh1 and eh2, it can 1467 submit the following request. 1469 POST /ecs/pv HTTP/1.1 1470 Host: alto.example.com 1471 Accept: multipart/related;type=application/alto-endpointcost+json, 1472 application/alto-error+json 1473 Content-Length: 227 1474 Content-Type: application/alto-endpointcostparams+json 1476 { 1477 "cost-type": { 1478 "cost-mode": "array", 1479 "cost-metric": "ane-path" 1480 }, 1481 "endpoints": { 1482 "srcs": [ "ipv4:192.0.2.2" ], 1483 "dsts": [ "ipv4:192.0.2.18" ] 1484 }, 1485 "ane-property-names": [ "max-reservable-bandwidth" ] 1486 } 1488 7.3.4. Capabilities 1490 The capabilities of the multipart Endpoint Cost Service resource are 1491 defined by a JSON object of type PVEndpointcostCapabilities, which is 1492 defined as the same as PVFilteredCostMapCapabilities. See 1493 Section 7.2.4. 1495 7.3.5. Uses 1497 If this resource supports "persistent-entity-id", it MUST also 1498 include the defining resources of persistent ANEs that may appear in 1499 the response. 1501 7.3.6. Response 1503 The response MUST indicate an error, using ALTO protocol error 1504 handling, as defined in Section 8.5 of [RFC7285], if the request is 1505 invalid. 1507 The "Content-Type" header of the response MUST be "multipart/related" 1508 as defined by [RFC7285] with the following parameters: 1510 type: The type parameter MUST be "application/alto- 1511 endpointcost+json" and is mandatory. 1513 start: The start parameter is as defined in Section 7.2.6. 1515 boundary: The boundary parameter is as defined in Section 5.1.1 of 1516 [RFC2046] and is mandatory. 1518 The body MUST consist of two parts: 1520 * The Path Vector part MUST include "Content-ID" and "Content-Type" 1521 in its header. The "Content-Type" MUST be "application/alto- 1522 endpointcost+json". The value of "Content-ID" MUST have the same 1523 format as the Part Content ID as specified in Section 6.6. 1525 The body of the Path Vector part MUST be a JSON object with the 1526 same format as defined in Section 11.5.1.6 of [RFC7285] when the 1527 "cost-type" field is present in the input parameters and MUST be a 1528 JSON object with the same format as defined in Section 4.2.3 of 1529 [RFC8189] if the "multi-cost-types" field is present. The JSON 1530 object MUST include the "vtag" field in the "meta" field, which 1531 provides the version tag of the returned EndpointCostMapData. The 1532 resource ID of the version tag MUST follow the format of 1534 resource-id '.' part-resource-id 1536 where "resource-id" is the resource Id of the Path Vector 1537 resource, and "part-resource-id" has the same value as the PART- 1538 RESOURCE-ID in the "Content-ID" of the Path Vector part. 1540 * The Unified Property Map part MUST also include "Content-ID" and 1541 "Content-Type" in its header. The "Content-Type" MUST be 1542 "application/alto-propmap+json". The value of "Content-ID" MUST 1543 have the same format as the Part Content ID as specified in 1544 Section 6.6. 1546 The body of the Unified Property Map part MUST be a JSON object 1547 with the same format as defined in Section 7.6 of 1548 [I-D.ietf-alto-unified-props-new]. The JSON object MUST include 1549 the "dependent-vtags" field in the "meta" field. The value of the 1550 "dependent-vtags" field MUST be an array of VersionTag objects as 1551 defined by Section 10.3 of [RFC7285]. The "vtag" of the Path 1552 Vector part MUST be included in the "dependent-vtags". If 1553 "persistent-entity-id" is requested, the version tags of the 1554 dependent resources that may expose the entities in the response 1555 MUST also be included. 1557 The PropertyMapData has one member for each ANEName that appears 1558 in the Path Vector part, which is an entity identifier belonging 1559 to the self-defined entity domain as defined in Section 5.1.2.3 of 1560 [I-D.ietf-alto-unified-props-new]. The EntityProps for each ANE 1561 has one member for each property that is both 1) associated with 1562 the ANE, and 2) specified in the "ane-property-names" in the 1563 request. If the Path Vector cost type is not included in the 1564 "cost-type" field or the "multi-cost-type" field, the "property- 1565 map" field MUST be present and the value MUST be an empty object 1566 ({}). 1568 A complete and valid response MUST include both the Path Vector part 1569 and the Property Map part in the multipart message. If any part is 1570 NOT present, the client MUST discard the received information and 1571 send another request if necessary. 1573 According to [RFC2387], the Path Vector part, whose media type is the 1574 same as the "type" parameter of the multipart response message, is 1575 the root object. Thus, it is the element the application processes 1576 first. Even though the "start" parameter allows it to be placed 1577 anywhere in the part sequence, it is RECOMMENDED that the parts 1578 arrive in the same order as they are processed, i.e., the Path Vector 1579 part is always put as the first part, followed by the Property Map 1580 part. When doing so, an ALTO server MAY choose not to set the 1581 "start" parameter, which implies the first part is the root object. 1583 Example: Consider the network in Figure 1. The response of the 1584 example request in Section 7.3.3 is as follows. 1586 HTTP/1.1 200 OK 1587 Content-Length: 845 1588 Content-Type: multipart/related; boundary=example-1; 1589 type=application/alto-endpointcost+json 1591 --example-1 1592 Content-ID: 1593 Content-Type: application/alto-endpointcost+json 1595 { 1596 "meta": { 1597 "vtag": { 1598 "resource-id": "ecs-pv.ecs", 1599 "tag": "ec137bb78118468c853d5b622ac003f1" 1600 }, 1601 "dependent-vtags": [ 1602 { 1603 "resource-id": "my-default-networkmap", 1604 "tag": "677fe5f4066848d282ece213a84f9429" 1605 } 1606 ], 1607 "cost-type": { "cost-mode": "array", "cost-metric": "ane-path" } 1608 }, 1609 "cost-map": { 1610 "ipv4:192.0.2.2": { "ipv4:192.0.2.18": ["ANE1"] } 1611 } 1612 } 1613 --example-1 1614 Content-ID: 1615 Content-Type: application/alto-propmap+json 1617 { 1618 "meta": { 1619 "dependent-vtags": [ 1620 { 1621 "resource-id": "ecs-pv.ecs", 1622 "tag": "ec137bb78118468c853d5b622ac003f1" 1623 } 1624 ] 1625 }, 1626 "property-map": { 1627 ".ane:ANE1": { "max-reservable-bandwidth": 100000000 } 1628 } 1629 } 1631 8. Examples 1633 This section lists some examples of Path Vector queries and the 1634 corresponding responses. Some long lines are truncated for better 1635 readability. 1637 8.1. Sample Setup 1639 ----- L1 1640 / 1641 PID1 +----------+ 10 Gbps +----------+ PID3 1642 192.0.2.0/28+-+ +------+ +---------+ +--+192.0.2.32/28 1643 | | MEC1 | | | | 2001:db8::3:0/16 1644 | +------+ | +-----+ | 1645 PID2 | | | +----------+ 1646 192.0.2.16/28+-+ | | NET3 1647 | | | 15 Gbps 1648 | | | \ 1649 +----------+ | -------- L2 1650 NET1 | 1651 +----------+ 1652 | +------+ | PID4 1653 | | MEC2 | +--+192.0.2.48/28 1654 | +------+ | 2001:db8::4:0/16 1655 +----------+ 1656 NET2 1658 Figure 10: Examples of ANE Properties 1660 In this document, Figure 10 is used to illustrate the message 1661 contents. There are 3 sub-networks (NET1, NET2 and NET3) and two 1662 interconnection links (L1 and L2). It is assumed that each sub- 1663 network has sufficiently large bandwidth to be reserved. 1665 8.2. Information Resource Directory 1667 To give a comprehensive example of the extension defined in this 1668 document, we consider the network in Figure 10. Assume that the ALTO 1669 server provides the following information resources: 1671 * "my-default-networkmap": A Network Map resource which contains the 1672 PIDs in the network. 1674 * "filtered-cost-map-pv": A Multipart Filtered Cost Map resource for 1675 Path Vector, which exposes the "max-reservable-bandwidth" property 1676 for the PIDs in "my-default-networkmap". 1678 * "ane-props": A filtered Unified Property resource that exposes the 1679 information for persistent ANEs in the network. 1681 * "endpoint-cost-pv": A Multipart Endpoint Cost Service for Path 1682 Vector, which exposes the "max-reservable-bandwidth" and the 1683 "persistent-entity-id" properties. 1685 * "update-pv": An Update Stream service, which provides the 1686 incremental update service for the "endpoint-cost-pv" service. 1688 * "multicost-pv": A Multipart Endpoint Cost Service with both Multi- 1689 Cost and Path Vector. 1691 Below is the Information Resource Directory of the example ALTO 1692 server. To enable the extension defined in this document, the "path- 1693 vector" cost type (Section 6.5) is defined in the "cost-types" of the 1694 "meta" field, and is included in the "cost-type-names" of resources 1695 "filtered-cost-map-pv" and "endpoint-cost-pv". 1697 { 1698 "meta": { 1699 "cost-types": { 1700 "path-vector": { 1701 "cost-mode": "array", 1702 "cost-metric": "ane-path" 1703 }, 1704 "num-rc": { 1705 "cost-mode": "numerical", 1706 "cost-metric": "routingcost" 1707 } 1708 } 1709 }, 1710 "resources": { 1711 "my-default-networkmap": { 1712 "uri": "https://alto.example.com/networkmap", 1713 "media-type": "application/alto-networkmap+json" 1714 }, 1715 "filtered-cost-map-pv": { 1716 "uri": "https://alto.example.com/costmap/pv", 1717 "media-type": "multipart/related; 1718 type=application/alto-costmap+json", 1719 "accepts": "application/alto-costmapfilter+json", 1720 "capabilities": { 1721 "cost-type-names": [ "path-vector" ], 1722 "ane-property-names": [ "max-reservable-bandwidth" ] 1723 }, 1724 "uses": [ "my-default-networkmap" ] 1725 }, 1726 "ane-props": { 1727 "uri": "https://alto.example.com/ane-props", 1728 "media-type": "application/alto-propmap+json", 1729 "accepts": "application/alto-propmapparams+json", 1730 "capabilities": { 1731 "mappings": { 1732 ".ane": [ "cpu" ] 1733 } 1734 } 1735 }, 1736 "endpoint-cost-pv": { 1737 "uri": "https://alto.exmaple.com/endpointcost/pv", 1738 "media-type": "multipart/related; 1739 type=application/alto-endpointcost+json", 1740 "accepts": "application/alto-endpointcostparams+json", 1741 "capabilities": { 1742 "cost-type-names": [ "path-vector" ], 1743 "ane-property-names": [ 1744 "max-reservable-bandwidth", "persistent-entity-id" 1745 ] 1746 }, 1747 "uses": [ "ane-props" ] 1748 }, 1749 "update-pv": { 1750 "uri": "https://alto.example.com/updates/pv", 1751 "media-type": "text/event-stream", 1752 "uses": [ "endpoint-cost-pv" ], 1753 "accepts": "application/alto-updatestreamparams+json", 1754 "capabilities": { 1755 "support-stream-control": true 1756 } 1757 }, 1758 "multicost-pv": { 1759 "uri": "https://alto.exmaple.com/endpointcost/mcpv", 1760 "media-type": "multipart/related; 1761 type=application/alto-endpointcost+json", 1762 "accepts": "application/alto-endpointcostparams+json", 1763 "capabilities": { 1764 "cost-type-names": [ "path-vector", "num-rc" ], 1765 "max-cost-types": 2, 1766 "testable-cost-type-names": [ "num-rc" ], 1767 "ane-property-names": [ 1768 "max-reservable-bandwidth", "persistent-entity-id" 1769 ] 1770 }, 1771 "uses": [ "ane-props" ] 1772 } 1773 } 1775 } 1777 8.3. Multipart Filtered Cost Map 1779 The following examples demonstrate the request to the "filtered-cost- 1780 map-pv" resource and the corresponding response. 1782 The request uses the "path-vector" cost type in the "cost-type" 1783 field. The "ane-property-names" field is missing, indicating that 1784 the client only requests for the Path Vector but not the ANE 1785 properties. 1787 The response consists of two parts. The first part returns the array 1788 of ANEName for each source and destination pair. There are two ANEs, 1789 where "L1" represents the interconnection link L1, and "L2" 1790 represents the interconnection link L2. 1792 The second part returns an empty Property Map. Note that the ANE 1793 entries are omitted since they have no properties (See Section 3.1 of 1794 [I-D.ietf-alto-unified-props-new]). 1796 POST /costmap/pv HTTP/1.1 1797 Host: alto.example.com 1798 Accept: multipart/related;type=application/alto-costmap+json, 1799 application/alto-error+json 1800 Content-Length: 153 1801 Content-Type: application/alto-costmapfilter+json 1803 { 1804 "cost-type": { 1805 "cost-mode": "array", 1806 "cost-metric": "ane-path" 1807 }, 1808 "pids": { 1809 "srcs": [ "PID1" ], 1810 "dsts": [ "PID3", "PID4" ] 1811 } 1812 } 1814 HTTP/1.1 200 OK 1815 Content-Length: 855 1816 Content-Type: multipart/related; boundary=example-1; 1817 type=application/alto-costmap+json 1819 --example-1 1820 Content-ID: 1821 Content-Type: application/alto-costmap+json 1822 { 1823 "meta": { 1824 "vtag": { 1825 "resource-id": "filtered-cost-map-pv.costmap", 1826 "tag": "d827f484cb66ce6df6b5077cb8562b0a" 1827 }, 1828 "dependent-vtags": [ 1829 { 1830 "resource-id": "my-default-networkmap", 1831 "tag": "c04bc5da49534274a6daeee8ea1dec62" 1832 } 1833 ], 1834 "cost-type": { 1835 "cost-mode": "array", 1836 "cost-metric": "ane-path" 1837 } 1838 }, 1839 "cost-map": { 1840 "PID1": { 1841 "PID3": [ "L1" ], 1842 "PID4": [ "L1", "L2" ] 1843 } 1844 } 1845 } 1846 --example-1 1847 Content-ID: 1848 Content-Type: application/alto-propmap+json 1850 { 1851 "meta": { 1852 "dependent-vtags": [ 1853 { 1854 "resource-id": "filtered-cost-map-pv.costmap", 1855 "tag": "d827f484cb66ce6df6b5077cb8562b0a" 1856 } 1857 ] 1858 }, 1859 "property-map": { 1860 } 1861 } 1863 8.4. Multipart Endpoint Cost Service Resource 1865 The following examples demonstrate the request to the "endpoint-cost- 1866 pv" resource and the corresponding response. 1868 The request uses the Path Vector cost type in the "cost-type" field, 1869 and queries the Maximum Reservable Bandwidth ANE property and the 1870 Persistent Entity property for two IPv4 source and destination pairs 1871 (192.0.2.34 -> 192.0.2.2 and 192.0.2.34 -> 192.0.2.50) and one IPv6 1872 source and destination pair (2001:db8::3:1 -> 2001:db8::4:1). 1874 The response consists of two parts. The first part returns the array 1875 of ANEName for each valid source and destination pair. As one can 1876 see in Figure 10, flow 192.0.2.34 -> 192.0.2.2 traverses NET2, L1 and 1877 NET1, and flows 192.0.2.34 -> 192.0.2.50 and 2001:db8::3:1 -> 1878 2001:db8::4:1 traverse NET2, L2 and NET3. 1880 The second part returns the requested properties of ANEs. Assume 1881 NET1, NET2 and NET3 has sufficient bandwidth and their "max- 1882 reservable-bandwidth" values are set to a sufficiently large number 1883 (50 Gbps in this case). On the other hand, assume there are no prior 1884 reservation on L1 and L2, and their "max-reservable-bandwidth" values 1885 are the corresponding link capacity (10 Gbps for L1 and 15 Gbps for 1886 L2). 1888 Both NET1 and NET2 have a mobile edge deployed, i.e., MEC1 in NET1 1889 and MEC2 in NET2. Assume the ANEName for MEC1 and MEC2 are "MEC1" 1890 and "MEC2" and their properties can be retrieved from the Property 1891 Map "ane-props". Thus, the "persistent-entity-id" property of NET1 1892 and NET3 are "ane-props.ane:MEC1" and "ane-props.ane:MEC2" 1893 respectively. 1895 POST /endpointcost/pv HTTP/1.1 1896 Host: alto.example.com 1897 Accept: multipart/related; 1898 type=application/alto-endpointcost+json, 1899 application/alto-error+json 1900 Content-Length: 362 1901 Content-Type: application/alto-endpointcostparams+json 1903 { 1904 "cost-type": { 1905 "cost-mode": "array", 1906 "cost-metric": "ane-path" 1907 }, 1908 "endpoints": { 1909 "srcs": [ 1910 "ipv4:192.0.2.34", 1911 "ipv6:2001:db8::3:1" 1912 ], 1913 "dsts": [ 1914 "ipv4:192.0.2.2", 1915 "ipv4:192.0.2.50", 1916 "ipv6:2001:db8::4:1" 1917 ] 1918 }, 1919 "ane-property-names": [ 1920 "max-reservable-bandwidth", 1921 "persistent-entity-id" 1922 ] 1923 } 1925 HTTP/1.1 200 OK 1926 Content-Length: 1432 1927 Content-Type: multipart/related; boundary=example-2; 1928 type=application/alto-endpointcost+json 1930 --example-2 1931 Content-ID: 1932 Content-Type: application/alto-endpointcost+json 1934 { 1935 "meta": { 1936 "vtags": { 1937 "resource-id": "endpoint-cost-pv.ecs", 1938 "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef" 1939 }, 1940 "cost-type": { 1941 "cost-mode": "array", 1942 "cost-metric": "ane-path" 1944 } 1945 }, 1946 "endpoint-cost-map": { 1947 "ipv4:192.0.2.34": { 1948 "ipv4:192.0.2.2": [ "NET3", "L1", "NET1" ], 1949 "ipv4:192.0.2.50": [ "NET3", "L2", "NET2" ] 1950 }, 1951 "ipv6:2001:db8::3:1": { 1952 "ipv6:2001:db8::4:1": [ "NET3", "L2", "NET2" ] 1953 } 1954 } 1955 } 1956 --example-2 1957 Content-ID: 1958 Content-Type: application/alto-propmap+json 1960 { 1961 "meta": { 1962 "dependent-vtags": [ 1963 { 1964 "resource-id": "endpoint-cost-pv.ecs", 1965 "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef" 1966 }, 1967 { 1968 "resource-id": "ane-props", 1969 "tag": "bf3c8c1819d2421c9a95a9d02af557a3" 1970 } 1971 ] 1972 }, 1973 "property-map": { 1974 ".ane:NET1": { 1975 "max-reservable-bandwidth": 50000000000, 1976 "persistent-entity-id": "ane-props.ane:MEC1" 1977 }, 1978 ".ane:NET2": { 1979 "max-reservable-bandwidth": 50000000000, 1980 "persistent-entity-id": "ane-props.ane:MEC2" 1981 }, 1982 ".ane:NET3": { 1983 "max-reservable-bandwidth": 50000000000 1984 }, 1985 ".ane:L1": { 1986 "max-reservable-bandwidth": 10000000000 1987 }, 1988 ".ane:L2": { 1989 "max-reservable-bandwidth": 15000000000 1990 } 1991 } 1993 } 1995 Under certain scenarios where the traversal order is not crucial, an 1996 ALTO server implementation may choose to not follow strictly the 1997 physical traversal order and may even obfuscate the order 1998 intentionally to preserve its own privacy or conform to its own 1999 policies. For example, an ALTO server may choose to aggregate NET1 2000 and L1 as a new ANE with ANE name "AGGR1", and aggregate NET2 and L2 2001 as a new ANE with ANE name "AGGR2". The "max-reservable-bandwidth" 2002 of "AGGR1" takes the value of L1, which is smaller than that of NET1, 2003 and the "persistent-entity-id" of "AGGR1" takes the value of NET1. 2004 The properties of "AGGR2" are computed in a similar way and the 2005 obfuscated response is as shown below. Note that the obfuscation of 2006 Path Vector responses is implementation-specific and is out of the 2007 scope of this document, and developers may refer to Section 11 for 2008 further references. 2010 HTTP/1.1 200 OK 2011 Content-Length: 1263 2012 Content-Type: multipart/related; boundary=example-2; 2013 type=application/alto-endpointcost+json 2015 --example-2 2016 Content-ID: 2017 Content-Type: application/alto-endpointcost+json 2019 { 2020 "meta": { 2021 "vtags": { 2022 "resource-id": "endpoint-cost-pv.ecs", 2023 "tag": "bb975862fbe3422abf4dae386b132c1d" 2024 }, 2025 "cost-type": { 2026 "cost-mode": "array", 2027 "cost-metric": "ane-path" 2028 } 2029 }, 2030 "endpoint-cost-map": { 2031 "ipv4:192.0.2.34": { 2032 "ipv4:192.0.2.2": [ "NET3", "AGGR1" ], 2033 "ipv4:192.0.2.50": [ "NET3", "AGGR2" ] 2034 }, 2035 "ipv6:2001:db8::3:1": { 2036 "ipv6:2001:db8::4:1": [ "NET3", "AGGR2" ] 2037 } 2038 } 2039 } 2040 --example-2 2041 Content-ID: 2042 Content-Type: application/alto-propmap+json 2044 { 2045 "meta": { 2046 "dependent-vtags": [ 2047 { 2048 "resource-id": "endpoint-cost-pv.ecs", 2049 "tag": "bb975862fbe3422abf4dae386b132c1d" 2050 }, 2051 { 2052 "resource-id": "ane-props", 2053 "tag": "bf3c8c1819d2421c9a95a9d02af557a3" 2054 } 2055 ] 2056 }, 2057 "property-map": { 2058 ".ane:AGGR1": { 2059 "max-reservable-bandwidth": 10000000000, 2060 "persistent-entity-id": "ane-props.ane:MEC1" 2061 }, 2062 ".ane:AGGR2": { 2063 "max-reservable-bandwidth": 15000000000, 2064 "persistent-entity-id": "ane-props.ane:MEC2" 2065 }, 2066 ".ane:NET3": { 2067 "max-reservable-bandwidth": 50000000000 2068 } 2069 } 2070 } 2072 8.5. Incremental Updates 2074 In this example, an ALTO client subscribes to the incremental update 2075 for the multipart Endpoint Cost Service resource "endpoint-cost-pv". 2077 POST /updates/pv HTTP/1.1 2078 Host: alto.example.com 2079 Accept: text/event-stream 2080 Content-Type: application/alto-updatestreamparams+json 2081 Content-Length: 112 2083 { 2084 "add": { 2085 "ecspvsub1": { 2086 "resource-id": "endpoint-cost-pv", 2087 "input": 2088 } 2089 } 2090 } 2092 Based on the server-side process defined in [RFC8895], the ALTO 2093 server will send the "control-uri" first using Server-Sent Event 2094 (SSE), followed by the full response of the multipart message. 2096 HTTP/1.1 200 OK 2097 Connection: keep-alive 2098 Content-Type: text/event-stream 2100 event: application/alto-updatestreamcontrol+json 2101 data: {"control-uri": "https://alto.example.com/updates/streams/123"} 2103 event: multipart/related;boundary=example-3; 2104 type=application/alto-endpointcost+json,ecspvsub1 2105 data: --example-3 2106 data: Content-ID: 2107 data: Content-Type: application/alto-endpointcost+json 2108 data: 2109 data: 2110 data: --example-3 2111 data: Content-ID: 2112 data: Content-Type: application/alto-propmap+json 2113 data: 2114 data: 2115 data: --example-3-- 2117 When the contents change, the ALTO server will publish the updates 2118 for each node in this tree separately, based on Section 6.7.3 of 2119 [RFC8895]. 2121 event: application/merge-patch+json, ecspvsub1.ecsmap@alto.example.com 2122 data: 2124 event: application/merge-patch+json, ecspvsub1.propmap@alto.example.com 2125 data: 2127 8.6. Multi-cost 2129 The following examples demonstrate the request to the "multicost-pv" 2130 resource and the corresponding response. 2132 The request asks for two cost types: the first is the Path Vector 2133 cost type, and the second is a numerical routing cost. It also 2134 queries the Maximum Reservable Bandwidth ANE property and the 2135 Persistent Entity property for two IPv4 source and destination pairs 2136 (192.0.2.34 -> 192.0.2.2 and 192.0.2.34 -> 192.0.2.50) and one IPv6 2137 source and destination pair (2001:db8::3:1 -> 2001:db8::4:1). 2139 The response consists of two parts. The first part returns a 2140 JSONArray that contains two JSONValue for each requested source and 2141 destination pair: the first JSONValue is a JSONArray of ANENames, 2142 which is the value of the Path Vector cost type, and the second 2143 JSONValue is a JSONNumber which is the value of the routing cost. 2144 The second part contains a Property Map that maps the ANEs to their 2145 requested properties. 2147 POST /endpointcost/mcpv HTTP/1.1 2148 Host: alto.example.com 2149 Accept: multipart/related; 2150 type=application/alto-endpointcost+json, 2151 application/alto-error+json 2152 Content-Length: 433 2153 Content-Type: application/alto-endpointcostparams+json 2155 { 2156 "multi-cost-types": [ 2157 { "cost-mode": "array", "cost-metric": "ane-path" }, 2158 { "cost-mode": "numerical", "cost-metric": "routingcost" } 2159 ], 2160 "endpoints": { 2161 "srcs": [ 2162 "ipv4:192.0.2.34", 2163 "ipv6:2001:db8::3:1" 2164 ], 2165 "dsts": [ 2166 "ipv4:192.0.2.2", 2167 "ipv4:192.0.2.50", 2168 "ipv6:2001:db8::4:1" 2169 ] 2170 }, 2171 "ane-property-names": [ 2172 "max-reservable-bandwidth", 2173 "persistent-entity-id" 2174 ] 2175 } 2177 HTTP/1.1 200 OK 2178 Content-Length: 1350 2179 Content-Type: multipart/related; boundary=example-4; 2180 type=application/alto-endpointcost+json 2182 --example-4 2183 Content-ID: 2184 Content-Type: application/alto-endpointcost+json 2186 { 2187 "meta": { 2188 "vtags": { 2189 "resource-id": "endpoint-cost-pv.ecs", 2190 "tag": "84a4f9c14f9341f0983e3e5f43a371c8" 2191 }, 2192 "multi-cost-types": [ 2193 { "cost-mode": "array", "cost-metric": "ane-path" }, 2194 { "cost-mode": "numerical", "cost-metric": "routingcost" } 2196 ] 2197 }, 2198 "endpoint-cost-map": { 2199 "ipv4:192.0.2.34": { 2200 "ipv4:192.0.2.2": [[ "NET3", "AGGR1" ], 3], 2201 "ipv4:192.0.2.50": [[ "NET3", "AGGR2" ], 2] 2202 }, 2203 "ipv6:2001:db8::3:1": { 2204 "ipv6:2001:db8::4:1": [[ "NET3", "AGGR2" ], 2] 2205 } 2206 } 2207 } 2208 --example-4 2209 Content-ID: 2210 Content-Type: application/alto-propmap+json 2212 { 2213 "meta": { 2214 "dependent-vtags": [ 2215 { 2216 "resource-id": "endpoint-cost-pv.ecs", 2217 "tag": "84a4f9c14f9341f0983e3e5f43a371c8" 2218 }, 2219 { 2220 "resource-id": "ane-props", 2221 "tag": "be157afa031443a187b60bb80a86b233" 2222 } 2223 ] 2224 }, 2225 "property-map": { 2226 ".ane:AGGR1": { 2227 "max-reservable-bandwidth": 10000000000, 2228 "persistent-entity-id": "ane-props.ane:MEC1" 2229 }, 2230 ".ane:AGGR2": { 2231 "max-reservable-bandwidth": 15000000000, 2232 "persistent-entity-id": "ane-props.ane:MEC2" 2233 }, 2234 ".ane:NET3": { 2235 "max-reservable-bandwidth": 50000000000 2236 } 2237 } 2238 } 2240 9. Compatibility with Other ALTO Extensions 2241 9.1. Compatibility with Legacy ALTO Clients/Servers 2243 The multipart Filtered Cost Map resource and the multipart Endpoint 2244 Cost Service resource has no backward compatibility issue with legacy 2245 ALTO clients and servers. Although these two types of resources 2246 reuse the media types defined in the base ALTO protocol for the 2247 accept input parameters, they have different media types for 2248 responses. If the ALTO server provides these two types of resources, 2249 but the ALTO client does not support them, the ALTO client will 2250 ignore the resources without incurring any incompatibility problem. 2252 9.2. Compatibility with Multi-Cost Extension 2254 The extension defined in this document is compatible with the multi- 2255 cost extension [RFC8189]. Such a resource has a media type of either 2256 "multipart/related; type=application/alto-costmap+json" or 2257 "multipart/related; type=application/alto-endpointcost+json". Its 2258 "cost-constraints" field must either be "false" or not present and 2259 the Path Vector cost type must be present in the "cost-type-names" 2260 capability field but must not be present in the "testable-cost-type- 2261 names" field, as specified in Section 7.2.4 and Section 7.3.4. 2263 9.3. Compatibility with Incremental Update 2265 This extension is compatible with the incremental update extension 2266 [RFC8895]. ALTO clients and servers MUST follow the specifications 2267 given in Sections 5.2 and 6.7.3 of [RFC8895] to support incremental 2268 updates for a Path Vector resource. 2270 9.4. Compatibility with Cost Calendar 2272 The extension specified in this document is compatible with the Cost 2273 Calendar extension [RFC8896]. When used together with the Cost 2274 Calendar extension, the cost value between a source and a destination 2275 is an array of Path Vectors, where the k-th Path Vector refers to the 2276 abstract network paths traversed in the k-th time interval by traffic 2277 from the source to the destination. 2279 When used with time-varying properties, e.g., maximum reservable 2280 bandwidth, a property of a single ANE may also have different values 2281 in different time intervals. In this case, if such an ANE has 2282 different property values in two time intervals, it MUST be treated 2283 as two different ANEs, i.e., with different entity identifiers. 2284 However, if it has the same property values in two time intervals, it 2285 MAY use the same identifier. 2287 This rule allows the Path Vector extension to represent both changes 2288 of ANEs and changes of the ANEs' properties in a uniform way. The 2289 Path Vector part is calendared in a compatible way, and the Property 2290 Map part is not affected by the calendar extension. 2292 The two extensions combined together can provide the historical 2293 network correlation information for a set of source and destination 2294 pairs. A network broker or client may use this information to derive 2295 other resource requirements such as Time-Block-Maximum Bandwidth, 2296 Bandwidth-Sliding-Window, and Time-Bandwidth-Product (TBP) (See 2297 [SENSE] for details). 2299 10. General Discussions 2301 10.1. Constraint Tests for General Cost Types 2303 The constraint test is a simple approach to query the data. It 2304 allows users to filter the query result by specifying some boolean 2305 tests. This approach is already used in the ALTO protocol. 2306 [RFC7285] and [RFC8189] allow ALTO clients to specify the 2307 "constraints" and "or-constraints" tests to better filter the result. 2309 However, the current syntax can only be used to test scalar cost 2310 types, and cannot easily express constraints on complex cost types, 2311 e.g., the Path Vector cost type defined in this document. 2313 In practice, developing a bespoke language for general-purpose 2314 boolean tests can be a complex undertaking, and it is conceivable 2315 that there are some existing implementations already (the authors 2316 have not done an exhaustive search to determine whether there are 2317 such implementations). One avenue to develop such a language may be 2318 to explore extending current query languages like XQuery [XQuery] or 2319 JSONiq [JSONiq] and integrating these with ALTO. 2321 Filtering the Path Vector results or developing a more sophisticated 2322 filtering mechanism is beyond the scope of this document. 2324 10.2. General Multi-Resource Query 2326 Querying multiple ALTO information resources continuously is a 2327 general requirement. Enabling such a capability, however, must 2328 address general issues like efficiency and consistency. The 2329 incremental update extension [RFC8895] supports submitting multiple 2330 queries in a single request, and allows flexible control over the 2331 queries. However, it does not cover the case introduced in this 2332 document where multiple resources are needed for a single request. 2334 This extension gives an example of using a multipart message to 2335 encode the responses from two specific ALTO information resources: a 2336 Filtered Cost Map or an Endpoint Cost Service, and a Property Map. By 2337 packing multiple resources in a single response, the implication is 2338 that servers may proactively push related information resources to 2339 clients. 2341 Thus, it is worth looking into the direction of extending the SSE 2342 mechanism as used in the incremental update extension [RFC8895], or 2343 upgrading to HTTP/2 [I-D.ietf-httpbis-http2bis] and HTTP/3 2344 [I-D.ietf-quic-http], which provides the ability to multiplex queries 2345 and to allow servers proactively send related information resources. 2347 Defining a general multi-resource query mechanism is out of the scope 2348 of this document. 2350 11. Security Considerations 2352 This document is an extension of the base ALTO protocol, so the 2353 Security Considerations [RFC7285] of the base ALTO protocol fully 2354 apply when this extension is provided by an ALTO server. 2356 The Path Vector extension requires additional scrutiny on three 2357 security considerations discussed in the base protocol: 2358 confidentiality of ALTO information (Section 15.3 of [RFC7285]), 2359 potential undesirable guidance from authenticated ALTO information 2360 (Section 15.2 of [RFC7285]), and availability of ALTO service 2361 (Section 15.5 of [RFC7285]). 2363 For confidentiality of ALTO information, a network operator should be 2364 aware that this extension may introduce a new risk: the Path Vector 2365 information, when used together with sensitive ANE properties such as 2366 capacities of bottleneck links, may make network attacks easier. For 2367 example, as the Path Vector information may reveal more fine-grained 2368 internal network structures than the base protocol, an attacker may 2369 identify the bottleneck link and start a distributed denial-of- 2370 service (DDoS) attack involving minimal flows to conduct the in- 2371 network congestion. Given the potential risk of leaking sensitive 2372 information, the Path Vector extension is mainly applicable in 2373 scenarios where 1) the ANE structures and ANE properties do not 2374 impose security risks to the ALTO service provider, e.g., not 2375 carrying sensitive information, or 2) the ALTO server and client have 2376 established a reliable trust relationship, for example, operated in 2377 the same administrative domain, or managed by business partners with 2378 legal contracts. 2380 Three risk types are identified in Section 15.3.1 of [RFC7285]: (1) 2381 Excess disclosure of the ALTO service provider's data to an 2382 unauthorized ALTO client; (2) Disclosure of the ALTO service 2383 provider's data (e.g., network topology information or endpoint 2384 addresses) to an unauthorized third party; and (3) Excess retrieval 2385 of the ALTO service provider's data by collaborating ALTO clients. 2386 To mitigate these risks, an ALTO server MUST follow the guidelines in 2387 Section 15.3.2 of [RFC7285]. Furthermore, an ALTO server MUST follow 2388 the following additional protections strategies for risk types (1) 2389 and (3). 2391 For risk type (1), an ALTO server MUST use the authentication methods 2392 specified in Section 15.3.2 of [RFC7285] to authenticate the identify 2393 of an ALTO client, and apply access control techniques to restrict 2394 unprivileged ALTO clients from retrieving sensitive Path Vector 2395 information. For settings where the ALTO server and client are not 2396 in the same trust domain, the ALTO server should reach agreements 2397 with the ALTO client on protecting the confidentiality before 2398 granting the access to Path Vector service with sensitive 2399 information. Such agreements may include legal contracts or Digital 2400 Right Management (DRM) techniques. Otherwise, the ALTO server MUST 2401 NOT offer the Path Vector service carrying sensitive information to 2402 the clients unless the potential risks are fully assessed and 2403 mitigated. 2405 For risk type (3), an ALTO service provider must be aware that 2406 persistent ANEs may be used as "landmarks" in collaborative 2407 inferences. Thus, they should only be used when exposing public 2408 service access points (e.g., API gateways, CDNi) and/or when the 2409 granularity is coarse-grained (e.g., when an ANE represents an AS, a 2410 data center or a WAN). Otherwise, an ALTO server MUST use dynamic 2411 mappings from ephemeral ANE names to underlying physical entities. 2412 Specifically, for the same physical entity, an ALTO server SHOULD 2413 assign a different ephemeral ANE name when the entity appears in the 2414 responses to different clients or even for different request from the 2415 same client. A RECOMMENDED assignment strategy is to generate ANE 2416 names from random numbers. 2418 Further, to protect the network topology from graph reconstruction 2419 (e.g., through isomorphic graph identification [BONDY]), the ALTO 2420 server SHOULD consider protection mechanisms to reduce information 2421 exposure or obfuscate the real information. When doing so, the ALTO 2422 server must be aware that information reduction/obfuscation may lead 2423 to potential Undesirable Guidance from Authenticated ALTO Information 2424 risk (Section 15.2 of [RFC7285]). 2426 Thus, implementations of ALTO servers involving reduction or 2427 obfuscation of the Path Vector information SHOULD consider reduction/ 2428 obfuscation mechanisms that can preserve the integrity of ALTO 2429 information, for example, by using minimal feasible region 2430 compression algorithms [NOVA] or obfuscation protocols 2431 [RESA][MERCATOR]. However, these obfuscation methods are 2432 experimental and their practical applicability of these methods to 2433 the generic capability provided by this extension is not fully 2434 assessed. The ALTO server MUST carefully verify that the deployment 2435 scenario satisfies the security assumptions of these methods before 2436 applying them to protect Path Vector services with sensitive network 2437 information. 2439 For availability of ALTO service, an ALTO server should be cognizant 2440 that using Path Vector extension might have a new risk: frequent 2441 requesting for Path Vectors might consume intolerable amounts of the 2442 server-side computation and storage, which can break the ALTO server. 2443 For example, if an ALTO server implementation dynamically computes 2444 the Path Vectors for each request, the service providing Path Vectors 2445 may become an entry point for denial-of-service attacks on the 2446 availability of an ALTO server. 2448 To mitigate this risk, an ALTO server may consider using 2449 optimizations such as precomputation-and-projection mechanisms 2450 [MERCATOR] to reduce the overhead for processing each query. Also, 2451 an ALTO server may also protect itself from malicious clients by 2452 monitoring the behaviors of clients and stopping serving clients with 2453 suspicious behaviors (e.g., sending requests at a high frequency). 2455 The ALTO service providers must be aware that providing incremental 2456 updates of the "max-reservable-bandwidth" may provide information 2457 about other consumers of the network. For example, a change of the 2458 value may indicate one or more reservations has been made or changed. 2459 To mitigate this risk, an ALTO server can batch the updates and/or 2460 add a random delay before publishing the updates. 2462 12. IANA Considerations 2464 12.1. ALTO Cost Metric Registry 2466 This document registers a new entry to the ALTO Cost Metric Registry, 2467 as instructed by Section 14.2 of [RFC7285]. The new entry is as 2468 shown below in Table 1. 2470 +============+====================+=========================+ 2471 | Identifier | Intended Semantics | Security Considerations | 2472 +============+====================+=========================+ 2473 | ane-path | See Section 6.5.1 | See Section 11 | 2474 +------------+--------------------+-------------------------+ 2476 Table 1: ALTO Cost Metric Registry 2478 12.2. ALTO Cost Mode Registry 2480 This document registers a new entry to the ALTO Cost Mode Registry, 2481 as instructed by Section 4 of [I-D.bw-alto-cost-mode]. The new entry 2482 is as shown below in Table 2. 2484 +============+====================+ 2485 | Identifier | Intended Semantics | 2486 +============+====================+ 2487 | array | See Section 6.5.2 | 2488 +------------+--------------------+ 2490 Table 2: ALTO Cost Mode Registry 2492 12.3. ALTO Entity Domain Type Registry 2494 This document registers a new entry to the ALTO Domain Entity Type 2495 Registry, as instructed by Section 12.2 of 2496 [I-D.ietf-alto-unified-props-new]. The new entry is as shown below 2497 in Table 3. 2499 +============+============+=============+===================+=======+ 2500 | Identifier |Entity | Hierarchy & |Media Type of |Mapping| 2501 | |Identifier | Inheritance |Defining Resoucrce |to ALTO| 2502 | |Encoding | | |Address| 2503 | | | | |Type | 2504 +============+============+=============+===================+=======+ 2505 | ane |See Section | None |application/alto- |false | 2506 | |6.2.2 | |propmap+json | | 2507 +------------+------------+-------------+-------------------+-------+ 2509 Table 3: ALTO Entity Domain Type Registry 2511 Identifier: See Section 6.2.1. 2513 Entity Identifier Encoding: See Section 6.2.2. 2515 Hierarchy: None 2517 Inheritance: None 2518 Media Type of Defining Resource: See Section 6.2.4. 2520 Mapping to ALTO Address Type: This entity type does not map to ALTO 2521 address type. 2523 Security Considerations: In some usage scenarios, ANE addresses 2524 carried in ALTO Protocol messages may reveal information about an 2525 ALTO client or an ALTO service provider. Applications and ALTO 2526 service providers using addresses of ANEs will be made aware of 2527 how (or if) the addressing scheme relates to private information 2528 and network proximity, in further iterations of this document. 2530 12.4. ALTO Entity Property Type Registry 2532 Two initial entries "max-reservable-bandwidth" and "persistent- 2533 entity-id" are registered to the ALTO Domain "ane" in the "ALTO 2534 Entity Property Type Registry", as instructed by Section 12.3 of 2535 [I-D.ietf-alto-unified-props-new]. The two new entries are shown 2536 below in Table 4 and their details can be found in Section 12.4.1 and 2537 Section 12.4.2. 2539 +==========================+====================+===================+ 2540 | Identifier | Intended | Media Type of | 2541 | | Semantics | Defining Resource | 2542 +==========================+====================+===================+ 2543 | max-reservable-bandwidth | See Section | application/alto- | 2544 | | 6.4.1 | propmap+json | 2545 +--------------------------+--------------------+-------------------+ 2546 | persistent-entity-id | See Section | application/alto- | 2547 | | 6.4.2 | propmap+json | 2548 +--------------------------+--------------------+-------------------+ 2550 Table 4: Initial Entries for ane Domain in the ALTO Entity 2551 Property Types Registry 2553 12.4.1. New ANE Property Type: Maximum Reservable Bandwidth 2555 Identifier: "max-reservable-bandwidth" 2557 Intended Semantics: See Section 6.4.1. 2559 Media Type of Defining Resource: application/alto-propmap+json 2561 Security Considerations: This property is essential for applications 2562 such as large-scale data transfers or overlay network 2563 interconnection to make better choice of bandwidth reservation. 2564 It may reveal the bandwidth usage of the underlying network and 2565 can potentially be leveraged to reduce the cost of conducting 2566 denial-of-service attacks. Thus, the ALTO server MUST consider 2567 protection mechanisms including only providing the information to 2568 authorized clients, and information reduction and obfuscation as 2569 introduced in Section 11. 2571 12.4.2. New ANE Property Type: Persistent Entity ID 2573 Identifier: "persistent-entity-id" 2575 Intended Semantics: See Section 6.4.2. 2577 Media Type of Defining Resource: application/alto-propmap+json 2579 Security Considerations: This property is useful when an ALTO server 2580 wants to selectively expose certain service points whose detailed 2581 properties can be further queried by applications. The entity IDs 2582 may consider sensitive information about the underlying network, 2583 and an ALTO server should follow the security considerations in 2584 Section 11 of [I-D.ietf-alto-unified-props-new]. 2586 13. References 2588 13.1. Normative References 2590 [I-D.bw-alto-cost-mode] 2591 Boucadair, M. and Q. Wu, "A Cost Mode Registry for the 2592 Application-Layer Traffic Optimization (ALTO) Protocol", 2593 Work in Progress, Internet-Draft, draft-bw-alto-cost-mode- 2594 01, 4 March 2022, . 2597 [I-D.ietf-alto-unified-props-new] 2598 Roome, W., Randriamasy, S., Yang, Y. R., Zhang, J. J., and 2599 K. Gao, "An ALTO Extension: Entity Property Maps", Work in 2600 Progress, Internet-Draft, draft-ietf-alto-unified-props- 2601 new-24, 28 February 2022, 2602 . 2605 [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 2606 Extensions (MIME) Part Two: Media Types", RFC 2046, 2607 DOI 10.17487/RFC2046, November 1996, 2608 . 2610 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2611 Requirement Levels", BCP 14, RFC 2119, 2612 DOI 10.17487/RFC2119, March 1997, 2613 . 2615 [RFC2387] Levinson, E., "The MIME Multipart/Related Content-type", 2616 RFC 2387, DOI 10.17487/RFC2387, August 1998, 2617 . 2619 [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, 2620 DOI 10.17487/RFC5322, October 2008, 2621 . 2623 [RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S., 2624 Previdi, S., Roome, W., Shalunov, S., and R. Woundy, 2625 "Application-Layer Traffic Optimization (ALTO) Protocol", 2626 RFC 7285, DOI 10.17487/RFC7285, September 2014, 2627 . 2629 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2630 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 2631 May 2017, . 2633 [RFC8189] Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost 2634 Application-Layer Traffic Optimization (ALTO)", RFC 8189, 2635 DOI 10.17487/RFC8189, October 2017, 2636 . 2638 [RFC8895] Roome, W. and Y. Yang, "Application-Layer Traffic 2639 Optimization (ALTO) Incremental Updates Using Server-Sent 2640 Events (SSE)", RFC 8895, DOI 10.17487/RFC8895, November 2641 2020, . 2643 [RFC8896] Randriamasy, S., Yang, R., Wu, Q., Deng, L., and N. 2644 Schwan, "Application-Layer Traffic Optimization (ALTO) 2645 Cost Calendar", RFC 8896, DOI 10.17487/RFC8896, November 2646 2020, . 2648 13.2. Informative References 2650 [BONDY] Bondy, J.A. and R.L. Hemminger, "Graph reconstruction—a 2651 survey", Journal of Graph Theory, Volume 1, Issue 3, pp 2652 227-268 , 1977, . 2654 [BOXOPT] Xiang, Q., Yu, H., Aspnes, J., Le, F., Kong, L., and Y.R. 2655 Yang, "Optimizing in the dark: Learning an optimal 2656 solution through a simple request interface", Proceedings 2657 of the AAAI Conference on Artificial Intelligence 33, 2658 1674-1681 , 2019, 2659 . 2661 [CLARINET] Viswanathan, R., Ananthanarayanan, G., and A. Akella, 2662 "CLARINET: WAN-Aware Optimization for Analytics Queries", 2663 In 12th USENIX Symposium on Operating Systems Design and 2664 Implementation (OSDI 16), USENIX Association, Savannah, 2665 GA, 435-450 , 2016, 2666 . 2668 [G2] Ros-Giralt, J., Bohara, A., Yellamraju, S., Langston, 2669 M.H., Lethin, R., Jiang, Y., Tassiulas, L., Li, J., Tan, 2670 Y., and M. Veeraraghavan, "On the Bottleneck Structure of 2671 Congestion-Controlled Networks", Proceedings of the ACM on 2672 Measurement and Analysis of Computing Systems, Volume 3, 2673 Issue 3, pp 1-31 , 2019, 2674 . 2676 [HUG] Chowdhury, M., Liu, Z., Ghodsi, A., and I. Stoica, "HUG: 2677 Multi-Resource Fairness for Correlated and Elastic 2678 Demands", 13th USENIX Symposium on Networked Systems 2679 Design and Implementation (NSDI 16) (Santa Clara, CA, 2680 2016), 407-424. , 2016, 2681 . 2683 [I-D.ietf-alto-performance-metrics] 2684 Wu, Q., Yang, Y. R., Lee, Y., Dhody, D., Randriamasy, S., 2685 and L. M. C. Murillo, "ALTO Performance Cost Metrics", 2686 Work in Progress, Internet-Draft, draft-ietf-alto- 2687 performance-metrics-26, 2 March 2022, 2688 . 2691 [I-D.ietf-httpbis-http2bis] 2692 Thomson, M. and C. Benfield, "HTTP/2", Work in Progress, 2693 Internet-Draft, draft-ietf-httpbis-http2bis-07, 24 January 2694 2022, . 2697 [I-D.ietf-quic-http] 2698 Bishop, M., "Hypertext Transfer Protocol Version 3 2699 (HTTP/3)", Work in Progress, Internet-Draft, draft-ietf- 2700 quic-http-34, 2 February 2021, 2701 . 2704 [JSONiq] "The JSON Query language", 2020, 2705 . 2707 [MERCATOR] Xiang, Q., Zhang, J., Wang, X., Liu, Y., Guok, C., Le, F., 2708 MacAuley, J., Newman, H., and Y.R. Yang, "Toward Fine- 2709 Grained, Privacy-Preserving, Efficient Multi-Domain 2710 Network Resource Discovery", IEEE/ACM IEEE Journal on 2711 Selected Areas of Communication 37(8): 1924-1940, 2019, 2712 . 2714 [MOWIE] Zhang, Y., Li, G., Xiong, C., Lei, Y., Huang, W., Han, Y., 2715 Walid, A., Yang, Y.R., and Z. Zhang, "MoWIE: Toward 2716 Systematic, Adaptive Network Information Exposure as an 2717 Enabling Technique for Cloud-Based Applications over 5G 2718 and Beyond", In Proceedings of the Workshop on Network 2719 Application Integration/CoDesign, ACM, Virtual Event USA, 2720 20-27. , 2020, . 2722 [NOVA] Gao, K., Xiang, Q., Wang, X., Yang, Y.R., and J. Bi, "An 2723 objective-driven on-demand network abstraction for 2724 adaptive applications", IEEE/ACM Transactions on 2725 Networking (TON) Vol 27, no. 2 (2019): 805-818., 2019, 2726 . 2728 [RESA] Xiang, Q., Zhang, J., Wang, X., Liu, Y., Guok, C., Le, F., 2729 MacAuley, J., Newman, H., and Y.R. Yang, "Fine-grained, 2730 multi-domain network resource abstraction as a fundamental 2731 primitive to enable high-performance, collaborative data 2732 sciences", Proceedings of the Super Computing 2018, 2733 5:1-5:13 , 2019, . 2735 [RFC2216] Shenker, S. and J. Wroclawski, "Network Element Service 2736 Specification Template", RFC 2216, DOI 10.17487/RFC2216, 2737 September 1997, . 2739 [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A 2740 Border Gateway Protocol 4 (BGP-4)", RFC 4271, 2741 DOI 10.17487/RFC4271, January 2006, 2742 . 2744 [SENSE] "Software Defined Networking (SDN) for End-to-End 2745 Networked Science at the Exascale", 2019, 2746 . 2748 [SEREDGE] Contreras, L., Baliosian, J., Martı́nez-Julia, P., and J. 2749 Serrat, "Computing at the Edge: But, what Edge?", In 2750 proceedings of the NOMS 2020 - 2020 IEEE/IFIP Network 2751 Operations and Management Symposium. pp. 1-9. , 2020, 2752 . 2754 [SWAN] Hong, C., Kandula, S., Mahajan, R., Zhang, M., Gill, V., 2755 Nanduri, M., and R. Wattenhofer, "Achieving High 2756 Utilization with Software-driven WAN", In Proceedings of 2757 the ACM SIGCOMM 2013 Conference on SIGCOMM (SIGCOMM '13), 2758 ACM, New York, NY, USA, 15-26. , 2013, 2759 . 2761 [UNICORN] Xiang, Q., Chen, S., Gao, K., Newman, H., Taylor, I., 2762 Zhang, J., and Y.R. Yang, "Unicorn: Unified Resource 2763 Orchestration for Multi-Domain, Geo-Distributed Data 2764 Analytics", 2017 IEEE SmartWorld, Ubiquitous Intelligence 2765 Computing, Advanced Trusted Computed, Scalable Computing 2766 Communications, Cloud Big Data Computing, Internet of 2767 People and Smart City Innovation 2768 (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) (Aug. 2017), 2769 1-6. , 2017, 2770 . 2772 [XQuery] "XQuery 3.1: An XML Query Language", 2017, 2773 . 2775 Appendix A. Acknowledgments 2777 The authors would like to thank discussions with Andreas Voellmy, 2778 Erran Li, Haibin Song, Haizhou Du, Jiayuan Hu, Qiao Xiang, Tianyuan 2779 Liu, Xiao Shi, Xin Wang, and Yan Luo. The authors thank Greg 2780 Bernstein, Dawn Chen, Wendy Roome, and Michael Scharf for their 2781 contributions to earlier drafts. 2783 The authors would also like to thank Tim Chown, Luis Contreras, Roman 2784 Danyliw, Benjamin Kaduk, Erik Kline, Suresh Krishnan, Murray 2785 Kucherawy, Warren Kumari, Danny Lachos, Francesca Palombini, Eric 2786 Vyncke, Samuel Weiler, and Qiao Xiang whose feedback and suggestions 2787 are invaluable to improve the practicability and conciseness of this 2788 document, and Mohamed Boucadair, Martin Duke, Vijay Gurbani, Jan 2789 Seedorf, and Qin Wu who provide great support and guidance. 2791 Appendix B. Revision Logs (To be removed before publication) 2793 B.1. Changes since -20 2795 Reivision -21 2797 * changes the normative requirement on protecting confidentiality of 2798 PV information with softer language 2800 B.2. Changes since -19 2802 Revision -20 2804 * changes the IANA registry information 2806 * adopts the comments from IESG reviews 2808 B.3. Changes since -18 2810 Revision -19 2812 * adds detailed examples for use cases 2814 * clarify terms with ambiguous meanings 2816 B.4. Changes since -17 2818 Revision -18 2820 * changes the specification for content-id to conform to [RFC2387] 2821 and [RFC5322] 2823 * adds IPv6 examples 2825 B.5. Changes since -16 2827 Revision -17 2829 * adds items for media type of defining resources in IANA 2830 considerations 2832 B.6. Changes since -15 2834 Revision -16 2836 * resolves the compatibility with the Multi-Cost extension (RFC 2837 8189) 2839 * adds media types of defining resources for ANE property types (for 2840 IANA registration) 2842 B.7. Changes since -14 2844 Revision -15 2846 * fixes the IDNits warnings, 2847 * fixes grammar issues, 2849 * addresses the comments in the AD review. 2851 B.8. Changes since -13 2853 Revision -14 2855 * addresses the comments in the chair review, 2857 * fixes most issues raised by IDNits. 2859 B.9. Changes since -12 2861 Revision -13 2863 * changes the abstract based on the chairs' reviews 2865 * integrates Richard's responds to WGLC reviews 2867 B.10. Changes since -11 2869 Revision -12 2871 * clarifies the definition of ANEs in a similar way as how Network 2872 Elements is defined in [RFC2216] 2874 * restructures several paragraphs that are not clear (Sec 3, Path 2875 Vector bullet, Sec 4.2, Sec 5.1.3, Sec 6.2.4, Sec 6.4.2, Sec 9.3) 2877 * uses "ALTO Entity Domain Type Registry" 2879 B.11. Changes since -10 2881 Revision -11 2883 * replaces "part" with "components" in the abstract; 2885 * identifies additional requirements (AR) derived from the flow 2886 scheduling example, and introduces how the extension addresses the 2887 additional requirements 2889 * fixes the inconsistent use of "start" parameter in multipart 2890 responses; 2892 * specifies explicitly how to handle "cost-constraints"; 2893 * uses the latest IANA registration mechanism defined in 2894 [I-D.ietf-alto-unified-props-new]; 2896 * renames "persistent-entities" to "persistent-entity-id"; 2898 * makes "application/alto-propmap+json" as the media type of 2899 defining resources for the "ane" domain; 2901 * updates the examples; 2903 * adds the discussion on ephemeral and persistent ANEs. 2905 B.12. Changes since -09 2907 Revision -10 2909 * revises the introduction which 2911 - extends the scope where the PV extension can be applied beyond 2912 the "path correlation" information 2914 * brings back the capacity region use case to better illustrate the 2915 problem 2917 * revises the overview to explain and defend the concepts and 2918 decision choices 2920 * fixes inconsistent terms, typos 2922 B.13. Changes since -08 2924 This revision 2926 * fixes a few spelling errors 2928 * emphasizes that abstract network elements can be generated on 2929 demand in both introduction and motivating use cases 2931 B.14. Changes Since Version -06 2933 * We emphasize the importance of the path vector extension in two 2934 aspects: 2936 1. It expands the problem space that can be solved by ALTO, from 2937 preferences of network paths to correlations of network paths. 2939 2. It is motivated by new usage scenarios from both application's 2940 and network's perspectives. 2942 * More use cases are included, in addition to the original capacity 2943 region use case. 2945 * We add more discussions to fully explore the design space of the 2946 path vector extension and justify our design decisions, including 2947 the concept of abstract network element, cost type (reverted to 2948 -05), newer capabilities and the multipart message. 2950 * Fix the incremental update process to be compatible with SSE -16 2951 draft, which uses client-id instead of resource-id to demultiplex 2952 updates. 2954 * Register an additional ANE property (i.e., persistent-entities) to 2955 cover all use cases mentioned in the draft. 2957 Authors' Addresses 2959 Kai Gao 2960 Sichuan University 2961 No.24 South Section 1, Yihuan Road 2962 Chengdu 2963 610000 2964 China 2965 Email: kaigao@scu.edu.cn 2967 Young Lee 2968 Samsung 2969 South Korea 2970 Email: younglee.tx@gmail.com 2972 Sabine Randriamasy 2973 Nokia Bell Labs 2974 Route de Villejust 2975 91460 Nozay 2976 France 2977 Email: sabine.randriamasy@nokia-bell-labs.com 2979 Yang Richard Yang 2980 Yale University 2981 51 Prospect Street 2982 New Haven, CT 2983 United States of America 2984 Email: yry@cs.yale.edu 2985 Jingxuan Jensen Zhang 2986 Tongji University 2987 4800 Caoan Road 2988 Shanghai 2989 201804 2990 China 2991 Email: jingxuan.n.zhang@gmail.com