idnits 2.17.1 draft-c2f-ippm-big-data-alt-mark-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document date (March 9, 2020) is 1509 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 8321 (Obsoleted by RFC 9341) == Outdated reference: A later version (-09) exists of draft-ietf-ippm-multipoint-alt-mark-06 Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IPPM Working Group M. Cociglio 3 Internet-Draft Telecom Italia 4 Intended status: Experimental C. Corbo 5 Expires: September 10, 2020 Politecnico di Torino 6 G. Fioccola 7 Huawei Technologies 8 M. Nilo 9 Telecom Italia 10 R. Sisto 11 Politecnico di Torino 12 March 9, 2020 14 The Big Data Approach for Multipoint Alternate Marking method 15 draft-c2f-ippm-big-data-alt-mark-00 17 Abstract 19 This document introduces a new approach for the Alternate Marking 20 method. It is called Big Data Multipoint Alternate Marking method 21 and, starting from the methodology described in RFC 8321 and draft- 22 ietf-ippm-multipoint-alt-mark, it explains how to implement 23 performance measurement analytics on the Network Management System by 24 analysing the raw data of the network nodes. 26 Requirements Language 28 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 29 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 30 document are to be interpreted as described in RFC 2119 [RFC2119]. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at https://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on September 10, 2020. 49 Copyright Notice 51 Copyright (c) 2020 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents 56 (https://trustee.ietf.org/license-info) in effect on the date of 57 publication of this document. Please review these documents 58 carefully, as they describe your rights and restrictions with respect 59 to this document. Code Components extracted from this document must 60 include Simplified BSD License text as described in Section 4.e of 61 the Trust Legal Provisions and are provided without warranty as 62 described in the Simplified BSD License. 64 Table of Contents 66 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 67 2. Scenario and Background . . . . . . . . . . . . . . . . . . . 3 68 3. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 5 69 3.1. Data collecting . . . . . . . . . . . . . . . . . . . . . 5 70 3.2. Sending data . . . . . . . . . . . . . . . . . . . . . . 7 71 3.3. Preprocessing . . . . . . . . . . . . . . . . . . . . . . 7 72 3.4. Results . . . . . . . . . . . . . . . . . . . . . . . . . 7 73 4. Security Considerations . . . . . . . . . . . . . . . . . . . 9 74 5. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 9 75 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 76 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 77 7.1. Normative References . . . . . . . . . . . . . . . . . . 9 78 7.2. Informative References . . . . . . . . . . . . . . . . . 9 79 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 10 81 1. Introduction 83 This document describes a scenario and a methodology that can be used 84 to get performance details from a monitored network. The approach is 85 inspired by the concepts illustrated in the Alternate Marking Method 86 (RFC 8321 [RFC8321]), Multipoint Alternate Marking Method 87 ([I-D.ietf-ippm-multipoint-alt-mark]), and Hash Sampling (RFC 5474 88 [RFC5474] and RFC 5475 [RFC5475]). 90 In general the performance measurement results are based on a 91 posteriori calculation and the method is called Big Data Multipoint 92 Alternate Marking performance measurement. 94 The kinds of measurements are specified on the Network Management 95 System (NMS) and they can be split into two main categories: per 96 cluster and end-to-end. 98 o The per cluster approach includes all the details that refer to 99 each single cluster and provides a list of parameters that 100 characterize it (packet loss, mean delay). 102 o The end-to-end approach provides more general information about 103 the entire path (packet loss, mean delay). 105 The results can be provided on demand, in a non real-time processing 106 environment, and each one of them refers to a single monitoring 107 period, even if it is possible to broaden the search to more periods. 109 The basic mechanism of the Big data approach here introduced is the 110 Packet sampling. Packet sampling, which is performed through Hashing 111 Sampling technique (RFC 5474 [RFC5474] and RFC 5475 [RFC5475]) 112 applied on all incoming traffic, without any flow distinction. 113 Nevertheless, thanks to data postprocessing, results are split by 114 flow afterwards, since the storage system memorizes the fields of the 115 packet headers that identify flows. The NMS, in fact, requires, as 116 input parameters, the flow identification fields as well as the 117 timestamp. 119 The use of hash sampling improves packet tracking performance and 120 thus overall performance. It allows to track the path followed by 121 each packet without further efforts by the NMS. 123 2. Scenario and Background 125 The service provider's network is made up of a main backbone network 126 surrounded by routers that handle customers traffic input and output. 127 The proposed methodology requires that the traffic is marked before 128 entering the backbone network, by means of the Alternate Marking 129 technique. The marking process can be made by the edge routers or by 130 the customers itself, keeping in mind that it requires that the 131 markers are synchronized. 133 ,..-..,_ 134 .'` '. 135 +--+ ,' `. +--+ 136 ----|R1| / Backbone \ |R2|---- 137 +--+/ Network \+--+ 138 [unmarked | | [unmarked 139 traffic] | [marked | traffic] 140 +--+\ traffic] /+--+ 141 ----|R3| \ / |R4|---- 142 +--+ `, ,' +--+ 143 '. ,'` 144 `''-''`` 146 Figure 1: Backbone Network 148 Only the marked traffic can be monitored. It is possible to mark 149 traffic partially; the results will not be affected by unmarked 150 packets and will refer only to the marked ones. In order to do that 151 a time reference period and a marking method must be fixed at the 152 beginning. The time reference period must consider the misalignment 153 between the marking source routers, clock error between network 154 devices and the interval we need to wait to avoid packets being out 155 of order because of network delay, as described in RFC 8321 [RFC8321] 156 and [I-D.ietf-ippm-multipoint-alt-mark]. 158 A possible marking method could use two bits of the header and set 159 them to 0x01, to identify a period, and to 0x10 to identify the next 160 one. This allows to distinguish between marked traffic and unmarked 161 traffic, instead of using just one bit, which can can generate 162 misunderstanding between the unmarked traffic (that has the marked 163 bit set to 0 by default) and the marked traffic (that alternates 164 between 0x0 and 0x1, with 0x0 as marker value and not as default). 165 As an alternative, it is possible to use just one bit to mark, and 166 use a filter based on IP address to distinguish the flow to be 167 monitored from the others. 169 To enable the Big Data approach for monitoring, the network nodes 170 require a packet collector, that is the agent installed on board of 171 the network node that collects measurements, based on the configured 172 Packet sampling criteria. 174 The portion of network to be monitored must be delimited by routers 175 with packet collector installed on. The rest of the network cannot 176 be monitored even if the traffic is marked. So, the size of the 177 monitored network depends on the network devices placement. However, 178 the size of the network surrounded by packet collectors must be less 179 than or equal to the size of the network with marked traffic. 181 3. Methodology 183 The method described here consists of the following steps: 185 1. Data collecting; 187 2. Sending data; 189 3. Preprocessing; 191 4. Results. 193 ___________ _____________ 194 | Packet | | | 195 |collector 1| -----> | | 196 |___________| |Preprocessing| 197 | | 198 | | ___________ 199 ___________ | | | | 200 | Packet | | (Grouping) | | Clusters | 201 |collector 2| ------> | | <-- |information| 202 |___________| | | | | 203 | | |___________| 204 | Results | 205 ___________ | | 206 | Packet | | | 207 |collector 3| -------> | | 208 |___________| |_____________| 210 Figure 2: Outline of the Methodology 212 3.1. Data collecting 214 The Data collecting phase implies that, on board of the network 215 nodes, the packet collector analyses data passing through a network 216 interface. A packet collector needs to be placed into each router 217 interface we want to monitor. 219 The agent is configured by setting: 221 o the reference hash, 223 o the maximum number of packets to store, 225 o the alternate marking period duration, 227 o the two alternate marking values that identify the marked flow, 228 o the interface to monitor, 230 o the flow to monitor (i.e identified by header IP fields). 232 We can select to monitor either a subset (e.g. filter on a particular 233 IP source or destination address range) or all the flows (filter 234 off). After that, the packet collector will only take care to verify 235 consistency between filter and identification fields and no more 236 information about flow will be carried out. 238 The Data collection is based on Hashing sampling described in RFC 239 5474 [RFC5474] and RFC 5475 [RFC5475]. 241 The process is recursive. Each incoming packet is hashed, compared 242 with the reference hash, and recorded if the number of bits that are 243 matching are the same of those required by the packet collector . 244 When the number of matched packets exceed the maximum number 245 requested in configuration, the number of bits to match is increased 246 by one. At this point all the previous stored packets could be 247 potentially discarded and must be rechecked. So data are stored 248 temporally and are subject to changes, discards and additions. After 249 the period ends, previous data are still subject to change, but after 250 a guard band (reasonably L/2 if L is the period duration) data are 251 stored permanently and ready to be sent. 253 Note that the packet collector (carried out with probe) selects the 254 packets based on the configured parameters, so it works with every 255 incoming packet, without distinction. This greatly increases the 256 probe configuration easiness. Otherwise the probe should save all 257 possible flows (potentially too many), which would be too expensive 258 for the device, and need to be reconfigured if a new flow is 259 available for performance monitoring. On the other hand, this 260 increases the amount of data collected. 262 Stored data include two kind of details: one refers to each single 263 packet and the other one is about aggregate measures. 265 The first set of data includes the fields that identify the flow 266 (IP header fields), packet hash, timestamp when the packets come 267 in, period to which data refer. 269 The second set of data reports network interface identification, 270 total counted packets, total hashed packets, mean timestamp based 271 on all the timestamp of all packets that passed through the 272 interface, period. 274 3.2. Sending data 276 The Sending data phase is separated from the previous one. Once the 277 data has been stored and collected as logs by the network device 278 following the provisions of the theoretical model, the sending system 279 has only the task of carrying data safely and reliably. It is 280 possible to use a synchronous mechanism, in which the sending system 281 periodically checks the availability of new data, or an asynchronous 282 mechanism. In the last case when a new batch of data is ready, an 283 alert wakes up the sending system that carries them to the 284 destination. 286 3.3. Preprocessing 288 The Preprocessing phase has two main goals: 290 o aggregate input data to produce a new record that is ready to be 291 postprocessed and that makes it easier to obtain performance 292 parameters; 294 o decrease the total amount of data to store. 296 Although this step is not mandatory, it is recommended to speed up 297 subsequent operations and to give a better shape to the stored data 298 in order to fit well with the last queries. 300 Preprocessing can be done after data has been stored into the NMS in 301 an iterative loop that parses that periodically or just before to be 302 sent to NMS, through a consolidator, that collects data that comes 303 from all network devices, parses them and then sends them to the NMS. 305 However, in this phase it is possible to group incoming data from all 306 devices and determine the path followed by each sampled packet. In 307 order to do that, if the data are grouped by hash and ordered them by 308 timestamp, it is possible to outline the path. 310 After providing to the NMS the topology information and Clusters 311 partition of the monitored network, it is also possible to track the 312 crossed cluster for each couple of sorted data, by analyzing the 313 interface ID available in the stored record and comparing them with 314 the edge that characterizes the clusters available in the monitored 315 network. 317 3.4. Results 319 The Results phase involves the preprocessed records lay into 320 database. When necessary the storage system can be queried, in a 321 deferred time. The records are organized to fit well with the 322 queries that care about timing and loss aspects. 324 Results are achieved by querying the storage system properly. 325 Certainly, input parameters that identify which flow we are 326 addressing are required. Additionally, time reference is needed to 327 select only the packets of interest. The Big data system is aware of 328 flow identification fields and performs packet flow grouping on the 329 fly. The results described below can refer to different flows, 330 depending on which parameters have been specified for the query. 332 It is possible to deduce the cluster mean delay D_i (mean delay 333 referred to cluster i), by analyzing each record, computing delay d_j 334 (delay referred to record j) as difference between the two available 335 timestamps, that correspond to the input timestamp (when the packet 336 has gone into the cluster) and the output timestamp (when the packet 337 has gone out of that cluster), and summing it with all other delays; 338 then the result is divided by the number of records that refer to the 339 same cluster: 341 D_i = [d_0 + d_1 +...+ d_(N_i-1)] / N_i 343 Where D_i is the mean delay related to cluster i, d_j the delay 344 related to record j, N_i the total number of records belonging to 345 cluster i. 347 It could also be computed the end-to-end mean delay AD as the sum of 348 all delays available in our database, and dividing it by all the 349 records: 351 AD = [ad_0 + ad_1 +...+ ad_(M-1)] / M 353 Where AD is the end-to-end mean delay, ad_j the delay related to 354 records j, and M the total number of records. 356 If necessary, after observing an unusual cluster delay, it could be 357 possible to compute also max/avg/min link delay, by analyzing records 358 again, and exploiting the difference between the two timestamps. 360 Additionally, also details about loss are available. Since the total 361 packets are counted by each node, the sum of the input packets must 362 be equal to the sum of the output packets inside each cluster. If 363 their difference is greater than 0, then a loss has occurred, and the 364 result is the total loss. The total packet loss per cluster: 366 PL_i = [p_(i,0) + p_(i,1) +...+ p_(i,K-1)] - [p_(o,0) + p_(o,1) +...+ 367 p_(o,L-1)] 368 Considering cluster i with K input nodes and L output nodes, the 369 calculation follows [I-D.ietf-ippm-multipoint-alt-mark]. 371 In the same way it is possible to get the entire packet loss, as the 372 sum of all the packet loss per cluster. The same measure can be 373 obtained by using only the hashed packets, but in this case, we get 374 an approximate measurement that might reflect or not the real one. 376 Notice that all these measurements refers to the flow we specify as 377 input of the query and that the specified flow can include or not all 378 the sampled packets (e.g. filter on ip_src=0.0.0.0/0, 379 ip_dst=0.0.0.0/0, port_src=/, port_dst=/, type=tcp, outlines a flow 380 that includes all the TCP packets in an IP network). 382 4. Security Considerations 384 tbc 386 5. Acknowledgements 388 The authors would like to thank Guido Marchetto for the precious 389 contribution. 391 6. IANA Considerations 393 tbc 395 7. References 397 7.1. Normative References 399 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 400 Requirement Levels", BCP 14, RFC 2119, 401 DOI 10.17487/RFC2119, March 1997, 402 . 404 [RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli, 405 L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi, 406 "Alternate-Marking Method for Passive and Hybrid 407 Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321, 408 January 2018, . 410 7.2. Informative References 412 [I-D.ietf-ippm-multipoint-alt-mark] 413 Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto, 414 "Multipoint Alternate Marking method for passive and 415 hybrid performance monitoring", draft-ietf-ippm- 416 multipoint-alt-mark-06 (work in progress), February 2020. 418 [RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., 419 Grossglauser, M., and J. Rexford, "A Framework for Packet 420 Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, 421 March 2009, . 423 [RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. 424 Raspall, "Sampling and Filtering Techniques for IP Packet 425 Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, 426 . 428 Authors' Addresses 430 Mauro Cociglio 431 Telecom Italia 432 Via Reiss Romoli, 274 433 Torino 10148 434 Italy 436 Email: mauro.cociglio@telecomitalia.it 438 Calogero Corbo 439 Politecnico di Torino 441 Email: corbocalo94@gmail.com 443 Giuseppe Fioccola 444 Huawei Technologies 445 Riesstrasse, 25 446 Munich 80992 447 Germany 449 Email: giuseppe.fioccola@huawei.com 450 Massimo Nilo 451 Telecom Italia 452 Via Reiss Romoli, 274 453 Torino 10148 454 Italy 456 Email: massimo.nilo@telecomitalia.it 458 Riccardo Sisto 459 Politecnico di Torino 460 Corso Duca degli Abruzzi, 24 461 Torino 10129 462 Italy 464 Email: riccardo.sisto@polito.it