idnits 2.17.1 draft-browne-sfc-nsh-kpi-stamp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 27, 2016) is 2736 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-10 -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group R. Browne 2 Internet Draft A. Chilikin 3 Intended status: Standards Track Intel 4 Expires May 2017 T. Mizrahi 5 Marvell 7 October 27, 2016 9 Network Service Header KPI Stamping 10 draft-browne-sfc-nsh-kpi-stamp-00.txt 12 Abstract 14 This draft describes a method of inserting Key Performance Indicators 15 (KPIs) into Network Service Header (NSH) encapsulated packets or 16 frames on service chains. This method may be used to monitor latency 17 and QoS configuration to identify problems with virtual links 18 (vlinks), Virtual Network Functions (VNFs) or Physical Network 19 Functions (PNFs) on the Rendered Service Path (RSP). 21 Status of this Memo 23 This Internet-Draft is submitted to IETF in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF), its areas, and its working groups. Note that 28 other groups may also distribute working documents as Internet- 29 Drafts. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 The list of current Internet-Drafts can be accessed at 37 http://www.ietf.org/ietf/1id-abstracts.txt. 39 The list of Internet-Draft Shadow Directories can be accessed at 40 http://www.ietf.org/shadow.html. 42 This Internet-Draft will expire on April 27, 2017. 44 Copyright Notice 46 Copyright (c) 2016 IETF Trust and the persons identified as the 47 document authors. All rights reserved. 49 This document is subject to BCP 78 and the IETF Trust's Legal 50 Provisions Relating to IETF Documents 51 (http://trustee.ietf.org/license-info) in effect on the date of 52 publication of this document. Please review these documents 53 carefully, as they describe your rights and restrictions with respect 54 to this document. Code Components extracted from this document must 55 include Simplified BSD License text as described in Section 4.e of 56 the Trust Legal Provisions and are provided without warranty as 57 described in the Simplified BSD License. 59 Table of Contents 61 1. Introduction...................................................3 62 2. Terminology....................................................3 63 2.1. Requirement Language......................................3 64 2.2. Definition of Terms.......................................4 65 2.3. Abbreviations.............................................5 66 3. NSH KPI Stamping...............................................6 67 3.1. Prerequisites.............................................8 68 3.2. Operation................................................10 69 3.2.1. Flow Selection......................................11 70 3.2.2. SCP Interface.......................................11 71 3.3. Performance Considerations...............................12 72 4. NSH KPIStamping Encapsulation.................................13 73 4.1. KPIstamping Encapsulation (Detection Mode)...............13 74 4.2. NSH Timestamping Encapsulation (Extended Mode)...........16 75 4.3. NSH QoS Stamping Encapsulation (Extended Mode)...........19 76 5. Hybrid Models.................................................22 77 5.1. Targeted VNF Stamp.......................................23 78 6. Fragmentation Considerations..................................23 79 7. Security Considerations.......................................24 80 8. Open Items for WG Discussion..................................24 81 9. IANA Considerations...........................................25 82 10. Contributors.................................................26 83 11. Acknowledgments..............................................26 84 12. References...................................................26 85 12.1. Normative References....................................26 86 12.2. Informative References..................................27 88 1. Introduction 90 Network Service Header (NSH), as defined by [NSH], defines a method 91 to insert a service-aware header in between payload and transport 92 headers. This allows a great deal of flexibility and programmability 93 in the forwarding plane allowing user flows to be programmed on-the- 94 fly for the appropriate Service Functions (SFs). 96 Whilst NSH promises a compelling vista of operational agility for 97 Service Providers, many service providers are concerned about losing 98 service and configuration visibility in the transition from physical 99 appliance SFs to virtualized SFs running in the Network Function 100 Virtualization (NFV) domain. This concern increases when we consider 101 that many service providers wish to run their networks seamlessly in 102 'hybrid' mode, whereby they wish to mix physical and virtual SFs and 103 run services seamlessly between the two domains. 105 This draft describes a generic method to monitor and debug service 106 chains in terms of application latency and QoS configuration of the 107 flows within a service chain. This method is compliant with hybrid 108 architectures in which VNFs and PNFs are freely mixed in the service 109 chain. This method also is flexible to monitor the performance and 110 configuration of an entire chain or part thereof as desired. Please 111 refer to [NSH] as background architecture for the method described in 112 this document. 114 In particular, this draft proposes mechanisms to detect and debug 115 performance issues based on timestamping flows within a chain and to 116 detect and debug QoS configuration on the chain. The method described 117 here is easily extensible to monitoring other KPIs also. 119 The method described in this draft is not an OAM protocol like 120 [Y.1731] or [Y.1564] for example. As such it does not define new OAM 121 packet types or operation. Rather it monitors the service chain 122 performance and configuration for subscriber payloads and indicates 123 subscriber QoE rather than out-of-band infrastructure metrics. 125 2. Terminology 127 2.1. Requirement Language 129 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 130 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 131 document are to be interpreted as described in [RFC2119]. 133 2.2. Definition of Terms 135 Classification: Locally instantiated policy and 136 customer/network/service profile matching of traffic flows for 137 identification of appropriate outbound forwarding actions. 139 First Stamping Node (FSN): Mark packets correctly. Must understand 5 140 tuple information in order to match Stamping Controller flow table. 142 Last Stamping Node (LSN): Reads all MD & export to system performance 143 statistics agent or repository. Should also send NSH header - the 144 Service Index (SI) will indicate if a PNF(s) was at the end of the 145 chain. The LSN changes the SPI in order that the underlay routes the 146 metadata back directly to the KPI database (KPIDB). 148 Network Node/Element: Device that forwards packets or frames based 149 on outer header information. In most cases is not aware of the 150 presence of NSH. 152 Network Overlay: Logical network built on top of existing network 153 (the underlay). Packets are encapsulated or tunneled to create the 154 overlay network topology. 156 Network Service Header: Data plane header added to frames/packets. 157 The header contains information required for service chaining, as 158 well as metadata added and consumed by network nodes and service 159 elements. 161 NSH Proxy: Acts as a gateway: removes and inserts SH on behalf of a 162 service function that is not NSH aware. 164 Service Classifier: Function that performs classification and 165 imposes an NSH. Creates a service path. Non-initial (i.e. 166 subsequent) classification can occur as needed and can alter, or 167 create a new service path. 169 Service Function (SF): A function that is responsible for specific 170 treatment of received packets. A service function can act at the 171 network layer or other OSI layers. A service function can be virtual 172 instance or be embedded in a physical network element. One of 173 multiple service functions can be embedded in the same network 174 element. Multiple instances of the service function can be enabled in 175 the same administrative domain. 177 Service Function Chain (SFC): A service function chain defines an 178 ordered set of service functions that must be applied to packets 179 and/or frames selected as a result of classification. The implied 180 order may not be a linear progression as the architecture allows for 181 nodes that copy to more than one branch. The term service chain is 182 often used as shorthand for service function chain. 184 Service Function Path (SFP): The instantiation of a SFC in the 185 network. Packets follow a service function path from a classifier 186 through the requisite service functions. 188 Stamping Controller SC: The SC may be part of the service chaining 189 application, SDN controller, NFVO or any MANO entity. For clarity we 190 define the SC separately here as the central logic that decides what 191 packets to stamp and how. The SC instructs the classifier on how to 192 build the NSH header. 194 Stamp Control Plane (SCP): the control plane between the FSN and the 195 SC. 197 Key Performance Indicator Database (KPIDB): external storage of 198 Metadata for reporting, trend analysis etc. 200 2.3. Abbreviations 202 DEI Drop Eligible Indicator 204 DSCP Differentiated Services Code Point 206 FSN First Stamping Node 208 KPI Key Performance Indicator 210 KPIDB Key Performance Indicator Database 212 LSN Last Stamping Node 214 MD Metadata 216 NFV Network Function Virtualization 218 NFVI-PoP NFV Infrastructure Point of Presence 220 NIC Network Interface Card 222 NSH Network Service Header 224 OAM Operations, Administration, and Maintenance 225 PCP Priority Code Point 227 PNF Physical Network Function 229 PNFN Physical Network Function Node 231 QoE Quality of Experience 233 QoS Quality of Service 235 QS QoS Stamp 237 RSP Rendered Service Path 239 SC Stamping Controller 241 SCL Service Classifier 243 SCP Stamp Control Plane 245 SI Service Index 247 SF Service Function 249 SFC Service Function Chain 251 SFN Service Function Node 253 SFP Service Function Path 255 SSI Stamp Service Index 257 TC Traffic Class 259 TS Timestamp 261 VLAN Virtual Local Area Network 263 VNF Virtual Network Function 265 vSwitch Virtual Switch 267 3. NSH KPI Stamping 269 A typical KPI stamping architecture is presented in Figure 1. 271 Stamping 272 Controller 273 | KPIDB 274 | SCP Interface | 275 ,---. ,---. ,---. ,---. 276 / \ / \ / \ / \ 277 ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFN ) 278 \ FSN / \ / \ / \ LSN / 279 `---' `---' `---' `---' 280 Figure 1: Logical roles in NSH KPI Stamping 282 The Stamping Controller (SC) will most probably be part of the SFC 283 controller but is explained separately in this document for clarity. 284 The SC is responsible for initiating start/stop stamp requests to the 285 SCL or FSN, and also for distributing NSH stamping policy into the 286 service chain via the Stamping Control Plane (SCP) interface. 288 The First Stamp Node (FSN) will typically be part of the SCL but 289 again is called out as separate logical entity for clarity. The FSN 290 is responsible for marking NSH MD fields for the correct flow with 291 the appropriate NSH fields. This tells all upstream nodes how to 292 behave in terms of stamping at VNF ingress, egress or both, or 293 ignoring the stamp NSH metadata completely. The FSN also writes the 294 Reference Time value, a (possibly inaccurate) estimate of the current 295 time-of-day, into the header, allowing the {chain:flow} performance 296 to be compared to previous samples for offline analysis. The FSN 297 should return an error to the SC if not synchronized to the current 298 time-of-day and forward the packet along the service-chain unchanged. 300 SF1, SF2 stamp the packets as dictated by the FSN and process the 301 payload as per normal. 303 Note 1: The exact location of the stamp creation may not be in 304 the VNF itself, as referenced in Section 3.3. 306 Note 2: Special cases exist where some of the SFs (PNFs or VNFs) are 307 NSH-unaware. This is covered in Section 5. 309 The Last Stamp Node (LSN) should strip the entire header and forward 310 the raw packet to the IP next hop. The LSN also exports NSH stamp 311 information to the KPI Database (KPIDB) for offline analysis; the LSN 312 may either export the stamping information of all packets, or a 313 subset based on packet sampling. In fully virtualized environments 314 the LSN will be co-located with the VNF that decrements the NSH 315 Service Index to zero. Corner cases exist whereby this is not the 316 case and is covered in section 5. 318 3.1. Prerequisites 320 Timestamping presents a set of prerequisites not required to QoS- 321 Stamp. In order to guarantee metadata accuracy, all servers hosting 322 VNFs should be synchronized from a centralized stable clock. As it is 323 assumed that PNFs do not timestamp there is no need for them to 324 synchronize. There are two possible levels of synchronization: 326 Level A: Low accuracy time-of-day synchronization, based on 327 NTP [RFC5905]. 329 Level B: High accuracy synchronization (typically on the order of 330 microseconds), based on [IEEE1588]. 332 Each platform SHOULD have a level A synchronization, and MAY have a 333 level B synchronization. 335 Level A requires each platform (including the Stamp Controller) to 336 synchronize its system real-time-clock to an NTP server. This is used 337 to mark the metadata in the chain, using the field 338 in the NSH KPIstamp header (Section 4.2). This timestamp is written 339 to the NSH header by the first SF in the chain. NTP accuracy can vary 340 by several milliseconds between locations. This is not an issue as 341 the Reference Time is merely being used as a reference inserted into 342 the KPIDB for performance monitoring. 344 Level B synchronization requires each platform to be synchronized to 345 a Primary Reference Clock (PRC) using the Precision Time Protocol 346 [IEEE1588]. A platform MAY also use Synchronous Ethernet ([G.8261], 347 [G.8262], [G.8264]), allowing more accurate frequency 348 synchronization. 350 If a SF is not synchronized at the moment of timestamping, it should 351 indicate synch status in the NSH header. This is described in more 352 detail in section 4. 354 By synchronizing the network in this way, the timestamping operation 355 is independent of the current RSP, whether the entire chain is served 356 by one NFVI-PoP or by multiple. Indeed the timestamp MD can indicate 357 where a chain has been moved due to a resource starvation event as 358 indicated in 0 below, between VNF 3 and VNF 4 at time B. 360 Delay 361 | v 362 | v 363 | x 364 | x x = reference time A 365 | xv v = reference time B 366 | xv 367 | xv 368 |______|______|______|______|______|_____ 369 VNF1 VNF2 VNF3 VNF4 VNF5 371 Figure 2: Flow performance in a service chain 373 For QoS Stamping it is desired that the SCL or FSN be synchronized in 374 order to provide reference time for offline analysis, but this is not 375 a hard requirement (they may be in holdover or free-run state for 376 example). Subsequent upstream platforms do not need to be 377 synchronized for QoS Stamping operation as described below 379 QoS stamping can be used to check consistency of configuration across 380 the entire chain or part thereof. This will allow quick 381 identification of QoS mismatches across multiple L2/L3 fields which 382 otherwise is a manual, expert-led consuming process. 384 | 385 | 386 | xy 387 | xy x = ingress QoS sum 388 | xv v = egress QoS sum 389 | xv y = egress QoS sum miss 390 | xv 391 |______|______|______|______|______|_____ 392 VNF1 VNF2 VNF3 VNF4 VNF5 394 Figure 3: Flow QoS Consistency in a service chain 396 Referring to figure 3 above, x, v and y are notional sum values of 397 the QoS configuration of the flow within a given chain. As the 398 encapsulation of the flow can change from hop to hop in terms of VLAN 399 header(s), MPLS labels, DSCP(s) these values are used to compare 400 consistency of configuration from for example payload DSCP through 401 overlay and underlay QoS settings in VLAN IEEE 802.1Q bits, TC MPLS 402 bits and infrastructure DSCPs. 404 The above figure indicates that at VNF4 in the chain, the egress QoS 405 marking is inconsistent. That is, the ingress QoS settings does not 406 match the egress. The method described here will indicate which QoS 407 field(s) is inconsistent, and whether this is ingress (whereby the 408 underlay has incorrectly marked and queued the packet) or egress 409 (where the VNF has incorrectly marked and queued the packet. 411 3.2. Operation 413 KPIstamping detection mode uses MD type 2. This involves the SFC 414 classifier stamping the flow at chain ingress, and no subsequent 415 stamps being applied, rather each VNF upstream can compare its local 416 condition with the ingress value and take appropriate action. 417 Therefore detection mode is very efficient in terms of header size 418 that does not grow after the classification. This is further 419 explained in section 4.1. 421 Section 3.5 of [NSH] (draft-ietf-sfc-nsh-10) defines NSH metadata 422 type 2 encapsulation as per the figure below In KPIstamped detection 423 and extended mode, flows will use this format. 425 0 1 2 3 426 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 427 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 428 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x2 | Next Protocol | 429 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 430 | Service Path ID | Service Index | 431 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 432 | MD Class |C| Type |R| Len | 433 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 434 | Variable Metadata | 435 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 436 Figure 5: NSH MD type 2 Encapsulation 438 3.2.1. Flow Selection 440 The SC should maintain a list of flows within each service chain to 441 be monitored. This flow table should be in the format SPI:5 tuple ID. 442 The SC should map these pairs to unique Flow IDs per service chain 443 within the extended NSH header specified in this draft. The SC should 444 instruct the FSN to initiate timestamping on flow table match. The SC 445 may also tell the classifier the duration of the timestamping 446 operation, either by a number of packets in the flow or by a time 447 duration. 449 In this way the system can monitor the performance of the all en- 450 route traffic, or an individual subscriber in a chain, or just a 451 specific application or QoS class the subscriber is running. 453 The SC should write the list of monitored flows into the KPIDB for 454 correlation of performance and configuration data. Thus, when the 455 KPIDB receives data from the LSN it understands to which flow the 456 data pertains. 458 The association of source IP to subscriber identity is outside the 459 scope of this draft and will vary by network application. For 460 example, the method of association of a source IP to IMSI in mobile 461 cores will be different to how a CPE with NAT function may be chained 462 in an enterprise NFV application. 464 3.2.2. SCP Interface 466 A new Stamp control plane (SCP) interface is required between the SC 467 and the FSN or classifier. This interface: 469 o Queries the SFC classifier for a list of active chains and 470 flows 472 o Communicates which chains and flows to stamp. This can be a 473 specific {chain:flow} combination or include wildcards for 474 monitoring subscribers across multiple chains or multiple flows 475 within one chain. 477 o How the stamp should be applied (ingress, egress, both or 478 specific). 480 o Typically SCP timestamps flows for a certain duration for trend 481 analysis, but only stamps one packet of each QoS class in a chain 482 periodically (perhaps once per day or after a network change). 483 Therefore timestamping is generally applied to a much larger set 484 of packets than QoS stamping 486 o When to stop stamping, either after a certain number of packets 487 or duration. 489 Exact specification of SCP is for further study. 491 3.3. Performance Considerations 493 This draft does not mandate a specific stamping implementation 494 method, and thus NSH KPI stamping can either be performed by hardware 495 mechanisms, or by software. If software-based stamping is used, 496 applying and operating on the stamps themselves incur an additional 497 small delay in the service chain. However, it can be assumed that 498 these additional delays are all relative for the flow in question. 499 This is only pertinent for timestamping mode, and not for QoS 500 stamping mode. Thus, whist the absolute timestamps may not be fully 501 accurate for normal non-timestamped traffic they can be assumed to be 502 relative. 504 It is assumed that the method described in this document would only 505 operate on a small percentage of user flows. The service provider may 506 choose a flexible policy in the SC to timestamp a selection of user- 507 plane every minute for example to highlight any performance issues. 508 Alternatively, the LSN may selectively export a subset of the 509 KPIstamps it receives, based on a predefined sampling method. Of 510 course the SC can stress test an individual flow or chain should a 511 deeper analysis be required. We can expect that this type of deep 512 analysis has an impact on the performance of the chain itself whilst 513 under investigation. The impact will be dependent on vendor 514 implementation and outside the scope of this document. 516 For QoS stamping the method described here is even less intrusive, as 517 you would not typically need to QoS stamp multiple packets in a flow 518 rather periodically (perhaps once per day) check one packet in a 519 chain per QoS class. 521 The KPIstamp may be applied at various parts of the NFV architecture. 522 The VNF, hypervisor, vSwitch or NIC are all potential locations that 523 can append the packet with the requested KPIstamp. Whilst it is 524 desirable to stamp as close as possible to the VNF for accuracy, the 525 exact location of the stamp application is outside the scope of this 526 document, but should be consistent across the individual SC domain. 528 4. NSH KPIStamping Encapsulation 530 KPI stamping uses NSH MD type 0x02 for detection of anomalies and 531 extended mode for root cause analysis of KPI violations. These are 532 further explained in this section. 534 4.1. KPIstamping Encapsulation (Detection Mode) 536 The generic NSH MD type 2 allocation for KPI Stamping (detection 537 mode) is shown below. This is the format we propose for KPI anomaly 538 detection. 540 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 541 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 542 |Ver|O|C|R|R|R|R|R|R| Length | MD type=0x2 | Next Protocol | 543 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 544 | Service Path Identifier | Service Index | 545 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 546 | MD Class=KPI Monitoring |C| Type=TSD |R| Len | 547 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 548 |C| KPIType | SI | Flow ID | 549 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 550 | Threshold KPI Value | 551 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 552 | | 553 | Ingress KPIStamp | 554 | | 555 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 556 Figure 6: Generic NSH KPI Encapsulation (Detection Mode) 558 Relevant fields in header that the FSN must implement: 560 o The O bit should not be set as we are operating on subscriber 561 packets 563 o The C bit should be set indicating critical metadata exists 565 o The MD type must be set to 0x2 567 o The MD Class must be set to 0x10 (General KPI Monitoring) as 568 requested in Section 9. The stamp type is defined as per below: 570 o Type = 0x00 Reserved. 572 o Type = 0x01 Timestamp Detection 574 o The MSB of the Type field must be set to zero. Thus if a 575 receiver along the path does not understand the KPIstamping 576 protocol it will pass the packet transparently and not drop. This 577 scheme allows for extensibility to the mechanism described in this 578 document to other KPI collections and operations. 580 In the first header the SFC classifier may program a KPI threshold 581 value. This is a value that when exceeded, requires the SF to set the 582 C bit and insert the current SI value into the SI field. The KPI type 583 is the type of KPI stamp inserted into the header as per section 9. 585 The flow ID is inserted into the header by the SFC classifier in 586 order to correlate flow data in the KPIDB for offline analysis. The 587 last two mandatory context headers are reserved for the KPIStamp. 588 This is the KPI value at the chain ingress at the SFC classifier. 590 As an example operation, say we are using KPI type 0x01 (timestamp) 591 when a service function (SFn) receives the packet it can compare 592 current local timestamp (it first checks that it is synchronized to 593 network PRC) with chain ingress timestamp to calculate the latency in 594 the chain. If this value exceeds the timestamp threshold, it then 595 sets the C bit inserts its SI and returns the NSH header to the 596 KPIDB. This effectively tells the system that at SFn the packet 597 violated the KPI threshold. All subsequent upstream SFs perform no 598 NSH KPI operation as the flow has already been marked in violation 599 via the C bit. Please refer to figure 9 for timestamp format. 601 When this occurs the SFC control plane system would then invoke the 602 KPI extended mode, which uses a more sophisticated (and intrusive) 603 method to isolate KPI violation root cause as described below. 605 Note: Whilst detection mode is a valuable tool for latency actions, 606 we feel that it is not justified to build the logic into the KPI 607 system for QoS configuration. As QoS stamping is done infrequently 608 and on a tiny percentage of user plane, it is more practical to use 609 extended mode only for service chain QoS verification. 611 The generic NSH MD type 2 KPI Stamping header extended mode is shown 612 below. This is the format we propose for performance monitoring of 613 service chain issues with respect to QoS configuration and latency. 615 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 616 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 617 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 618 |Ver|O|C|R|R|R|R|R|R| Length | MD type=0x2 | NextProto | 619 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 620 | Service Path ID | Service Index | 621 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 622 | MD Class=KPI Monitoring |C| Type=KPI |R| Len | 623 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 624 |I|E|T|R|R|R|SSI| Service Index | Flow ID | 625 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 626 | Reference Time | 627 | | 628 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 629 |I|E|K|K|K|K|K|K| Reserved | 630 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 631 | KPI Value (LSN) | 632 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 633 \ \ 634 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 635 |I|E|K|K|K|K|K|K| Reserved | 636 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 637 | KPI Value (FSN) | 638 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 639 Figure 7: Generic KPI Encapsulation (Extended Mode) 641 As per section 9, we propose a new MD class 0x10 to indicate KPI MD. 642 Within this class we define 2 types for QoS and timestamp MD to be 643 reported along the service chain. The K bits are KPI specific bits, 644 for example, SYN for timestamping. 646 4.2. NSH Timestamping Encapsulation (Extended Mode) 648 The NSH timestamping encapsulation is shown below. 650 0 1 2 3 651 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 652 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 653 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x2 | NextProto | 654 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 655 | Service Path ID | Service Index | 656 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 657 | MD Class=KPI Monitoring |C| Type=TS |R| Len | 658 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 659 |I|E|T|R|R|R|SSI| Service Index | Flow ID | 660 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 661 | Reference Time (T bit is set) | 662 | | 663 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 664 |I|E|R|R|R| Syn | Service Index | Reserved | 665 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 666 | Ingress Timestamp (I bit is set)(LSN) | 667 | | 668 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 669 | Egress Timestamp (E bit is set)(LSN) | 670 | | 671 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 672 . . 673 . . 674 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 675 |I|E|R|R|R| Syn | Service Index | Reserved | 676 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 677 | Ingress Timestamp (I bit is set) (FSN) | 678 | | 679 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 680 | Egress Timestamp (E bit is set) (FSN) | 681 | | 682 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 683 Figure 8: NSH Timestamp Encapsulation (Extended Mode) 685 Relevant fields in header that the FSN must implement: 687 o The O bit should not be set as we are operating on subscriber 688 packets 690 o The C bit should be set indicating critical metadata exists 692 o The MD type must be set to 0x2 694 o The MD Class must be set to 0x10 (General KPI Monitoring) as 695 requested in Section 9. The stamp type is defined as per below: 697 o Type = 0x00 Reserved. 699 o Type = 0x02 Timestamp Extended 701 o Type = 0x03 QoSStamp Extended 703 o The MSB of the Type field must be set to zero. Thus if a 704 receiver along the path does not understand the KPIstamping 705 protocol it will pass the packet transparently and not drop. This 706 scheme allows for extensibility to the mechanism described in this 707 document to other KPI collections and operations. 709 The FSN KPIstamp metadata starts with Stamping Configuration Header. 710 This header contains the Stamp Service Index (SSI) field which must 711 be set to one of the following values: 713 o 0x0 KPIstamp mode, no Service index specified in the Stamp 714 Service Index field. 716 o 0x1 KPIUstamp Hybrid mode is selected, Stamp Service Index 717 contains LSN Service index. This is used when PNFs or NSH-unaware 718 SFs are used at the tail of the chain. If SSI=0x1, then the value 719 in the type field informs the chain which SF should act as the 720 LSN. 722 o 0x2 KPIstamp Specific mode is selected, Stamp Service Index 723 contains the targeted Service Index. In this case the Stamp 724 Service Index field indicates which SF is to be stamped. Both 725 ingress and egress stamps are performed when the SI=SSI on the 726 chain. For timestamping mode, the FSN will also apply the 727 Reference Time and Ingress Timestamp. This will indicate the delay 728 along the entire service chain to the targeted SF. This method may 729 also be used as a light implementation to monitor end-to-end 730 service chain performance whereby the targeted SF is the LSN. This 731 is not applicable to QoSStamping mode. 733 The Flow ID is a unique 16 bit identifier written into the header by 734 the classifier. This allow 65536 flows to be concurrently stamped on 735 any given NSH service chain (SPI). Flow IDs are not written by 736 subsequent SFs in the chain. The FSN may export monitored flow IDs to 737 the KPIDB for correlation. 739 The E bit should be set if Egress stamp is requested. 741 The I bit should be set if Ingress stamp is requested. 743 The T bit should be set if Reference Time follows Stamping 744 Configuration Header. 746 Reference Time is the wall clock of the FSN, and may be used for 747 historical comparison of SC performance. If the FSN is not Level A 748 synchronized (see Section 3.1) it should inform the SC over the SCP 749 interface. The Reference Time is represented in 64-bit NTP format 750 [RFC5905]. 752 Each stamping Node adds stamping metadata which consist of Stamping 753 Reporting Header and timestamps. 755 The E bit should be set if Egress stamp is reported. 757 The I bit should be set if Ingress stamp is reported. 759 With respect to timestamping mode, the Syn bits are an indication of 760 the synchronization status of the node performing the timestamp and 761 must be set to one of the following values: 763 o In Synch: 0x00 765 o In holdover: 0x01 767 o In free run: 0x02 769 o Out of Synch: 0x03 771 If the network node is out of synch or in free run no timestamp is 772 applied by the node (but other timestamp MD is applied) and the 773 packet is processed normally. 775 If FSN is out of synch or in free run timestamp request rejected and 776 not propagated though the chain. The FSN should inform the SC in such 777 an event over the SCP interface. 779 The outer service index value is copied into the stamp metadata to 780 help cater for hybrid chains that's are a mix of VNFs and PNFs or 781 through SFs that do not understand NSH. Thus if a flow transits 782 through a PNF or an NSH-unaware node the delta in the inner service 783 index between timestamps will indicate this. 785 The Ingress Timestamp and Egress Timestamp are represented in 64-bit 786 NTP format [RFC5905]. The corresponding bits (I and E) reported in 787 the Stamping Reporting Header of the node's metadata. 789 The 64-bit timestamp format [RFC5905] is presented below: 791 0 1 2 3 792 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 793 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 794 | Seconds | 795 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 796 | Fraction | 797 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 798 Figure 9: NTP [RFC5905] 64-bit Timestamp Format 800 4.3. NSH QoS Stamping Encapsulation (Extended Mode) 802 Packets have a variable QoS stack. That is for example the same 803 payload IP can have a very different stack in the access part of the 804 network to the core. This is most apparent in mobile networks where 805 for example in an access circuit we would have 2 layers of 806 infrastructure IP header (DSCP) - one transport-based and the other 807 IPsec-based, in addition to multiple MPLS and VLAN tags. The same 808 packet as it leaves the PGW Gi egress interface may be very much 809 simplified in terms of overhead and related QoS fields. 811 Because of this variability we need to build extra meaning into the 812 QoS headers - they are not for example all PTP timestamps of a fixed 813 length as in the case of timestamping, rather they are variable 814 lengths and types. Also they can be changed on the underlay at any 815 time without knowledge by the SFC system. Therefore each VNF must be 816 able to ascertain and record its ingress and egress QoS configuration 817 on the fly. 819 The suggested QoS type, lengths are as below. The type is 4 bits 820 long. 822 Q Type(QT) Value Length Comment 824 IVLAN 0x01 4 Bits Ingress VLAN (PCP + DEI) 826 EVLAN 0x02 4 Bits Egress VLAN 828 IQINQ 0x03 8 Bits Ingress QinQ (2x PCP+DEI) 830 EQINQ 0x04 8 Bits Egress QinQ 832 IMPLS 0x05 3 Bits Ingress Label 834 EMPLS 0x06 3 Bits Egress Label 836 IMPLS 0x07 6 Bits 2 Ingress Labels (2x EXP) 838 EMPLS 0x08 6 Bits 2 Egress Labels 840 IDSCP 0x09 8 Bits Ingress DSCP 842 EDSCP 0x0A 8 Bits Egress DSCP 844 For stacked headers such as MPLS and 802.1ad, we extract the QoS 845 relevant data from the header and insert into one QoS value in order to 846 be more efficient on packet size. This for MPLS we represent both EXP 847 fields in one QoS value, and both 802.1p priority and drop precedence in 848 one QoS value as indicated above. 850 0 1 2 3 851 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 852 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 853 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x2 | NextProto=0x0 | 854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 855 | Service Path ID | Service Index | 856 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 857 | MD Class= KPI |C| Type= QoS |R| Len | 858 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 859 |R|R|T|R|R|R|SSI| Service Index | Flow ID | 860 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 861 | Reference Time (T bit is set) | 862 | | 863 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 864 |R|R|R|R|R|R|R|R| Service Index | Reserved | 865 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 866 | QT | QoS Value |R|R|R|E| QT | QoS Value |R|R|R|E| 867 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 868 . . 869 . . 870 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 871 |R|R|R|R|R|R|R|R| Service Index | Reserved | 872 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 873 | QT | QoS Value |R|R|R|E| QT | QoS Value |R|R|R|E| 874 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 875 Figure 10: NSH QoS Configuration Encapsulation (Extended Mode) 877 The encapsulation above is very similar to that detailed in section 878 4.1 with the following exceptions 880 - I and E bits are not required as we wish to walk the full QoS 881 stack at ingress and egress at every SF. 883 - Syn status bits are not required 885 - The QT (QoS Type) and QoS value are as outlined in the table above 887 - The E bit at the tail of each QoS context field indicates if this 888 is the last egress QoS stamp for a given SF. This should coincide 889 with SI=0 at the LSN, whereby the packet is truncated and the NSH 890 MD sent to the KPIDB and the subscriber raw IP packet forwarded to 891 the underlay next hop. 893 Note: It is possible to compress the frame structure to better 894 utilize the header, but this would come at the expense of crossing 895 byte boundaries. For ease of implementation, and that QoS stamping is 896 applied on an extremely small subset of user plane traffic, we 897 believe the above structure is a pragmatic compromise between header 898 efficiency and ease of implementation. 900 5. Hybrid Models 902 A hybrid chain may be defined as a chain whereby there is a mix of 903 NSH-aware and NSH-unaware SFs. This may be the case if some PNFs are 904 used in the chain or if VNFs are used that do not support NSH. 906 Example 1. PNF in the middle 908 Stamp 909 Controller 910 | KPIDB 911 | SCP Interface | 912 ,---. ,---. ,---. ,---. 913 / \ / \ / \ / \ 914 ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFN ) 915 \ FSN / \ / \ PNF1/ \ LSN / 916 `---' `---' `---' `---' 917 Figure 11: Hybrid chain with PNF in middle 919 In this example the FSN begins operation and sets the SI to 3, SF1 920 decrements this to 2 and passes the flow to an SFC proxy (not shown). 922 The proxy strips the NSH header and passes to the PNF. On receipt 923 back from the PNF the Proxy decrements the SI and passes the packet 924 onto the LSN with a SI=1. 926 After the LSN processes the traffic it knows it is the last node on 927 the chain from the SI value and exports the entire NSH header and all 928 metadata to the KPIDB. The payload is forwarded to the next hop on 929 the underlay minus the NSH header. The TS information packet may be 930 given a new SPI to act as a homing tag to transport the timestamp 931 data back to the KPIDB. 933 Example 2. PNF at the end 935 Stamp 936 Controller 937 | KPIDB 938 | SCP Interface | 939 ,---. ,---. ,---. ,---. 940 / \ / \ / \ / \ 941 ( SCL )-------->( SF1 )--------->( SF2 )--------->( PNFN ) 942 \ FSN / \ / \ LSN / \ / 943 `---' `---' `---' `---' 944 Figure 12: Hybrid Chain with PNF at end 946 In this example the FSN begins operation and sets the SI to 3, the 947 SSI field set to 0x1, and the type to 1. Thus when SF2 receives the 948 packet with SI=1, it understands that it is expected to take on the 949 role of the LSN as it is the last NSH-aware node in the chain. 951 5.1. Targeted VNF Stamp 953 For the majority of flows within the service chain, stamps (ingress, 954 egress or both) will be carried out at each hop until the SI 955 decrements to zero and the NSH header and Stamp MD is exported to the 956 KPIDB. There may exist however the need to just test a particular VNF 957 (perhaps after a scale out operation, software upgrade or underlay 958 change for example). In this case the FSN should mark the NSH header 959 as follows: 961 SSI field is set to 0x2. Type is set to the expected SI at the SF in 962 question. When outer SI is equal to the SSI, stamps are applied at SF 963 ingress and egress, and the NSH header and MD are exported to the 964 KPIDB. 966 6. Fragmentation Considerations 968 The method described in this draft does not support fragmentation. 969 The SC should return an error should a stamping request from an 970 external system exceed MTU limits and require fragmentation. 972 Depending on the length of the payload and the type of KPIstamp and 973 chain length, this will vary for each packet. 975 In most service provider architectures we would expect a SI << 10, 976 and that may include some PNFs in the chain which do not add 977 overhead. Thus for typical IMIX packet sizes we expect to able to 978 perform timestamping on the vast majority of flows without 979 fragmenting. Thus the classifier can have a simple rule to only allow 980 KPIstamping on packet sizes less than 1200 bytes for example. 982 7. Security Considerations 984 The security considerations of NSH in general are discussed in [NSH]. 986 The use of in-band timestamping, as defined in this document, can be 987 used as a means for network reconnaissance. By passively 988 eavesdropping to timestamped traffic, an attacker can gather 989 information about network delays and performance bottlenecks. 991 The NSH timestamp is intended to be used by various applications to 992 monitor the network performance and to detect anomalies. Thus, a man- 993 in-the-middle attacker can maliciously modify timestamps in order to 994 attack applications that use the timestamp values. For example, an 995 attacker could manipulate the SFC classifier operation, such that it 996 forwards traffic through 'better' behaving chains. Furthermore, if 997 timestamping is performed on a fraction of the traffic, an attacker 998 can selectively induce synthetic delay only to timestamped packets, 999 causing systematic error in the measurements. 1001 Similarly, if an attacker can modify QoS stamps, erroneous values may 1002 be imported into the KPIDB, resulting is further misconfiguration and 1003 subscriber QoE impairment. 1005 An attacker that gains access to the SCP can enable time and QoS 1006 stamping for all subscriber flows, thereby causing performance 1007 bottlenecks, fragmentation, or outages. 1009 As discussed in previous sections, NSH timestamping relies on an 1010 underlying time synchronization protocol. Thus, by attacking the time 1011 protocol an attack can potentially compromise the integrity of the 1012 NSH timestamp. A detailed discussion about the threats against time 1013 protocols and how to mitigate them is presented in [RFC7384]. 1015 8. Open Items for WG Discussion 1017 o Specification and operation of SCP 1018 o AOB 1020 9. IANA Considerations 1022 MD Class Allocation 1024 MD classes are defined in [NSH]. 1026 IANA is requested allocate a new MD class value: 1028 0x10 KPI General Monitoring, stamping types and QoS types. 1030 NSH Stamping MD Types 1032 IANA is requested to set up a registry of "NSH KPIstamping MD 1033 Types". These are 7-bit values. Registry entries are assigned by 1034 using the "IETF Review" policy defined in [RFC5226]. 1036 IANA is requested to allocate two new types as follows: 1038 o Type = 0x00 Reserved. 1040 o Type = 0x01 Timestamp Detection 1042 o Type = 0x02 Timestamp Extended 1044 o Type - 0x03 QoSStamp Extended 1046 QoS Types (QT) 1048 o IVLAN 0x01 1050 o EVLAN 0x02 1052 o IQINQ 0x03 1054 o EQINQ 0x04 1056 o IMPLS 0x05 1058 o EMPLS 0x06 1060 o IMPLS 0x07 1062 o EMPLS 0x08 1063 o IDSCP 0x09 1065 o EDSCP 0x0A 1067 10. Contributors 1069 This document originated as draft-browne-sfc-nsh-timestamp-00 and had 1070 the following co-authors and contributors. We would like to thank and 1071 recognize them and their contributions. 1073 Yoram Moses 1075 Technion 1077 moses@ee.technion.ac.il 1079 Brendan Ryan 1081 Intel Corporation 1083 brendan.ryan@intel.com 1085 11. Acknowledgments 1087 This document was prepared using 2-Word-v2.0.template.dot. 1089 The authors would like to thank Ramki Krishnan and Anoop Ghanwani 1090 from Dell for their reviews and comments on this draft. 1092 12. References 1094 12.1. Normative References 1096 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 1097 Requirement Levels", BCP 14, RFC 2119, March 1997. 1099 [NSH] Quinn, P., Elzur, U., "Network Service Header", draft- 1100 ietf-sfc-nsh-10 (work in progress), Septermber 2016. 1102 12.2. Informative References 1104 [IEEE1588] IEEE TC 9 Instrumentation and Measurement Society, 1105 "1588 IEEE Standard for a Precision Clock 1106 Synchronization Protocol for Networked Measurement and 1107 Control Systems Version 2", IEEE Standard, 2008. 1109 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing 1110 an IANA Considerations Section in RFCs", BCP 26, RFC 1111 5226, May 2008. 1113 [RFC5905] Mills, D., Martin, J., Burbank, J., Kasch, W., 1114 "Network Time Protocol Version 4: Protocol and 1115 Algorithms Specification", RFC 5905, June 2010. 1117 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols 1118 in Packet Switched Networks", RFC 7384, October 2014. 1120 [Y.1731] ITU-T Recommendation G.8013/Y.1731, "OAM Functions and 1121 Mechanisms for Ethernet-based Networks", August 2015. 1123 [Y.1564] ITU-T Recommendation Y.1564, "Ethernet service 1124 activation test methodology", March 2011. 1126 [G.8261] ITU-T Recommendation G.8261/Y.1361, "Timing and 1127 synchronization aspects in packet networks", August 1128 2013. 1130 [G.8262] ITU-T Recommendation G.8262/Y.1362, "Timing 1131 characteristics of a synchronous Ethernet equipment 1132 slave clock", January 2015. 1134 [G.8264] ITU-T Recommendation G.8264/Y.1364, "Distribution of 1135 timing information through packet networks", May 2014. 1137 Authors' Addresses 1139 Rory Browne 1140 Intel 1141 Dromore House 1142 Shannon 1143 Co.Clare 1144 Ireland 1146 Email: rory.browne@intel.com 1147 Andrey Chilikin 1148 Intel 1149 Dromore House 1150 Shannon 1151 Co.Clare 1152 Ireland 1154 Email: andrey.chilikin@intel.com 1156 Tal Mizrahi 1157 Marvell 1158 6 Hamada St. 1159 Yokneam, 20692 Israel 1161 Email: talmi@marvell.com