idnits 2.17.1 draft-browne-sfc-nsh-timestamp-01.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (June 1, 2016) is 2878 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Outdated reference: A later version (-28) exists of draft-ietf-sfc-nsh-05 -- Obsolete informational reference (is this intentional?): RFC 5226 (Obsoleted by RFC 8126) Summary: 0 errors (**), 0 flaws (~~), 2 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group R. Browne 2 Internet Draft A. Chilikin 3 Intended status: Standards Track B. Ryan 4 Expires: December 2016 Intel 5 T. Mizrahi 6 Marvell 7 Y. Moses 8 Technion 9 June 1, 2016 11 Network Service Header Timestamping 12 draft-browne-sfc-nsh-timestamp-01.txt 14 Abstract 16 This draft describes a method of timestamping Network Service Header 17 (NSH) encapsulated packets or frames on service chains in order to 18 measure accurately hop-by-hop performance delays of application flows 19 carried within the chain. This method may be used to monitor 20 performance and highlight problems with virtual links (vlinks), 21 Virtual Network Functions (VNFs) or Physical Network Functions (PNFs) 22 on the Rendered Service Path (RSP). 24 Status of this Memo 26 This Internet-Draft is submitted to IETF in full conformance with the 27 provisions of BCP 78 and BCP 79. 29 Internet-Drafts are working documents of the Internet Engineering 30 Task Force (IETF), its areas, and its working groups. Note that 31 other groups may also distribute working documents as Internet- 32 Drafts. 34 Internet-Drafts are draft documents valid for a maximum of six months 35 and may be updated, replaced, or obsoleted by other documents at any 36 time. It is inappropriate to use Internet-Drafts as reference 37 material or to cite them other than as "work in progress." 39 The list of current Internet-Drafts can be accessed at 40 http://www.ietf.org/ietf/1id-abstracts.txt. 42 The list of Internet-Draft Shadow Directories can be accessed at 43 http://www.ietf.org/shadow.html. 45 This Internet-Draft will expire on December 1, 2016. 47 Copyright Notice 49 Copyright (c) 2016 IETF Trust and the persons identified as the 50 document authors. All rights reserved. 52 This document is subject to BCP 78 and the IETF Trust's Legal 53 Provisions Relating to IETF Documents 54 (http://trustee.ietf.org/license-info) in effect on the date of 55 publication of this document. Please review these documents 56 carefully, as they describe your rights and restrictions with respect 57 to this document. Code Components extracted from this document must 58 include Simplified BSD License text as described in Section 4.e of 59 the Trust Legal Provisions and are provided without warranty as 60 described in the Simplified BSD License. 62 Table of Contents 64 1. Introduction...................................................2 65 2. Terminology....................................................3 66 2.1. Requirement Language......................................3 67 2.2. Definition of Terms.......................................3 68 2.3. Abbreviations.............................................5 69 3. NSH Timestamping...............................................6 70 3.1. Prerequisites.............................................7 71 3.2. Operation.................................................8 72 3.3. Performance Considerations................................9 73 4. NSH Timestamping Encapsulation................................10 74 5. Hybrid Models.................................................14 75 5.1. Targeted VNF Timestamp...................................16 76 6. Fragmentation Considerations..................................16 77 7. Security Considerations.......................................16 78 8. Open Items for WG Discussion..................................17 79 9. IANA Considerations...........................................17 80 10. Acknowledgments..............................................17 81 11. References...................................................18 82 11.1. Normative References....................................18 83 11.2. Informative References..................................18 85 1. Introduction 87 Network Service Header (NSH), as defined by [NSH], defines a method 88 to insert a service-aware header in between payload and transport 89 headers. This allows a great deal of flexibility and programmability 90 in the forwarding plane allowing user flows to be programmed on-the- 91 fly for the appropriate Service Functions (SFs). 93 Whilst NSH promises a compelling vista of operational agility for 94 Service Providers, many service providers are concerned about losing 95 service visibility in the transition from physical appliance SFs to 96 virtualized SFs running in the Network Function Virtualization (NFV) 97 domain. This concern increases when we consider that many service 98 providers wish to run their networks seamlessly in 'hybrid' mode, 99 whereby they wish to mix physical and virtual SFs and run services 100 seamlessly between the two domains. 102 This draft describes a generic method to monitor and debug service 103 chains and application performance of the flows within a service 104 chain. This method is compliant with hybrid architectures in which 105 VNFs and PNFs are freely mixed in the service chain. This method also 106 is flexible to monitor the performance of an entire chain or part 107 thereof as desired. Please refer to [NSH] as background architecture 108 for the method described in this document. 110 The method described in this draft is not an OAM protocol like 111 [Y.1731] or [Y.1564] for example. As such it does not define new OAM 112 packet types or operation. Rather it monitors the service chain 113 performance for subscriber payloads and indicates subscriber QoE 114 rather than out-of-band infrastructure metrics. 116 2. Terminology 118 2.1. Requirement Language 120 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 121 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 122 document are to be interpreted as described in [RFC2119]. 124 2.2. Definition of Terms 126 Classification: Locally instantiated policy and 127 customer/network/service profile matching of traffic flows for 128 identification of appropriate outbound forwarding actions. 130 First TS Node (FTSN): Must mark packet correctly. Must understand 5 131 tuple information in order to match TS Controller flow table. 133 Last TS Node (LTSN): must read all MD & export to system performance 134 statistics agent or repository. Should also send NSH header - the 135 Service Index (SI) will indicate if a PNF(s) was at the end of the 136 chain. The LTSN changes the SPI in order that the underlay routes the 137 metadata back directly to the TSDB. 139 Network Node/Element: Device that forwards packets or frames based 140 on outer header information. In most cases is not aware of the 141 presence of NSH. 143 Network Overlay: Logical network built on top of existing network 144 (the underlay). Packets are encapsulated or tunneled to create the 145 overlay network topology. 147 Network Service Header: Data plane header added to frames/packets. 148 The header contains information required for service chaining, as 149 well as metadata added and consumed by network nodes and service 150 elements. 152 NSH Proxy: Acts as a gateway: removes and inserts SH on behalf of a 153 service function that is not NSH aware. 155 Service Classifier: Function that performs classification and 156 imposes an NSH. Creates a service path. Non-initial (i.e. 157 subsequent) classification can occur as needed and can alter, or 158 create a new service path. 160 Service Function (SF): A function that is responsible for specific 161 treatment of received packets. A service function can act at the 162 network layer or other OSI layers. A service function can be virtual 163 instance or be embedded in a physical network element. One of 164 multiple service functions can be embedded in the same network 165 element. Multiple instances of the service function can be enabled in 166 the same administrative domain. 168 Service Function Chain (SFC): A service function chain defines an 169 ordered set of service functions that must be applied to packets 170 and/or frames selected as a result of classification. The implied 171 order may not be a linear progression as the architecture allows for 172 nodes that copy to more than one branch. The term service chain is 173 often used as shorthand for service function chain. 175 Service Function Path (SFP): The instantiation of a SFC in the 176 network. Packets follow a service function path from a classifier 177 through the requisite service functions. 179 TS Controller: The TS Controller may be part of the service chaining 180 application, SDN controller, NFVO or any MANO entity. For clarity we 181 define the TS Controller separately here as the central logic that 182 decides what packets to timestamp and how. The TS Controller 183 instructs the classifier on how to mark the NSH header. 185 Timestamp Control Plane (TSCP): the control plane between the FTSN 186 and the TS Controller. 188 Timestamp Database (TSDB): external storage of Metadata for 189 reporting, trend analysis etc. 191 2.3. Abbreviations 193 FTSN First Timestamp Node 195 LTSN Last Timestamp Node 197 MD Metadata 199 NFV Network Function Virtualization 201 NFVI-PoP NFV Infrastructure Point of Presence 203 NIC Network Interface Card 205 NSH Network Service Header 207 OAM Operations, Administration, and Maintenance 209 PNF Physical Network Function 211 PNFN Physical Network Function Node 213 QoE Quality of Experience 215 RSP Rendered Service Path 217 SCL Service Classifier 219 SI Service Index 221 SF Service Function 223 SFC Service Function Chain 225 SFN Service Function Node 227 SFP Service Function Path 229 TS Timestamp 231 TSCP Timestamp Control Plane 232 TSDB Timestamp Database 234 TSSI Timestamp Service Index 236 VNF Virtual Network Function 238 vSwitch Virtual Switch 240 3. NSH Timestamping 242 As a generic architecture, please refer to Figure 1 below. 244 TS 245 Controller 246 | TSDB 247 | TSCP Interface | 248 ,---. ,---. ,---. ,---. 249 / \ / \ / \ / \ 250 ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFN ) 251 \ FTSN/ \ / \ / \ LTSN/ 252 `---' `---' `---' `---' 253 Figure 1 Logical roles in NSH Timestamping 255 The TS Controller will most probably be part of the SFC controller 256 but is explained separately in this document for clarity. The TS 257 Controller is responsible for initiating start/stop timestamp 258 requests to the SCL or FTSN, and also for distributing timestamp NSH 259 policy into the service chain via the Timestamping Control Plane 260 (TSCP) interface. 262 The First Timestamp Node (FTSN) will typically be part of the SCL but 263 again is called out as separate logical entity for clarity. The FTSN 264 is responsible for marking NSH MD Type 0x2 fields for the correct 265 flow with the appropriate NSH fields. This tells all upstream nodes 266 how to behave in terms of timestamping at VNF ingress, egress or 267 both, or ignoring the timestamp NSH metadata completely. The FTSN 268 also writes the Reference Time value, a (possibly inaccurate) 269 estimate of the current time-of-day, into the header, allowing the 270 {chain,flow} performance to be compared to previous samples for 271 offline analysis. The FTSN should return an error to the TS 272 Controller if not synchronized to the current time-of-day and forward 273 the packet along the service-chain unchanged. 275 SF1, SF2 timestamp the packets as dictated by the FTSN and process 276 the payload as per normal. 278 Note 1: The exact location of the timestamp creation may not be in 279 the VNF itself, as referenced in Section 3.3. 281 Note 2: Special cases exist where some of the SFs (PNFs or VNFs) are 282 NSH-unaware. This is covered in Section 5. 284 The Last Timestamp Node (LTSN) should strip the entire header and 285 forward the packet to the IP next hop. The LTSN also exports NSH 286 timestamp information to the Timestamp Database (TSDB) for offline 287 analysis; the LTSN may either export the timestamping information of 288 all packets, or a subset based on packet sampling. In fully 289 virtualized environments the LTSN will be co-located with the VNF 290 that decrements the NSH Service Index to zero. Corner cases exist 291 whereby this is not the case and is covered in section 5. 293 3.1. Prerequisites 295 In order to guarantee metadata accuracy, all servers hosting VNFs 296 should be synchronized from a centralized stable clock. As PNFs do 297 not timestamp there is no need for them to synchronize. There are two 298 possible levels of synchronization: 300 Level A: Low accuracy time-of-day synchronization, based on 301 NTP [RFC5905]. 303 Level B: High accuracy synchronization (typically on the order of 304 microseconds), based on [IEEE1588]. 306 Each platform SHOULD have a level A synchronization, and MAY have a 307 level B synchronization. 309 Level A requires each platform (including the TS Controller) to 310 synchronize its system real-time-clock to an NTP server. This is used 311 to mark the metadata in the chain, using the field 312 in the NSH timestamp header (Section 4.). This timestamp is written 313 to the NSH header by the first SF in the chain. NTP accuracy can vary 314 by several milliseconds between locations. This is not an issue as 315 the Reference Time is merely being used as a reference inserted into 316 the TSDB for performance monitoring. 318 Level B synchronization requires each platform to be synchronized to 319 a Primary Reference Clock (PRC) using the Precision Time Protocol 320 [IEEE1588]. A platform MAY also use Synchronous Ethernet ([G.8261], 321 [G.8262], [G.8264]), allowing more accurate frequency 322 synchronization. 324 If a SF is not synchronized at the moment of timestamping, it should 325 indicate synch status in the NSH header. This is described in more 326 detail in section 4. 328 By synchronizing the network in this way, the timestamping operation 329 is independent of the current RSP, whether the entire chain is served 330 by one NFVI-PoP or by multiple. Indeed the timestamp MD can indicate 331 where a chain has been moved due to a resource starvation event as 332 indicated in Figure 2 below, between VNF 3 and VNF 4 at time B. 334 Delay 335 | v 336 | v 337 | x 338 | x x = reference time A 339 | xv v = reference time B 340 | xv 341 | xv 342 |______|______|______|______|______|_____ 343 VNF1 VNF2 VNF3 VNF4 VNF5 344 Figure 2 Flow performance in a service chain 346 3.2. Operation 348 Section 3.5 of [NSH] defines NSH metadata type 2 encapsulation as per 349 the figure below. Please refer to the draft for a detailed 350 explanation. Timestamped flows will use this format. 352 0 1 2 3 353 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 354 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 355 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x2 | Next Protocol | 356 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 357 | Service Path ID | Service Index | 358 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 359 | TLV Class | Type |R|R|R| Len | 360 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 361 | Variable Metadata | 362 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 363 Figure 3 NSH MD type 2 Encapsulation 365 Flow Selection 366 The TS Controller should maintain a list of flows within each service 367 chain to be monitored. This flow table should be in the format SPI:5 368 tuple ID. The TS Controller should map these pairs to unique Flow IDs 369 per service chain within the extended NSH header specified in this 370 draft. The TS Controller should instruct the FTSN to initiate 371 timestamping on flow table match. The TS Controller may also tell the 372 classifier the duration of the timestamping operation, either by a 373 number of packets in the flow or by a time duration. 375 In this way the system can monitor the performance of the all en- 376 route traffic, or an individual subscriber in a chain, or just a 377 specific application the subscriber is running. 379 The TS Controller should write the list of monitored flows into the 380 TSDB for correlation of performance data. Thus, when the TSDB 381 receives data from the LTSN it understands to which flow the data 382 pertains. 384 The association of source IP to subscriber identity is outside the 385 scope of this draft and will vary by network application. For 386 example, the method of association of a source IP to IMSI in mobile 387 cores will be different to how a CPE with NAT function may be chained 388 in an enterprise NFV application. 390 TSCP Interface 392 A new timestamp control plane (TSCP) interface is required between 393 the TS Controller and the FTSN or classifier. This interface: 395 o Communicates which chains and flows to timestamp. This can be a 396 specific {chain,flow} combination or include wildcards for 397 monitoring subscribers across multiple chains or multiple flows 398 within one chain. 400 o How the timestamp should be applied (ingress, egress, both or 401 specific). 403 o When to stop timestamping. 405 Exact specification of TSCP is for further study. 407 3.3. Performance Considerations 409 This draft does not mandate a specific timestamping implementation 410 method, and thus NSH timestamping can either be performed by hardware 411 mechanisms, or by software. If software-based timestamping is used, 412 applying and operating on the timestamps themselves incur an 413 additional small delay in the service chain. However, it can be 414 assumed that these additional delays are all relative for the flow in 415 question. Thus, whist the absolute timestamps may not be fully 416 accurate for normal non-timestamped traffic they can be assumed to be 417 relative. 419 It is assumed that the monitoring method described in this document 420 would only operate on a small percentage of user flows. The service 421 provider may choose a flexible policy in the TS Controller to 422 timestamp a selection of user-plane every minute for example to 423 highlight any performance issues. Alternatively, the LTSN may 424 selectively export a subset of the timestamps it receives, based on a 425 predefined sampling method. Of course the TS Controller can stress 426 test an individual flow or chain should a deeper analysis be 427 required. We can expect that this type of deep analysis has an impact 428 on the performance of the chain itself whilst under investigation. 429 The impact will be dependent on vendor implementation and outside the 430 scope of this document. 432 The timestamp may be applied at various parts of the NFV 433 architecture. The VNF, hypervisor (assuming no SRIOV pass-through), 434 vSwitch or NIC are all potential locations that can append the packet 435 with the requested timestamp. Whilst it is desirable to timestamp as 436 close as possible to the VNF for performance accuracy, the exact 437 location of the timestamp application is outside the scope of this 438 document, but should be consistent across the individual TS 439 Controller domain. 441 4. NSH Timestamping Encapsulation 443 The NSH timestamping encapsulation is shown below in figure 4: 445 0 1 2 3 446 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 447 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 448 |Ver|O|C|R|R|R|R|R|R| Length | MD-type=0x2 | NextProto=0x0 | 449 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 450 | Service Path ID | Service Index | 451 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 452 | TLV Class=0x10 |C| Type=0x01 |R|R|R| Len | 453 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 454 |I|E|T|R|R|R|TSI|TS Service Indx| Flow ID | 455 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 456 | Reference Time (T bit is set) | 457 | | 458 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 459 |I|E|R|R|R| Syn | Service Index | Reserved | 460 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 461 | Ingress Timestamp (I bit is set)(LTSN) | 462 | | 463 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 464 | Egress Timestamp (E bit is set)(LTSN) | 465 | | 466 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 467 . . 468 . . 469 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 470 |I|E|R|R|R| Syn | Service Index | Reserved | 471 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 472 | Ingress Timestamp (I bit is set) | 473 | | 474 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 475 | Egress Timestamp (E bit is set) | 476 | | 477 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 478 |I|E|R|R|R| Syn | Service Index | Reserved | 479 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| 480 | Ingress Timestamp (I bit is set) (FTSN) | 481 | | 482 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 483 | Egress Timestamp (E bit is set) (FTSN) | 484 | | 485 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 486 Figure 4 NSH Timestamp Encapsulation 488 Relevant fields in header that the FTSN must implement: 490 o The O bit should not be set as we are operating on subscriber 491 packets 493 o The C bit should be set indicating critical metadata exists 495 o The MD type must be set to 0x2 497 o The TLV Class must be set to 0x10 (General KPI Monitoring) as 498 requested in Section 9. The timestamp type is defined to be 0x01: 500 o Type = 0x00 Reserved. 502 o Type = 0x01 Timestamp. 504 o The MSB of the Type field must be set to zero. Thus if a receiver 505 along the path does not understand the timestamping protocol it 506 will pass the packet transparently and not drop. This scheme 507 allows for extensibility to the mechanism described in this 508 document to other KPI collections and operations. 510 The FTSN timestamp metadata starts with Stamping Configuration 511 Header. This header contains the Timestamp Service Index (TSI) field 512 which must be set to one of the following values: 514 o 0x0 Timestamp mode, no Service index specified in the TS Service 515 Index field. 517 o 0x1 Timestamp Hybrid mode is selected, Timestamp Service Index 518 contains LTSN Service index. This is used when PNFs or NSH-unaware 519 SFs are used at the tail of the chain. If TSI=0x1, then the value 520 in the type field informs the chain which SF should act as the 521 LTSN. 523 o 0x2 Timestamp Specific mode is selected, Timestamp Service Index 524 contains the targeted Service Index. In this case the Timestamp 525 Service Index field indicates which SF is to be timestamped. Both 526 ingress and egress timestamps are performed when the SI=TSSI on 527 the chain. In this mode the FTSN will also apply the Reference 528 Time and Ingress Timestamp. This will indicate the delay along the 529 entire service chain to the targeted SF. This method may also be 530 used as a light implementation to monitor end-to-end service chain 531 performance whereby the targeted SF is the LTSN. 533 The Flow ID is a unique 16 bit identifier written into the header by 534 the classifier. This allow 65536 flows to be concurrently timestamped 535 on any given NSH service chain (SPI). Flow IDs are not written by 536 subsequent SFs in the chain. The FTSN exports monitored flow IDs to 537 the TSDB for correlation. 539 The E bit should be set if Egress timestamp is requested. 541 The I bit should be set if Ingress timestamp is requested. 543 The T bit should be set if Reference Time follows Stamping 544 Configuration Header. 546 Reference Time is the wall clock of the FTSN, and may be used for 547 historical comparison of SC performance. If the FTSN is not Level A 548 synchronized (see Section 3.1.) it should inform the TS controller 549 over the TSCP interface. The Reference Time is represented in 64-bit 550 NTP format [RFC5905]. 552 Each Timestamping Node adds timestamping metadata which consist of 553 Stamping Reporting Header and timestamps. 555 The E bit should be set if Egress timestamp is reported. 557 The I bit should be set if Ingress timestamp is reported. 559 The Syn bits are an indication of the synchronization status of the 560 node performing the timestamp and must be set to one of the following 561 values: 563 o In Synch: 0x00 565 o In holdover: 0x01 567 o In free run: 0x02 569 o Out of Synch: 0x03 570 If the network node is out of synch or in free run no timestamp is 571 applied by the node (but other timestamp MD is applied) and the 572 packet is processed normally. 574 If FTSN is out of synch or in free run timestamp request rejected and 575 not propagated though the chain. The FTSN should inform the TS 576 controller in such an event over the TSCP interface. 578 The outer service index value is copied into the timestamp metadata 579 to help cater for hybrid chains that's are a mix of VNFs and PNFs or 580 through SFs that do not understand NSH. Thus if a flow transits 581 through a PNF or an NSH-unaware node the delta in the inner service 582 index between timestamps will indicate this. 584 The Ingress Timestamp and Egress Timestamp are represented in 64-bit 585 NTP format [RFC5905]. The corresponding bits (I and E) reported in 586 the Stamping Reporting Header of the node's metadata. 588 The 64-bit timestamp format [RFC5905] is presented below: 590 0 1 2 3 591 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 592 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 593 | Seconds | 594 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 595 | Fraction | 596 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 597 Figure 5 NTP [RFC5905] 64-bit Timestamp Format 599 5. Hybrid Models 601 A hybrid chain may be defined as a chain whereby there is a mix of 602 NSH-aware and NSH-unaware SFs. This may be the case if some PNFs are 603 used in the chain or if VNFs are used that do not support NSH. 605 Example 1. PNF in the middle 607 TS 608 Controller 609 | TSDB 610 | TSCP Interface | 611 ,---. ,---. ,---. ,---. 612 / \ / \ / \ / \ 613 ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFN ) 614 \ FTSN/ \ / \ PNF1/ \ LTSN/ 615 `---' `---' `---' `---' 616 Figure 6 Hybrid chain with PNF in middle 618 In this example the FTSN begins operation and sets the SI to 3, SF1 619 decrements this to 2 and passes the flow to an SFC proxy (not shown). 621 The proxy strips the NSH header and passes to the PNF. On receipt 622 back from the PNF the Proxy decrements the SI and passes the packet 623 onto the LTSN with a SI=1. 625 After the LTSN processes the traffic it knows it is the last node on 626 the chain from the SI value and exports the entire NSH header and all 627 metadata to the TSDB. The payload is forwarded to the next hop on the 628 underlay minus the NSH header. The TS information packet is given a 629 new SPI which acts as a homing tag to transport the timestamp data 630 back to the TSDB. 632 Example 2. PNF at the end 634 TS 635 Controller 636 | TSDB 637 | TSCP Interface | 638 ,---. ,---. ,---. ,---. 639 / \ / \ / \ / \ 640 ( SCL )-------->( SF1 )--------->( SF2 )--------->( PNFN ) 641 \ FTSN/ \ / \ LTSN/ \ / 642 `---' `---' `---' `---' 643 Figure 7 Hybrid Chain with PNF at end 645 In this example the FTSN begins operation and sets the SI to 3, the 646 TSI field set to 0x1, and the type to 1. Thus when SF2 receives the 647 packet with SI=1, it understands that it is expected to take on the 648 role of the LTSN as it is the last NSH-aware node in the chain. 650 5.1. Targeted VNF Timestamp 652 For the majority of flows within the service chain, timestamps 653 (ingress, egress or both) will be carried out at each hop until the 654 SI decrements to zero and the NSH header and TS MD is exported to the 655 TSDB. There may exist however the need to just test a particular VNF 656 (perhaps after a scale out operation or software upgrade for 657 example). In this case the FTSN should mark the NSH header as 658 follows: 660 TSI field is set to 0x2. Type is set to the expected SI at the SF in 661 question. When outer SI is equal to the TSSI, timestamps are applied 662 at SF ingress and egress, and the NSH header and MD are exported to 663 the TSDB. 665 6. Fragmentation Considerations 667 The method described in this draft does not support fragmentation. 668 The TS Controller should return an error should a timestamping 669 request from an external system exceed MTU limits and require 670 fragmentation. 672 Depending on the length of the payload and the type of timestamp and 673 chain length, this will vary for each packet. 675 In most service provider architectures we would expect a SI << 10, 676 and that may include some PNFs in the chain which do not add 677 overhead. Thus for typical IMIX packet sizes we expect to able to 678 perform timestamping on the vast majority of flows without 679 fragmenting. 681 7. Security Considerations 683 The security considerations of NSH in general are discussed in [NSH]. 685 The use of in-band timestamping, as defined in this document, can be 686 used as a means for network reconnaissance. By passively 687 eavesdropping to timestamped traffic, an attacker can gather 688 information about network delays and performance bottlenecks. 690 The NSH timestamp is intended to be used by various applications to 691 monitor the network performance and to detect anomalies. Thus, a man- 692 in-the-middle attacker can maliciously modify timestamps in order to 693 attack applications that use the timestamp values. For example, an 694 attacker could manipulate the SFC classifier operation, such that it 695 forwards traffic through 'better' behaving chains. Furthermore, if 696 timestamping is performed on a fraction of the traffic, an attacker 697 can selectively induce synthetic delay only to timestamped packets, 698 causing systematic error in the measurements. 700 An attacker that gains access to the TSCP can enable timestamping for 701 all subscriber flows, thereby causing performance bottlenecks, 702 fragmentation, or outages. 704 As discussed in previous sections, NSH timestamping relies on an 705 underlying time synchronization protocol. Thus, by attacking the time 706 protocol an attack can potentially compromise the integrity of the 707 NSH timestamp. A detailed discussion about the threats against time 708 protocols and how to mitigate them is presented in [RFC7384]. 710 8. Open Items for WG Discussion 712 o Specification and operation of TSCP 714 o AOB 716 9. IANA Considerations 718 TLV Class Allocation 720 TLV classes are defined in [NSH]. 722 IANA is requested allocate a new TLV class value: 724 0x10 KPI General Monitoring and timestamping type. 726 NSH Timestamping TLV Type 728 IANA is requested to set up a registry of "NSH Timesamping TLV 729 Types". These are 7-bit values. Registry entries are assigned by 730 using the "IETF Review" policy defined in [RFC5226]. 732 IANA is requested to allocate two new types as follows: 734 o Type = 0x00 Reserved. 736 o Type = 0x01 Timestamp. 738 10. Acknowledgments 740 The authors would like to thank Ron Parker of Affirmed Networks and 741 Seungik Lee of ETRI for their reviews of this draft. 743 This document was prepared using 2-Word-v2.0.template.dot. 745 11. References 747 11.1. Normative References 749 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 750 Requirement Levels", BCP 14, RFC 2119, March 1997. 752 [NSH] Quinn, P., Elzur, U., "Network Service Header", draft- 753 ietf-sfc-nsh-05 (work in progress), May 2016. 755 11.2. Informative References 757 [IEEE1588] IEEE TC 9 Instrumentation and Measurement Society, 758 "1588 IEEE Standard for a Precision Clock 759 Synchronization Protocol for Networked Measurement and 760 Control Systems Version 2", IEEE Standard, 2008. 762 [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing 763 an IANA Considerations Section in RFCs", BCP 26, RFC 764 5226, May 2008. 766 [RFC5905] Mills, D., Martin, J., Burbank, J., Kasch, W., 767 "Network Time Protocol Version 4: Protocol and 768 Algorithms Specification", RFC 5905, June 2010. 770 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols 771 in Packet Switched Networks", RFC 7384, October 2014. 773 [Y.1731] ITU-T Recommendation G.8013/Y.1731, "OAM Functions and 774 Mechanisms for Ethernet-based Networks", August 2015. 776 [Y.1564] ITU-T Recommendation Y.1564, "Ethernet service 777 activation test methodology", March 2011. 779 [G.8261] ITU-T Recommendation G.8261/Y.1361, "Timing and 780 synchronization aspects in packet networks", August 781 2013. 783 [G.8262] ITU-T Recommendation G.8262/Y.1362, "Timing 784 characteristics of a synchronous Ethernet equipment 785 slave clock", January 2015. 787 [G.8264] ITU-T Recommendation G.8264/Y.1364, "Distribution of 788 timing information through packet networks", May 2014. 790 Authors' Addresses 792 Rory Browne 793 Intel 794 Dromore House 795 Shannon 796 Co.Clare 797 Ireland 799 Email: rory.browne@intel.com 801 Andrey Chilikin 802 Intel 803 Dromore House 804 Shannon 805 Co.Clare 806 Ireland 808 Email: andrey.chilikin@intel.com 810 Brendan Ryan 811 Intel 812 Dromore House 813 Shannon 814 Co.Clare 815 Ireland 817 Email: brendan.ryan@intel.com 819 Tal Mizrahi 820 Marvell 821 6 Hamada St. 822 Yokneam, 20692 Israel 824 Email: talmi@marvell.com 825 Yoram Moses 826 Department of Electrical Engineering 827 Technion - Israel Institute of Technology 828 Technion City, Haifa, 32000, Israel 830 Email: moses@ee.technion.ac.il