idnits 2.17.1 draft-ietf-pwe3-fc-encap-16.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (May 3, 2011) is 4741 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Downref: Normative reference to an Informational RFC: RFC 3985 ** Obsolete normative reference: RFC 4447 (Obsoleted by RFC 8077) -- Possible downref: Non-RFC (?) normative reference: ref. 'FC-BB-6' -- Possible downref: Non-RFC (?) normative reference: ref. 'FC-FS-2' Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 INTERNET-DRAFT David L. Black (ed.) 2 PWE3 WG EMC Corporation 3 Intended Status: Standard Track Linda Dunbar(ed.) 4 Expires: November 2011 Huawei Technologies 5 Moran Roth 6 Infinera 7 Ronen Solomon 8 Orckit-Corrigent 9 May 3, 2011 11 Encapsulation Methods for Transport of 12 Fibre Channel Traffic over MPLS Networks 14 draft-ietf-pwe3-fc-encap-16.txt 16 Status of this Memo 18 This Internet-Draft is submitted to IETF in full conformance with 19 the provisions of BCP 78 and BCP 79. 21 Internet-Drafts are working documents of the Internet Engineering 22 Task Force (IETF), its areas, and its working groups. Note that 23 other groups may also distribute working documents as Internet- 24 Drafts. 26 Internet-Drafts are draft documents valid for a maximum of six months 27 and may be updated, replaced, or obsoleted by other documents at any 28 time. It is inappropriate to use Internet-Drafts as reference 29 material or to cite them other than as "work in progress." 31 The list of current Internet-Drafts can be accessed at 32 http://www.ietf.org/ietf/1id-abstracts.txt 34 The list of Internet-Draft Shadow Directories can be accessed at 35 http://www.ietf.org/shadow.html 37 This Internet-Draft will expire on November 3, 2011. 39 Copyright Notice 41 Copyright (c) 2011 IETF Trust and the persons identified as the 42 document authors. All rights reserved. 44 This document is subject to BCP 78 and the IETF Trust's Legal 45 Provisions Relating to IETF Documents 46 (http://trustee.ietf.org/license-info) in effect on the date of 47 publication of this document. Please review these documents 48 carefully, as they describe your rights and restrictions with 49 respect to this document. Code Components extracted from this 50 document must include Simplified BSD License text as described in 51 Section 4.e of the Trust Legal Provisions and are provided without 52 warranty as described in the Simplified BSD License. 54 Abstract 56 A Fibre Channel pseudowire (PW) is used to carry Fibre Channel 57 traffic over an MPLS network. This enables service providers to take 58 advantage of MPLS to offer "emulated" Fibre Channel services. This 59 document specifies the encapsulation of Fibre Channel traffic within 60 a pseudowire. It also specifies the common procedures for using a PW 61 to provide a Fibre Channel service. 63 Conventions used in this document 65 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 66 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 67 document are to be interpreted as described in [RFC-2119]. 69 Table of Contents 71 1. Introduction...................................................3 72 1.1. Transparency..............................................3 73 1.2. Bandwidth Efficiency......................................4 74 1.3. Reliability...............................................5 75 2. Reference Model................................................5 76 3. Encapsulation..................................................8 77 3.1. The Control Word.........................................10 78 3.2. MTU Requirements.........................................11 79 3.3. Mapping of FC traffic to PW packets......................11 80 3.3.1. FC Data Frames (PT=0) and FC Login Frames (PT=1)....11 81 3.3.2. FC Primitive Sequences and Primitive Signals (PT=2).12 82 3.3.3. FC PW Control Frames (PT=6).........................14 83 3.4. PW failure mapping.......................................15 84 4. Signaling of FC Pseudowires...................................15 85 5. Timing Considerations.........................................15 86 6. Security Considerations.......................................17 87 7. Applicability Statement.......................................17 88 8. IANA Considerations...........................................18 89 9. Acknowledgments...............................................20 90 10. Normative References.........................................20 91 11. Informative references.......................................21 92 Authors' Addresses...............................................22 94 1. Introduction 96 Fibre Channel (FC) is a high-speed communications technology, used 97 primarily for Storage Area Networks (SANs). Within a single site 98 (e.g., data center), an FC-based SAN connects servers to storage 99 systems, and FC can be extended across sites. When FC is extended 100 across multiple sites, the most common usage is storage replication 101 in support of recovery from disasters (e.g., flood or fire that takes 102 a site out of operation). This is particularly the case over longer 103 distances where network latency results in unacceptable performance 104 for a server whose storage is not at the same site. Fibre Channel is 105 standardized by INCITS Technical Committee T11 [T11] and multiple 106 methods for encapsulating and transporting FC traffic over other 107 networks have been developed [FC-BB-6]. 109 FCIP, as described in [RFC3821] and [FC-BB-6], interconnects 110 otherwise isolated FC SANs over IP Networks. FCIP uses FC Frame 111 Encapsulation [RFC3643] to encapsulate FC frames for tunneling over 112 an IP-based network. Since IP networks may drop or reorder packets, 113 FCIP relies on TCP to retransmit dropped frames and restore the 114 delivery order of reordered frames. Due to possible delay variation 115 and TCP timeouts, special timing mechanisms are required to ensure 116 correct Fibre Channel operation over FCIP [FC-BB-6]. 118 MPLS networks can be provisioned and operated with very low loss 119 rates and very low probability of reordering, making it possible to 120 directly interconnect Fibre Channel ports over MPLS. A Fibre Channel 121 pseudowire (FC PW) is a method to transparently transport FC traffic 122 over an MPLS network resulting in behavior similar to a pair of FC 123 ports that are directly connected by a physical FC link. The result 124 is simpler control processing by comparison to FCIP. 126 This document specifies the encapsulation of FC traffic into an MPLS 127 pseudowire and related PW procedures to transport FC traffic over 128 MPLS PWs. The complete FC pseudowire specification consists of this 129 document and the FC PW portion of the T11 [FC-BB-6] standard. The 130 following subsections describe some of the requirements for 131 transporting FC traffic over an MPLS network. 133 1.1. Transparency 135 Transparent extension of an FC link is a key requirement for 136 transporting FC traffic over a PW. This requires the FC PW to emulate 137 an FC Link between two FC ports, similar to the approach defined for 138 FC over GFPT in [FC-BB-6]. GFPT is an Asynchronous Transparent 139 Generic Framing Procedure specified by ITU-T, see [FC-BB-6] for 140 details and reference to the ITU-T specifications. This results in 141 transparent forwarding of FC traffic over the MPLS network from both 142 the FC Fabric and the network operator points of view. 144 Transparency distinguishes the FC PW approach from FCIP. An FC PW 145 logically connects the FC port on the FC link attached to one end of 146 the PW directly with the FC port on the far end of the FC link 147 attached to the other end of the PW, whereas FCIP introduces FC 148 B_Ports at both ends of the extended FC link; each FC B_Port is 149 connected to an FC E_Port in an FC switch on the same side of the 150 link extension. 152 1.2. Bandwidth Efficiency 154 The bandwidth allocated to a PW may be less than the rate of the 155 attached FC port. When there is no data exchange on a native FC link, 156 Idle Primitive Signals are continuously exchanged between the two FC 157 ports. In order to improve the bandwidth efficiency across the MPLS 158 network, it is necessary for the FC PW PE to suppress (or drop) the 159 Idle Primitive signals generated by its adjacent FC ports. The far 160 end FC PW PE regenerates Idle Primitive signals to send to its 161 adjacent FC port as required, see [FC-BB-6]. 163 FC link control protocols require an FC port to continuously send the 164 same FC Primitive Sequence [FC-FS-2] until a reply is received or 165 some other event occurs. To improve bandwidth efficiency, the FC PW 166 PE encapsulates a subset of repeated FC Primitive Sequences to send 167 across the WAN [FC-BB-6]. For example, in a sequence of identical 168 received primitives, only every fourth primitive may be sent across 169 the MPLS network. Alternatively, a time-based approach may be used to 170 send a copy of the repeated FC Primitive Sequence once every few 171 milliseconds. The far end FC PW PE regenerates the FC link behavior 172 by continuously sending the Primitive Sequence most recently received 173 from the WAN until a new primitive signal, primitive sequence or data 174 frame is received from the WAN. 176 The sending FC PW PE may unilaterally choose any convenient subset 177 for sending the same FC Primitive Sequence. This is acceptable 178 because the receiving FC PW PE generates a continuous stream of the 179 most recently received FC Primitive Sequence on the outgoing native 180 FC link, independent of the arrival rate of that FC Primitive 181 Sequence from the WAN. In practice, a 10:1 reduction in FC Primitive 182 Sequence transmission rate achieves 90% of the bandwidth benefits 183 without loss of FC functionality and sending a copy every few 184 milliseconds does not pose a serious risk of exceeding the timeouts 185 specified in Section 5 below. 187 These bandwidth efficiency techniques may cause changes in the FC 188 traffic that traverses an FC PW (e.g., number of IDLE signals or 189 number of identical Primitive Sequences), but the far end FC PW PE's 190 regeneration of FC link behavior on the attached FC port is 191 transparent to the FC ports connected to each PW PE. 193 1.3. Reliability 195 Fibre Channel does not employ a native frame retransmission protocol, 196 and treats most frame delivery failures as errors. FC SAN traffic 197 requires a very low frame loss rate because the typical result of a 198 failure to deliver a frame is an I/O operation failure. Recovery from 199 such I/O failures involves I/O operation retries after what may be a 200 significant delay (30 second and 60 second timeouts are common). In 201 addition, such retries are likely to be logged as errors indicating 202 possible problems with FC equipment or cables. Hence, drops, errors 203 and discards of FC frames must be very rare for an FC PW. 205 FC SAN implementations have limited tolerance for frame reordering. 206 Any reordering affecting more than a few frames within a single 207 higher level operation (e.g., a read or write I/O) is usually treated 208 as an error by the destination FC port, resulting in discards of the 209 frames involved; some deployed FC implementations treat all such 210 within-operation frame reordering as errors that result in frame 211 discards. As a result, FC frame reordering must be minimized for an 212 FC PW. 214 The FC PW does not compensate for frame drops, discards or 215 reordering. The MPLS network that hosts the FC PW is expected to be 216 designed and operated in a fashion that makes such events very rare. 218 In contrast to the TTL field in an IP packet, FC uses a constant 219 delivery timeout value (R_A_TOV) for which 10 seconds is the default. 220 Each FC frame must be delivered or discarded within that timeout 221 period after it is sent, see Section 5. 223 2. Reference Model 225 An FC PW extends a native FC link over an MPLS network. This document 226 specifies the PW encapsulation for FC. Figure 1 describes the 227 reference models (derived from [RFC3985]) that support the FC PW. FC 228 traffic is received by PE1's FC attachment channel, encapsulated at 229 PE1, transported across MPLS network, decapsulated at PE2, and 230 transmitted onward via the PE2's FC attachment channel. This document 231 assumes that a pseudowire can be provisioned statically or via a 232 signaling protocol as defined in [RFC4447]. 234 |<-------------- Emulated Service ----------------->| 235 | | 236 | |<------- Pseudowire -------->| | 237 | | | | 238 | | |<-- MPLS Tunnel -->| | | 239 | V V V V | 240 V AC +----+ +----+ AC V 241 +-----+ | | PE1|===================| PE2| | +-----+ 242 | |----------|............PW1..............|----------| | 243 | CE1 | | | | | | | | CE2 | 244 | |----------|............PW2..............|----------| | 245 +-----+ ^ | | |===================| | | ^ +-----+ 246 ^ | +----+ +----+ | | ^ 247 | | Provider Edge 1 Provider Edge 2 | | 248 | | | | 249 Customer | | Customer 250 Edge 1 | | Edge 2 251 | | 252 | | 253 Native FC service Native FC service 255 Figure 1: PWE3 FC Interface Reference Configuration 257 The following reference model describes the termination point of each 258 end of the PW within the PE: 260 +-----------------------------------+ 261 | PE | 262 +---+ +-+ +-----+ +------+ +------+ +-+ 263 | | |P| | | |PW ter| | MPLS | |P| 264 | |<==|h|<=| NSP |<=|minati|<=|Tunnel|<=|h|<== From network 265 | | |y| | | |on | | | |y| 266 | C | +-+ +-----+ +------+ +------+ +-+ 267 | E | | | 268 | | +-+ +-----+ +------+ +------+ +-+ 269 | | |P| | | |PW ter| | MPLS | |P| 270 | |==>|h|=>| NSP |=>|minati|=>|Tunnel|=>|h|==> To network 271 | | |y| | | |on | | | |y| 272 +---+ +-+ +-----+ +------+ +------+ +-+ 273 | | 274 +-----------------------------------+ 275 Figure 2: PW reference diagram 277 The Native Service Processing (NSP) function includes the following 278 functionality: 280 o Idle Suppression: any FC Idle signals received from the source 281 PE's attached FC port are suppressed and re-generated at the 282 destination PE to send on its attached FC port when there is no 283 other FC traffic to send; 285 o FC Primitive Sequence Reduction: a subset of repetitive FC 286 Primitive Sequences received from the attached FC port at the 287 source PE is selected for WAN transmission, with the destination 288 PE sending the FC Primitive Sequence most recently received from 289 the WAN on the destination PE's attached FC port continuously 290 until a new packet is received from the WAN; and 292 o Flow Control: the Alternate Simple Flow Control (ASFC) protocol is 293 used for buffer management in concert with the peer PW PE's NSP 294 function so that FC traffic is not dropped. ASFC is a simple 295 pause/resume protocol that allows operation repetition; the 296 receiver responds to the first pause or resume operation in an 297 identical sequence of operations, and ignores the rest of the 298 sequence. 300 The NSP flow control functionality is required to extend FC's credit- 301 based flow control to address situations where the number of buffer 302 credits available to an FC link is insufficient to utilize the 303 available bandwidth over the additional distance and latency 304 represented by the FC pseudowire. The NSPs avoid this problem by 305 inserting ASFC into FC's link flow control used on the attached FC 306 ports, see [FC-BB-6]. 308 In contrast, Idle Suppression and FC Primitive Sequence Reduction are 309 bandwidth optimizations that are included in the NSP for clarity in 310 this document. Analogous optimizations are not treated as part of 311 the NSP by other pseudowires (e.g., ATM idle frame suppression is not 312 considered to be an NSP function by [RFC4717]). 314 The NSP function is specified in detail by [FC-BB-6]. 316 3. Encapsulation 318 This specification provides port to port transport of FC encapsulated 319 traffic. There are a number of port types defined by Fibre Channel, 320 including: 322 o An N_port is a port on the node (e.g. host or storage device) used 323 with both FC-P2P (Point to Point) or FC-SW (Switched fabric) 324 topologies. Also known as a Node port. 326 o An NL_port is a port on the node used with an FC-AL (Arbitrated 327 Loop) topology. Also known as a Node Loop port. 329 o An F_port is a port on the switch that connects to a node point- 330 to-point (i.e. connects to an N_port). Also known as a Fabric 331 port. An F_port is not loop capable. 333 o An FL_port is a port on the switch that connects to a FC-AL loop 334 (i.e. to NL_ports). Also known as Fabric Loop port. 336 o An E_port is a port used to connect two Fibre Channel switches. 337 Also known as an Expansion port. When E_ports between two switches 338 are connected to form a link, that link is referred to as an 339 inter-switch link (ISL). 341 Among the port types listed above, only the following FC connections 342 (as specified in [FC-BB-6]) are supported by an FC PW over MPLS: 344 - N_Port to N_Port, established by an FC PLOGI (Port Login) 345 operation 347 - N_Port to F_Port, established by an FC FLOGI (Fabric Login) 348 operation 350 - E_Port to E_Port, established by an FC ELP (Exchange Link 351 Parameters) operation 353 FC traffic flowing over an FC PW is subdivided into four payload 354 types (PT) that are encoded in the PW Control Word (see Section 3.1): 356 1. FC login traffic (PT = 1): FC login operations and responses that 357 establish connections between FC ports. The three FC login 358 operations are PLOGI, FLOGI, and ELP. These operations and their 359 responses may require the NSP to allocate buffer resources, see 360 the specification of Login Exchange Monitors in [FC-BB-6]. 362 2. FC data traffic (PT = 0): All FC frames other than those involved 363 in an FC login operation. 365 3. FC Primitive Sequences and Signals (PT = 2): Native FC link 366 control operations - 4-character primitive sequences and signals 367 that are not encapsulated in FC frames. See [FC-BB-6] and 368 [FC-FS-2]. 370 4. FC PW Control (PT = 6): FC PW control operations exchanged only 371 between the endpoints of the PW. FC PW control operations are used 372 for ASFC flow control, ping (e.g., for round trip latency 373 measurement) and reporting native FC link errors, see [FC-BB-6]. 375 This FC PW specification is limited to use with FC service classes 2, 376 3 and F (see [FC-FS-2]). Other FC service classes (e.g., 1, 4 and 6) 377 MUST NOT be used with an FC PW. Numbered FC service classes are used 378 for end-to-end FC traffic, whereas service class F is used for inter- 379 switch traffic in an FC switched fabric. 381 This FC PW specification is limited to native FC attachment links 382 that employ an 8b/10b transmission code (see [FC-FS-2]). The 383 protocol specified in this document converts a received 10b code to 384 its 8b counterpart for PW encapsulation, and hence does not support 385 attached FC links that use a 64b/66b transmission code (e.g., 10GFC, 386 16GFC); such links MUST NOT be attached to an FC PW PE unless their 387 link speed can be negotiated to one that uses 8b/10b encoding. If an 388 invalid 10b code that cannot be converted to an 8b code is received 389 from an FC link, the PE sends an FC PW control frame to report the 390 error, see [FC-BB-6]. 392 3.1. The Control Word 394 The Generic PW Control Word, as defined in "PWE3 Control Word" 395 [RFC4385] MUST be used for FC PW to facilitate the transport of short 396 packets (by setting the Length field as detailed below), and convey 397 the flag bits defined below. The structure of the Control Word for 398 the FC PW is as follows: 400 1 2 3 401 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 402 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 403 |0 0 0 0| PT |X|0 0| Length | Sequence Number | 404 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 406 Figure 3 - Control Word Structure 408 The first four bits of the PW Control Word MUST be set to 0 by the 409 ingress PE to indicate PW data. 411 Three of the four Flags bits are used to convey the PT - Payload Type 412 indication. The 3-bit binary value in this field identifies the 413 payload type carried by a PW packet. The following types are defined: 415 PT = 0: FC data frame. 417 PT = 1: FC login frame. 419 PT = 2: FC Primitive Sequence(s) and/or Primitive Signal(s). 421 PT = 6: FC PW Control Frame (refer to [FC-BB-6] for usage). 423 Packets with other values in the PT field are not valid for the FC PW 424 and MUST be discarded by the receiving FC PW PE. 426 X - This flag bit is not used by this version of the protocol. It 427 SHOULD be set to zero by the sender and MUST be ignored by the 428 receiver. 430 The fragmentation bits (bits 8-9) are not used by the FC PW protocol. 431 These bits may be used in the future for FC specific indications as 432 defined in [RFC4385]. The fragmentation bits SHOULD be set to zero by 433 the ingress PE and MUST be ignored by the egress PE. 435 The Length field enables recovery of the original pseudowire packet 436 when a short packet is padded to the minimum 64 octet packet size 437 required for Ethernet, see [RFC4385]. The Length field MUST be used 438 for packets shorter than 64 octets, MUST be set to zero for longer 439 packets, and MUST be processed according to the rules specified in 440 [RFC4385]. 442 The sequence number is not used for the FC PW and MUST be set to 0 by 443 the ingress PE, and MUST be ignored by the egress PE. 445 3.2. MTU Requirements 447 The MPLS network MUST be able to transport the largest Fibre Channel 448 frame after encapsulation, including the overhead associated with the 449 encapsulation. The maximum FC frame size is 2164 octets without PW 450 and MPLS labels (refer to Figure 4); this maximum size is a constant 451 value that is required for all FC implementations [FC-FS-2]. The MPLS 452 network SHOULD accommodate frames of up to 2500 octets in order to 453 support possible future increases in the maximum FC frame size. 455 Fragmentation, as described in [RFC4623], SHALL NOT be used for an FC 456 PW, therefore the network MUST be configured with a minimum MTU that 457 is sufficient to transport the largest encapsulated FC frame. 459 3.3. Mapping of FC traffic to PW packets 461 FC frames, Primitive Sequences, and Primitive Signals are transported 462 over the PW. All packet types are carried over a single PW. In 463 addition to the PW Control Word, an FC Encapsulation Header is 464 included in the PW packet. This FC Encapsulation Header is not used 465 in this version of the protocol; it SHOULD be set to zero by the 466 sender and MUST be ignored by the receiver. 468 3.3.1. FC Data Frames (PT=0) and FC Login Frames (PT=1) 470 FC data frames and FC login frames share a common encapsulation 471 format, except that the PT field in the FC PW control word is set to 472 0 for data frames and is set to 1 for login frames. An FC login frame 473 contains an FC PLOGI, FLOGI or ELP operation or response that 474 requires special processing by the NSP in support of flow control, 475 see [FC-BB-6]. 477 Each FC data frame or login frame is mapped to a PW packet, including 478 the Start Of Frame (SOF) delimiter, frame header, CRC field and the 479 End Of Frame (EOF) delimiter, as shown in figure 4. 481 1 2 3 482 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 483 +---------------------------------------------------------------+ 484 | FC PW Control Word | 485 +---------------------------------------------------------------+ 486 | FC Encapsulation Header | 487 +---------------+-----------------------------------------------+ 488 | SOF Code | Reserved | 489 +---------------+-----------------------------------------------+ 490 | | 491 +----- FC Frame ----+ 492 | | 493 +---------------------------------------------------------------+ 494 | CRC | 495 +---------------+-----------------------------------------------+ 496 | EOF Code | Reserved | 497 +---------------+-----------------------------------------------+ 499 Figure 4 - FC frame (SOF/Data/CRC/EOF) encapsulation in PW packet 501 The SOF and EOF frame delimiters are each encoded into a single octet 502 as specified in [RFC3643], except that the codes for delimiters that 503 apply only to FC service class 4 (SOFi4, SOFc4, SOFn4, EOFdt, EOFdti, 504 EOFrt, EOFrti - see [FC-FS-2]) MUST NOT be used. 506 The CRC in the frame is obtained directly from the FC attachment 507 channel, so that the PW PE is not required to re-calculate the CRC or 508 to check the CRC in the received frame. The CRC will be checked by 509 the FC port that receives the frame, ensuring that coverage is 510 provided for data errors that occur between the PW endpoints. This 511 CRC behavior differs from the FCS retention technique for PWs defined 512 in [RFC4720] which states that "as usual, the FCS MUST be examined at 513 the ingress PE, and errored frames MUST be discarded." 515 3.3.2. FC Primitive Sequences and Primitive Signals (PT=2) 517 FC Primitive Sequences and Primitive Signals are FC ordered sets. On 518 an 8b/10b-coded FC link, an ordered set consists of four 10b 519 characters, starting with the K28.5 character, followed by three 520 Dxx.y data characters. All FC ordered sets start with a K28.5 control 521 character, but the three following Dxx.y data characters differ 522 depending on the ordered set. A Kxx.y control character has a 523 different 10b code from the corresponding Dxx.y data character, but 524 uses the same 8b code (e.g., K28.5 and D28.5 both use the 8b code 525 0xBC). Here are two examples of ordered sets: 527 o Idle(IDLE) is K28.5 - D21.4 - D21.5 - D21.5. This FC primitive 528 signal is sent when the FC link is idle; it is suppressed by the 529 FC PW NSP and not sent over the WAN. 531 o Link Reset Response(LRR) is K28.5 - D21.1 - D31.5 - D9.2 (this FC 532 primitive sequence is used as part of FC link initialization and 533 recovery). 535 Each ordered set is encapsulated in a PW packet containing the 536 encoded K28.5 control character [FC-BB-6], followed by three encoded 537 data characters, as shown in Figure 5. 539 1 2 3 540 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 541 +---------------------------------------------------------------+ 542 | FC PW Control Word | 543 +---------------------------------------------------------------+ 544 | FC Encapsulation Header | 545 +---------------+---------------+---------------+---------------+ 546 | K28.5 | Dxx.y | Dxx.y | Dxx.y | 547 +---------------+---------------+---------------+---------------+ 548 | | 549 +---- ----+ 550 | | 551 +---------------+---------------+---------------+---------------+ 552 | K28.5 | Dxx.y | Dxx.y | Dxx.y | 553 +---------------+---------------+---------------+---------------+ 555 Figure 5 - FC Ordered Sets encapsulation in PW packet 557 The K28.5 10b control character received from the PE's attached FC 558 link is encoded for the FC PW as its 8b counterpart (0xBC). Because 559 the same 8b value (0xBC) is used to encode a D28.5 data word, the 560 receiving FC PW PE: 562 o MUST check for presence of an 8b K28.5 value (0xBC) at the start 563 of each ordered set (see Figure 5), and MUST send that value as a 564 10b K28.5 character on the attached FC link. 566 o MUST send the following three Dxx.y 8b values as Dxx.y 10b 567 characters on the attached FC link and MUST NOT send any of these 568 Dxx.y 8b values as 10b Kxx.y characters on the attached FC link. 570 A PW packet may contain one or more encoded FC Ordered sets [FC-BB- 571 6]. The Length field in the FC PW Control Word is used to indicate 572 the packet length when the PW packet contains multiple Ordered Sets. 574 For this reason, FC PW packets that contain FC Ordered Sets MUST NOT 575 be larger than 60 octets (8 octets of header words plus at most 13 576 ordered sets), in order to ensure that the Length field contains a 577 non-zero value, see [RFC4385]. 579 Idle Primitive Signals could be carried over the PW in the same 580 manner as Primitive Sequences. However, [FC-BB-6] requires that Idle 581 Primitive Signals be dropped by the Ingress PE and re-generated by 582 the egress PE in order to reduce bandwidth consumption (see [FC-BB-6] 583 for further details). 585 The egress PE extracts the Primitive Sequence or Primitive Signal 586 from the received PW packet. For a Primitive Sequence, the PE 587 continues transmitting the same FC Ordered Set to its attached FC 588 port until an FC frame or another ordered set is received over the 589 PW; see Section 1.2 above for discussion of ingress PE transmission 590 behavior for Primitive Sequences. A Primitive Signal is sent once, 591 except that Idle Primitive Signals are sent continuously when there 592 is nothing else to send. 594 3.3.3. FC PW Control Frames (PT=6) 596 FC PW Control Frames are transported over the PW, by encapsulating 597 each frame in a PW packet with PT=6 in the Control Word. FC PW 598 Control Frame payloads are generated and terminated by the 599 corresponding FC entity. FC PW Control frames are used for FC PW flow 600 control (ASFC), ping and transmission of error indications. [FC-BB-6] 601 specifies the generation and processing of FC PW Control Frames. FC 602 PW Control Frames are always shorter than 64 octets, and hence the 603 Length field in the FC Control Word indicates their length. 605 1 2 3 606 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 607 +---------------------------------------------------------------+ 608 | FC PW Control Word | 609 +---------------------------------------------------------------+ 610 | FC Encapsulation Header | 611 +---------------------------------------------------------------+ 612 | | 613 +----- FC PW Control Frame ----+ 614 | | 615 +---------------------------------------------------------------+ 617 Figure 6 - FC PW Control frame encapsulation in PW packet 619 3.4. PW failure mapping 621 PW failures are detected through PW signaling failure, PW status 622 notifications as defined in [RFC4447], or through PW OAM mechanisms 623 and MUST be mapped to emulated signal failure indications. Sending 624 the FC link failure indication to its attached FC link is performed 625 by the NSP, as defined by [FC-BB-6]. 627 4. Signaling of FC Pseudowires 629 RFC4447 specifies the use of the MPLS Label Distribution Protocol, 630 LDP, as a protocol for setting up and maintaining pseudowires. This 631 section describes the use of specific fields and error codes used to 632 control FC PW. 634 The PW Type field in the PWid FEC element and PW generalized ID FEC 635 elements MUST be set to the "FC Port Mode" value in section 8 below. 637 The Control Word is REQUIRED for FC pseudowires. Therefore the C-Bit 638 in the PWid FEC element and PW generalized ID FEC elements MUST be 639 set. If the C-Bit is not set, the pseudowire MUST NOT be established 640 and a Label Release MUST be sent with an "Illegal C-Bit" status code 641 [RFC4447]. 643 The Fragmentation Indicator (Parameter ID = 0x09) is specified in 644 [RFC4446] and its usage is defined in [RFC4623]. Since fragmentation 645 is not used in FC PW, the fragmentation indicator parameter MUST be 646 omitted from the Interface Parameter Sub-TLV. 648 The Interface MTU Parameter (Parameter ID = 0x01) is specified in 649 [RFC4447]. Since all FC interfaces have the same MTU, this parameter 650 MUST be omitted from the Interface Parameter Sub-TLV. 652 The FCS Retention Indicator (Parameter ID = 0x0A) is specified in 653 [RFC4720]. Since the CRC treatment defined in this document differs 654 from one that is specified in [RFC4720], this parameter MUST be 655 omitted from the Interface Parameter Sub-TLV. 657 5. Timing Considerations 659 Correct Fibre Channel link operation requires that the FC link 660 latency between CE1 and CE2 (refer to Figure 1) be: 662 o no more than one-half of the R_T_TOV (Receiver Transmitter Timeout 663 Value, default value: 100 milliseconds) of the attached devices 664 for Primitive Sequences; 666 o no more than one-half of the E_D_TOV (Error Detect Timeout Value, 667 default value: 2 seconds) of the attached devices for frames; and 669 o within the R_A_TOV (Resource Allocation Timeout Value, default 670 value: 10 seconds) of the attached fabric(s), if any. The FC 671 standards require that the E_D_TOV value for each FC link be set 672 so that the R_A_TOV value for the fabric is respected when the 673 worst case latency occurs for each link, see [FC-FS-2]. 675 An FC PW MUST adhere to these three timing requirements and MUST NOT 676 be used in environments where high or variable latency may cause 677 these requirements to be violated. 679 These three timeout values are ordered (R_T_TOV < E_D_TOV < R_A_TOV), 680 so adherence to one-half of R_T_TOV for all FC PW traffic is 681 sufficient. See [FC-FS-2] for definitions of the FC timeout values. 683 The R_T_TOV is used by the FC link initialization protocol. If an FC 684 PW's latency exceeds one-half R_T_TOV, initialization of the FC link 685 that is encapsulated by the FC PW may fail, leaving that FC link in a 686 non-operational state. 688 The E_D_TOV is used to detect failures of operational FC links. If an 689 FC PW's latency exceeds the one-half E_D_TOV requirement, the FC link 690 that is encapsulated by the FC PW may fail. The usual FC response to 691 such a link failure is to attempt to recover the FC link by 692 initializing it. That initialization will also fail if the FC PW 693 latency exceeds one-half R_T_TOV (a tighter requirement). 695 The R_A_TOV is used to determine when FC communication resources 696 (e.g., values that identify FC frames) may be reused. If an FC PW's 697 violation of the one-half E_D_TOV requirement is sufficient to also 698 cause the FC fabric to violate the R_A_TOV requirement, then FC reuse 699 of frame identification values after an R_A_TOV timeout may result in 700 multiple FC frames with the same identification values, causing 701 incorrect Fibre Channel operation. For example, if two such frames 702 are swapped between I/O operations, the result may corrupt data in 703 the I/O operations. 705 The PING and PING_ACK FC PW control frames defined in Section 6.4.7 706 of [FC-BB-6] SHOULD be used to measure the current FC pseudowire 707 latency between the CE devices. If the measured latency violates any 708 of the timing requirements, then the FC PW PE MUST generate a WAN 709 Down event as specified in [FC-BB-6]. 711 The WAN Down event causes the PE to continuously send NOS (an FC 712 primitive sequence) on the native FC link to the FC Port at the other 713 end of that link (typically an E_Port on a switch in this case). 714 This immediately causes the FC link that is carried by the PW to 715 become non-operational, halting transmission of FC traffic. However, 716 it is not necessary to tear down the pseudowire itself in this 717 situation (e.g., destroy the MPLS path set up by LDP). 719 The Transparent FC-BB initialization state machine in [FC-BB-6] 720 specifies the protocol used to attempt to recover from a WAN Down 721 event (i.e., bring the WAN back up). If that protocol brings the WAN 722 back up, FC traffic will resume and the standard FC link recovery 723 protocol will bring the encapsulated FC link back up. If the previous 724 pseudowire was destroyed, attempts will be made to re-establish the 725 path via LDP as part of recovering from the WAN Down event. If the PW 726 round-trip latency remains above R_T_TOV, the initialization protocol 727 for the FC PW will repeatedly time out in attempting to recover from 728 the WAN Down event, preventing recovery of the FC link carried by the 729 PW, see [FC-BB-6]. 731 6. Security Considerations 733 The FC PW is an MPLS pseudowire; for MPLS pseudowire security 734 considerations, see the security considerations sections of [RFC3985] 735 and [RFC4385]. 737 The protocols used to implement security in a Fibre Channel fabric 738 are defined in [FC-SP]. These protocols operate at higher layers of 739 the FC hierarchy and are transparent to the FC PW. 741 The FC timing requirements (see Section 5) create an exposure of the 742 FC PW to inserted latency. Injection of latency sufficient to cause 743 the round trip time for an FC PW to exceed R_T_TOV (default: 100ms) 744 may cause the FC PW to fail in an active fashion because the FC link 745 initialization protocol repeatedly times out. OAM functionality for 746 deployed FC PWs SHOULD monitor for persistence of this situation and 747 respond accordingly (e.g., shut down the FC PW in order to avoid 748 wasting WAN bandwidth on an FC PW whose FC link cannot be 749 successfully initialized due to excessive latency). 751 7. Applicability Statement 753 FC PW allows the transparent transport of FC traffic between Fibre 754 Channel ports while saving network bandwidth by removing FC Idle 755 Signals and reducing the number of FC Primitive Sequences. 757 o The pair of CE devices operates as if they were directly connected 758 by an FC link. In particular they react to Primitive Sequences on 759 their local FC links as specified by the FC standards. 761 o The FC PW carries only FC data frames, FC Primitive Signals and a 762 subset of the copies of an FC Primitive Sequence. Idle Primitive 763 Signals are suppressed, and long streams of the same Primitive 764 Sequence are reduced over the PW thus saving bandwidth. 766 o The PW PE MUST generate Idle Primitive Signals to the attached FC 767 link when there is no other traffic to transmit on the attached FC 768 link [FC-FS-2]. 770 o The PW PE MUST send Primitive Sequences continuously to the 771 attached FC port, as required by the FC standards [FC-FS-2]. 773 FC PW traffic should only traverse MPLS networks that are provisioned 774 based on traffic engineering to provide dedicated bandwidth for FC PW 775 traffic. The MPLS network should enforce ingress traffic policing so 776 that delivery of FC PW traffic can be assured. To extend FC across a 777 network that does not satisfy these requirements, FCIP SHOULD be used 778 instead of an FC PW, see [RFC3821] and [FC-BB-6]. 780 This document does not provide any mechanisms for protecting an FC PW 781 against network outages. As a consequence, resilience of the emulated 782 FC service to such outages is dependent upon the underlying MPLS 783 network, which should be protected against failures. When a network 784 outage is detected, the PE SHOULD use a WAN Down event (as specified 785 in [FC-BB-6]) to convey the PW status to the CE, to enable faster 786 outage handling. 788 8. IANA Considerations 790 IANA is requested to assign a new MPLS Pseudowire (PW) type as 791 follows: 793 PW type Description Reference 794 -------- -------------- ---------- 795 0x001F FC Port Mode RFC XXXX 797 The above value is suggested as the next available value and has been 798 reserved for this purpose by IANA. 800 RFC Editor: Please replace RFC XXXX above with the RFC number of this 801 document and remove this note. 803 IANA should reserve the following Pseudowire Interface Parameters 804 Sub-TLV Types that were tentatively allocated for FC PW and restrict 805 them to prevent future allocation, citing this RFC as the reference 806 for that reservation and restriction. These Sub-TLV types were used 807 for the FC PW Selective Retransmission protocol, which the working 808 group has decided to eliminate. This action prevents future use of 809 these values for other purposes, as there is at least one 810 implementation of the Selective Retransmission protocol that has been 811 deployed. 813 Parameter ID Length Reference 814 --------- --------- ---------- 815 0x12 4 RFC XXXX 816 0x13 4 RFC XXXX 817 0x14 4 RFC XXXX 818 0x15 4 RFC XXXX 820 RFC Editor: Please replace RFC XXXX above with the RFC number of this 821 document and remove this note. 823 9. Acknowledgments 825 Previous versions of this document were authored by Moran Roth, Ronen 826 Solomon and Munefumi Tsurusawa; their efforts and contributions are 827 gratefully acknowledged. The authors would like to thank Stewart 828 Bryant, Elwyn Davies, Steve Hanna, Dave Peterson, Yaakov Stein, 829 Alexander Vainshtein, and the members of the IESG for helpful 830 comments on this document. 832 The protocol specified in this document is intended to be used in 833 conjunction with the Fibre Channel pseudowire portion of the FC-BB-6 834 specification developed by INCITS Technical Committee T11. The 835 authors would like to thank the members of both the IETF and T11 836 organizations who have supported and contributed to this work. 838 This document was prepared using 2-Word-v2.0.template.dot. 840 10. Normative References 842 [RFC3643] Weber, R., et al, "Fibre Channel (FC) Frame 843 Encapsulation", RFC 3643, December 2003. 845 [RFC3985] Bryant, S., et al, "Pseudo Wire Emulation Edge-to-Edge 846 (PWE3) Architecture", RFC 3985, March 2005. 848 [RFC4446] Martini, L., "IANA Allocations for Pseudowire Edge to 849 Edge Emulation (PWE3)", RFC 4446, April 2006. 851 [RFC4447] Martini, L., et al, "Pseudowire Setup and Maintenance 852 using the Label Distribution Protocol (LDP)", RFC4447, 853 April 2006. 855 [RFC4385] Bryant, S., et al, "Pseudowire Emulation Edge-to- 856 Edge(PWE3) Control Word for use over an MPLS PSN", 857 RFC4385, February 2006. 859 [RFC4623] Malis, A., Townsley, M., "PWE3 Fragmentation and 860 Reassembly", RFC 4623, August 2006. 862 [RFC4720] Malis, A., et al, "Pseudowire Emulation Edge-to-Edge 863 (PWE3) Frame Check Sequence Retention", RFC 4720, 864 November 2006. 866 [FC-BB-6] "Fibre Channel Backbone-6" (FC-BB-6), T11 Project 867 2159-D, Rev 1.02, October 2010. 869 RFC Editor: FC-BB-6 is a work in progress. Please treat [FC-BB-6] as a 870 normative reference to a work in progress, and proceed as follows: 871 1. Assign an RFC number to this draft and communicate that 872 number to the authors of this draft, one of whom (David Black) 873 is the T11 designated liaison to IETF. 874 2. Place a reference hold on this draft until FC-BB-6 is published 875 as an ANSI standard. 876 3. When FC-BB-6 is published as an ANSI standard, the draft authors 877 will provide an update to the FC-BB-6 reference that includes 878 an ANSI standard number. Update the FC-BB-6 reference using 879 that information, remove the reference hold due to FC-BB-6, and 880 remove this note. 882 [RFC-2119] Bradner, S., "Key words for use in RFCs to Indicate 883 requirement Levels", BCP 14, RFC 2119, March 1997. 885 [FC-FS-2] "Fibre Channel - Framing and Signaling-2 (FC-FS-2)", 886 ANSI INCITS 424:2007, August 2007. 888 11. Informative references 890 [RFC3821] M. Rajogopal, E. Rodriguez, "Fibre Channel over TCP/IP 891 (FCIP)", RFC 3821, July 2004. 893 [RFC4717] Martini, L., et al, "Encapsulation Methods for 894 Transport of Asynchronous Transfer Mode (ATM) over 895 MPLS Networks", RFC 4717, December 2006. 897 [T11] INCITS Technical Committee T11, http://www.t11.org, 898 visited January, 2011. 900 [FC-SP] "Fibre Channel - Security Protocols" (FC-SP), ANSI 901 INCITS 426:2007, February 2007. 903 Authors' Addresses 905 David L. Black (ed.) 906 EMC Corporation 907 176 South Street 908 Hopkinton, MA 01748 909 Phone: +1 (508) 293-7953 910 Email: david.black@emc.com 912 Linda Dunbar (ed.) 913 Huawei Technologies 914 1700 Alma Drive, Suite 500 915 Plano, TX 75075, USA 916 Phone: +1 (972) 543-5849 917 Email: ldunbar@huawei.com 919 Moran Roth 920 Infinera Corporation 921 169 Java Drive 922 Sunnyvale, CA 94089 923 Phone: (408) 572-5200 924 Email: MRoth@infinera.com 926 Ronen Solomon 927 Orckit-Corrigent Systems 928 126, Yigal Alon st. 929 Tel Aviv, ISRAEL 930 Phone: +972-3-6945316 931 Email: ronens@corrigent.com 933 Intellectual Property Statement 935 The IETF Trust takes no position regarding the validity or scope of 936 any Intellectual Property Rights or other rights that might be 937 claimed to pertain to the implementation or use of the technology 938 described in any IETF Document or the extent to which any license 939 under such rights might or might not be available; nor does it 940 represent that it has made any independent effort to identify any 941 such rights. 943 Copies of Intellectual Property disclosures made to the IETF 944 Secretariat and any assurances of licenses to be made available, or 945 the result of an attempt made to obtain a general license or 946 permission for the use of such proprietary rights by implementers or 947 users of this specification can be obtained from the IETF on-line IPR 948 repository at http://www.ietf.org/ipr 950 The IETF invites any interested party to bring to its attention any 951 copyrights, patents or patent applications, or other proprietary 952 rights that may cover technology that may be required to implement 953 any standard or specification contained in an IETF Document. Please 954 address the information to the IETF at ietf-ipr@ietf.org. 956 Disclaimer of Validity 958 All IETF Documents and the information contained therein are provided 959 on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 960 REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE 961 IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL 962 WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY 963 WARRANTY THAT THE USE OF THE INFORMATION THEREIN WILL NOT INFRINGE 964 ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 965 FOR A PARTICULAR PURPOSE. 967 Acknowledgment 969 Funding for the RFC Editor function is currently provided by the 970 Internet Society.