idnits 2.17.1 draft-tsiang-srp-02.txt: ** The Abstract section seems to be numbered Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity -- however, there's a paragraph with a matching beginning. Boilerplate error? == No 'Intended status' indicated for this document; assuming Proposed Standard Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 28 instances of too long lines in the document, the longest one being 5 characters in excess of 72. ** The abstract seems to contain references ([2], [1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 1703 has weird spacing: '...ram for the ...' -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 2114 looks like a reference -- Missing reference section? '2' on line 2116 looks like a reference -- Missing reference section? '5' on line 2124 looks like a reference -- Missing reference section? '3' on line 2118 looks like a reference -- Missing reference section? '4' on line 2121 looks like a reference -- Missing reference section? '6' on line 2126 looks like a reference Summary: 7 errors (**), 0 flaws (~~), 2 warnings (==), 9 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 INTERNET DRAFT D. Tsiang 3 Expires 11/1/2000 G. Suwala 4 Cisco Systems 5 5/1/2000 7 The Cisco SRP MAC Layer Protocol 9 1. Status of this Memo 11 This document is an Internet-Draft and is in full conformance with 12 all provisions of Section 10 of RFC2026 except that the right to 13 produce derivative works is not granted. 15 Internet-Drafts are working documents of the Internet Engineering 16 Task Force (IETF), its areas, and its working groups. Note that 17 other groups may also distribute working documents as Internet- 18 Drafts. Internet-Drafts are draft documents valid for a maximum of 19 six months and may be updated, replaced, or obsoleted by other 20 documents at any time. It is inappropriate to use Internet- Drafts 21 as reference material or to cite them other than as "work in 22 progress." 24 The list of current Internet-Drafts can be accessed at 25 http://www.ietf.org/ietf/1id-abstracts.txt 27 The list of Internet-Draft Shadow Directories can be accessed at 28 http://www.ietf.org/shadow.html. 30 2. Abstract 32 This document specifies the MAC layer protocol, "Spatial Reuse 33 Protocol" (SRP) for use with ring based media. This is a second 34 version of the protocol (V2). 36 The primary requirements for SRP are as follows: 38 - Efficient use of bandwidth using: 40 spatial reuse of bandwidth 41 local reuse of bandwidth 42 minimal protocol overhead 44 - Support for priority traffic 45 - Scalability across a large number of nodes or stations attached to 46 a ring 48 - "Plug and play" design without a software based station management 49 transfer (SMT) protocol or ring master negotiation as seen in other 50 ring based MAC protocols [1][2] 52 - Fairness among nodes using the ring 54 - Support for ring based redundancy (error detection, ring wrap, 55 etc.) similar to that found in SONET BLSR specifications. 57 - Independence of physical layer (layer 1) media type. 59 This document defines the terminology used with SRP, packet formats 60 the protocol format, protocol operation and associated protocol 61 finite state machines. See the last two pages for the table of 62 contents. 64 3. Differences between SRP V1 and V2 66 This document pertains to SRP V2. SRP V1 was a previously published 67 draft specification. The following lists V2 feature differences from 68 V1: 70 - Reduction of the header format from 4 bytes to 2 bytes. 72 - Replacement of the keepalive packet with a new control packet that 73 carries usage information in addition to providing a keepalive 74 function. 76 - Change bit value of inner ring to be 1 and outer to be 0. 78 - Reduction in the number of TTL bits from 11 to 8. 80 - Removal of the DS bit. 82 - Change ordering of CRC transmission to be most significant octet 83 first (was least significant octet in V1). The SRP CRC is now the 84 same as in [5]. 86 - Addition of the SRP cell mode to carry ATM cells over SRP. 88 - Changes to the SRP-fa to increase the usage field width and to 89 remove the necessity of adding a fixed constant when propagating 90 usage messages. 92 4. Terms and Taxonomy 94 4.1. Ring Terminology 96 SRP uses a bidirectional ring. This can be seen as two symmetric 97 counter-rotating rings. Most of the protocol finite state machines 98 (FSMs) are duplicated for the two rings. 100 The bidirectional ring allows for ring-wrapping in case of media or 101 station failure, as in FDDI [1] or SONET/SDH [3]. The wrapping is 102 controlled by the Intelligent Protection Switching (IPS) protocol. 104 To distinguish between the two rings, one is referred to as the 105 "inner" ring, the other the "outer" ring. The SRP protocol operates 106 by sending data traffic in one direction (known as "downstream") and 107 it's corresponding control information in the opposite direction 108 (known as "upstream") on the opposite ring. Figure 1 highlights this 109 graphically. 111 FIGURE 1. Ring Terminology 113 {outer_data 114 ----- inner_ctl} 115 ---------------->| N |----------------- 116 | ---------------| 1 |<-------------- | 117 | | {inner_data ----- | | 118 | | outer_ctl} | | 119 ----- ----- 120 | N | | N | 121 | 6 | | 2 | 122 ----- ----- 123 ^ | ^ | 124 o | | i | | 125 u | | n | | 126 t | | n | | 127 e | | e | | 128 r | | r | | 129 | v | v 130 ----- ----- 131 | N | | N | 132 | 5 | | 3 | 133 ----- ----- 134 | | | | 135 | | ----- | | 136 | -------------->| N |--------------- | 137 -----------------| 4 |<---------------- 138 ----- 140 4.2. Spatial Reuse 142 Spatial Reuse is a concept used in rings to increase the overall 143 aggregate bandwidth of the ring. This is possible because unicast 144 traffic is only passed along ring spans between source and 145 destination nodes rather than the whole ring as in earlier ring based 146 protocols such as token ring and FDDI. 148 Figure 2 below outlines how spatial reuse works. In this example, 149 node 1 is sending traffic to node 4, node 2 to node 3 and node 5 to 150 node 6. Having the destination node strip unicast data from the ring 151 allows other nodes on the ring who are downstream to have full access 152 to the ring bandwidth. In the example given this means node 5 has 153 full bandwidth access to node 6 while other traffic is being 154 simultaneously transmitted on other parts of the ring. 156 4.3. Fairness 158 Since the ring is a shared media, some sort of access control is 159 necessary to ensure fairness and to bound latency. Access control can 160 be broken into two types which can operate in tandem: 162 Global access control - controls access so that everyone gets a 163 fair share of the global bandwidth of the ring. 165 Local access control - grants additional access beyond that 166 allocated globally to take advantage of segments of the ring 167 that are less than fully utilized. 169 As an example of a case where both global and local access are 170 required, refer again to Figure 2. Nodes 1, 2, and 5 will get 1/2 of 171 the bandwidth on a global allocation basis. But from a local 172 perspective, node 5 should be able to get all of the bandwidth since 173 its bandwidth does not interfere with the fair shares of nodes 1 and 174 2. 176 FIGURE 2. Global and Local Re-Use 178 . . . . . . . . . . . . . . . . . 179 . . 180 ----- . 181 ---------------->| N |----------------- . 182 | ---------------| 1 |<-------------- | . 183 | | ----- | | . 184 | | | | . 185 ----- ----- . 186 . .>| N | | N |. .. . 187 . | 6 | | 2 | . . 188 . ----- ----- . . 189 . ^ | ^ | . . 190 . o | | i | | . . 191 . u | | n | | . . 192 . t | | n | | . . 193 . e | | e | | . . 194 . r | | r | | . . 195 . | v | v . . 196 . ----- ----- . . 197 . . | N | | N |<. . . 198 | 5 | | 3 | . 199 ----- ----- . 200 | | | | . 201 | | ----- | | . 202 | -------------->| N |--------------- | . 203 -----------------| 4 |<---------------- . 204 ----- . 205 ^ . 206 . . 207 . . . . .<. . . . . . . . . . . . 209 4.4. Transit Buffer 211 To be able to detect when to transmit and receive packets from the 212 ring, SRP makes use of a transit (sometimes referred as insertion) 213 buffer as shown in Figure 3 below. High priority packets and low 214 priority packets can be placed into separate fifo queues. 216 FIGURE 3. Transit buffer 218 ^^ || 219 || vv 220 |----| |----| 221 | | | | 222 |----|Rx |----|Tx 223 | |Buffer | |Buffer 224 |----| |----| 225 | | | | 226 |----| |----| 227 | | | | 228 |----| |----| 229 | | | | 230 |----| |----| 231 ^^ Transit || 232 || Buffer || 233 || |------| vv 234 | H | 235 ===========>|------|==========> 236 | L | 237 |------| 239 5. SRP Overview 241 5.1. Receive Operation Overview 243 Receive Packets entering a node are copied to the receive buffer if a 244 Destination Address (DA) match is made. If a DA matched packet is 245 also a unicast, then the packet will be stripped. If a packet does 246 not DA match or is a multicast and the packet does not Source Address 247 (SA) match, then the packet is placed into the Transit Buffer (TB) 248 for forwarding to the next node if the packet passes Time To Live and 249 Cyclic Redundancy Check (CRC) tests. 251 5.2. Transmit Operation Overview 253 Data sent from the node is either forwarded data from the TB or 254 transmit data originating from the node via the Tx Buffer. High 255 priority forwarded data always gets sent first. High priority 256 transmit data may be sent as long as the Low Priority Transit Buffer 257 (LPTB) is not full. 259 A set of usage counters monitor the rate at which low priority 260 transmit data and forwarded data are sent. Low priority data may be 261 sent as long as the usage counter does not exceed an allowed usage 262 governed by the SRP-fa rules and the LPTB has not exceeded the low 263 priority threshold. 265 5.3. SRP Fairness Algorithm (SRP-fa) Overview 267 If a node experiences congestion, then it will advertise to upstream 268 nodes via the opposite ring the value of its transmit usage counter. 269 The usage counter is run through a low pass filter function to 270 stabilize the feedback. Upstream nodes will adjust their transmit 271 rates so as not to exceed the advertised values. Nodes also 272 propagate the advertised value received to their immediate upstream 273 neighbor. Nodes receiving advertised values who are also congested 274 propagate the minimum of their transmit usage and the advertised 275 usage. 277 Congestion is detected when the depth of the low priority transit 278 buffer reaches a congestion threshold. 280 Usage messages are generated periodically and also act as keepalives 281 informing the upstream station that a valid data link exists. 283 5.4. Intelligent Protection Switching (IPS) Protocol Overview 285 An SRP Ring is composed of two counter-rotating, single fiber rings. 286 If an equipment or fiber facility failure is detected, traffic going 287 towards and from the failure direction is wrapped (looped) back to go 288 in the opposite direction on the other ring (subject to the 289 protection hierarchy). The wrap around takes place on the nodes 290 adjacent to the failure, under control of the IPS protocol. The wrap 291 re-routes the traffic away from the failed span. 293 An example of the data paths taken before and after a wrap are shown 294 in Figures 4 and 5. Before the fiber cut, N4 sends to N1 via the 295 path N4->N5->N6->N1. 297 If there is a fiber cut between N5 and N6, N5 and N6 will wrap the 298 inner ring to the outer ring. After the wraps have been set up, 299 traffic from N4 to N1 initially goes through the non-optimal path 300 N4->N5->N4->N3->N2->N1->N6->N1. 302 Subsequently a new ring topology is discovered and a new optimal path 303 is used N4->N3->N2-N1 as shown in Figure 6. Note that the topology 304 discovery and the subsequent optimal path selection are not part of 305 the IPS protocol. 307 FIGURE 4. Data path before wrap, N4 -> N1 309 ----- 310 ################>| N |----------------- 311 # ---------------| 1 |<-------------- | 312 # | ----- | | 313 # | | | 314 ----- ----- 315 | N | | N | 316 | 6 | | 2 | 317 ----- ----- 318 ^ | ^ | 319 # | | | 320 # | | | 321 # | | | 322 # | | | 323 # | | | 324 # v | v 325 ----- ----- 326 | N | | N | 327 | 5 | | 3 | 328 ----- ----- 329 # | | | 330 # | ----- | | 331 # -------------->| N |--------------- | 332 #################| 4 |<---------------- 333 ----- 335 The ring wrap is controlled through SONET BLSR [3][4] style IPS 336 signaling. It is an objective to perform the wrapping as fast as in 337 the SONET equipment or faster. 339 The IPS protocol processes the following request types (in the order 340 of priority, from highest to lowest): 342 1. Forced Switch (FS): operator originated, performs a 343 protection switch on a requested span (wraps at both ends 344 of the span) 346 2. Signal Fail (SF): automatic, caused by a media Signal 347 Failure or SRP keep-alive failure - performs a protection 349 FIGURE 5. Data path after the wrap, N4 -> N1 351 ----- 352 ################>| N |----------------- 353 # ###############| 1 |<############## | 354 # # ----- # | 355 # v # | 356 ----- ----- 357 | N | | N | 358 | 6 | | 2 | 359 ----- ----- 360 ^ # wrap ^ | 361 ### # | 362 _________ # | 363 fiber cut # | 364 --------- # | 365 ### # | 366 # v wrap # v 367 ----- ----- 368 | N | | N | 369 | 5 | | 3 | 370 ----- ----- 371 # # # | 372 # # ----- # | 373 # ##############>| N |############### | 374 #################| 4 |<---------------- 375 ----- 377 switch on a requested span 379 3. Signal Degrade (SD): automatic, caused by a media Signal 380 Degrade (e.g. excessive Bit Error Rate) - performs a 381 protection switch on a requested span 383 4. Manual Switch (MS): operator originated, like Forced 384 Switched but of a lower priority. 386 5. Wait to Restore (WTR): automatic, entered after the working 387 channel meets the restoration criteria after SF or SD 388 condition disappears. IPS waits WTR period before 389 restoring traffic in order to prevent protection switch 390 oscillations. 392 If a protection (either automatic or operator originated) is 393 requested for a given span, the node on which the protection has been 394 requested issues a protection request to the node on the other end of 395 the span using both the short path (over the failed span, as the 396 FIGURE 6. Data path after the new topology is discovered 398 ----- 399 -----------------| N |----------------- 400 | ---------------| 1 |<############## | 401 | | ----- # | 402 | v # | 403 ----- ----- 404 | N | | N | 405 | 6 | | 2 | 406 ----- ----- 407 ^ | wrap ^ | 408 -- # | 409 _________ # | 410 fiber cut # | 411 --------- # | 412 -- # | 413 | v wrap # v 414 ----- ----- 415 | N | | N | 416 | 5 | | 3 | 417 ----- ----- 418 | | # | 419 | | ----- # | 420 | -------------->| N |############### | 421 -----------------| 4 |<---------------- 422 ----- 424 failure may be unidirectional) and the long path (around the ring). 426 As the protection requests travel around the ring, the protection 427 hierarchy is applied. If the requested protection switch is of the 428 highest priority e.g. Signal Fail request is of higher priority than 429 the Signal Degrade than this protection switch takes place and the 430 lower priority switches elsewhere in the ring are taken down, as 431 appropriate. If a lower priority request is requested, it is not 432 allowed if a higher priority request is present in the ring. The only 433 exception is multiple SF and FS switches, which can coexist in the 434 ring. 436 All protection switches are performed bidirectionally (wraps at both 437 ends of a span for both transmit and receive directions, even if a 438 failure is only unidirectional). 440 6. Packet Formats 442 This section describes the packet formats used by SRP. Packets can be 443 sent over any point to point link layer (e.g. SONET/SDH, ATM, point 444 to point ETHERNET connections). The maximum transfer unit (MTU) is 445 9216 octets. The minimum transfer unit for data packets is 55 446 octets. The maximum limit was designed to accommodate the large IP 447 MTUs of IP over AAL5. SRP also supports ATM cells. ATM cells over 448 SRP are 55 octets. The minimum limit corresponds to ATM cells 449 transported over SRP. The minimum limit does not apply to control 450 packets which may be smaller. 452 These limits include everything listed in Figure 7: but are exclusive 453 of the frame delineation (e.g. for SRP over SONET/SDH, the flags used 454 for frame delineation are not included in the size limits). 456 The following packet and cell formats do not include any layer 1 457 frame delineation. For SRP over POS, there will be an additional 458 flag that delineates start and end of frame. 460 6.1. Overall Packet Format 462 The overall packet format is show below in Figure 7: 464 FIGURE 7. Overall Packet Format 466 --------------------------------- 467 | SRP Header | 468 --------------------------------- 469 | Dest. Addr. | 470 --------------------------------- 471 | Source Addr. | 472 --------------------------------- 473 | Protocol Type | 474 --------------------------------- 475 | Payload | 476 | | 477 | | 478 | | 479 --------------------------------- 480 | FCS | 481 --------------------------------- 483 The frame check sequence (FCS) is a 32-bit cyclic redundancy check 484 (CRC) as specified in RFC-1662 and is the same CRC as used in Packet 485 Over SONET (POS - specified in RFC-2615). The generator polynomial 486 is : 488 CRC-32: 490 x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 + 491 x2 + x + 1 493 The FCS is computed over the destination address, source address, 494 protocol type and payload. It does not include the SRP header. 496 Note that the packet format after the SRP header is identical to 497 Ethernet Version 2. 499 6.2. Generic Packet Header Format 501 Each packet has a fixed-sized header. The packet header format is 502 shown in Figure 8. 504 FIGURE 8. Detailed Packet Header Format 506 0 1 2 3 507 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 508 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 509 | Time to Live |R| MOD | PRI |P| | 510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Destination Address | 511 | | 512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 513 | | 514 + Source Address +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 515 | | Protocol Type | 516 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 517 | | 518 + + 519 | Payload | 520 . . 521 . . 522 . . 523 | | 524 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 526 The fields are described below. 528 6.2.1. Time To Live (TTL) 530 This 8 bit field is a hop-count that must be decremented every time a 531 node forwards a packet. If the TTL reaches zero it is stripped off 532 the ring. This allows for a total node space of 256 nodes on a ring. 533 However, due to certain failure conditions (e.g. when the ring is 534 wrapped) the total number of nodes that are supported by SRP is 128. 535 When a packet is first sent onto the ring the TTL should be set to at 536 least twice the total number of nodes on the ring. 538 6.2.2. Ring Identifier (R) 540 This single bit field is used to identify which ring this packet is 541 designated for. The designation is as follows: 543 TABLE 1. Ring Indicator Values 545 Outer Ring 0 546 Inner Ring 1 548 6.2.3. Priority Field (PRI) 550 This three bit field indicates the priority level of the SRP packet 551 (0 through 7). The higher the value the higher the priority. Since 552 there are only two queues in the transit buffer (HPTB and LPTB) a 553 packet is treated as either low or high priority once it is on the 554 ring. Each node determines the threshold value for determining what 555 is considered a high priority packet and what is considered a low 556 priority packet. However, the full 8 levels of priority in the SRP 557 header can be used prior to transmission onto the ring (transmit 558 queues) as well as after reception from the ring (receive queues). 560 6.2.4. MODE 562 This three bit field is used to identify the mode of the packet. The 563 following modes are defined in Table 2 below. 565 TABLE 2. MODE Values 567 Value Description 569 000 Reserved 570 001 Reserved 571 010 Reserved 572 011 ATM cell 573 100 Control Message (Pass to host) 574 101 Control Message (Locally Buffered for host) 575 110 Usage Message 576 111 Packet Data 578 These modes will be further explained in later sections. 580 6.2.5. Parity Bit (P-bit) 582 The parity bit is used to indicate the parity value over the 15 bits 583 of the SRP header to provide additional data integrity over the 584 header. Odd parity is used (i.e. the number of ones including the 585 parity bit shall be an odd number). 587 6.2.6. Destination Address 589 The destination address is a globally unique 48 bit address assigned 590 by the IEEE. 592 6.2.7. Source Address 594 The source address is a globally unique 48 bit address assigned by 595 the IEEE. 597 6.2.8. Protocol Type 599 The protocol type is a two octet field like that used in EtherType 600 representation. Current defined values relevant to SRP are defined in 601 Table 3 below. 603 TABLE 3. Defined Protocol Types 605 Value Protocol Type 607 0x2007 SRP Control 608 0x0800 IP version 4 609 0x0806 ARP 611 6.3. SRP Cell Format 613 SRP also supports the sending of ATM cells. The detailed cell format 614 is shown below: 616 FIGURE 9. SRP Cell Format 618 0 1 2 3 619 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 620 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 621 | Time to Live |R| MOD | PRI |P| VPI/VCI | 622 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 623 | VCI | PTI |C| HEC | | 624 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 625 | | 626 . . 627 . ATM Payload . 628 . ( 48 Bytes ) +-+-+-+-+-+-+-+-+ 629 | | 630 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 632 Packet nodes would typically ignore (never receive or strip) and 633 always forward ATM-cells. The idea is that ATM switches and routers 634 could coexist in a ring. Note that SRP cells do not contain an FCS. 635 Data integrity is handled at the AAL layer. 637 6.4. SRP Usage Packet Format 639 SRP usage packets are sent out periodically to propagate allowed 640 usage information to upstream nodes. SRP usage packets also perform 641 a keepalive function. SRP usage packets should be sent approximately 642 every 106 usec. 644 If a receive interface has not seen a usage packet within the 645 keepalive timeout interval it will trigger an L2 keepalive timeout 646 interrupt/event. The IPS software will subsequently mark that 647 interface as faulty and initiate a protection switch around that 648 interface. The keepalive timeout interval should be set to 16 times 649 the SRP usage packet transmission interval. 651 FIGURE 10. Usage Packet Format 653 0 1 2 3 654 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 655 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 656 | Time to Live |R| MOD | PRI |P| | 657 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Originator MAC Address + 658 | | 659 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 660 | Reserved | Usage | 661 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 663 A USAGE of all ones indicates a value of NULL. 665 6.5. SRP Control Packet Format 667 If the MODE bits are set to 10X (SRP control) then this indicates a 668 control message. Control messages are always received and stripped by 669 the adjacent node. They are by definition unicast, and do not need 670 any addressing information. The destination address field for 671 control packets should be set to 0's. The source address field for a 672 control packet should be set to the source address of the 673 transmitting node. 675 Two types of controls messages are defined : Pass to host and Locally 676 buffered. Pass to host messages can be passed to the host software by 677 whatever means is convenient. This is most often the same path used 678 to transfer data packets to the host. Locally buffered control 679 messages are usually reserved for protection messages. These are 680 normally buffered locally in order to not contend for resources with 681 data packets. The actual method of handling these messages is up to 682 the implementor. 684 The control packet format is shown in Figure 11. 686 FIGURE 11. Control Packet Format 688 0 1 2 3 689 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 690 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 691 | Time to Live |R| MOD | PRI |P| | 692 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Destination Address | 693 | | 694 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 695 | | 696 + Source Address +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 697 | | Protocol Type = 0x2007 | 698 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 699 | Control Ver | Control Type | Control Checksum | 700 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 701 | Control TTL | | 702 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 703 . . 704 . Payload . 705 . . 706 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 708 The priority (PRI) value should be set to 0x7 (all one's) when 709 sending control packets and should be queued to the highest priority 710 transmit queue available. The Time to Live is not relevant since all 711 packets will be received and stripped by the nearest downstream 712 neighbor and can be set to any value (preferably this should be set 713 to 001). 715 6.5.1. Control Ver 717 This one octet field is the version number associated with the 718 control type field. Initially, all control types will be version 0. 720 6.5.2. Control Type 722 This one octet field represents the control message type. Table 4 723 contains the currently defined control types. 725 TABLE 4. Control Types 727 Control Type Description 729 0x01 Topology Discovery 731 0x02 IPS message 733 0x03- 734 0xFF Reserved 736 6.5.3. Control TTL 738 The Control TTL is a control layer hop-count that must be decremented 739 every time a node forwards a control packet. If a node receives a 740 control packet with a control TTL <= 1, then it should accept the 741 packet but not forward it. 743 Note that the control layer hop count is separate from the SRP L2 TTL 744 which is always set to 1 for control messages. 746 The originator of the control message should set the initial value of 747 the control TTL to the SRP L2 TTL normally used for data packets. 749 6.5.4. Control Checksum 751 The checksum field is the 16 bit one's complement of the one's 752 complement sum of all 16 bit words starting with the control version. 753 If there are an odd number of octets to be checksummed, the last 754 octet is padded on the right with zeros to form a 16 bit word for 755 checksum purposes. The pad is not transmitted as part of the 756 segment. While computing the checksum, the checksum field itself is 757 replaced with zeros. This is the same checksum algorithm as that 758 used for TCP. The checksum does not cover the 32 bit SRP FCS. 760 6.5.5. Payload 762 The payload is a variable length field dependent on the control type. 764 6.5.6. Addressing 766 All nodes must have a globally unique IEEE 48 bit MAC address. A 767 multicast bit is defined using canonical addressing conventions i.e. 768 the multicast bit is the least significant bit of the most 769 significant octet in the destination address. It is acceptable but 770 not advisable to change a node's MAC address to one that is known to 771 be unique within the administrative layer 2 domain (that is the SRP 772 ring itself along with any networks connected to the SRP ring via a 773 layer 2 transparent bridge). 775 FIGURE 12. Multicast bit position 777 Destination Address 778 0 1 2 3 779 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 780 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 781 | |M| | 782 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 783 ^ 784 |----Multicast bit 786 Note that for SONET media, the network order is MSB of each octet 787 first, so that as viewed on the line, the multicast bit will be the 788 8th bit of the destination address sent. (For SRP on Ethernet media, 789 the multicast bit would be sent first). 791 6.6. Topology Discovery 793 Each node performs topology discovery by sending out topology 794 discovery packets on one or both rings. The node originating a 795 topology packet marks the packet with the egressing ring id, appends 796 the node's mac binding to the packet and sets the length field in the 797 packet before sending out the packet. This packet is a point-to-point 798 packet which hops around the ring from node to node. Each node 799 appends its mac address binding, updates the length field and sends 800 it to the next hop on the ring. If there is a wrap on the ring, the 801 wrapped node will indicate a wrap when appending its mac binding and 802 wrap the packet. When the topology packets travel on the wrapped 803 section with the ring identifier being different from that of the 804 topology packet itself, the mac address bindings are not added to the 805 packet. 807 Eventually the node that generated the topology discovery packet gets 808 back the packet. The node makes sure that the packet has the same 809 ingress and egress ring id before excepting the packet. A topology 810 map is changed only after receiving two topology packets which 811 indicate the same new topology (to prevent topology changes on 812 transient conditions). 814 Note that the topology map only contains the reachable nodes. It does 815 not correspond to the failure-free ring in case of wraps and ring 816 segmentations. 818 FIGURE 13. Topology Packet Format 820 Topology 822 0 1 2 3 823 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 824 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 825 | Time to Live |R| MOD | PRI |P| | 826 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Destination Address | 827 | | 828 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 829 | | 830 + Source Address +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 831 | | Protocol Type = 0x2007 | 832 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 833 | Control Ver=0 | Control Type=1| Control Checksum | 834 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 835 | Control TTL | Topology Length | 836 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 837 | Originator's Globally Unique | 838 + MAC Address (48 bits) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 839 | | MAC Type | | 840 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 841 | MAC Address (48 bits) | 842 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 843 | | Other MAC bindings | 844 +-+-+-+-+-+-+-+-+ + 845 | | 846 + + 848 Note that the Source address should be set to the source address of 849 the TRANSMITTING node (which is not necessarily the ORIGINATING 850 node). 852 6.6.1. Topology Length 854 This two octet field represents the length of the topology message in 855 octets starting with the first MAC Type/MAC Address binding. 857 6.6.2. Topology Originator 858 A topology discovery packet is determined to have been originated by 859 a node if the originator's globally unique MAC address of the packet 860 is that node's globally unique MAC address (assigned by the IEEE). 862 Because the mac addresses could be changed at a node, the IEEE MAC 863 address ensures that a unique identifier is used to determine that 864 the topology packet has gone around the ring and is to be consumed. 866 6.6.3. MAC bindings 867 Each MAC binding shall consist of a MAC Type field followed by the 868 node's 48 bit MAC address. The first MAC binding shall be the MAC 869 binding of the originator. Usually the originator's MAC address will 870 be it's globally unique MAC Address but some implementations may 871 allow this value to be overridden by the network administrator. 873 6.6.4. MAC Type Format 875 This 8 bit field is encoded as follows: 877 TABLE 5. MAC Type Format 879 Bit Value 881 0 Reserved 882 1 Ring ID (1 or 0) 883 2 Wrapped Node (1) / Unwrapped Node (0) 884 3-7 Reserved 886 Determination of whether a packet's egress and ingress ring ID's are 887 a match should be done by using the Ring ID found in the MAC Type 888 field of the last MAC binding as the ingress ring ID rather than the 889 R bit found in the SRP header. Although they should be the same, it 890 is better to separate the two functions as some implementations may 891 not provide the SRP header to upper layer protocols. 893 The topology information is not required for the IPS protection 894 mechanism. This information can be used to calculate the number of 895 nodes in the ring as well as to calculate hop distances to nodes to 896 determine the shortest path to a node (since there are two counter- 897 rotating rings). 899 The implementation of the topology discovery mechanism could be a 900 periodic activity or on "a need to discover" basis. In the periodic 901 implementation, each node generates the topology packet periodically 902 and uses the cached topology map until it gets a new one. In the need 903 to discover implementation, each node generates a topology discovery 904 packet whenever they need one e.g., on first entering a ring or 905 detecting a wrap. 907 6.7. Intelligent Protection Switching (IPS) 909 IPS is a method for automatically recovering from various ring 910 failures and line degradation scenarios. The IPS packet format is 911 outlined in Figure 14 below. 913 FIGURE 14. IPS Packet Format 915 0 1 2 3 916 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 917 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 918 | Time to Live |R| MOD | PRI |P| | 919 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Destination Address | 920 | | 921 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 922 | | 923 + Source Address +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 924 | | Protocol Type = 0x2007 | 925 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 926 | Control Ver=0 | Control Type=2| Control Checksum | 927 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 928 | Control TTL | | 929 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 930 | Originator MAC Address | 931 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 932 | Ips Octet | Rsvd Octet | 933 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 935 The IPS specific fields are detailed below. 937 6.7.1. Originator MAC Address 939 This is the MAC address of the originator of the IPS message. It is 940 not necessarily the same as the SRP Header Source Address as a node 941 may be simply propagating an IPS message (see the section "SRP IPS 942 Protocol Rules" Rule P.8 as an example). 944 6.7.2. IPS Octet 946 The IPS octet contains specific protection information. The format of 947 the IPS octet is as follows: 949 FIGURE 15. 950 IPS Octet Format: 952 Bits Values (values not listed are reserved) 954 0-3 IPS Request Type 956 1101 - Forced Switch (FS) 957 1011 - Signal Fail (SF) 958 1000 - Signal Degrade (SD) 959 0110 - Manual Switch (MS) 960 0101 - Wait to Restore (WTR) 961 0000 - No Request (IDLE) 963 4 Path indicator 965 0 - short (S) 966 1 - long (L) 968 5-7 Status Code 970 010 - Protection Switch Completed - traffic Wrapped (W) 971 000 - Idle (I) 973 The currently defined request types with values, hierarchy and 974 interpretation are as used in SONET BLSR [3], [4], except as noted. 976 6.8. Circulating packet detection (stripping) 978 Packets continue to circulate when transmitted packets fail to get 979 stripped. Unicast packets are normally stripped by the destination 980 station or by the source station if the destination station has 981 failed. Multicast packets are only stripped by the source station. If 982 both the source and destination stations drop out of the ring while a 983 unicast packet is in flight, or if the source node drops out while 984 its multicast packet is in flight, the packet will rotate around the 985 ring continuously. 987 The solution to this problem is to have a TTL or Time To Live field 988 in each packet that is set to at least twice the number of nodes in 989 the ring. As each node forwards the packet, it decrements the TTL. If 990 the TTL reaches zero it is stripped off of the ring. 992 The ring ID is used to qualify all stripping and receive decisions. 993 This is necessary to handle the case where packets are being wrapped 994 by some node in the ring. The sending node may see its packet on the 995 reverse ring prior to reaching its destination so must not source 996 strip it. The exception is if a node is in wrap. Logically, a node 997 in wrap "sees" the packet on both rings. However the usual 998 implementation is to receive the packet on one ring and to transmit 999 it on the other ring. Therefore, a node that is in the wrap state 1000 ignores the ring ID when making stripping and receiving decisions. 1002 A potential optimization would be to allow ring ID independent 1003 destination stripping of unicast packets. One problem with this is 1004 that packets may be delivered out of order during a transition to a 1005 wrap condition. For this reason, the ring ID should always be used as 1006 a qualifier for all strip and receive decisions. 1008 7. Packet acceptance and stripping 1010 A series of decisions based on the type of packet (mode), source and 1011 destination addresses are made on the MAC incoming packets. Packets 1012 can either be control or data packets. Control packets are stripped 1013 once the information is extracted. The source and destination 1014 addresses are checked in the case of data packets. The rules for 1015 reception and stripping are given below as well as in the flow chart 1016 in Figure 16. 1018 1. Decrement TTL on receipt of a packet, discard if it gets to 1019 zero; do not forward. 1021 2. Strip unicast packets at the destination station. Accept 1022 and strip "control" packets. 1024 3. Do not process packets other than for TTL and forwarding if 1025 they have the "wrong" ring_id for the direction in which 1026 they are received unless the node is in wrap. If the node 1027 is in wrap then ignore the ring_id. 1029 4. Do not process packets other than for TTL and forwarding if 1030 the mode is not supported by the node (e.g. reserved modes, 1031 or ATM cell mode for packet nodes). 1033 5. Packets accepted by the host because of the destination 1034 address should be discarded at the upper level if there is 1035 CRC error. 1037 6. Control messages are point to point between neighbors and 1038 should always be accepted and stripped. 1040 7. Packets whose source address is that of the receiving 1041 station and whose ring_id matches should be stripped. If a 1042 node is in wrap then ignore the ring_id. 1044 FIGURE 16. SRP Receive Flowchart (Packet node) 1046 if (MODE == 4,5)----------------------------------->[to host]--->| 1047 | | 1048 v | 1049 if (MODE == 6)------------------------------------->[strip]----->| 1050 | | 1051 v | 1052 if (!WRAPPED | 1053 & WRONG_RING_ID)----------------------------------------------|--->| 1054 | | | 1055 v | | 1056 if (MODE == 0,1,2,3)---------------------------------------------|--->| 1057 | | | 1058 v | | 1059 if (DA MATCH)------------------>if !(SA MATCH)----->[to host]--->| | 1060 | | | | 1061 | v | | 1062 | if (unicast)------->[to host]--->| | 1063 | | | | 1064 | v | | 1065 if (SA MATCH)----------------------->[strip]-------------------->| | 1066 | | | 1067 | | v 1068 |------------------------------>|<-----------------------|----| 1069 | | 1070 v | 1071 if (ttl < 2)------->[strip]----->| 1072 | | 1073 v | 1074 [decrement ttl] | 1075 | | 1076 [fwd pkt to tb] | 1077 | v 1078 |<-----------------------| 1079 v 1080 [back to top] 1082 Notes: Host is responsible for discarding CRC errored packets. 1083 Conditionals (if statements) branch to the right if true 1084 and branch down if false. 1086 7.1. Transmission and forwarding with priority 1088 A node can transmit four types of packets: 1090 1. High priority packets from the high priority transit 1091 buffer. 1093 2. Low priority packets from the low priority transit buffer. 1095 3. High priority packets from the host Tx high priority fifo. 1097 4. Low priority packets from the host Tx low priority fifo. 1099 High priority packets from the transit buffer are always sent first. 1100 High priority packets from the host are sent as long as the low 1101 priority transit buffer is not full. Low priority packets are sent 1102 as long as the transit buffer has not crossed the low priority 1103 threshold and the SRP-fa rules allow it (my_usage < allowed_usage). 1104 If nothing else can be sent, low priority packets from the low 1105 priority transit buffer are sent. 1107 This decision tree is shown in Figure 17. 1109 FIGURE 17. SRP transmit flowchart 1111 if (TB_High has pkt)----------->[send pkt from TB_high]-->| 1112 | | 1113 v | 1114 if (TB_Low full)------------------------------------------|---->| 1115 | | | 1116 v | | 1117 if (Tx_High has pkt)----------->[send pkt from Tx_high]-->| | 1118 | | | 1119 v | | 1120 if (TB_Low > Hi threshold)--------------------------------|---->| 1121 | | | 1122 v | | 1123 if (my_usage >= allowed_usage)----------------------------|---->| 1124 | | | 1125 v | | 1126 if (Tx_Low has pkt)------------>[send pkt from Tx_low]--->| | 1127 | | | 1128 | | v 1129 |<------------------------------------------------|-----| 1130 | | 1131 v | 1132 if (TB_Low has pkt)------------>[send pkt from TB_low]--->| 1133 | v 1134 |<------------------------------------------------| 1135 | 1136 v 1137 [Go to Top] 1139 Notes: Conditionals (if statements) branch to the right if true 1140 and branch down if false. 1142 7.2. Wrapping of Data 1144 Normally, transmitted data is sent on the same ring to the downstream 1145 neighbor. However, if a node is in the wrapped state, transmitted 1146 data is sent on the opposite ring to the upstream neighbor. 1148 8. SRP-fa Rules Of Operation 1150 The SRP-fa governs access to the ring. The SRP-fa only applies to 1151 low priority traffic. High priority traffic does not follow SRP-fa 1152 rules and may be transmitted at any time as long as there is 1153 sufficient transit buffer space. 1155 The SRP-fa requires three counters which control the traffic 1156 forwarded and sourced on the SRP ring. The counters are my_usage 1157 (tracks the amount of traffic sourced on the ring), forward_rate 1158 (amount of traffic forwarded on to the ring from sources other than 1159 the host) and allowed_usage (the current maximum transmit usage for 1160 that node). 1162 With no congestion all nodes build up allowed usage periodically. 1163 Each node can send up to max_usage. Max_usage is a per node 1164 parameter than limits the maximum amount of low priority traffic a 1165 node can send. 1167 When a node sees congestion it starts to advertise its my_usage which 1168 has been low pass filtered (lp_my_usage). 1170 Congestion is measured by the transit buffer depth crossing a 1171 congestion threshold. 1173 A node that receives a non-null usage message (rcvd_usage) will set 1174 its allowed usage to the value advertised. However, if the source of 1175 the rcvd_usage is the same node that received it then the rcvd_usage 1176 shall be treated as a null value. When comparing the rcvd_usage 1177 source address the ring ID of the usage packet must match the 1178 receiver's ring ID in order to qualify as a valid compare. The 1179 exception is if the receive node is in the wrap state in which case 1180 the usage packet's ring ID is ignored. 1182 Nodes that are not congested and that receive a non-null rcvd_usage 1183 generally propagate rcvd_usage to their upstream neighbor else 1184 propagate a null value of usage (all 1's). The exception is when an 1185 opportunity for local reuse is detected. Additional spatial reuse 1186 (local reuse) is achieved by comparing the forwarded rate (low pass 1187 filtered) to allow_usage. If the forwarded rate is less than the 1188 allowed usage, then a null value is propagated to the upstream 1189 neighbor. 1191 Nodes that are congested propagate the smaller of lp_my_usage and 1192 rcvd_usage. 1194 Convergence is dependent upon number of nodes and distance. 1195 Simulation has shown simulation convergence within 100 msec for rings 1196 of several hundred miles. 1198 8.1. SRP-fa pseudo-code 1200 A more precise definition of the fairness algorithm is shown below: 1202 Variables: 1204 lo_tb_depth low priority transit buffer depth 1206 my_usage count of octets transmitted by host 1207 lp_my_usage my_usage run through a low pass filter 1208 my_usage_ok flag indicating that host is allowed to transmit 1210 allow_usage the fair amount each node is allowed to transmit 1212 fwd_rate count of octets forwarded from upstream 1213 lp_fwd_rate fwd_rate run through a low pass filter 1215 congested node cannot transmit host traffic without the TB buffer 1216 filling beyond its congestion threshold point. 1218 rev_usage the usage value passed along to the upstream neighbor 1220 Constants: 1222 MAX_ALLOWANCE = configurable value for max allowed usage for this node 1224 DECAY_INTERVAL = 8000 octet times @ OC-12, 32,000 octet times @ OC-48 1226 AGECOEFF = 4 // Aging coeff for my_usage and fwd_rate 1228 LP_FWD = 64 // Low pass filter for fwd_rate 1229 LP_MU = 512 // Low pass filter for my usage 1230 LP_ALLOW = 64 // Low pass filter for allow usage auto increment 1232 NULL_RCVD_INFO = All 1's in rcvd_usage field 1234 TB_LO_THRESHOLD // TB depth at which no more lo-prio host traffic 1235 // can be sent 1237 MAX_LRATE = AGECOEFF * DECAY_INTERVAL = 128,000 for OC-48, 32000 for OC-12 1239 THESE ARE UPDATED EVERY CLOCK CYCLE: 1240 ===================================== 1242 my_usage is incremented by 1 for every octet that is transmitted by 1243 the host (does not include data transmitted from the 1244 Transit Buffer). 1246 fwd_rate is incremented by 1 for every octet that enters the 1247 Transit Buffer 1249 if ((my_usage < allow_usage) && 1250 !((lo_tb_depth > 0) && (fwd_rate < my_usage)) && 1251 (my_usage < MAX_ALLOWANCE)) 1252 // true means OK to send host packets 1253 my_usage_ok = true; 1255 UPDATED WHEN USAGE_PKT IS RECEIVED: 1256 =================================== 1258 if (usage_pkt.SA == my_SA) && 1259 [(usage_pkt.RI == my_RingID) || (node_state == wrapped)] 1260 rcvd_usage = NULL_RCVD_INFO; 1261 else 1262 rcvd_usage = usage_pkt.usage; 1264 THE FOLLOWING IS CALCULATED EVERY DECAY_INTERVAL: 1265 ================================================== 1267 congested = (lo_tb_depth > TB_LO_THRESHOLD/2) 1269 lp_my_usage = ((LP_MU-1) * lp_my_usage + my_usage) / LP_MU 1271 my_usage is decremented by min(allow_usage/AGECOEFF, my_usage/AGECOEFF) 1273 lp_fwd_rate = ((LP_FWD-1) * lp_fwd_rate + fwd_rate) / LP_FWD 1275 fwd_rate is decremented by fwd_rate/AGECOEFF 1277 (Note: lp values must be calculated prior to decrement of non-lp values). 1279 if (rcvd_usage != NULL_RCVD_INFO) 1280 allow_usage = rcvd_usage; 1281 else 1282 allow_usage += (MAX_LRATE - allow_usage) / (LP_ALLOW); 1284 if (congested) 1285 { 1286 if (lp_my_usage < rcvd_usage) 1287 rev_usage = lp_my_usage; 1288 else 1289 rev_usage = rcvd_usage; 1290 } 1291 else if ((rcvd_usage != NULL_RCVD_INFO) && 1292 (lp_fwd_rate > allow_usage) 1293 rev_usage = rcvd_usage; 1294 else 1295 rev_usage = NULL_RCVD_INFO 1297 if (rev_usage > MAX_LRATE) 1298 rev_usage = NULL_RCVD_INFO; 1300 8.2. Threshold settings 1302 The low priority transit buffer (TB_LO_THRESHOLD) is currently sized 1303 to about 4.4 msec or 320 KB at OC12 rates. The TB_HI_THRESHOLD is 1304 set to about 870 usec higher than the TB_LO_THRESHOLD or at 458 KB at 1305 OC12 rates. 1307 The high priority transit buffer needs to hold 2 to 3 MTUs or about 1308 30KB. 1310 9. SRP Synchronization 1312 Each node operates in "free-run" mode. That is, the receive clock is 1313 derived from the incoming receive stream while the transmit clock is 1314 derived from a local oscillator. This eliminates the need for 1315 expensive clock synchronization as required in existing SONET 1316 networks. Differences in clock frequency are accommodated by 1317 inserting a small amount of idle bandwidth at each nodes output. 1319 The clock source for the transmit clock shall be selected to deviate 1320 by no more than 20 ppm from the center frequency. The overall 1321 outgoing rate of the node shall be rate shaped to accommodate the 1322 worst case difference between receive and transmit clocks of adjacent 1323 nodes. This works as follows: 1325 A transit buffer slip count (tb_cnt) keeps track of the amount of 1326 octets inserted into the TB minus the amount of octets transmitted 1327 and is a positive integer. 1329 To account for a startup condition where a packet is being inserted 1330 into an empty TB and the node was otherwise idle the tb_cnt is reset 1331 if the transmit interface is idle. Idle is defined as no data being 1332 sent even though there is opportunity to send (i.e. the transmit 1333 interface is not prohibited from transmitting by the physical layer). 1335 An interval counter defines the sample period over which rate shaping 1336 is performed. This number should be sufficiently large to get an 1337 accurate rate shaping. 1339 A token_bucket counter implements the rate shaping and is a signed 1340 integer. We increment this counter by one of two fixed values called 1341 quantums each sample period. Quantum1 sets the rate at (Line_rate - 1342 Delta) where delta is the clock inaccuracy we want to accommodate. 1344 Quantum2 sets the rate at (Line_rate + Delta). If at the beginning 1345 of a sample period, tb_cnt >= sync_threshold, then we set the rate to 1346 Quantum2. This will allow us to catch up and causes the TB slip count 1347 to eventually go < sync_threshold. If tb_cnt is < sync_threshold 1348 then we set the rate to Quantum1. 1350 When the input rate and output rates are exactly equal, the tb_cnt 1351 will vary between sync_threshold > tb_cnt >= 0. This will vary for 1352 each implementation dependent upon the burst latencies of the design. 1353 The sync_threshold value should be set so that for equal transmit and 1354 receive clock rates, the transmit data rate is always Line_rate-Delta 1355 and will be implementation dependent. 1357 The token_bucket is decremented each time data is transmitted. When 1358 token_bucket reaches a value <= 0, a halt_transmit flag is asserted 1359 which halts further transmission of data (halting occurs on a packet 1360 boundary of course which can cause token_bucket to become a negative 1361 number). 1363 9.1. SRP Synchronization Examples 1365 Assume an interval of 2^^18 or 262144 clock cycles. A Quantum1 value 1366 must be picked such that the data rate will = (LINE_RATE - DELTA). A 1367 Quantum2 value must be picked and used if the tb_cnt shows that the 1368 incoming rate is greater than the outgoing rate and is = (LINE_RATE + 1369 DELTA). Assume that the source of the incoming and outgoing rate 1370 clocks are +/- 100 ppm. 1372 For an OC12c SPE rate of 600 Mbps and a system clock rate of 800 Mbps 1373 (16 bits @ 50 Mhz). The system clock rate is the rate at which the 1374 system transmits bytes to the framer (in most cases the framer 1375 transmit rate is asynchronous from the rate at which the system 1376 transfers data to the framer). 1378 Quantum1/Interval * 800 Mbps = 600 Mbps(1 - Delta) 1379 Quantum1 = Interval * (600/800) * (1 - Delta) 1380 Quantum1 = Interval * (600/800) * (1 - 1e-4) = 196588 1382 Quantum2/Interval * 800 Mbps = 600 Mbps(1 + Delta) 1383 Quantum2 = Interval * (600/800) * (1 + Delta) 1384 Quantum2 = Interval * (600/800) * (1 + 1e-4) = 196628 1386 Note: The actual data rate for OC-12c is 599.04 Mbps. 1388 10. IPS Protocol Description 1390 An SRP ring is composed of two counter-rotating, single fiber rings. 1391 If an equipment or fiber facility failure is detected, traffic going 1392 towards and from the failure direction is wrapped (looped) back to go 1393 in the opposite direction on the other ring. The wrap around takes 1394 place on the nodes adjacent to the failure, under software control. 1395 This way the traffic is re-routed from the failed span. 1397 Nodes communicate between themselves using IPS signaling on both 1398 inner and outer ring. 1400 The IPS octet contains specific protection information. The format of 1401 the IPS octet is as follows: 1403 FIGURE 18. 1404 IPS Octet format: 1406 0-3 IPS Request Type 1408 1101 - Forced Switch (FS) 1409 1011 - Signal Fail (SF) 1410 1000 - Signal Degrade (SD) 1411 0110 - Manual Switch (MS) 1412 0101 - Wait to Restore (WTR) 1413 0000 - No Request (IDLE) 1415 4 Path indicator 1417 0 - short (S) 1418 1 - long (L) 1420 5-7 Status Code 1422 010 - Protection Switch Completed -traffic Wrapped (W) 1423 000 - Idle (I) 1425 The IPS control messages are shown in this document as: 1427 {REQUEST_TYPE, SOURCE_ADDRESS, WRAP_STATUS, PATH_INDICATOR} 1429 10.1. The IPS Request Types 1431 The following is a list of the request types, from the highest to the 1432 lowest priority. All requests are signaled using IPS control 1433 messages. 1435 1. Forced Switch (FS - operator originated) 1437 This command performs the ring switch from the working 1438 channel to the protection, wrapping the traffic on the node 1439 at which the command is issued and at the adjacent node to 1440 which the command is destined. Used for example to add 1441 another node to the ring in a controlled fashion. 1443 2. Signal Fail (SF - automatic) 1445 Protection caused by a media "hard failure" or SRP keep- 1446 alive failure. SONET examples of SF triggers are: Loss of 1447 Signal (LOS), Loss of Frame (LOF), Line Bit Error Rate 1448 (BER) above a preselected SF threshold, Line Alarm 1449 Indication Signal (AIS). Note that the SRP keep-alive 1450 failure provides end-to-end coverage and as a result SONET 1451 Path triggers are not necessary. 1453 3. Signal Degrade (SD - automatic) 1455 Protection caused by a media "soft failure". SONET example 1456 of a SD is Line BER or Path BER above a preselected SD 1457 threshold. 1459 4. Manual Switch (MS - operator originated) 1461 Like the FS, but of lower priority. Can be used for example 1462 to take down the WTR. 1464 5. Wait to Restore (WTR - automatic) 1466 Entered after the working channel meets the restoration 1467 threshold after an SD or SF condition disappears. IPS waits 1468 WTR timeout before restoring traffic in order to prevent 1469 protection switch oscillations. 1471 10.2. SRP IPS Protocol States 1473 Each node in the IPS protocol is in one of the following states for 1474 each of the rings: 1476 10.2.1. Idle 1478 In this mode the node is ready to perform the protection switches and 1479 it sends to both neighboring nodes "idle" IPS messages, which include 1480 "self" in the source address field {IDLE, SELF, I, S} 1482 10.2.2. Pass-through 1484 Node participates in a protection switch by passing the wrapped 1485 traffic and long path signaling through itself. This state is entered 1486 based on received IPS messages. If a long path message with not null 1487 request is received and if the node does not strip the message (see 1488 Protocol Rules for stripping conditions) the node decrements the TTL 1489 and retransmits the message without modification. Sending of the 1490 Idle messages is stopped in the direction in which the message with 1491 not null request is forwarded. 1493 10.2.3. Wrapped 1495 Node participates in a protection switch with a wrap present. This 1496 state is entered based on a protection request issued locally or 1497 based on received IPS messages. 1499 10.3. IPS Protocol Rules 1501 10.3.1. SRP IPS Packet Transfer Mechanism 1503 R T.1: 1505 IPS packets are transferred in a store and forward mode between 1506 adjacent nodes (packets do not travel more than 1 hop between nodes 1507 at a time). Received packet (payload portion) is passed to software 1508 based on interrupts. 1510 R T.2: 1512 All IPS messages are sent to the neighboring nodes periodically on 1513 both inner and outer rings. The timeout period is configurable 1-600 1514 sec (default 1 sec). It is desirable (but not required) that the 1515 timeout is automatically decreased by a factor of 10 for the short 1516 path protection requests. 1518 10.3.2. SRP IPS Signaling and Wrapping Mechanism 1520 R S.1: 1522 IPS signaling is performed using IPS control packets as defined in 1523 Figure 14 "IPS Packet Format". 1525 R S.2: 1527 Node executing a local request signals the protection request on both 1528 short (across the failed span) and long (around the ring) paths after 1529 performing the wrap. 1531 R S.3: 1533 Node executing a short path protection request signals an idle 1534 request with wrapped status on the short (across the failed span) 1535 path and a protection request on the long (around the ring) path 1536 after performing the wrap. 1538 R S.4: 1540 A node which is neither executing a local request nor executing a 1541 short path request signals IDLE messages to its neighbors on the ring 1542 if there is no long path message passing through the node on that 1543 ring. 1545 R S.5: 1547 Protection IPS packets are never wrapped. 1549 R S.6 1551 If the protocol calls for sending both short and long path requests 1552 on the same span (for example if a node has all fibers disconnected), 1553 only the short path request should be sent. 1555 R S.7 1557 A node wraps and unwraps only on a local request or on a short path 1558 request. A node never wraps or unwraps as a result of a long path 1559 request. Long path requests are used only to maintain protection 1560 hierarchy. (Since the long path requests do not trigger protection, 1561 there is no need for destination addresses and no need for topology 1562 maps) 1564 In Figure 19, Node A detects SF (local request/ self-detected 1565 request) on the span between Node A and Node B and starts sourcing 1566 {SF, A, W, S} on the outer ring and {SF, A, W, L} on the inner ring. 1567 Node B receives the protection request from Node A (short path 1568 request) and starts sourcing {IDLE, B, W, S} on the inner ring and 1569 {SF, B, W, L} on the outer ring. 1571 FIGURE 19. SRP IPS Signaling 1573 {SF,A,W,S} 1574 ------------------------------- 1575 | -----X--------------------- | 1576 | | fiber | | 1577 | v cut {IDLE,B,W,S}| v 1578 ----- ----- 1579 | A | | B | 1580 | | | | 1581 ----- ----- 1582 ^ | {SF,A,W,L} i ^ | o {SF,B,W,L} 1583 | | n | | u 1584 | | n | | t 1585 | | e | | e 1586 | v r | v r 1588 10.4. SRP IPS Protocol Rules 1590 R P.1: 1592 Protection Request Hierarchy is as follows (Highest priority to the 1593 lowest priority). In general a higher priority request preempts a 1594 lower priority request within the ring with exceptions noted as 1595 rules. The 4 bit values below correspond to the REQUEST_TYPE field in 1596 the IPS packet. 1598 1101 - Forced Switch (FS) 1599 1011 - Signal Fail (SF) 1600 1000 - Signal Degrade (SD) 1601 0110 - Manual Switch (MS) 1602 0101 - Wait to Restore (WTR) 1603 0000 - No Request (IDLE): Lowest priority 1605 R P.2: 1607 Requests >= SF can coexist. 1609 R P.3: 1611 Requests < SF can not coexist with other requests. 1613 R P.4: 1615 A node always honors the highest of {short path request, self 1616 detected request} if there is no higher long path message passing 1617 through the node. 1619 R P.5: 1621 When there are more requests of priority < SF, the first request to 1622 complete long path signaling will take priority. 1624 R P.6: 1626 A Node never forwards an IPS packet received by it which was 1627 originally generated by the node itself (it has the node's source 1628 address). 1630 R P.7: 1632 Nodes never forward packets with the PATH_INDICATOR set to SHORT. 1634 R P.8 1636 When a node receives a long path request and the request is >= to the 1637 highest of {short path request, self detected request}, the node 1638 checks the message to determine if the message is coming from its 1639 neighbor on the short path. If that is the case then it does not 1640 enter pass-thru and it strips the message. 1642 R P.9: 1644 When a node receives a long path request, it strips (terminates) the 1645 request if it is a wrapped node with a request >= than that in the 1646 request; otherwise it passes it through and unwraps. 1648 R P.10: 1650 Each node keeps track of the addresses of the immediate neighbors 1651 (the neighbor node address is gleaned from the short path IPS 1652 messages). 1654 R P.11: 1656 When a wrapped node (which initially detected the failure) discovers 1657 disappearance of the failure, it enters WTR (user-configurable WTR 1658 time-period). WTR can be configured in the 10-600 sec range with a 1659 default value of 60 sec. 1661 R P.12: 1663 When a node is in WTR mode, and detects that the new neighbor (as 1664 identified from the received short path IPS message) is not the same 1665 as the old neighbor (stored at the time of wrap initiation), the node 1666 drops the WTR. 1668 R P.13: 1670 When a node is in WTR mode and long path request Source is not equal 1671 to the neighbor Id on the opposite side (as stored at the time of 1672 wrap initiation), the node drops the WTR. 1674 R P.14: 1676 When a node receives a local protection request of type SD or SF and 1677 it cannot be executed (according to protocol rules) it keeps the 1678 request pending. (The request can be kept pending outside of the 1679 protection protocol implementation). 1681 R P.15: 1683 If a local non-failure request (WTR, MS, FS) clears and if there are 1684 no other requests pending, the node enters idle state. 1686 R P.16 1688 If there are two failures and two resulting WTR conditions on a 1689 single span, the second WTR to time out brings both the wraps down 1690 (after the WTR time expires a node does not unwrap automatically but 1691 waits till it receives idle messages from its neighbor on the 1692 previously failed span) 1694 R P.17 1696 If a short path FS request is present on a given side and a SF/SD 1697 condition takes place on the same side, accept and process the SF/SD 1698 condition ignoring the FS. Without this rule a single ended wrap 1699 condition could take place. (Wrap on one end of a span only). 1701 10.5. State Transitions 1703 Figure 20 shows the simplified state transition diagram for the IPS 1704 protocol: 1706 FIGURE 20. Simplified State Transitions Diagram 1708 Local FS,SF,SD,MS req. 1709 --------- or Rx{REQ,SRC,W,S} from mate 1710 | IDLE |------------------------------------------- 1711 | |<---------------------------------------- | 1712 --------- Local REQ clears | | 1713 ^ | or Rx{IDLE,SRC,I,S} | | 1714 | | | | 1715 | | | | 1716 | | | | 1717 | | | | 1718 Rx{IDLE,SRC,I,S}| | Rx{REQ,SRC,W,L} | | 1719 | | | | 1720 | | | | 1721 | v Local FS,SF,SD,MS REQ > Active req. | v 1722 --------- or Rx{REQ,SRC,W,S},REQ > Active req. --------- 1723 | PASS |------------------------------------>| WRAPPED | 1724 | THRU |<------------------------------------| | 1725 --------- --------- 1726 Forwards Tx{REQ,SELF,W,S} for local REQ 1727 {REQ,SRC,W,L} Tx{IDLE,SELF,W,S} for mate REQ 1728 & Tx{REQ,SELF,W,L} 1730 Legend: Mate = node on the other end of the affected span 1731 REQ = {FS | SF | SD | MS} 1733 10.6. Failure Examples 1735 10.6.1. Signal Failure - Single Fiber Cut Scenario 1737 Sample scenario in a ring of four nodes A, B, C and D, with 1738 unidirectional failure on a fiber from A to B, detected on B. Ring is 1739 in the Idle state (all nodes are Idle) prior to failure. 1741 Signal Fail Scenario 1743 1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on 1745 FIGURE 21. An SRP Ring with outer ring fiber cut 1747 fiber cut 1748 ---------X----------------------------- 1749 | ----------------------------------- | 1750 | | | | 1751 | v | v 1752 ----- ----- 1753 | A | | B | 1754 | | | | 1755 ----- ----- 1756 ^ | ^ | 1757 o | | i | | 1758 u | | n | | 1759 t | | n | | 1760 e | | e | | 1761 r | | r | | 1762 | v | v 1763 ----- ----- 1764 | D | | C | 1765 | | | | 1766 ----- ----- 1767 | | | | 1768 | | | | 1769 | ----------------------------------- | 1770 --------------------------------------- 1772 both rings (in both directions) 1774 2. B detects SF on the outer ring, transitions to Wrapped 1775 state (performs a wrap), Tx towards A on the inner 1776 ring/short path: {SF, B, W, S} and on the outer ring/long 1777 path: Tx {SF, B, W, L} 1779 3. Node A receives protection request on the short path, 1780 transitions to Wrapped state, Tx towards B on short path: 1781 {IDLE, A, W, S} (message does not go through due to the 1782 failure) and on the long path: Tx {SF, A, W, L} 1784 4. As the nodes D and C receive a switch request, they enter a 1785 pass-through mode (in each direction) which mean they stop 1786 sourcing the Idle messages and start passing the messages 1787 between A an B 1789 5. Steady state is reached 1791 Signal Fail Clears 1793 1. SF on B clears, B does not unwrap, sets WTR timer, Tx {WTR, 1794 B, W, S} on inner and Tx {WTR, B, W, L} 1796 2. Node A receives WTR request on the short path, does not 1797 unwrap, Tx towards B on short path: {IDLE, A, W, S} 1798 (message does not go through due to the failure) and on the 1799 long path: Tx {WTR, A, W, L} 1801 3. Nodes C and D relay long path messages without changing the 1802 IPS octet 1804 4. Steady state is reached 1806 5. WTR times out on B. B transitions to idle state (unwraps) 1807 Tx {IDLE, B, I, S} on both inner and outer rings 1809 6. A receives Rx {IDLE, B, I, S} and transitions to Idle 1811 7. As idle messages reach C and D the nodes enter the idle 1812 state (start sourcing the Idle messages) 1814 8. Steady state it reached 1816 10.6.2. Signal Failure - Bidirectional Fiber Cut Scenario 1818 Sample scenario in a ring of four nodes A, B, C and D, with a 1819 bidirectional failure between A and B. Ring is in the Idle state 1820 (all nodes are Idle) prior to failure. 1822 Signal Fail Scenario 1824 1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on 1825 both rings (in both directions) 1827 2. A detects SF on the outer ring, transitions to Wrapped 1828 state (performs a wrap), Tx towards B on the inner 1829 ring/short path: {SF, A, W, S} and on the outer ring/long 1830 path: Tx {SF, A, W, L} 1832 3. B detects SF on the outer ring, transitions to Wrapped 1833 state (performs a wrap), Tx towards A on the inner 1835 FIGURE 22. An SRP Ring with bidirectional fiber cut 1837 fiber cut 1838 ---------X----------------------------- 1839 | -------X--------------------------- | 1840 | | fiber cut | | 1841 | v | v 1842 ----- ----- 1843 | A | | B | 1844 | | | | 1845 ----- ----- 1846 ^ | ^ | 1847 o | | i | | 1848 u | | n | | 1849 t | | n | | 1850 e | | e | | 1851 r | | r | | 1852 | v | v 1853 ----- ----- 1854 | D | | C | 1855 | | | | 1856 ----- ----- 1857 | | | | 1858 | | | | 1859 | ----------------------------------- | 1860 --------------------------------------- 1862 ring/short path: {SF, B, W, S} and on the outer ring/long 1863 path: Tx {SF, B, W, L} 1865 4. As the nodes D and C receive a switch request, they enter a 1866 pass-through mode (in each direction) which mean they stop 1867 sourcing the Idle messages and start passing the messages 1868 between A an B 1870 5. Steady state is reached 1872 Signal Fail Clears 1874 1. SF on A clears, A does not unwrap, sets WTR timer, Tx {WTR, 1875 A, W, S} towards B and Tx {WTR, A, W, L} on the long path 1877 2. SF on B clears, B does not unwrap. Since it now has a short 1878 path WTR request present from A it acts upon this request. 1879 It keeps the wrap, Tx {IDLE, B, W, S} towards A and Tx 1880 {WTR, B, W, L} on the long path 1882 3. Nodes C and D relay long path messages without changing the 1883 IPS octet 1885 4. Steady state is reached 1887 5. WTR times out on A. A enters the idle state (drops wraps) 1888 and starts transmitting idle in both rings 1890 6. B sees idle request on short path and enters idle state 1892 7. Remaining nodes in the ring enter the idle state 1894 8. Steady state is reached 1896 10.6.3. Failed Node Scenario 1898 FIGURE 23. An SRP Ring with a failed node 1900 --------------------------------------- 1901 | ----------------------------------- | 1902 | | | | 1903 | v | v / 1904 ----- ----/ 1905 | A | | C/| failed 1906 | | | / | node C 1907 ----- -/--- 1908 ^ | /^ | 1909 o | | i | | 1910 u | | n | | 1911 t | | n | | 1912 e | | e | | 1913 r | | r | | 1914 | v | v 1915 ----- ----- 1916 | D | | B | 1917 | | | | 1918 ----- ----- 1919 | | | | 1920 | | | | 1921 | ----------------------------------- | 1922 --------------------------------------- 1924 Sample scenario in a ring where node C fails. Ring is in the Idle 1925 state (all nodes are Idle) prior to failure. 1927 Node Failure (or fiber cuts on both sides of the node) 1929 1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on 1930 both rings (in both directions) 1932 2. Based on the source field of the idle messages, all nodes 1933 identify the neighbors and keep track of them 1935 3. B detects SF on the outer ring, transitions to Wrapped 1936 state (performs a wrap), Tx towards C on the inner 1937 ring/short path: {SF, B, W, S} and on the outer ring/long 1938 path: Tx {SF, B, W, L} 1940 4. A detects SF on the inner ring, transitions to Wrapped 1941 state (performs a wrap), Tx towards C on the outer 1942 ring/short path: {SF, A, W, S} and on the inner ring/long 1943 path: Tx {SF, A, W, L} 1945 5. As the nodes on the long path between A and B receive a SF 1946 request, they enter a pass-through mode (in each 1947 direction), stop sourcing the Idle messages and start 1948 passing the messages between A an B 1950 6. Steady state is reached 1952 Failed Node and One Span Return to Service 1954 Note: Practically the node will always return to service with one 1955 span coming after the other (with the time delta potentially close to 1956 0). Here, a node is powered up with the fibers connected and fault 1957 free. 1959 1. Node C and a span between A and C return to service (SF 1960 between A and C disappears) 1962 2. Node C, not seeing any faults starts to source idle 1963 messages {IDLE, C, I, S} in both directions. 1965 3. Fault disappears on A and A enters a WTR (briefly) 1967 4. Node A receives idle message from node C. Because the long 1968 path protection request {SF, B, W, L} received over the 1969 long span is not originating from the short path neighbor 1970 (C), node A drops the WTR and enters a PassThrough state 1971 passing requests between C and B 1973 5. Steady state is reached 1975 Second Span Returns to Service 1977 The scenario is like the Bidirectional Fiber Cut fault clearing 1978 scenario. 1980 10.6.4. Bidirectional Fiber Cut and Node Addition Scenarios 1982 FIGURE 24. An SRP Ring with a failed node 1984 wrap 1985 ----->|-------------------------------- 1986 | -<--|------------------------------ | 1987 | | | | 1988 | v | v 1989 ----- ---- 1990 | A | | C | Added 1991 | | | | node 1992 ----- ----- 1993 ^ | ^ | 1994 o | | i | | 1995 u | | n | | 1996 t | | n | | 1997 e | | e --- wrap 1998 r | | r ^ | 1999 | v | v 2000 ----- ----- 2001 | D | | B | 2002 | | | | 2003 ----- ----- 2004 | | | | 2005 | | | | 2006 | ----------------------------------- | 2007 --------------------------------------- 2009 Sample scenario in a ring where initially nodes A and B are 2010 connected. Subsequently fibers between the nodes A and B are 2011 disconnected and a new node C is inserted. 2013 Bidirectional Fiber Cut 2015 1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on 2016 both rings (in both directions) 2018 2. Fibers are removed between nodes A and B 2020 3. B detects SF on the outer ring, transitions to Wrapped 2021 state (performs a wrap), Tx towards A on the inner 2022 ring/short path: {SF, B, W, S} and on the outer ring/long 2023 path: Tx {SF, B, W, L} 2025 4. A detects SF on the inner ring, transitions to Wrapped 2026 state (performs a wrap), Tx towards B on the inner 2027 ring/short path: {SF, A, W, S} and on the outer ring/long 2028 path: Tx {SF, A, W, L} 2030 5. As the nodes on the long path between A and B receive a SF 2031 request, they enter a pass-through mode (in each 2032 direction), stop sourcing the Idle messages and start 2033 passing the messages between A an B 2035 6. Steady state is reached 2037 Node C is Powered Up and Fibers Between Nodes A and C are Reconnected 2039 This scenario is identical to the returning a Failed Node to Service 2040 scenario. 2042 Second Span Put Into Service 2044 Nodes C and B are connected. The scenario is identical to 2045 Bidirectional Fiber Cut fault clearing scenario. 2047 11. SRP over SONET/SDH 2049 Although SRP is media independent it is worth noting how SRP is used 2050 with a layer 1 media type. SRP over SONET/SDH is the first media type 2051 perceived for SRP applications. 2053 Flag delimiting on SONET/SDH uses the octet stuffing method defined 2054 for POS. The flags (0x7E) are packet delimiters required for 2055 SONET/SDH links but may not be necessary for SRP on other media 2056 types. End of a packet is delineated by the flag which could also be 2057 the same as the next packet's starting flag. If the flag (0x7E) or 2058 an escape character (0x7D) are present anywhere inside the packet, 2059 they have to be escaped by the escape character when used over 2060 SONET/SDH media. 2062 SONET/SDH framing plus POS packet delimiting allows SRP to be used 2063 directly over fiber or through an optical network (including WDM 2064 equipment). 2066 SRP may also connect to a SONET/SDH ring network via a tributary 2067 connection to a SONET/SDH ADM (Add Drop Multiplexor). The two SRP 2068 rings may be mapped into two STS-Nc connections. SONET/SDH networks 2069 typically provide fully redundant connections, so SRP mapped into two 2070 STS-Nc connections will have two levels of protection. The SONET/SDH 2071 network provides layer 1 protection, and SRP provides layer 2 2072 protection. In this case it is recommended to hold off the SRP Signal 2073 Fail IPS triggers (which correspond to failures which can be 2074 protected by SONET/SDH) for about 100 msec in order to allow the 2075 SONET/SDH network to protect. Only if a failure persists for over 100 2076 msec (indicating SONET/SDH protection failure) should the IPS 2077 protection take place. 2079 Since multiple protection levels over the same physical 2080 infrastructure are not very desirable, an alternate way of connecting 2081 SRP over a SONET/SDH network is configuring SONET/SDH without 2082 protection. Since the connection is unprotected at layer 1, SRP would 2083 be the sole protection mechanism. 2085 Hybrid SRP rings may also be built where some parts of the ring 2086 traverse over a SONET/SDH network while other parts do not. 2088 Connections to a SONET/SDH network would have to be synchronized to 2089 network timing by some means. This can be accomplished by locking 2090 the transmit connection to the frequency of the receive connection 2091 (called loop timing) or via an external synchronization technique. 2093 Connections made via dark fiber or over a WDM optical network should 2094 utilize internal timing as clock synchronization is not necessary in 2095 this case. 2097 12. Pass-thru mode 2099 An optional mode of operation is pass-thru mode. In pass-thru mode, 2100 a node transparently forwards data. The node does not source 2101 packets, and does not modify any of the packets that it forwards. 2102 Data should continue to be sorted into high and low priority transit 2103 buffers with high priority transit buffers always emptied first. The 2104 node does not source any control packets (e.g. topology discovery or 2105 IPS) and basically looks like a signal regenerator with delay (caused 2106 by packets that happened to be in the transit buffer when the 2107 transition to pass-thru mode occurred). 2109 A node can enter pass-thru mode because of an operator command or due 2110 to a error condition such as a software crash. 2112 13. References 2114 [1] ANSI X3T9 FDDI Specification 2116 [2] IEEE 802.5 Token Ring Specification 2118 [3] Bellcore GR-1230, Issue 2, Nov. 1995, "SONET Bidirectional 2119 Line-Switched Ring Equipment Generic Criteria". 2121 [4] ANSI T1.105.01-1995 "Synchronous Optical Network (SONET) 2122 Automatic Protection Switching" 2124 [5] RFC 2615 "PPP over SONET/SDH" 2126 [6] RFC 1662 "PPP in HDLC-like Framing" 2128 14. Security Considerations 2130 As in any shared media, packets that traverse a node are available to 2131 that node if that node is misconfigured or maliciously configured. 2132 Additionally, it is possible for a node to not only inspect packets 2133 meant for another node but to also prevent the intended node from 2134 receiving the packets due to the destination stripping scheme used to 2135 obtain spatial reuse. Topology discovery should be used to detect 2136 duplicate MAC addresses. 2138 15. Patent Notice 2140 The IETF has been notified of intellectual property rights claimed in 2141 regard to some or all of the specification contained in this 2142 document. For more information consult the online list of claimed 2143 rights. 2145 16. Acknowledgments 2147 The authors would like to acknowledge Hon Wah Chin who came up with 2148 the original version of the SRP-fa. Besides the authors, the 2149 original conceivers of SRP include Hon Wah Chin, Graeme Fraser, Tony 2150 Bates, Bruce Wilford, Feisal Daruwalla, and Robert Broberg. 2152 17. Author's Address 2154 Comments should be sent to the authors at the following addresses: 2156 David Tsiang 2157 Cisco Systems 2158 170 W. Tasman Drive 2159 San Jose, CA 95134 2160 Phone: (408) 526-8216 2161 EMail: tsiang@cisco.com 2163 George Suwala 2164 Cisco Systems 2165 170 W. Tasman Drive 2166 San Jose, CA 95134 2167 Phone: (408) 525-8674 2168 EMail: gsuwala@cisco.com 2170 18. Table Of Contents 2172 1. Status of this Memo ........................................... 1 2174 2. Abstract ...................................................... 1 2176 3. Differences between SRP V1 and V2 ............................. 2 2178 4. Terms and Taxonomy ............................................ 3 2179 4.1. Ring Terminology ....................................... 3 2180 4.2. Spatial Reuse .......................................... 4 2181 4.3. Fairness ............................................... 4 2182 4.4. Transit Buffer ......................................... 5 2184 5. SRP Overview .................................................. 6 2185 5.1. Receive Operation Overview ............................. 6 2186 5.2. Transmit Operation Overview ............................ 6 2187 5.3. SRP Fairness Algorithm (SRP-fa) Overview ............... 7 2188 5.4. Intelligent Protection Switching (IPS) Protocol 2189 Overview ..................................................... 7 2191 6. Packet Formats ................................................ 11 2192 6.1. Overall Packet Format .................................. 11 2193 6.2. Generic Packet Header Format ........................... 12 2194 6.2.1. Time To Live (TTL) .............................. 13 2195 6.2.2. Ring Identifier (R) ............................. 13 2196 6.2.3. Priority Field (PRI) ............................ 13 2197 6.2.4. MODE ............................................ 13 2198 6.2.5. Parity Bit (P-bit) .............................. 14 2199 6.2.6. Destination Address ............................. 14 2200 6.2.7. Source Address .................................. 14 2201 6.2.8. Protocol Type ................................... 14 2202 6.3. SRP Cell Format ........................................ 15 2203 6.4. SRP Usage Packet Format ................................ 15 2204 6.5. SRP Control Packet Format .............................. 16 2205 6.5.1. Control Ver ..................................... 17 2206 6.5.2. Control Type .................................... 17 2207 6.5.3. Control TTL ..................................... 18 2208 6.5.4. Control Checksum ................................ 18 2209 6.5.5. Payload ......................................... 18 2210 6.5.6. Addressing ...................................... 18 2211 6.6. Topology Discovery ..................................... 19 2212 6.6.1. Topology Length ................................. 20 2213 6.6.2. Topology Originator ............................. 20 2214 6.6.3. MAC bindings .................................... 21 2215 6.6.4. MAC Type Format ................................. 21 2216 6.7. Intelligent Protection Switching (IPS) ................. 21 2217 6.7.1. Originator MAC Address .......................... 22 2218 6.7.2. IPS Octet ....................................... 22 2219 6.8. Circulating packet detection (stripping) ............... 23 2221 7. Packet acceptance and stripping ............................... 24 2222 7.1. Transmission and forwarding with priority .............. 26 2223 7.2. Wrapping of Data ....................................... 27 2225 8. SRP-fa Rules Of Operation ..................................... 27 2226 8.1. SRP-fa pseudo-code ..................................... 28 2227 8.2. Threshold settings ..................................... 31 2229 9. SRP Synchronization ........................................... 31 2230 9.1. SRP Synchronization Examples ........................... 32 2232 10. IPS Protocol Description ..................................... 33 2233 10.1. The IPS Request Types ................................. 34 2234 10.2. SRP IPS Protocol States ............................... 35 2235 10.2.1. Idle ........................................... 35 2236 10.2.2. Pass-through ................................... 35 2237 10.2.3. Wrapped ........................................ 35 2238 10.3. IPS Protocol Rules .................................... 35 2239 10.3.1. SRP IPS Packet Transfer Mechanism .............. 35 2240 10.3.2. SRP IPS Signaling and Wrapping Mechanism ....... 36 2241 10.4. SRP IPS Protocol Rules ................................ 37 2242 10.5. State Transitions ..................................... 39 2243 10.6. Failure Examples ...................................... 40 2244 10.6.1. Signal Failure - Single Fiber Cut Scenario ..... 40 2245 10.6.2. Signal Failure - Bidirectional Fiber Cut 2246 Scenario ................................................ 42 2247 10.6.3. Failed Node Scenario ........................... 44 2248 10.6.4. Bidirectional Fiber Cut and Node Addition 2249 Scenarios ............................................... 46 2251 11. SRP over SONET/SDH ........................................... 47 2253 12. Pass-thru mode ............................................... 48 2255 13. References ................................................... 49 2257 14. Security Considerations ...................................... 49 2259 15. Patent Notice ................................................ 49 2261 16. Acknowledgments .............................................. 49 2263 17. Author's Address ............................................. 50 2265 18. Table Of Contents ............................................ 51