idnits 2.17.1 draft-ietf-st2-state-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Cannot find the required boilerplate sections (Copyright, IPR, etc.) in this document. Expected boilerplate is as follows today (2024-04-26) according to https://trustee.ietf.org/license-info : IETF Trust Legal Provisions of 28-dec-2009, Section 6.a: This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 2: Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. IETF Trust Legal Provisions of 28-dec-2009, Section 6.b(i), paragraph 3: This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about Internet-Drafts being working documents. ** The document seems to lack a 1id_guidelines paragraph about 6 months document validity. ** The document seems to lack a 1id_guidelines paragraph about the list of current Internet-Drafts. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. ** Bad filename characters: the document name given in the document, 'draft-ietf-st2-state-01.ps', contains other characters than digits, lowercase letters and dash. ** Missing revision: the document name given in the document, 'draft-ietf-st2-state-01.ps', does not give the document revision number == Mismatching filename: the document gives the document name as 'draft-ietf-st2-state-01.ps', but the file name used is 'draft-ietf-st2-state-00' ** The document is more than 15 pages and seems to lack a Table of Contents. == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 1) being 1437 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Abstract section. ** The document seems to lack a Security Considerations section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The document seems to lack an Authors' Addresses Section. ** There are 137 instances of weird spacing in the document. Is it really formatted ragged-right, rather than justified? ** There are 321 instances of too long lines in the document, the longest one being 918 characters in excess of 72. ** There are 214 instances of lines with control characters in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == Line 14 has weird spacing: '...nes for the r...' == Line 66 has weird spacing: '...This section ...' == Line 67 has weird spacing: '...ream is estab...' == Line 68 has weird spacing: '...sources resul...' == Line 69 has weird spacing: '...ay also be un...' == (132 more instances...) == Couldn't figure out when the document was first submitted -- there may comments or warnings related to the use of a disclaimer for pre-RFC5378 work that could not be issued because of this. Please check the Legal Provisions document at https://trustee.ietf.org/license-info to determine if you need the pre-RFC5378 disclaimer. -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? '1' on line 1431 looks like a reference -- Missing reference section? '2' on line 111 looks like a reference Summary: 16 errors (**), 0 flaws (~~), 10 warnings (==), 3 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Internet-Draft November, 1995 2 ST Working Group M. Rajagopal and Sharon Sergeant 3 File: draft-ietf-st2-state-01.ps Expires: May, 1996 5 Internet Stream Protocol Version 2 (ST2) 6 Protocol State Machines - Version ST2+ 8 Status of this Memo 9 This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its Areas and Working Groups. Note that other groups may also distribute working documents as Internet-Drafts. 10 Internet-Drafts are draft documents valid for a maximum of six months. Internet-Drafts may be updated, replaced, or obsoleted by other documents at any time. It is not appropriate to use Internet-Drafts as reference material or cite them other than as "work in progress". 11 To learn the current status of any Internet-Draft, please check the "lid-abstracts.txt" listing contained in the Internet-Drafts Shadow directories on ds.internic.net (US East Coast), nic.nordu.net (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific Rim). 13 Abstract: 14 This memo contains a description of state machines for the revised specification of the Internet STream Protocol Version 2 (ST2). The revised version of ST2 specified in this memo is called ST2+ and described in RFC 1819. The state machines in this document are descriptions of the ST2+ protocol states and message sequences specifications for normal behavior. Exception processsing issues are defined and discussed for protocol compliance and implementation options. 15 Editor's Note: 16 This memo is available both in ASCII format (file: draft-ietf-st2-state-01.txt) and in PostScript (file: draft-ietf-st2-state-01.ps). The PostScript version contains the essential state diagrams and is absolutely required. 18 TABLE OF CONTENTS 20 1. Introduction 5 21 2. ST2 Agent Architecture 8 22 2.1 ST2 Origin, Intermediate and Target Agent Roles 8 23 2.2 Stream Data Flow through an ST2 Agent 9 24 2.3 Origin, Next Hop, Previous Hop and Target Finite State Machines 11 25 2.4 Stream Finite State Machines 13 26 2.4.1 Externally Communicating FSMs 13 27 2.4.2 Internally Communicating FSMs 13 28 2.5 Queues between External Communicating FSMs 16 29 2.6 Queues Inside an Agent 16 30 3. Stream Finite State Machines 18 31 3.1 Assumptions 19 32 3.2 State Machine Model Conventions 19 33 3.2.1 Notations 19 34 3.2.2Transmissions and Receptions 20 35 3.2.3 Application-Initiated Transitions 20 36 3.2.4 Predicates 20 37 3.2.5 Naming conventions 20 38 3.3 Normal Behavior versus Exception Processing 20 39 3.3.1 Individual Stream FSM Issues versus the complete context of the ST2 Agent 40 Stream Databases and the ST2 Agent FSM 20 41 3.3.2 Recovery, Retry and Timer Failures 21 42 3.3.3 Normal Behavior of the Atomic Individual Stream FSMs 23 43 3.3.4 Exception Processing not Fully Covered in Atomic Individual Stream FSMs 23 44 3.4 Individual Stream State Machines 26 45 3.41 Glossary 26 46 3.42 Origin State Machine (OSM) 28 47 3.43 Next Hop State Machine (NHSM) 33 48 3.44 Previous Hop State Machine (PHSM) 38 49 3.45 The Target State Machine (TSM) 41 50 4. ST2 Agent FSMs 44 51 4.1 Control Message Traversal and Agent Database Context 44 52 4.2 ST2 Dispatcher role for incoming Packet-switching, ACKnowledgement and PDU validation 46 53 4.3 ST2 Dispatcher functions for outgoing Packet switching, timer and retry settings 49 54 4.4 Retry FSM - RFSM for datalink reliability of PDU transmissions 50 55 4.5 Agent, Neighbor and Stream Supervision 53 56 4.5.1 The Control FSM for Agent and Stream Supervision 53 57 4.5.2 NFDSM- Neighbor Failure Detection State Machine 54 58 4.5.3 Service Model Interactions 56 59 5. Exception Processing 58 60 Appendix A 61 61 Appendix B 65 63 SECTION\x131 65 Introduction 66 This section gives a brief overview of the ST2 protocol and the protocol FiniteState Machine (FSM) issues addressed in this document. It is assumed that the reader is familiar with the ST2+ Protocol Specification document listed in [1]. Unless otherwise stated ST2 in this document refers to the enhanced ST2 protocol (ST2+). 67 The Internet Stream Protocol, Version 2 (ST2) is a connection-oriented internetworking protocol that operates at the same layer as connectionless IP. An ST2 stream is established as a connection between an Origin sending data to one or more Targets requiring reliable delivery of that data. Each network node, in the path between the Origin and Targets, participates in resource reservation negotiations during stream setup. The resource reservation request is based on a Flow Specification sent by the Origin. The FlowSpec provides a definition of the delivery requirements as either predictive or guaranteed in character. 68 If the reservation negotiations are successful, paths of reserved resources result in a Quality of Service (QOS) reservation for the stream data. This QOS is only established, monitored and maintained by nodes with ST2 Agent capabilities. Each ST2 Agent is called a hop in the ST2 stream routing tree. ST2 Agents that are one hop away from a given node are called Previous-Hops in the upstream direction and Next-Hops in the down stream direction. 69 Data transfer in the ST2 stream is simplex in the downstream direction. As such, a single Origin sending data to many Targets is similar to a media broadcast model. However, each ST2 Agent may simultaneously need to perform Origin, Previous-Hop, Next-Hop and Target functions for a number of different streams. These streams may be part of a conference ( as in the telephone model) or Group of related streams, such that resource reservation and routing issues may be interrelated. The streams may also be unrelated to each other, but ranked by Precedence within an internetwork in the event that limited or changing resources are reallocated. Some streams may request an automatic Recovery option in the event of network failure, Change to the QOS after the original setup or allow Targets to Join the stream, with or without Notifying the Origin. 70 Thus, every ST2 Agent may be required to support a complex web of intersecting streams with competing QOS requirements, changing resource allocations or members. In addition to standard network management, IP routing and packet-switching duties, an ST2 Agent also supports the ST2 protocol and ST2 QOS features for routing, resource management and packet-switching. An ST2 Agent is a packet-switching router, supporting ST2 components in the context of what is known as the ST2 Service Model. 72 Individual implementations of this ST2 Service Model will have various QOS algorithms.These algorithms may be based on the architecture of the underlying router model and the resource management goals of the internetwork topology, as well as other issues concerning interactions between supporting services. Such issues are discussed throughout the ST2+ Specification document. This Protocol State Machine document addresses the ST2 protocol in any ST2 Agent, regardless of the QOS algorithms chosen to support the ST2 Service Model. 73 Stream Control Message Protocol (SCMP) messages form a request-response protocol where the particulars of the Flow Specification, as well as other Protocol Data Unit (PDU) parameters, are interpreted by the chosen QOS Service Model algorithms for routing, Local Resource Management (LRM) and packet-switching. The ST2+ Specification explicitly defines all required and allowable, functions and sequences of SCMP message operations, such that stream FSM behavior can also be explicitly described. 74 Stream setup and tear down has four fundemental states: IDLE, ESTABLISHED, ADD and CHANGE. The basic transition from IDLE to ESTABLISHED is through a CONNECT request with an ACCEPT response. Additional CONNECT and JOIN requests may result in an ADD stream state, while a CHANGE request would result in a CHANGE stream state. DISCONNECT and REFUSE messages may remove one or more Targets while the stream is in any state. 75 Each ST2 Agent maintains state information describing the streams flowing through it. It can actively gather and distribute such information. If, for example, an Intermediate ST2 Agent fails, the neighboring Agents can recognize this via HELLO messages that are periodically exchanged between the ST2 Agents that share streams. STATUS packets can be used to ask other ST2 Agents about a particular stream. These agents then send back a STATUS-RESPONSE message. NOTIFY messages serve to inform ST2 Agents of additional mangement information. The failure of an ST2 neighbor will result in stream DISCONNECTs for all Targets that had an Origin or an upstream Previous-Hop and REFUSEs for all Origins that had downstream Targets or a Next-hop. 76 Each SCMP request-response sequence is defined with next hop ACKnowledgement , ERROR, timeout or retry circumstances. Some SCMP requests also have end-to-end response requirements. This exception processing is designed to resolve incomplete functions during times of network or ST2 Agent failure. QOS Service Model algorithms for packet-switching, routing and LRM services may also initiate exception processing functions in the form of API responses to the ST2 protocol request for services. 77 ST2 Reason Codes are used to inform other ST2 Agents of the source and type of problem such that the correct response sequences will be followed. These Reason Codes are inserted in the appropriate SCMP PDUs and available to the ST2 Agent management functions. Thus an ST2 Agent not only manages the normal request- response protocol between the Origin and Targets of each stream, but also is actively involved in the detection and distribution of error and QOS implications. The fundemental stream state transitions can become somewhat complicated by these ST2 neighbor , Target management and exception processing issues. The ST2 Agent should anticipate, detect and filter conflicting SCMP messages. 78 It is assumed that a sophisticated internetwork would be subject to frequent stream membership changes, network failures, multiple routes and resource allocation activities, requiring interpretation and mangement by each ST2 Agent. Simultaneous, or possibly conflicting, request conditions suggests that SCMP message filtering and FIFO queues be used for all SCMP messages destined for an individual stream. In addition, ST2 Agent management functions and exception processing features can provide a hierarchy to protect the individual stream FSMs from unnecessary complications. 79 The ST2 Agent architecture and FSM models contained in this document have been chosen to illustrate a method for an ST2+ protocol implementation. There are many alternative techniques. Every effort has been made t note the relevant tradeoffs between protocol requirements and implementation choices, suchthat the atomic components of this model may be rearranged to accomodate platform and implementation issues. Every effort has been made to ensure that the described state tables accurately reflect state transitions of the ST2+ version of SCMP, and that the described state diagrams accurately reflect the state transition tables. If there are discrepancies, the tables take precedence over the diagrams and the protocol specification takes precedence over the tables. 81 Outline of rest of this Document: 83 Section 2 : ST2 Agent Architecture 84 Section 3: Stream Finite State Machines 85 Section 4: Agent Finite State Machines 86 Section 5: Exception Processing Issues 87 SECTION\x132 88 ST2 Agent Architecture 89 The architecture of an ST2 Agent for all of the ST2 Finite State Machines (FSMs) is described in this section. The architectural descriptions are necessarily at a high level and are meant to serve as a guide to the protocol implementer. The state machine models are expected to provide the implementer with useful information such as valid message sequences. The ST2+ Specification provides all detailed supporting documentation. 90 2.1 ST2 Origin, Intermediate and Target Agent Roles 91 The following internetwork diagram of ST2 Agents (Figure 1) indicates the Origin (O), Intermediate (I) and Target (T) roles of each ST2 Agent in relation to two ST2 conferences. Conference 1 (C1) has 4 participants, all of which are sending and receiving data from each other. Intermediate Agents I1, I2, I3 and I4 each have an ST2 application with an Origin sending data to all other members, and simultaneously receiving data as a Target for each of the other members. 92 Each ST2 Agent participating in C1, as illustrated, has all four streams to manage, representing a fully meshed stream conference with Targets and Origins communicating along the same paths. Conference 2 (C2) has 3 participants and is also a fully meshed set of streams for each member of the conference. All ST2 Agents in the C2 illustration also have multiple ST2 neighbors, streams and interfaces to manage. In addition, since both conferences are being conducted simultaneously, several Agents are managing streams from both conferences, which may be Grouped for Resource or Fatesharing characteristics. Appendix B includes additional discussions about the possible scenarios that may occur in such an internetwork of conferences. 93 The ST2 FSMs of any individual stream for the Origin, Intermediate and Target ST2 Agents will be separately modeled. However, since each ST2 Agent may be required to manage multiple FSMs for any one stream, as well as all competitive, intersecting streams in the internetwork topology, the interactions of the individual FSMs is very important. Fortunately, ST2 neighbor communications and SCMP exception processing can be used to create a first line of defense for the ST2 Agent individual stream FSMs. Normal operations for individual streams may be protected and simplified. Network errors and conflicting message sequences can be filtered out of the fundemental stream state transitions. These complexities will be addressed in incremental stages as the ST2 Agent architecture and FSMs are discussed. 94 Figure 1. Internetwork of ST2 Conferences 95 2.2 Stream Data Flow through an ST2 Agent 96 In the following diagram(Figure 2) , an ST2 Agent is depicted with an ST2 Dispatcher sending and receiving ST2 PDUs from interface queues (and PDU surrogates from application interfaces), representing a high order ST2 message management scheme. This dispatcher unpacks or forwards incoming PDUs, creates and forwards outgoing PDUs. 97 The forwarding of data or the forwarding of certain command sequences that are not following a negotiatied QOS path (i.e., JOIN and/or JOIN flooding messages) requires a packet-forwarding mechanism, separate from the stream operations that unpack, interpret or create PDUs. 98 The ST2 PDU validation and delivery functions manage information about the the messaging success or failure, i.e. the Retry, timeout, ERROR , or ACK status of messages. This information concerns a datalink reliability that is separate, but not unrelated to, the state of an individual stream. State transitions that occur as a result of time-outs and retransmissions are first taken into consideration as an ST2 neighbor communication and datalink reliability issue. 99 Figure 2. ST2 Agent Architecture 100 Some SCMP FSM message transitions may occur while any individual stream state exists. The ST2 Agent management function is the Control FSM that interprets and sends messages with information relevant to many streams , i.e., HELLO, STATUS, STATUS-RESPONSE , NOTIFY. 101 The filtering of these messages is relevant to the architectural assumptions for the individual stream FSM transitions. Such message transitions and implications are described in detail in Section 4. 102 The first level of filtering is designed to determine whether the incoming PDU (or PDU surrogate) is data or one of the JOIN sequences where the destination is, in fact, another ST2 Agent or an application. Such PDUs are then forwarded to the destination. Data delivery to a resident Target or replication for multiple Next-hop Targets are other issues to consider in a packet forwarding function. The efficient packet switching of ST2 PDUs through Intermediate hops is the main emphasis for this filtering priority. 103 Second-level filtering occurs when an ST2 Agent validates incoming SCMP PDUs and sends the required ACKnowledge (or ERROR PDU, if there are semantic errors) to the originator of the incoming PDU. Conversely, incoming ACKnowledgement and ERROR PDUs trigger the Retry FSM, where all the timeout and retry values are updated or a signal is generated to the appropriate stream FSM for specialized exception processing. 104 All SCMP PDUs, except ACK, ERROR, HELLO, STATUS and STATUS-RESPONSE, require an ACK from the next hop Agent upon receipt. Signaling the implicit occurrence of a datalink failure to an FSM or queueing the requests and responses to the resident stream FSMs occurs only after such protocol message formalities have been accounted for. 105 The Control FSM is primarily concerned with ST2 neighbor status. The implications of these issues with an implementations's QOS services and ST2 auxiliary functions will be discussed in detail in Section 4. 106 Once an ST2 Dispatcher has validated and filtered the PDUs, the individual stream SCMP messages are separately queued into Requests (CONNECT, CHANGE, JOIN) and Responses (ACCEPT, DISCONNECT, JOIN-REJECT, REFUSE). Requests must wait for the completion of any preceding Requests for the same stream, while Responses must be handled immediately without regard to Request state transitions or queues. 107 The convention of discussing issues in terms of individual stream state transistions will be used throughout Section 3 (Stream FSMs) as a way of simplifying the discussion. Section 4 (ST2 Agent FSMs) and Section 5 (Exception Processing) will provide more detail about the architecture, filtering and queues used with the individual stream FSMs, such that network failures can be managed with competing and intersecting streams. Appendix A contains tables providing specific categories of the ST2 message flow characteristics. 108 2.3 Origin, Next Hop, Previous Hop and Target Finite State Machines 109 Discussion surrounding the individual stream state machines most often refers to normal (typical) behavior rather than all pathological cases. Error control and recovery in the architecture is expected to alleviate many problems in the individual stream FSMs, but will be discussed in detail in Section 5. 110 However, the architecture of a particular Agent may be such that ST2 intra-Agent communications are actually between multiple processors, where Next-hop and Previous-hop FSM communications require the concept of multiple ST2 Agents within what might otherwise appear to be one ST2 Agent. As such, the rationale for the approach to filtering and queueing of the SCMP messages , not just the particular method of illustration, is very important to the Finite State Machine Model architecture. 111 Communicating Finite State Machine (CFSM) models have been extensively used in the trade to formally describe protocol behavior [2]. Many variations of the basic CFSM model exist and our model is also a variation of the basic model. Our model uses the basic CFSM model with FIFO queues combined with predicates. The model describes the ST2 protocol behavior and consists of ST2 SCMP messages along with a number of predicates. These predicates are not part of the formal ST2 Protocol specifications but are useful mechanisms that simplify the state machine specifications 112 ST2 Agents - Origin, Intermediate and Target - are all modeled separately using state machines. Because a stream diverges in a tree-like graph, every Intermediate ST2 Agent has to communicate with one upstream ST2 Agent and one or more downstream Agents. Therefore an ST2 Intermediate Agent (Fig. 4) is modeled with 2 or more state machines, one for communicating with an upstream neighbor and a separate state machine for communicating with each downstream neighbor. These machines are called the Previous Hop State Machine (PHSM) and Next Hop State Machine (NHSM) respectively. An Intermediate Agent will therefore have exactly one PHSM and one or more NHSMs for each stream. Note that, it is possible to have more than one NHSM per physical interface, when that interface has more than one Agent on the associated communications link. 113 The state machine model architecture at the Origin is similar to an Intermediate (Fig. 4). The Origin may have one or more NHSMs. There is no PHSM in this case. However, in the place of the PHSM we have a Origin State Machine (OSM) which interfaces with the application layer above it. 114 The Target is modeled with one PHSM (Fig. 4). There are no NHSMs here. However, in the place of a NHSM we have a Target State Machine (TSM) that interfaces with the upper layer application protocol. 115 Because the role of each ST Agent (Origin, Intermediate, or Target) is different, the finite state machine models are not identical. However, the model for communication between FSMs inside or outside an Agent is uniform. 116 Details of the individual stream machine states and transitions are given in Section 3. 117 The ST2 individual stream state model architecture is further described here with the help of an example. 118 Consider a stream topology shown in Figure 3. The figure shows an ST2 Origin (O) connected to 2 Intermediate Agents (I1 and I3). I1 is also connected to I2 and a target T2. I2 is connected to Target T1 and I3 is connected to Target T3. With reference to this example configuration we define a number of state machines. The arrangement of the state machines for this example configuration is shown in Fig. 4. 119 The Origin is modeled with one OSM and 2 NHSMs (one per next hop). Each Target is modeled with one PHSM and one TSM. I1 and I3 are both modeled with one PHSM and 2 NHSMs; I2 is modeled with one PHSM and 2 NHSMs. 120 Figure 3. Example Stream Configuration 121 2.4 Stream Finite State Machines 122 2.4.1 Externally Communicating FSMs 123 Communication between two ST2 agents is External Communication and always happens between a NHSM and a PHSM pair. We note that, in the case of Origin and Target Agent as direct neighbors, it is possible for a Target to be directly connected to an Origin . It is also possible that one Target Agent is an Intermediate Agent for another Target. in which case an Agent will have a PHSM communicating with a TSM and one or more NHSMs. 124 2.4.2 Internally Communicating FSMs 125 Communicating entities inside an ST2 Agent is different for each Agent type, i.e., Origin, Intermediate or Target. However, all FSMs inside an Agent communicate via a Message Separator/Combiner box (MS/C). The function of the MS/C box is described later in this section.. Internal Communication within the Origin occurs: 126 7 between the OSM and an Upper Layer Protocol and 127 7 between the OSM and one or more NHSMs via a MS/C box (Note that the NHSMs themselves do not communicate with each other) 128 Internal Communication within a Target occurs: 129 7 between the TSM and an Upper Layer Protocol and 130 7 between the TSM and a PHSM via a MS/C box 131 Internal Communication within an Intermediate Agent occurs: 132 7 between a PHSM & and one or more NHSMs via a MS/C box (Note that the NHSMs themselves do not communicate with each other) 133 For the example configuration below we have: 134 7 Origin: 135 - an OSM communicating with 2 NHSMs 136 - an OSM communicating with the Upper Layer Protocol 137 7 All Targets: 138 - a TSM communicating with a PHSM 139 - a TSM communicating with the Upper Layer Protocol 140 7 Intermediate Agents: 141 - a PHSM communicating with a NHSM inside both I3 and I2 142 - a PHSM communicating with 2 NHSMs inside I1 143 Figure 4. State Machines at the Origin, Intermediate and Target Agents 144 Figure 5. Implicit Queues between an External Communicating FSM pair 145 2.5 Queues between External Communicating FSMs 146 For the purposes of modelling, we assume that messages are filtered and queued in FIFO queues for the case of external Communicating FSM pairs, i.e. between any two ST2 Agents. However, as indicated in the previous discussion and diagrams of the ST2 dispatcher and filtering hierarchy, it is somewhat more complex in reality. The concept shown in Figure 5 allows the discussion of the inter-Agent and intra-Agent state machines to focus on the basic individual stream issues without regard to message and neighbor management issues. 147 2.6 Queues Inside an Agent 148 Recall that each Agent is modeled with at least 2 state machines for each stream. These state machines also need to communicate just like the external communicating FSM pairs described above. The queue model in this case also requires filtering mechanisms. This model requires a message Separating and Combining function shown as Message Separator/Combiner (MS/C) box in Fig. 6. 149 Figure 6 pictorially describes the multi-stage FIFO queue model for an Agent. Implicit FIFO queues are assumed between the PHSM and the MS/C, and also between the MS/C and one or more NHSMs. Use of such FIFO queues eliminates the need for a separate synchronizing state machine that would normally be required to synchronize the flows . 150 Figure 6. Queues between Internal Communicating FSMs inside an Agent 151 The function of this Message Separator/Combiner box is many: 152 7 Performing a multicasting function by replicating an OSM or PHSM message and sending them to different NHSMs 153 7 Combining messages coming from different TSMs or NHSMs and sending them to the appropriate OSM or PHSM 154 Designing the Agent to contain separate upstream and downstream state machines (PHSM and NHSMs respectively) with FIFO queues as shown in Fig. x, offers several benefits: 155 7 It simplifies the Agent design considerably by separating the neighbor upstream and downstream communications 156 7 Use of FIFO queues simplifies the Agent management since no other synchronization mechanisms need to be used to streamline messages flowing through the Agent. 157 The Agent architecture model is described in more detail in Section 4. 158 SECTION\x133 159 Stream Finite State Machines 160 Each ST2 Agent must maintain state for each stream supported by that Agent. There are many ways to represent the state that must be maintained by Agents. This section presents a reference set of state machines described in diagram and table form. 161 Implementations may support machines based on this section or may even support a completely different set of state machines. In particular, the full implications of the Section 4 (ST2 Agent FSMs) and Section 5 (Exception Processing) have not been detailed in these diagrams and tables. These individual stream FSM represent normal operations of the atomic request-response scenarios. The later sections will discuss the tradeoffs involved in adding the specialized exception processing issues into these atomic FSMs. 162 This section represents stream state through four state machines. The defined machines are: 163 7 The Origin State Machine, or OSM. It represents the state of a stream at the Origin Agent. 164 7 The Next-Hop State Machine, or NHSM. It represents the state of the stream for Targets reached via a particular next-hop. 165 7 The Previous-Hop State Machine, or PHSM. It represents the state of a stream at an Intermediate Agent or a Target Agent. The OSM is essentially a special case of the PHSM, where the delivery of SCMP to the Origin is via an API. 166 7 The Target State Machine, or TSM. It represents the state of a stream for a particular target application at the Target Agent. This state machine is essentially a special case of the NHSM, where there is only a ever a single Target and delivery of SCMP to the Target is via an API. 167 A number of NHSMs related to the same stream, could conceivably all be running in parallel -one for each next hop. In some cases, where there is a network-level multipoint link (e.g., ethernet), it is also possible we may have more than one NHSM associated with the same physical interface. 168 The model assumes that a data engine separate from the control engine exists. 169 A Message Separator/Combiner (MS/C) box separates all downstream messages modifying the Targetlist and placing them in the respective NHSM FIFO queues. The MS/C box also functions as a combiner of messages flowing up stream. In this role it multiplexes all local messages and places them in the PHSM FIFO queues. Note that the MS/C relies on separate routing and LRM functions to determine the appropriate separation since route and resource computation is not part of ST2 protocol. Full-duplex FIFO queues are assumed between the MS/C box and PHSM, and also between the MS/C box and the NHSMs. 170 The multi-machine Agent model breaks the complexity that results with only one gigantic model with the aid of the FIFO queue buffers and a MS/C box. The FIFO queues eliminate the need for a separate synchronizing state machine while reducing the complexity. The MS/C reduces the explicit next-hop identification modelling that would otherwise be required.. 171 The Intermediate Agent PHSM always communicates with a NHSM on the upstream side and the NHSM always communicates with a PHSM on the downstream side. 172 3.1 Assumptions 173 Some basic assumptions were made as part of the development of the enclosed state machines. These included: 174 7 All state machines exist as part of an ST2 Agent and that the Agent will instantiate state machines as needed to represent state on a per stream basis. 175 7 The ST2 Agent implements logic that unpacks incoming SCMP packets, validates the contents, updates the Agent databases and routes the message signal to the appropriate stream and it's associated state machine. 176 7 Detection and handling of messages that are broken, duplicates, or not valid for a particular stream state does not affect stream state and is not represented in the state machines. The mechanisms to prevent such misleading signals to individual state machines will be discussed in the later sections. 177 7 All reliable delivery of intra- and inter-Agent SCMP messages is handled by the ST2 Agent independent of the described state machines except in the case where stream state is dependent on the outcome of the message delivery. 178 7 All communication within the same Agent should follow the same Request Response paradigm as inter-Agent messages in order to be as reliable as SCMP communications. This assumes that all API communications and intra-agent communications, whether with FSM signals or MS/C messages, creates the reliability available with the ACK, timeout and retry paradigms. Options for accomplishing this in the architecture are discussed in the later sections. 179 7 The described state tables accurately reflect state transitions of the ST2+ version of SCMP, and that the described state diagrams accurately reflect the state transition tables. If there are discrepancies, the tables take precedence over the diagrams and the protocol specification takes precedence over the tables. 180 3.2 State Machine Model Conventions 181 3.2.1 Notations 182 Tables show states, events, output, and transitions. 183 Diagrams show states, events, and transitions. Initial states are indicated by an asterisk "*". 184 Messages that trigger events are proceeded by a plus sign "+". 185 Outputs are proceeded by a minus sign "-". 186 Transitions are represented by arrows in the diagrams and by ">>" in the tables. 187 3.2.2 Transmissions and Receptions 188 In all the state machine models, we follow the standard convention of prefixing message transition labels, with a + or - symbol, to explicitly indicate a transmission and reception respectively. The prefixes are not part of the message syntax. In addition, the tables will show both transmitted and received messages, but the diagrams show only recieived messages. This simplifies the diagrams and alleviates some of the complexity of error handling, as discussed in Section 4 (Agent FSMs) and Section 5 (Exception Processing). 189 3.2.3 Application-Initiated Transitions 190 State transitions are sometimes dictated by conditions outside the scope of the protocol specification. Predicates are mechanisms that allow such transitions to occur. For example, terminating a protocol session (a result of many conditions) should allow the Agent to transition to either the initial state or some idle state. This decision is of course Application-initiated but a means should nevertheless exist to allow transitioning to the correct state. In the ST2 protocol there is no message which accomplishes this. Thus, a close predicate is introduced in the FSMs to carry out this function. 191 3.2.4 Predicates 192 Predicates allow a state machine to express conditions and control not explicitly possible with the protocol messages. Generally speaking, they add clarity to the state diagram while reducing the complexity in terms of states. The addition of control predicates allows user defined change of states. These predicates are meant to give hints to the protocol implementer and are not part of the ST2 protocol. For example, a close predicate is implicit in every state machine as a means to transfer control to the Init state. Some triggers and events are combinations of implicit and explicit message conditions. This is particularly true for the timeout and retry mechanisms, as well as the requirement that responses from all Targets in a Request be complete before the Request state can complete. 193 3.2.5 Naming conventions 194 All state names are in bold and start out with a capital letter followed by the lower case letters. All message names are in capitals usually prefixed with a + or - sign. Predicates are in bold and lower case string. 195 3.3 Normal Behavior versus Exception Processing 196 These state models describe the protocol under normal conditions. Detailed error handling will be discussed in both Section 4 and Section 5. Also, control messages STATUS, STATUS-RESPONSE, NOTIFY and ERROR are not shown. Otherwise, if a core message transition is not specified from a state, it implicitly means that this message is not allowed from that state. 197 3.3.1 Individual Stream FSM Issues versus the complete context of the ST2 Agent 198 Stream Databases and the ST2 Agent FSM 199 The OSM, NHSM, PHSM and TSM diagrams and tables in this section do not represent state as the complete context of the stream as it would be defined by all possible combinations of conditions. The G-bit (all targets), the S-bit (stream recovery), the I-bit and E-bit ( change failure-stream teardown) involve combinations of FSM and stream database managment issues. There are multiple dimensions inherent in the partitioning and management of the TargetList in relation to the particula NHSM and ST2 Agent neighbor associated with each Target or set of Targets. Multiple SCMPPDUs for the same transaction or SCMP propagation failures may be due to MTU size limitations. Such issues involve additional data planes of information that define the context of the individual stream FSMs. 200 The diagrams and tables in this section reflect an atomic grouping of the core ST2 stream setup and teardown procedures. This perspective delineates three classes of Responses. The first class is a Response that does not have any signifigance for state change, where these Responses are not specifically either the last one required to complete the TargetList of the current Request, nor a deletion of the last of all Targets associated with that stream's FSM. Completion of the Responses for the current TargetList is the second class. Removal of the last of all Targets for that stream's FSM is the third class.All api references are illustrative and are not intended to fully define the application interface. 201 Class 1 Responses: api_accept, api_disconnect, api_refuse, ACCEPT, DISCONNECT, REFUSE, and RetryTimeout indicate that an individual TargetList member has signaled a Response. api_disconnect, api_refuse, DISCONNECT and REFUSE may also be a Request to delete a Target( whether or not it is in the current TargetList of an Add or Change state transaction). Individual and/or Global Target deletion may occur at any time, but any Global (G-bit set) Target Response or deletion Request falls into one of the second two classes. 202 Class 2 Responses:api_accept_last, api_disconnect_last, api_refuse_last, ACCEPT_LAST, DISCONNECT_LAST and REFUSE_LAST, RetryTimeout_last, E2E_Timeout_last are only relevant to the current stream Request and refer to the completion of the Request state by occurring as the Response that incidently completes the TargetList . 203 Class 3 responses:api_disconnect_all, api_refuse_all, DISCONNECT_ALL and REFUSE_ALL refer to the Requests or Responses that remove the last active Target from that FSM for that stream. 204 These classes represent the delineation of the asynchronous Request/Response activity that may occur. Network changes or conditions may result in interruptions of any individual stream stream FSM operations. The ST2 Agent Retry FSM, Control FSM, Routing function or LRM may initiate stream teardown for any Target or TargetList at any point. The OSM, NHSM, PHSM and TSM diagrams and tables in this section define stream state as it relates to atomic setup and teardown functions. Every attempt has been made to delineate the atomic SCMP request-response specifications such that implementors may reorganize the Agent architecture to address implementation-specific issues. 205 3.3.2 Recovery, Retry and Timer Failures 206 The following table provides a quick reference for ST2 recovery and retry implications across ST2 Agent FSMs as atomic conditions of the request-response specifications. These conditions are explicit in that an ST2 Agent neighbor ACK will terminate the neighbor ACK timer and retry count for that transaction. Any allowable End to End response will also terminate the neighbor ACK timer and retry count for that transactiont, as well as the End to End response timer. Implicit conditions occur when any of the timer or the retry count values have been exhausted. The general paradigm is an implicit REFUSE for unsatisfied downstream requests and an implicit DISCONNECT for unsatisfied upstream requests. The secondary consequence of a timeout is that explicit REFUSE and DISCONNECT messages may also be issued. 207 However, each table entry below has its own variation of the basic paradigm. In addition, the ST2 specification indicates many secondary and tertiary implications for SCMP message failures. As a particular example, once any ST2 Agent has completed a downstream request-response scenario, any upstream propagation problem may or may not cause the stream to be torn down. The I-bit in CHANGE processing and the S-bit in all transactions are examples of causes for the secondary and tertiary implications. 209 Table 1: 210 ST2 Agent FSM 211 SCMP 212 message 213 Nbr ACK timer 214 Nbr Retry count 215 End to End explicit Responses 216 End to End 217 Response 218 timer 219 Retry 220 ACCEPT 221 ToAccept 222 NAccept 223 ACK 224 Retry 225 CHANGE 226 ToChange 227 NChange 228 ACCEPT/REFUSE 229 ToChangeResp 230 Retry 231 CONNECT 232 ToConnect 233 NConnect 234 ACCEPT/REFUSE 235 ToConnectResp 236 Retry 237 DISCONNECT 238 ToDisconnect 239 Ndisconnect 240 ERROR 241 Retry 242 JOIN 243 ToJoin 244 NJoin 245 CONNECT/JOIN-REFUSE 246 ToJoinResp 247 Retry 248 JOIN-REJECT 249 ToJoinReject 250 NJoinReject 251 Retry 252 NOTIFY 253 ToNotify 254 NNotify 255 Retry 256 REFUSE 257 ToRefuse 258 NRefuse 259 Control -NRetryRoute 260 RecoveryCONNECT 261 ToConnect 262 NConnect 263 ACCEPT/REFUSE 264 ToRetryRoute 265 Control -NStatus 266 STATUS 267 STATUS-RESPONSE 268 ToStatusResp 269 Control - 270 HellotimerHolddown 271 HELLO 272 implicit not explicit ACK - DefaultRecoveryTimeout 273 implicit not explicit retry -HellolossFactor 275 3.3.3 Normal Behavior of the Atomic Individual Stream FSMs 276 The exception processing issues that are most represented in this section include Reason Codes: 277 1 NoError No error has occurred. 278 5 ApplAbort The application aborted the stream abnormally. 279 6 ApplDisconnect The application closed the stream normally. 280 7 ApplRefused Applications refused requested connection or change. 281 25 JoinAuthFailure Join failed due to stream authorization level. 282 The following Reason Codes are illustrated for Requests (CONNECT, CHANGE), but not Responses propagations that would result in special exception processing (i.e, ACCEPT-ACK, DISCONNECT-ACK, REFUSE-ACK or JOIN-REJECT-ACK failures):, 283 38 ResponseTimeout Control message has been acknowledged but not 284 answered by an appropriate control message. 285 41 RetransTimeout An acknowledgment has not been received after 286 several retransmissions. 287 3.3.4 Exception Processing not Fully Covered in Atomic Individual Stream FSMs 288 The following management and exception processing issues are to be addressed in detail in Sections 4 and 5: 289 Retry and Timeout Failures for Responses (e.g., an ACCEPT-ACK, DISCONNECT-ACK, JOIN-REJECT-ACK, REFUSE-ACK) : 290 38 ResponseTimeout Control message has been acknowledged but not 291 answered by an appropriate control message. 292 41 RetransTimeout An acknowledgment has not been received after 293 several retransmissions. 294 ST2 Dispatcher, ACKknowledgement and ERROR functions with Reason Codes: 295 4 AckUnexpected An unexpected ACK was received. 296 13 CksumBadCtl Control PDU has a bad message checksum. 297 14 CksumBadST PDU has a bad ST Header checksum. 298 15 DuplicateIgn Control PDU is a duplicate and is being acknowledged. 299 16 DuplicateTarget Control PDU contains a duplicate target, or an attempt to add an existing target. 300 23 InvalidSender Control PDU has an invalid SenderIPAddress field. 301 24 InvalidTotByt Control PDU has an invalid TotalBytes field. 302 26 LnkRefUnknown Control PDU contains an unknown LnkReference. 303 31 OpCodeUnknown Control PDU has an invalid OpCode field. 304 32 PCodeUnknown Control PDU has a parameter with an invalid PCode. 305 33 ParmValueBad Control PDU contains an invalid parameter value. 306 35 ProtocolUnknown Control PDU contains an unknown next-higher 307 layer protocol identifier. 308 36 RecordRouteSize RecordRoute parameter is too long to permit 309 message to fit a network's MTU. 310 37 RefUnknown Control PDU contains an unknown Reference. 311 45 SAPUnknown Control PDU contains an unknown next-higher 312 layer SAP (port). 313 46 SIDUnknown Control PDU contains an unknown SID. 314 48 STVer3Bad A received PDU is not ST Version 3. 315 49 StreamExists A stream with the given SID already exists. 316 51 TargetExists A CONNECT was received that specified an 317 existing target. 318 52 TargetUnknown A target is not a member of the specified stream. 319 53 TargetMissing A target parameter was expected and is not 320 included, or is empty. 321 54 TruncatedCtl Control PDU is shorter than expected. 322 55 TruncatedPDU A received ST PDU is shorter than the ST Header 323 indicates. 324 56 UserDataSize UserData parameter too large to permit a 325 message to fit into a network's MTU. 326 Control FSM issues with neighbor failure and stream recovery with Reason Codes: 327 12 CantRecover Unable to recover failed stream. 328 22 IntfcFailure A network interface failure has been detected. 329 27 NetworkFailure A network failure has been detected. 330 39 RestartLocal The local ST agent has recently restarted. 331 40 RestartRemote The remote ST agent has recently restarted. 332 47 STAgentFailure An ST agent failure has been detected. 333 Routing issues with Reason Codes: 334 9 BadMcastAddress IP Multicast address is unacceptable in CONNECT 335 28 NoRouteToAgent Cannot find a route to an ST agent. 336 29 NoRouteToHost Cannot find a route to a host. 337 30 NoRouteToNet Cannot find a route to a network. 338 34 PathConvergence Two branches of the stream join during the 339 CONNECT setup. 340 42 RouteBack Route to next-hop through same interface as 341 previous-hop and is not previous-hop. 342 43 RouteInconsist A routing inconsistency has been detected. 343 44 RouteLoop A routing loop has been detected. 344 LRM issues with Reason Codes: 345 10 CantGetResrc Unable to acquire (additional) resources. 346 11 CantRelResrc Unable to release excess resources. 347 17 FlowSpecMismatch FlowSpec in request does not match 348 existing FlowSpec. 349 18 FlowSpecError An error occurred while processing the FlowSpec. 350 19 FlowVerUnknown Control PDU has a FlowSpec Version Number that 351 is not supported. 352 20 GroupUnknown Control PDU contains an unknown Group Name. 353 21 InconsistGroup An inconsistency has been detected with the 354 streams forming a group. 355 50 StreamPreempted The stream has been preempted by one with a 356 higher precedence. 357 Miscellaneous errors with Reason Codes: 358 2 ErrorUnknown An error not contained in this list has been 359 detected. 360 3 AccessDenied Access denied. 361 8 AuthentFailed The authentication function failed. 362 3.4 Individual Stream State Machines 363 3.4.1 Glossary 364 All individual stream FSMs have the following 4 states in common: 365 Init: The stream is not active. 366 Establd: The stream is established and may or may not have Target members. 367 Add: The stream is currently adding Targets as the result of a Connect of Join initiated Connect. 368 Change: The stream is currently attempting to Change according to the new FlowSpec. 369 A list of predicates, api interactions and combination conditions include the following: 370 api_close - the Origin api explicitly terminates a stream, since a stream with no Targets at the Origin may remain Established 371 api_open - the Origin api explicitly establishes a stream to initiate all database setup functions whether or not any Targets are initially specified. 372 api_connect- the Origin api adds Targets. 373 api_change - the Origin api initiates a CHANGE to the FlowSpec. 374 api_disconnect - the Origin api initiates a DISCONNECT to Targets. 375 accept_api - the OSM propagates an ACCEPT received from either a TSM or a NHSM to the Origin api. 376 notify_api - the OSM propagates a NOTIFY received from either a TSM or a NHSM to 377 the Origin api. 378 refuse_api - the OSM propagates a REFUSE received from either a TSM or a NHSM to 379 the Origin api. 380 nexthop_open - the first time each unique NHSM is invoked for each unique stream in an Agent, the Agent explicitly establishes a NHSM database and Establd state. 381 prevhop_open - the first time the PHSM is invoked for each unique stream in an Agent, the Agent explicitly establishes a PHSM database and Establd state. 382 api_join - the Target api initiates a JOIN request. 383 api_refuse - the Target api initiates a REFUSE to the TSM. 384 api_refuse_change - the Target api initiates a REFUSE of a CHANGE request to the TSM. 385 connect_api - the TSM propagates a CONNECT to the Target api. 386 change_api - the TSM propagates a CHANGE to the Target api. 387 join_reject_api - the TSM propagates a JOIN_REJECT to the Target api. 388 disconnect_api - the TSM propagates a DISCONNECT to the Target api. 389 JOIN_AUTH - a PHSM or OSM JOIN is authorized. 390 JOIN_NOT_AUTH - a PHSM or OSM JOIN is not authorized. 391 CH_DISC - a CHANGE failure when the I-bit is set results in stream teardown as an enforced DISCONNECT downstream. 392 CH_REF - a CHANGE failure when the I-bit is set results in stream teardown as an enforced REFUSE upstream. 393 RetryTimeout - an FSM recieves an implicit REFUSE response to a CONNECT or CHANGE request to one Target in the TargetList by exceeding the ACK retry and timeout values (i.e., ToChange/NChange, ToConnect/NConnect timers and retry counts) for that particular transaction. 394 The Add and Change states cannot transition back to the Establd state until all Targets have given implicit or explicit responses. 395 ACCEPT_LAST - the last Target in the TargetList for a CONNECT or CHANGE has responded with an ACCEPT. 396 DISC_LAST - the last Target in the TargetList for a CONNECT or CHANGE has responded with a DISCONNECT. 397 REFUSE_LAST - the last Target in the TargetList for a CONNECT or CHANGE has responded with a REFUSE. 398 RetryTimeout_last - the last Target in the TargetList for a CONNECT or CHANGE has responded with an implicit REFUSE by exceeding the ACK retry and timeout values (i.e., ToChange/NChange, ToConnect/NConnect timers and retry counts). 399 E2E_Timeout_last - the last Target in the TargetList for a CONNECT or CHANGE has responded with an implicit REFUSE by exceeding the End-to-End timeout value (i.e., ToChangeResp, ToConnectResp timers). 400 Except in the case of the OSM (which must be explicitly closed by the Origin api), the Establd, Add and Change states cannot transition back to the Init state until all Targets in the unique FSM TargetList have given implicit or explicit stream teardown instructions. 401 DISC_ALL -a DISCONNECT has been received for the last Target in the entire TargetList for a stream FSM (as opposed to the TargetList for a particular CONNECT or CHANGE request). 402 REFUSE_ALL - a REFUSE has been received for the last Target in the entire TargetList for a stream FSM (as opposed to the TargetList for a particular CONNECT or CHANGE request). 403 3.4.2 Origin State Machine (OSM) 404 The Origin State Machine (OSM) is a high level state machine which communicates with one or more NHSMs. The OSM also talks to the Upper Layer Protocol via primitives. This OSM to Upper Layer Interface is outside the scope of this document and the draft assumes that such an Interface exists but does not specify the Interface. All ST2 Dispatcher and MS/C Box diagrams have indicated that API messages could be included. The actual mechanism used for API communications should be decided by implementation factors. 405 The OSM consists of a small number of states: Init, Establd, Add and Change. 406 Init: The initial state is called Init. An api_open predicate moves the control to the Establd state. An api_close is required to return the stream to the Init state. 407 Establd: The Establd state is the stable state from which all api_connect, JOIN and api_change requests may cause a transition to the Add or Change states. All CONNECT, JOIN and CHANGE requests that occur while a stream is in either an Add or Change state will be queued up until the stream returns to the Establd state. Data transfer may occur to established Targets. The removal of Targets from previous operations or current operations may occur in the Establd, Add or Change states with an api_disconnect or a REFUSE. 408 It is possible for an Application at the Origin to add new Targets to an existing stream any time after the stream has been established. A JOIN message received by an OSM indicates that the Origin Agent happens to be the first Agent for that stream in the path between the JOIN originator and the Origin. JOIN messages from potential Targets require the authorization process to determine if the JOIN will be allowed. The OSM (or PHSM in the case of an Intermediate Agent) then issues either a -JOIN-REJECT message or a -CONNECT message. Issuing a JOIN-REJECT brings it back to the Establd state. Issuing a -CONNECT message moves the control to the Add state awaiting an ACCEPT or REFUSE response. 409 Figure 7. Origin State Machine (OSM) 411 Table 2: OSM 412 OSM 413 Init 414 Estbld 415 Add 416 Change 417 +ACCEPT 418 - 419 - 420 >> Self 421 -accept_api 422 >> Self 423 -accept_api 424 +ACCEPT_LAST 425 - 426 - 427 >> Estbld 428 -accept_api 429 >> Estbld 430 -accept_api 431 +JOIN_AUTHORIZED 432 - 433 >> Add 434 -CONNECT 435 (-notify_api) 436 >> Self 437 -queue 438 >> Self 439 -queue 440 +JOIN_NOT_AUTHORIZED 441 - 442 >> Self 443 -JOIN_REJ 444 -notify_api 445 >> Self 446 -queue 447 >> Self 448 -queue 449 +REFUSE 450 - 451 >> Self 452 -refuse_api 453 >> Self 454 -refuse_api 455 >> Self 456 -refuse_api 457 (-CH_DISC) 458 +REFUSE_LAST 459 - 460 - 461 >> Estbld 462 -refuse_api 463 >> Estbld 464 -refuse_api 465 (-CH_DISC) 466 +api_open 467 >> Estbld 468 - 469 - 470 - 471 +api_change 472 - 473 >> Change 474 -CHANGE 475 >> Self 476 -queue 477 >> Self 478 -queue 479 +api_connect 480 - 481 >> Add 482 -CONNECT 483 >> Self 484 -queue 485 >> Self 486 -queue 487 +api_disconnect 488 - 489 >> Self 490 -DISCON 491 >> Self 492 -DISCON 493 >> Self 494 -DISCON 495 +api_close 496 - 497 >> Init 498 -DISCON 499 >> Init 500 -DISCON 501 >> Init 502 -DISCON 503 +RetryTimeout 504 - 505 - 506 >>self 507 -DISCON 508 -refuse_api 509 >>self 510 -DISCON 511 -refuse_api 512 (-CH_DISC) 513 +RetryTimeout_last 515 >> Estbld 516 -DISCON 517 -refuse_api 518 >> Estbld 519 -DISCON 520 -refuse_api 521 (-CH_DISC) 522 +E2ETimeout_last 523 - 524 - 525 >> Estbld 526 -DISCON 527 -refuse_api 528 >> Estbld 529 -DISCON 530 -refuse_api 531 (-CH_DISC) 532 An ST2 Agent processes a JOIN request when that Agent has the designated stream active in its databases and FSMs. The authorization level associated with the stream is examined to determine whether to transition from the Establd state to the Add state. 533 7 level 0 (JOIN_NOT_AUTHORIZED) JN bits=00 534 It is not allowed to join the stream. No further actions are taken. 535 7 level 1 (JOIN_AUTHORIZED, NOTIFY Origin)JN bits=01 536 The ST2 Agent sends a CONNECT message with a TargetList containing the Target that requested to join the stream. This results in adding the Target to the stream. When the ST2 Agent which is already part in the stream receives the ACCEPT message indicating that the new Target has been added, it does not propagate the ACCEPT message backwards. Instead, it issues a NOTIFY message with ReasonCode(TargetJoined) to inform the Origin of the new Target. 537 7 level 2(JOIN_AUTHORIZED)JN bits=10 538 The ST2 Agent sends a CONNECT message with a TargetList containing the Target that requested to join the stream. When the ST2 Agent which is already part in the stream receives the ACCEPT message indicating that the new Target has been added, it does not propagate the ACCEPT message backwards (in the OSM an accept_api), nor does it notify the Origin (notify_api). 539 The Origin also checks that 540 1. the SID is valid 541 2. the Targets are not already members of the stream 542 3. the FlowSpec of the new Target, if present, matches the FlowSpec of the existing stream, i.e it requires an equal or smaller amount of resources to be allocated. If the FlowSpec of the new target does not match the FlowSpec of the existing stream, it is simply ignored. 543 If this validation is complete and the stream JOIN option allows authorization to be completed,the ST2 Agent at the Origin transitions to the Add state and then issues a CONNECT message that contains the SID, the FlowSpec, and the TargetList specifying the new Targets. 544 If this is not the case, a JOIN-REJECT message is sent to the Target with the appropriate ReasonCode (e.g., JoinAuthFailure, DuplicateTarget or RouteLoop). 545 Add:Once in the Establd state the api may issue an api_connect. A transition to Add will create a CONNECT message that is placed in the FIFO queue between the OSM and the MS/C box. The CONNECT message contains the SID, an updated FlowSpec, and a TargetList. The MS/C box will then make a copy of the CONNECT message, partition the Targetlist parameter and place it the NHSMs queues.The spliting (or separating) information is derived from the implementation's routing and LRM functions. 546 This model has placed the routing and LRM functions in the MS/C box, such that OSM (or PHSM in the case of an Intermediate Agent) may find that the Response has been initiatedf by either the local functions or a remote Agent. 547 If multiple next-hops are to be reached through a network that supports network level multipoint (e.g., an ethernet link), a different CONNECT message must nevertheless be sent to each next-hop since each will have a different TargetList. In this case, we have one physical interface but more than one or more logical NHSMs associated with this interface. 548 Once in the Add state the OSM waits to get ACCEPT or REFUSE responses. The stream will not transition back to the Establd state until all Targets have responded. The expiration of the retry timer and count (if the next hop is not ACKing the request) or the expiration of the end-to-end timer will be interpreted as an implicit REFUSE. 549 The OSM will record the status of each response from each Target. As each ACCEPT is received, the OSM updates its database and records the status of each Target and the resources that were successfully allocated along the path to it, as specified in the FlowSpec contained in the ACCEPT message. The Application may then use the information to either adopt or terminate the portion of the stream to each Target. When either an ACCEPT or REFUSE ( explicit or implicit by timeout failure) from all Targets has been received at the Origin, the stream state returns to Establd and any additional queued up requests may then be processed. 550 When an ACCEPT is received by the OSM, the path to the Target is considered to be established and the ST2 Agent is allowed to forward the data along this path. When a REFUSE reaches the OSM, the OSM notifies the Application that the Target is no longer part of the stream. If there are no remaining Targets, the Application may wish to terminate the stream or keep the stream active to allow stream joining. 551 To be fairly sure that all Targets receive the data with the desired quality of service, an Application should send the data only after the whole stream has been established. Depending on the local API, an Application may not be prevented to send data before the completion of all stream Targets. 552 For each new Target in the TargetList, processing is much the same as for the original CONNECT. The CONNECT is acknowledged, propagated, and network resources are reservedHowever, it may be possible to route to the new Targets using previously allocated paths or an existing multicast group. In that case, additional resources do not need to be reserved but more next-hops might have to be added to an existing multicast group. Intermediate or Target ST2 Agents that are not already nodes in the stream behave as in the case of stream setup. 553 The OSM may issue a -DISCONNECT as a result of an api_disconnect. This message may be processed in any state. The OSM then records this fact and appropriately updates its database. 554 A +REFUSE message may arrive at the OSM asynchronously at any time.This message is sent as a result of an intermediate Agent failure or a Target leaving a stream. The database update processing that occurs after receiving a +REFUSE is the same as that which occurs after sending a -DISCONNECT message. Therefore, sending a -DISCONNECT or receiving a +REFUSE leads to the same state transitions. 555 Change:The Application at the Origin may wish to change the FlowSpec of an established stream. To do so, it informs the ST2 Agent at the Origin of the new FlowSpec and of the list of Targets relative to the change with an api_change. The Origin then issues one CHANGE message with the new FlowSpec per next-hop and sends it to the relevant next-hop Agents. The control flow to the Change state is very similar to the control to the Add state from the Establd state. Depending on the CHANGE options selected and the resources avalailable in each of the stream paths, the CHANGE may result in either a simple refusal of any change or the disconnect of the entire stream. A REFUSE response to a CHANGE request may simply mean that the stream for that Target is unchanged, depending on the I-bit setting. In all other cases, a REFUSE results in the teardown of the stream for that TargetList. 556 3.4.3 Next Hop State Machine (NHSM) 557 The NHSM is pictorially shown in Figure 8. This model is common to the Origin as well as an Intermediate Agent .The NHSM consists of the same fundamental states as the OSM: Init, Establd, Add and Change. 558 Init:The state machine for each next hop enters its Init state at Agent start-up time. A dot indicates that this is the initial state. A nexthop_open predicate moves control to the Establd state when the next hop associated with an NHSM is required by Targets in a stream. 559 Establd:Once in the Establd state a number of things can happen. Targets may be added by the Origin or Targets may request to join the stream. However, the processing of a JOIN request is always handled by either an OSM or a PHSM. Within each ST2 Agent, the ST2 Dispatcher examines incoming JOIN requests and determines whether the stream referenced is a stream that that Agent supports. If not, the JOIN is forwarded on towards the Origin. Once a JOIN request reaches an Agent that can process the JOIN, the ST2 Dispatcher ACKs the JOIN and queues it up to the resident OSM or PHSM. The NHSM only sees the resultant CONNECT when stream authorization has completed successfully and the OSM or PHSM has issued a CONNECT through the MS/C Box. 560 As previously described in the OSM, an ST2 Agent can handle only one stream Add or Change at a time. If such a stream operation is already underway, further requests are queued and handled when the previous operation has been completed. Either a DISCONNECT or REFUSE for all Targets transfers control from the Establd state to the Init state. 561 Add: A CONNECT that has been propagated from the NHSM Add state to the next hop Agent PHSM and will require a response in the form an ACK . If not, the timeout and retry mechanisms of the Retry FSM will invoke an implicit refuse, a RetryTimeout signal. Every PDU has a unique reference number, so that all ACKs may be matched to the appropriate Request or Response. 562 The ACK, if it is received, does not need to be reported to the NHSM. However, if the ACK is not received and the retries are exhausted, a RetryTimeout signal will be reported to the NHSM and interpreted as a REFUSE. Otherwise, the NHSM will record all Target explicit responses until the last Target in the TargetList has sent an ACCEPT or REFUSE (or an implicit REFUSE due to Retry exhaustion or a DISCONNECT from the Origin or a Control FSM). 563 An Origin DISCONNECT may be due to the exhaustion of the end-to-end timer. A DISCONNECT signal from a Control FSM may be due to the failure of an upstream or a downstream nieghbor. 564 The Application at the Origin may specify a set of Targets that are to be removed from the stream with an appropriate ReasonCode (ApplDisconnect). The Targets are partitioned by the MS/C box into multiple DISCONNECT messages based on the next-hop to the individual Targets. If the TargetList is too long to fit into one DISCONNECT message, it is partitioned. 566 Figure 8. Next Hop State Machine (NHSM) 567 If, after deleting the specified Targets, any next-hop has no remaining Targets, then those resources associated with that next-hop agent may be released. Note that the network resources may not actually be released if network multicasting is being used since they may still be required for traffic to other next-hops in the multicast group. 568 When the DISCONNECT reaches a Target, the Target sends an ACK to the upstream NHSM and notifies the Application (at target) that it is no longer part of the stream and for which reason. The ST2 Agent at the Target deletes the stream from its database after performing any necessary management and accounting functions. Note that the stream is not deleted if the ST2 Target Agent is also an Intermediate Agent for the stream and there are remaining downstream Targets. 569 The ST2 Dispatcher and MS/C Box process the PDU, PDU surrogates and api message contents into a common database within the Agent, such that the Agent OSM/PHSM and NHSM/TSM pairs may be updated with simultaneous signals, rather than waiting for the signal to be propagated by the individual stream FSMs through the MS\C Box. The tradeoff of such implementation choices revolves around exception processing conditions. The basic states in each of the FSM models are very similar. Thus in the following table, all messages sent (-) by the NHSM are explicitly noted, even though there is an opportunity for such signalling to be managed more simply or simultaneously through the architecture. The filtering and exception processing mechanisms will be discussed in more detail in a later section. 571 NHSM 572 Init 573 Estbld 574 Add 575 Change 576 +nexthop_open 577 >>Estbld 578 - 579 - 580 - 581 +ACCEPT 582 - 583 - 584 >> Self 585 -ACCEPT 586 >> Self 587 -ACCEPT 588 +ACCEPT_LAST 589 - 590 - 591 >> Estbld 592 -ACCEPT 593 >> Estbld 594 -ACCEPT 595 +CHANGE 596 - 597 >> Change 598 -CHANGE 599 >> Self 600 -queue 601 >> Self 602 -queue 603 +CONNECT 604 - 605 >> Add 606 -CONNECT 607 >> Self 608 -queue 609 >> Self 610 -queue 611 +DISCONNECT 612 - 613 >> Self 614 -DISCON 615 >> Self 616 -DISCON 617 >> Self 618 -DISCON 619 +DISCONNECT_LAST 620 - 621 - 622 >>Establd 623 -DISCON 624 >>Establd 625 -DISCON 626 +DISCONNECT_ALL 627 - 628 >>Init 629 -DISCON 630 >>Init 631 -DISCON 632 >>Init 633 -DISCON 634 +REFUSE 635 - 636 >> Self 637 -REFUSE 638 >> Self 639 -REFUSE 640 >> Self 641 -REFUSE 642 (-CH_DISC) 643 (-CH_REF) 644 +REFUSE_LAST 645 - 646 - 647 >> Estbld 648 -REFUSE 649 >> Estbld 650 -REFUSE 651 (-CH_DISC) 652 (-CH_REF) 653 +REFUSE_ALL 654 - 655 >>Init 656 -REFUSE 657 >>Init 658 -REFUSE 659 >>Init 660 -REFUSE 661 (-CH_DISC) 662 (-CH_REF) 663 +RetryTimeout 664 - 665 - 666 >> Self 667 -REFUSE 668 >> Self 669 -REFUSE 670 (-CH_DISC) 671 (-CH_REF) 672 +RetryTimeout_Last 673 - 674 - 675 >> Estbld 676 -REFUSE 677 >> Estbld 678 -REFUSE 679 (-CH_DISC) 680 (-CH_REF) 681 Table 3: NHSM 682 The CONNECT message contains the SID, an updated FlowSpec, and a TargetList. In general, the FlowSpec and TargetList depend on both the next-hop and the intervening network. Each TargetList is a subset of the original TargetList, identifying the targets that are to be reached through the next-hop to which the CONNECT message is being sent. If the TargetList causes a too long CONNECT message to be generated, the CONNECT message is partitioned. 683 Each ST agent knows the MTU of the networks to which it is connected, and those MTUs restrict the size of the SCMP message it can send. SCMP messages with long TargetList can cause the size of the SCMP message to exceed the network MTU. The ST agent which receives an SCMP message bigger than its MTU must break the original message into multiple fragments, each carrying part of the TargetList. If the original SCMP message contains any Userdata parameters, these parameters are replicated in each fragment for delivery to all targets. Applications that support a large number of receivers may avoid using long target lists by exploiting the stream joining functions. 684 If an Application at a Target does not wish to participate in the stream, it sends a REFUSE message back to the Origin with a ReasonCode (ApplDisconnect). When an NHSM receives a REFUSE message with ReasonCode (ApplDisconnect), the acknowledgement has already been sent by the ST2 Dispatcher as an ACK to the next-hop. The Agent considers which resources are to be released, deletes the Target entry from the internal database, and propagates the REFUSE message back to the OSM or PHSM. 685 If, after deleting the specified Target, the next-hop has no remaining Targets, then those resources associated with that next-hop agent may be released. Note that network resources may not actually be released if network multicasting is being used since they may still be required for traffic to other next-hops in the multicast group. 686 Change: The Application at the Origin may wish to change the FlowSpec of an established stream. To do so, it informs the OSM of the new FlowSpec and of the list of Targets relative to the change. The OSM then issues one CHANGE message with the new FlowSpec per next-hop and sends it with the correct Targetlist. The MS/C box then places copies (as required) of this in the NHSM queues.This takes the control to the Change state from the Establd state. CHANGE messages are structured and processed similar to CONNECT messages. 687 A next-hop agent that is an Intermediate Agent that receives a CHANGE message similarly determines if it can implement the new FlowSpec along the path to each of its next-hop agents, and if so, it propagates the CHANGE messages along the established paths. If this process succeeds, the CHANGE messages will eventually reach the Targets, which will each respond with an ACCEPT (or REFUSE) message that is propagated back to the OSM. 688 At this point the Application decides whether all replies have been received. If the change to the FlowSpec is in a direction that makes fewer demands of the involved networks, then the change has a high probability of success along the path of the established stream. Each ST2 agent receiving the CHANGE message makes the necessary request changes to the network resource allocations, and if successful, propagates the CHANGE message along the established paths. If the change cannot be made, but the I-bit indicates that stream should be torn down, then the ST2 Agent must recover using DISCONNECT and REFUSE messages as in the case of a network failure. Note that a failure to change the resources requested for specific Targets should not cause other targets in the stream to be deleted. 689 Data Forwarding: Once the Application or OSM determines that the stream is established Data may be transferred to the targets. An Application is not guaranteed that the data reaches its destinations: ST2 is unreliable and it does not make any attempt to recover from packet loss, e.g. due to the underlying network. In case the data reaches its destination, it does it accordingly to the negotiated quality of service. An ST2 Agent forwards the data only along already established paths to Targets. 690 Since a path is considered to be established when the ST2 next-hop agent on the path sends an ACCEPT message, it implies that the target and all other intermediate ST2 Agents on the path to the Target are ready to handle the incoming data packets. In no case will an ST2 Agent forward data to a next-hop Agent that has not explicitly accepted the stream. 691 At the end of the connection setup phase, the Origin, each Target, and each Intermediate ST2 Agent has a database entry that allows it to forward the data packets from the Origin to the Targets and to recover from failures of the Intermediate Agents or networks. The database should be optimized to make the packet forwarding task most efficient. The time critical operation is an Intermediate Agent receiving a packet from the previous-hop Agent and forwarding it to the next- hop Agents. The database entry must also contain the FlowSpec, utilization information, the address of the Origin and previous-hop, and the addresses of the Targets and next-hops, so it can perform enforcement and recover from failures. An ST2 Agent receives data packets encapsulated by an ST header. A data packet received by an ST2 Agent contains the SID. This SID was selected at the Origin so that it is globally unique and thus can be used as an index into the database, to obtain quickly the necessary! 692 replication and forwarding inform 694 ation. 695 The forwarding information will be network and implementation specific, but must identify the next-hop Agents. It is suggested that the cached information for a next-hop Agent include the local network address of the next- hop. If the data packet must be forwarded to multiple next- hops across a single network that supports multicast, the database may specify the next-hops by a (local network) multicast address. If the network does not support multicast, or the next-hops are on different networks, multiple copies of the data packet must be sent. 696 No data fragmentation is supported during the data transfer phase. The Application is expected to segment its PDUs according to the minimum MTU over all paths in the stream. The Application receives information on the MTUs relative to the paths to the Targets as part of the FlowSpec contained in the ACCEPT message. The minimum MTU over all paths has to be calculated from the MTUs relative to the single paths. If the Application at the Origin sends a too large data packet, the ST2 Agent at the Origin generates an error and it does not forward the data. 697 3.4.4 Previous Hop State Machine (PHSM) 698 The Previous Hop State Machine Model is common to a Target or Intermediate Agent. A PHSM communicates with an upstream NHSM and downstream with one or more NHSMs and/or a TSM via a MS/C box. When a CONNECT message is received, the Intermediate ST2 Agent invokes the routing function, reserves resources via the Local Resource Manager, and then propagates the CONNECT messages to its next-hops. For the most part the Intermediate Agent behaves like a relay. In the cases when the Intermediate Agent is not able to successfully send out a -CONNECT message to a downstream PHSM, a -REFUSE message from the PHSM is sent to the upstream NHSM.. 699 The PHSM consists of a small number of states: Init, Establd, Add and Change. 700 Init: The Application initially takes control from the Init state to the Establd state via the phsm_open predicate. A DISCONNECT or REFUSE of all Targets in a stream will take the stream from the Establd to a terminating state which is also the Init state. 701 Establd:Once in the Establd state, Targets may be added or changed by the Origin or Targets may request to join the stream. The processing of a JOIN request is always handled by either an OSM or a PHSM. Within each ST2 Agent, the ST2 Dispatcher examines incoming JOIN requests and determines whether the stream referenced is a stream that that Agent supports. If not, the JOIN is forwarded on towards the Origin. Once a JOIN request reaches an Agent that can process the JOIN, the ST2 Dispatcher ACKs the JOIN and queues it up to the resident OSM or PHSM. When stream authorization has completed successfully, the PHSM issues a CONNECT through the MS/C Box to either a NHSM or a TSM. 702 As previously described in the OSM, an ST2 Agent can handle only one stream Add or Change at a time. If such a stream operation is already underway, further requests are queued and handled when the previous operation has been completed. Either a DISCONNECT or REFUSE for all Targets transfers control from the Establd state to the Init state. 703 Add: Once in the Establd state the previous hop may relay a CONNECT message. A transition to Add will create a CONNECT message that is placed in the FIFO queue between the PHSM and the MS/C box. The CONNECT message contains the SID, an updated FlowSpec, and a TargetList. The MS/C box will then make a copy of the CONNECT message, partition the Targetlist parameter and place it the NHSM and/or TSM queues.The spliting (or separating) information is derived from the implementation's routing and LRM functions. 704 Once in the Add state the OSM waits to get ACCEPT or REFUSE responses. The stream will not transition back to the Establd state until all Targets have responded. The expiration of the retry timer and count (if the next hop is not ACKing the request) or the expiration of the end-to-end timer will be interpreted as an implicit refuse. 705 Change:The Application at the Origin may wish to change the FlowSpec of an established stream. To do so, it informs the ST2 Agent at the Origin of the new FlowSpec and of the list of Targets relative to the change and this message will be propagated to through the NHSMs to the PHSMs and TSMs. The control flow to the Change state is very similar to the previous FSM discussions. 706 Figure 9. Previous Hop State Machine (PHSM) 708 PHSM 709 Init 710 Estbld 711 Add 712 Change 713 + prevhop_open 714 >> Estbld 715 - 716 - 717 - 718 +ACCEPT 719 - 720 - 721 >> Self 722 -ACCEPT 723 >> Self 724 -ACCEPT 725 +ACCEPT_LAST 726 - 727 - 728 >> Estbld 729 -ACCEPT 730 >> Estbld 731 -ACCEPT 732 +CHANGE 733 - 734 >> Change 735 -CHANGE 736 >> Self 737 -queue 738 >> Self 739 -queue 740 +CONNECT 741 - 742 >> Add 743 -CONNECT 744 >> Self 745 -queue 746 >> Self 747 -queue 748 +JOIN_AUTHORIZED 749 - 750 >> Add 751 -CONNECT 752 >> Self 753 -queue 754 >> Self 755 -queue 756 +JOIN_NOT_AUTHORIZED 757 - 758 >> Self 759 -JOIN-REJ 760 >> Self 761 -queue 762 >> Self 763 -queue 764 +DISCONNECT 765 - 766 >> Self 767 -DISCON 768 >> Self 769 -DISCON 770 >> Self 771 -DISCON 772 +DISCONNECT_LAST 773 - 774 - 775 >>Establd 776 -DISCON 777 >>Establd 778 -DISCON 779 +DISCONNECT_ALL 780 - 781 >>Init 782 -DISCON 783 >>Init 784 -DISCON 785 >>Init 786 -DISCON 787 +REFUSE 788 - 789 >> Self 790 -REFUSE 791 >> Self 792 -REFUSE 793 >> Self 794 -REFUSE 795 (-CH_DISC) 796 (-CH_REF) 797 +REFUSE_LAST 798 - 799 - 800 >> Estbld 801 -REFUSE 802 >> Estbld 803 -REFUSE 804 (-CH_DISC) 805 (-CH_REF) 806 +REFUSE_ALL 807 - 808 >>Init 809 -REFUSE 810 >>Init 811 -REFUSE 812 >>Init 813 -REFUSE 814 (-CH_DISC) 815 (-CH_REF) 816 +RetryTimeout 817 - 818 - 819 >> Self 820 -REFUSE 821 >> Self 822 -REFUSE 823 (-CH_DISC) 824 (-CH_REF) 825 +RetryTimeout_Last 826 - 827 - 828 >> Estbld 829 -REFUSE 830 >> Estbld 831 -REFUSE 832 (-CH_DISC) 833 (-CH_REF) 834 +E2ETimeout_last 835 - 836 - 837 >> Estbld 838 -DISCON 839 -NOTIFYi 840 - 841 Table 4: PHSM 842 3.5 The Target State Machine (TSM) 843 The Target State Machine (TSM) is a high level state machine which communicates with a PHSM, or OSM if residing the same Agent as the Origin. The TSM also talks to the Upper Layer Protocol via primitives. The TSM consists of a small number of states: Init, Establd, Add and Change. 844 Init: The Application initially takes control from the Init state to the Establd state via the tsm_open predicate. An Application may request to join an existing stream. It has to collect information on the stream including the stream ID (SID) and the IP address of the stream's Origin. This can be done out-of- band, e.g. via regular IP. The information is then passed to the local ST2 Agent together with the FlowSpec. The Application directs the TSM to generate a JOIN message containing the Application's request to join the stream and sends it to the PHSM which in turn sends it upstream toward the stream Origin. 845 An ST2 Agent receiving a JOIN message for which that Agent has a matching stream , responds with an ACK. The ACK message must identify the JOIN message to which it corresponds by including the Reference number indicated by the Reference field of the Join message. If the ST2 Agent is not traversed by the stream that has to be joined, it propagates the JOIN message toward the stream's Origin. Eventually, an ST2 Agent traversed by the stream or the stream's Origin itself is reached. In any case, the TSM will eventually receive a JOIN-REJECT or CONNECT response. This is shown as transitions to the Establd state and the Add state respectively. 846 Add: The TSM may receive a +CONNECT message any time . The ST2 Agent reserves local resources and inquires from the specified Application process whether or not it is willing to accept the connection. In particular, the Application must be presented with parameters from the CONNECT, such as the SID, FlowSpec, Options, and Group, to be used as a basis for its decision. The Application is identified by a combination of the NextPcol field and the SAP field included in the correspondent (usually single remaining) target of the TargetList. The contents of the SAP field may specify the port or other local identifier for use by the protocol layer above the host ST layer. Subsequently received data packets will carry the SID, that can be mapped into this information and be used for their delivery. 847 The TSM responds with an ACCEPT or REFUSE - a result of the Upper Layer Protocol decision. 848 Change: The TSM may receive a CHANGE message any time it is in a Establd state. This happens always after a CONNECT. The TSM again responds with an ACCEPT or REFUSE after informing the Upper Layer Protocol. 849 The TSM may any time want to terminate its membership in the stream. This is handled by the TSM sending out a REFUSE message. On the other hand it is possible for an Origin or IntermediateAgent to disconnect the Target from the stream. This is accomplished by the Agent or Origin sending a -DISCONNECT message. 850 Figure 10. Target State Machine (TSM) 852 TSM 853 Init 854 Estbld 855 Add 856 Change 857 +tsm_open 858 >>Estbld 859 - 860 - 861 - 862 +api_accept 863 - 864 - 865 >> Estbld 866 -ACCEPT 867 >> Estbld 868 -ACCEPT 869 +CHANGE 870 - 871 >> Change 872 -change_api 873 >>self 874 -queue 875 >>self 876 -queue 877 +CONNECT 878 - 879 >> Add 880 -connect_api 881 - 882 - 883 +DISCONNECT 884 - 885 >> Init 886 -disconnect_api 887 >> Init 888 -discon_api 889 >> Init 890 -discon_api 891 +JOIN_REJECT 892 - 893 >> Init 894 -join_reject_api 895 - 896 - 897 +api_join 898 - 899 >> Self 900 -JOIN 901 - 902 - 903 +api_refuse 904 - 905 >> Init 906 -REFUSE 907 >> Init 908 -REFUSE 909 >> Init 910 -REFUSE 911 +api_refuse_change 912 - 913 - 914 - 915 >> Estbld 916 -REFUSE 917 Table 5: TSM 918 SECTION\x134 919 ST2 Agent FSMs 920 Section 2 summarized the ST2 Agent architecture, describing the ST2 Dispatcher and MS/C Box roles in the filtering, validation, queuing and creation of ST2 PDUs, as well as the CFSM relationships between the individual stream OSM, NHSM, PHSM and TSM partners. Section 3 detailed the normal operations of these individual stream FSMs. The roles of the Retry FSM and the Control FSM will now be discussed in more detail. First, the SCMP messages will described according to how they traverse the ST2 Agents and impact the individual Agent databases and states. 921 4.1 Control Message Traversal and Agent Database Context 922 ST2 Agent stream database entries are intitated by the first CONNECT for that Stream Id. The information initially correlated to each StreamId entry includes: 923 ST2 Neighbor Previous Hop and Next Hops 924 FlowSpec, Group, MulticastAddress, Origin, TargetList, ACK and Response timers 925 Stream Options for NoRecovery(S-bit) and Join Authorization Level (J-bit, N-bit) 926 Routing results for each Target's Next Hop 927 LRM results for each Next Hop's resource allocation 928 Each Agent database is modified when the CONNECT Responses indicate some variation specified by a downstream Agent or Target Response. Subsequent Requests can also modify the database and include additional CONNECT, JOIN and CHANGE requests. Origin, Network or Agent Recovery and LRM initiated stream teardown can occur in the form of explicit DISCONNECTs, REFUSEs and Recovery initiated CONNECTs or implicit conditions detected through the HELLO, STATUS, STATUS-RESPONSE and NOTIFY messages. The database context is then augmented with the history of Reason Codes, prior stream characteristics, G-bit (Global stream TargetList), I-bit and E-bit (CHANGE characteristics), J-bit and N-bit (Join level)and R-bit (Restarted Agent) values. 929 While all control messages may have an indirect effect on stream state and databases , only ACCEPT, CHANGE, CONNECT, DISCONNECT, JOIN-REJECT, JOIN and REFUSE directly affect each Agent's defintion of each stream. ACK, ERROR, HELLO, NOTIFY, STATUS and STATUS-RESP are the control messages that are primarily used to maintain ST2 Agent databases for datalink, neighbor and network management functions. 930 Thus as the ST2 control messages traverse the Agents, the databases are updated to include the additional information as approriate to the Agent and stream FSM implementations. Table 6. lists the control message with the scope of the control message flow in the following categories : 931 End-to-End - messages flow between the Origin and the Targets. 932 End-to-Intermediate & Intermediate-to-End- control messages can flow between a Target and an Intermediate (both directions) or between an Intermediate and Origin (both directions) 933 Local - a message traverses Upstream or Downstream but limited to one hop only. 935 Table 6: Control Message Direction 936 Message Direction 937 End-to-End 938 End-to-Intermediate & 939 Intermediate-to-End 940 Local 941 Message 942 Origin-to-Target (D) or 943 Target-to-Origin (U) 944 Target-to-Interm (T->I) or 945 Interm-to-Target (I->T) or 946 Interm-to-Origin (I->O) or 947 Origin-to-Interm (O->I) 948 U = {O, I} 949 D = {I, T} 950 ACCEPT 951 Upstream 952 T->I, I->O 953 ACK 954 Either 955 CHANGE 956 Downstream 957 O->I, I->T 958 CONNECT 959 Downstream 960 O->I, I->T 961 DISCONNECT 962 Downstream 963 O->I, I->T 964 ERROR 965 Either 966 HELLO 967 Either 968 JOIN-REJECT 969 Downstream 970 O->I, I->T 971 JOIN 972 Upstream 973 T->I 974 NOTIFY 975 Either 976 Either 977 Either 978 REFUSE 979 Upstream 980 T->I, I->O 981 STATUS 982 Either 983 STATUS-RESP 984 Either 985 As the ST2 PDUs traverse the network, each Agent presumably has a platform specific interface-to-packet-switching function that must intercept the ST2 packets for ST2 functions. The ST2 Dispatcher represents this PDU validation, filtering and packetswitching. The ST2 Dispatcher in this model is organized as the Agent packet-switcher, rather than as a per-interface or per-nex-hop packet-switcher. This function may be reorganized as a distributed function if the Agent platform architecture requires such distribution. 986 4.2 ST2 Dispatcher role for incoming Packet-switching, ACKnowledgement and PDU validation 987 An ST2 Dispatcher can validate an ST2 PDU for ST2 header and PDU semantic validity and then rapidly switch Data packets , .i.e to a local Target application SAP or to the appropriate next hop interface for remote Targets. 988 When the PDU semantics are in error, an ERROR PDU with the corresponding Reason Code and the offending PDU contents are returned to the SenderIpAddress (instead of an ACK for those SCMP messages that require an ACK). The PDU in ERROR is then discarded. The following Reason Codes detail the inconsistencies reported in an ERROR Response: 989 2 ErrorUnknown An error not contained in this list has been 990 detected. 991 8 AuthentFailed The authentication function failed. 992 13 CksumBadCtl Control PDU has a bad message checksum. 993 14 CksumBadST PDU has a bad ST Header checksum. 994 23 InvalidSender Control PDU has an invalid SenderIPAddress field. 995 24 InvalidTotByt Control PDU has an invalid TotalBytes field. 996 * 26 LnkRefUnknown Control PDU contains an unknown LnkReference. 997 31 OpCodeUnknown Control PDU has an invalid OpCode field. 998 32 PCodeUnknown Control PDU has a parameter with an invalid PCode. 999 33 ParmValueBad Control PDU contains an invalid parameter value. 1000 35 ProtocolUnknown Control PDU contains an unknown next-higher 1001 layer protocol identifier. 1002 37 RefUnknown Control PDU contains an unknown Reference. 1003 45 SAPUnknown Control PDU contains an unknown next-higher 1004 layer SAP (port). 1005 * 46 SIDUnknown Control PDU contains an unknown SID. 1006 48 STVer3Bad A received PDU is not ST Version 3. 1007 54 TruncatedCtl Control PDU is shorter than expected. 1008 55 TruncatedPDU A received ST PDU is shorter than the ST Header 1009 indicates. 1010 Figure 11. 1011 In some cases, SCMP PDUs may also be forwarded without impacting the local stream FSMs. A JOIN whose SID is not active on this Agent is sent to the appropriate neighbor, as indicated by the Origin to which the JOIN is directed. Also, when the PDU is a CHANGE or DISCONNECT for an active SIDand an unknown Target, but with a JOIN level of 2,, the PDU is forwarded to all of the stream next hops. This satisfies the flooding mechanism required to support a stream whose Origin and branches do not explicitly know all downstream Targets. 1012 The next level of PDU analysis involves Agent and stream consistency. The PDU is examined for content consistency with both Agent and stream database information.The following detected inconsistencies may result: 1013 3 AccessDenied Access denied. 1014 4 AckUnexpected An unexpected ACK was received. 1015 15 DuplicateIgn Control PDU is a duplicate and is being acknowledged. 1016 16 DuplicateTarget Control PDU contains a duplicate target, or an attempt to add an existing target. 1017 49 StreamExists A stream with the given SID already exists. 1018 51 TargetExists A CONNECT was received that specified an 1019 existing target. 1020 52 TargetUnknown A target is not a member of the specified stream. 1021 53 TargetMissing A target parameter was expected and is not 1022 included, or is empty. 1023 Most SCMP PDUs (except ACK, ERROR, HELLO, STATUS, STATUS-RESPONSE,) will trigger an ACK to the ST2 neighbor that sent the PDU. 1024 CONNECT, CHANGE and JOIN Requests will be directed to the appropriate stream PHSM. 1025 Incoming Responses are first correlated with any corresponding Request Reference so that the appropriate next hop or Response timer may be terminated. Then ACCEPT and REFUSE messages are queued up to the appropriate stream NHSM, while DISCONNECT and JOIN-REJECT messages arequeued up to the appropriate stream PHSM. 1026 ACK and ERROR messages are correlated with a PDU Reference, terminating the appropriate timers, and then queued up to the stream Retry FSM. 1027 HELLO, STATUS and STATUS-RESPONSE messages are correlated with a PDU Reference so that the appropriate timers may be terminated and then queued up to the Control FSM. 1028 4.3 ST2 Dispatcher functions for outgoing Packet switching, timer and retry settings 1029 Figure 12. 1030 The ST2 Dispatcher also has the role of packaging and forwarding outgoing PDUs to the apprropriate interfaces. The outgoing PDU must be given it's own PDU Reference number and any correlated PDU Referencer number, as well as the semantics and context of the PDU database entries. This Agent architecture model assumes that the Agent and stream databases are the intra-Agent repository of all activities, such that the ST2 Dispatcher can efficiently create and distribute the PDUs. However, it is entirely possible that the accumulated contents of a PDU has exceeded an outgoing MTU restriction and the PDU would be trunkated with the following Reason Codes: 1031 6 UserDataSize UserData parameter too large to permit a 1032 message to fit into a network's MTU. 1033 36 RecordRouteSize RecordRoute parameter is too long to permit 1034 message to fit a network's MTU. 1035 4.4 Retry FSM- RFSM for datalink reliability of PDU transmissions 1036 Figure 13. Retry State Machine (RFSM) 1037 The Retry FSM has four states - Init, Set-timers (and check retry count), Ack-wait and Resp-wait. The general paradigm for the Retry FSM is to move from the Init state to the Set-timers state whenthere is a PDU requiring an ACK and/or a Response, then transition to the appropriate 1038 state to wait for the resultant ACKS, Responses and/or timeouts. PDUs requiring ACKS cycle through resends by NAccept, NChange, NConnect, NDisconnect, NJoin, NJoinReject, NNotify and NRefuse configured counts. 1039 Since all ACKs and Responses are correlated by PDU Reference numbers, packets are correlated to the outstanding Retry FSM by the same mechanism. Either an ACK or a RetryTimeout that is correlated to an ACCEPT, DISCONNECT, JOIN-REJECT, NOTIFY or REFUSE results in the Retry FSM transitionsto Init. Such PDUs have no end to end response rquirements and generally have no secondary error processing when it can be assumed that the neighbor Agent and/or link level reliability is gone.. An ACCEPT is an exception. The failure of an ACCEPT is an implicit REFUSE upstream and DISCONNECT downstreamsince the end to end Response has now failed to completely traverse the stream Agents. 1040 An ACK Response to a CHANGE, CONNECT or JOIN results in a transtion to the Resp-wait state until either a Response is received or the Respsonse timers expire. 1041 A legitimate Response, or a RetryTimeout or an E2ETimeout on CHANGE, CONNECT or JOIN causes the transition to the Init state with the signal to be replicated for individaul stream FSM. 1042 In fact, only an OSM or a PHSM acting as an Origin for a JOIN will instantiate the Response-wait state of the Retry FSM. This could be inidcated by the implementation's databases, or as some implementor's prefer, variations of the Retry FSM could be incorporated into each of the OSM, NHSM, PHSM, TSM and Control FSMs. Section 5 will discuss some of the secondary and tertiary exception processing issues that may influence this implementation decision. 1044 RFSM 1045 Init 1046 Set-timersd 1047 Ack-wait 1048 Response-wait 1049 +Ack_timeout 1050 >>Set-timer 1051 -resend PDU 1052 +ERROR 1053 >>Set-timer 1054 -resend PDU 1055 +ACK-ACCEPT 1056 >>Init 1057 +ACK-CHANGE 1058 >>Response- wait 1059 +ACK-CONNECT 1060 >>Response- 1061 wait 1062 +ACK-DISCONNECT 1063 >>Init 1064 +ACK-JOIN 1065 >>Response-wait 1066 +ACK-JOIN-REJECT 1067 >>Init 1068 +ACK-NOTIFY 1069 >>Init 1070 +ACK-REFUSE 1071 >>Init 1072 +RetryTimeout-ACCEPT 1073 >>Init 1074 -DISCONNECT 1075 -REFUSE 1076 +RetryTimeout-CHANGE 1077 >>Init 1078 -RetryTimeout 1079 +RetryTimeout-CONNECT 1080 >>Init 1081 -RetryTimeout 1082 +RetryTimeout-DISCONNECT 1083 >>Init 1084 +RetryTimeout-JOIN 1085 >>Init 1086 -RetryTimeout 1087 +RetryTimeout-JOIN-REJECT 1088 >>Init 1089 +RetryTimeout-NOTIFY 1090 >>Init 1091 +RetryTimeout-REFUSE 1092 >>Init 1093 +E2ETimeout 1094 >>Init 1095 -E2ETimeout 1096 >>Init 1097 -E2ETimeout 1098 >>Init 1099 -E2ETimeout 1100 +ACCEPT 1101 - 1102 >> Init 1103 -ACCEPT 1104 >> Init 1105 -ACCEPT 1106 >>Init 1107 -ACCEPT 1108 +REFUSE 1109 - 1110 >> Init 1111 -REFUSE 1112 >>Ini 1113 -REFUSE 1114 >> Init 1115 -REFUSE 1116 +CONNECT 1117 >> Init 1118 -CONNECT 1119 >> Init 1120 -CONNECT 1121 >> Init 1122 -CONNECT 1123 +JOIN-REJECT 1124 >> Init 1125 -JOIN-REJECT 1126 >> Init 1127 -JOIN-REJECT 1128 >> Init 1129 -JOIN- 1130 REJECT 1131 +STATUS-RESPONSE 1132 >> Init 1133 -STATUS-RESPONSE 1134 >> Init 1135 -STATUS- 1136 RESPONSE 1137 >> Init 1138 -STATUS- 1139 RESPONSE 1140 Table 7: PHSM 1141 4.5 Agent , Neighbor and Stream Supervision 1142 4.5.1 The Control FSM for Agent and Stream Supervision 1144 Figure 14. .CFSM 1145 Each ST2 Agent must monitor its own status, network conditions, neighbor Agent status and supervise the Recovery of streams whenever required and possible during a network failure.This CFSM is intended to be a general approach to these issues, rather than a fully specified FSM since the particular network, platform and implementation architecture will determine detail FSM considerations. 1146 What this CFSM model does suggest is that the CFSM provides a superstructure for the management of the Neighbor Detection Failure FSM, any Agent NOTIFY, STATUS or STATUS-RESPONSE implications, Service Model management (including application issues, routing and LRM or other Agent implementation specific issues), as well as datalink statistics analysis (e.g., broken or dropped PDUs or accumulated routing errors). 1147 At the very least, stream Recovery requires careful analysis of the possible recursions in Agent, neighbor failure detection, routing and LRM conditions. The ST2+ specification defines parameters for a configured number of times that Recovery should be attempted (NRetryRoute), the configured time to wait for each Response (ToRetryRoute) and variations in the exception processing. 1148 During the course of a stream setup, the CONNECT contains a Recovery Timeout, as specified by the Origin. The resultant ACCEPTs contain the individual Agent's "supportable" Recovery Timeout such that the stream Recovery Timeout becomes the smallest Recovery Timeout for all Targets. The HELLO timer must be smaller than the smallest Recovery Timeout for all streams between these Agents, but an Agent may have various HELLO timers between different Agents, such that the management funtion of such timers should fall into the CFSM also. A Round Trip Time (RTT) estimation function is available with STATUS and STATUS_RESPONSE messages to aid in this area. 1149 The CFSM relies on the Nieghbor Detection Failure FSM (NDFSM) as the primary notification vehicle for stream and neighbor management. During the initial stream setup of any stream NHSM and PHSM, the CFSM is signalled to begin monitoring of the FSM neighbor Agents involved in the stream. The sending of HELLOs is begun once an ACCEPT is forwarded upstream. The receiving of HELLOs is acceptable as soon as an ACCEPT is received. HELLOs are terminated once an ACK is sent or received for the DISCONNECT or REFUSE associated with the last of all streams and Targets for that neighbor.This requires signalling and coordination with the ST2 Dispatcher, Retry FSM and database context, especially when the Restarted bit is active for either the local Agent of a neighbor. 1150 Agent network "inspection and repair" functions might also exist in the CFSM to extend the mechanisms of the NDFSM before attempting Recovery and/or stream teardown. 1151 Group management for Bandwidth-sharing, Fate-sharing, Path-sharing and Subnet resource- sharing can be intiated by any ST2 Agent and it may be adviseable to incorporate optimization algorithms in the CFSM tointeract with routing and LRM function, thus allowing the CFSM to monitor and gauge the impact on the stream Recovery analysis. 1152 4.5.2 The Nieghbor Detection Failure FSM for Neighbor Management 1153 This FSM has a more atomic focus in that ST2 neighbor HELLOs are maintained and monitored only while there are one or more shared streams active. When the neighbor HELLOs and subsequent STATUS inquiry fails or the neighbor R-bit has been set, the neighbor is considered down and the streams involved in that neighbor relationship must be examined for Recovery conditions. 1155 Figure 15. NFDSM 1157 NFDSM 1158 Inactive 1159 Up 1160 Verify 1161 Down 1162 +Start Monitoring 1163 >> Up 1164 >> Self 1165 >> Up 1166 >> Up 1167 +End Monitoring 1168 - 1169 >>Self 1170 >> Self 1171 >> Self 1172 +End Monitoring Last 1173 - 1174 >>Inactive 1175 >>Inactive 1176 >>Inactive 1177 +Xmit Timeout 1178 - 1179 >> Self 1180 -Hello 1181 >> Self 1182 -Hello 1183 >> Self 1184 -Hello 1185 +Hello & Recovery Timeout 1186 - 1187 >> Self 1188 - 1189 - 1190 +NO Hello & Recovery Timeout 1191 - 1192 >> Verify 1193 -Status (SID) 1194 - 1195 - 1196 +Status Timeout 1197 - 1198 - 1199 >> Down 1200 -N_Down 1201 - 1202 +R-bit Set 1203 - 1204 >> Down 1205 -N_Down 1206 >> Down 1207 -N_Down 1208 >> Self 1209 +R-bit Clear 1210 - 1211 >> Self 1212 >> Up 1213 >> Up 1214 Table 8: NFDSM 1215 4.5.3 Service Model Interactions 1216 Figure 16. MS/C Box Communications inside an Agent 1217 The optimization of route and LRM functions can affect the selection from multiple path routes to a Target on initial CONNECTs, as well as CHANGE and Recovery procedures. This document's model follows a sequential process of integrating the route and LRM services with the MS/C Box. for the atomic individual stream FSMs. 1218 Additional algorithms may be used in the CFSM, such that Options and Group factors may be optimized in relation to the stream Recovery decisions. 1219 SECTION\x135 1220 Exception Processing 1221 All the exception processing conditions have been referenced in the preceding sections. Not all have been spelled out in detail. The general paradigms fall into several categories and all of this document's models are based on a suggested approach. The secondary and tertiary conditions of some apects of exception processsing are especially subject to implementation preferences. 1222 The first topic of discussion might be the category SCMP datalink reliability as generally characterized in the Retry FSM. This document favors maintaining a coordinating Retry FSM versus incorporating the Retry states in each of the OSM, NHSM, PHSM and TSM, which is naturally an alternative. 1223 ERROR message generation for PDU semantics problems has discussed in Section 5 as an ST2 Dispatcher function. A special case occurs in PDU construction when the MTU size is exceeded, i.e.: 1224 6 UserDataSize UserData parameter too large to permit a 1225 message to fit into a network's MTU. 1226 36 RecordRouteSize RecordRoute parameter is too long to permit 1227 message to fit a network's MTU. 1229 However, all of the analysis and potential REFUSE message or signal generation, still seems best suited to the ST2 Dispatcher. 1230 5.1 Additional Exception Processing 1231 5.1.1 ST2 Dispatcher detected inconsistencies with Reason Codes: 1232 The following errors can also be detected by the ST2 Dispatcher with the careful analysis of all Agent and stream database values: 1233 3 AccessDenied Access denied. 1234 4 AckUnexpected An unexpected ACK was received. 1235 15 DuplicateIgn Control PDU is a duplicate and is being acknowledged. 1236 16 DuplicateTarget Control PDU contains a duplicate target, or an attempt to add an existing target. 1237 49 StreamExists A stream with the given SID already exists. 1238 51 TargetExists A CONNECT was received that specified an 1239 existing target. 1240 52 TargetUnknown A target is not a member of the specified stream. 1241 53 TargetMissing A target parameter was expected and is not 1242 included, or is empty. 1243 This means that the atomic FSMs do not have to incorporate this logic and simplifies core FSM paradigms. 1244 5.1.2 Control FSM issues with neighbor failure and stream recovery with Reason Codes: 1245 The details of these specific instances can be intertwined with Retry, Routing and LRM failures. 1246 12 CantRecover Unable to recover failed stream. 1247 22 IntfcFailure A network interface failure has been detected. 1248 27 NetworkFailure A network failure has been detected. 1249 39 RestartLocal The local ST agent has recently restarted. 1250 40 RestartRemote The remote ST agent has recently restarted. 1251 47 STAgentFailure An ST agent failure has been detected. 1252 5.1.3 Retry and Timeout Failures with Reason Codes: 1253 38 ResponseTimeout Control message has been acknowledged but not 1254 answered by an appropriate control message. 1255 41 RetransTimeout An acknowledgment has not been received after 1256 several retransmissions. 1257 5.1.4 Routing issues with Reason Codes: 1258 Routing issues initiate special exception processing requirements. Some of these have been addressed in the ST2+ specification, but each implementation should consider the network and platform architecture, also. 1259 9 BadMcastAddress IP Multicast address is unacceptable in CONNECT 1260 28 NoRouteToAgent Cannot find a route to an ST agent. 1261 29 NoRouteToHost Cannot find a route to a host. 1262 30 NoRouteToNet Cannot find a route to a network. 1263 34 PathConvergence Two branches of the stream join during the 1264 CONNECT setup. 1265 42 RouteBack Route to next-hop through same interface as 1266 previous-hop and is not previous-hop. 1267 43 RouteInconsist A routing inconsistency has been detected. 1268 44 RouteLoop A routing loop has been detected. 1269 5.1.5 LRM issues with Reason Codes: 1270 Optimization of routing and LRM issues can also initiate special exception processing requirements. Some of these have been addressed in the ST2+ specification, but each implementation should also consider the network and platform architecture. 1271 10 CantGetResrc Unable to acquire (additional) resources. 1272 11 CantRelResrc Unable to release excess resources. 1273 17 FlowSpecMismatch FlowSpec in request does not match 1274 existing FlowSpec. 1275 18 FlowSpecError An error occurred while processing the FlowSpec. 1276 19 FlowVerUnknown Control PDU has a FlowSpec Version Number that 1277 is not supported. 1278 20 GroupUnknown Control PDU contains an unknown Group Name. 1279 21 InconsistGroup An inconsistency has been detected with the 1280 streams forming a group. 1281 50 StreamPreempted The stream has been preempted by one with a 1282 higher precedence. 1283 SECTION\x136 1284 Appendix A 1285 6.1 ST Control Message Flow 1286 Control Message Types 1287 ST2 control messages are generally of the request -response type. Table 1 summarizes these control messages alphabetically. The table has three major columns. 1288 7 Message type 1289 7 Response 1290 7 Possible causes for message 1291 6.1.1 Message Type 1292 Under the Message Type each control message is categorized either as a: 1293 - Request message 1294 - Response message 1295 It is possible for a message to be more than one type depending on the usage, although this is not apparent from this table. 1296 6.1.2 Response 1297 The response to each control message is given in the next major column under Response. Note that the response to a message can be interpreted to mean either: 1298 1. a response to another control message 1299 2. a response to indicate the condition of receipt of the message, driven primarily by the error control function 1300 The second interpretation of response includes positive acknowledgments and negative acknowledgments (error response). Thus, this major column has the following categories: 1301 - Error Response 1302 - Mandatory Response 1303 - Other response following mandatory response. 1304 An X or an entry in the table indicates classification of a message under a particular category shown under each major column. 1305 6.1.3 Possible causes for message 1306 Finally, a control message might have been sent in response to another control message. This is shown in the last column. Note that it is possible that independently a number of control messages may be the cause for this control message in question Note that an entry does not necessarily mean that is the only cause. A blank entry in this column for instance means that the message was not invoked by another message. 1307 For example, an ACCEPT message is a Response Type message to either a CONNECT or a CHANGE message. It will be acknowledged (Mandatory response) with an ACK. It may be responded with an ERROR in case of error conditions. The state diagrams illustrate this sequencing more completely. It may be noted that the sequencing of messages gives the protocol semantics. 1308 . 1309 Table 9:\x11Message Types: Requests, Responses and Others 1310 Type 1311 Response 1312 Possible causes for message: 1313 Message 1314 Req. 1315 Resp. 1316 Error Resp. 1317 Man-datory Resp. 1318 Other response following Mandatory Resp. 1319 ACCEPT 1320 X 1321 ERROR 1322 ACK 1323 1. CONNECT 1324 2. CHANGE 1325 ACK 1326 X 1327 ERROR 1328 1. ACCEPT 1329 2. CHANGE 1330 3. CONNECT 1331 4. DISCONNECT 1332 5. JOIN 6.JOIN-REJECT 1333 7. NOTIFY 1334 8. REFUSE 1335 CHANGE 1336 X 1337 ERROR 1338 ACK 1339 ACCEPT 1340 REFUSE 1341 1.Origin Change stream 1342 CONNECT 1343 X 1344 X 1345 ERROR 1346 ACK 1347 ACCEPT 1348 REFUSE 1349 1.Target JOIN 1350 2. Origin Connect TargetList 1351 DISCONNECT 1352 X 1353 ERROR 1354 ACK 1355 1.Origin Disconnect TargetList 1356 2. Intermediate Agent detects upstream failure or is acting as an Origin 1357 ERROR 1358 X 1359 Errored mesgs.: 1360 1. ACCEPT 1361 2. ACK 1362 3. CHANGE 1363 4. CONNECT 1364 5. DISCONNECT 1365 6. HELLO 1366 7. JOIN 1367 8. JOIN-REJECT 1368 9. NOTIFY 1369 10.REFUSE 1370 11.STATUS 1371 12. STATUS-RESPONSE 1372 HELLO 1373 X 1374 ERROR 1375 1.Periodic Message 1376 JOIN-REJECT 1377 X 1378 ERROR 1379 ACK 1380 1. JOIN 1381 JOIN 1382 X 1383 ERROR 1384 ACK 1385 CONNECT 1386 JOIN-REJECT 1387 Target joining stream 1388 NOTIFY 1389 X 1390 ERROR 1391 ACK 1392 1. Information message 1393 2. Notification upstream to JOIN for some authorization levels 1394 REFUSE 1395 X 1396 X 1397 ERROR 1398 ACK 1399 1.Target leaving stream 2. CONNECT 1400 3. CHANGE 4.Intermediate 1401 Agent detects 1402 downstream stream failure 1403 STATUS 1404 X 1405 ERROR 1406 STATUS-RESPONSE 1407 STATUS-RESPONSE 1408 X 1409 ERROR 1410 1.STATUS 1411 APPENDIX B 1412 The following internetwork diagram of ST2 Agents indicates the Origin (O), Intermediate (I) and Target (T) roles of each ST2 Agent in relation to two ST2 conferences. Conference 1 (C1) has 4 participants, all of which are sending and receiving data from each other. Intermediate Agents 1, 2, 3 and 4 each have an ST2 application with an Origin sending data to all other members, and simultaneously receiving data as a Target for each of the other members. 1413 Figure 17. 1414 Figure 18. 1415 Each ST2 Agent participating in C1, as illustrated, has all four streams to manage, representing a fully meshed stream conference with Targets and Origins communicating along the same paths. 1416 There are a number of other possible routes for each individual stream in C1. The above paths through Agents I6, I9 and I10 were chosen to illustrate a simple routing scheme for such a conference. Agents I8 and I12 could have just as easily been involved, if there were no other routing metrics to consider other than number of hops. However, it is also possible that the resources available at any Agent or interface may not actually be equal , such that Agents I7 and I12 became involved in branches of some of the streams. In such alternative routing and resource circumstances, some of the Intermediate Agents might only maintain one stream in the conference. However, in this illustration, Agent I9 happens to have an ST2 neighbor for every stream and the need to manage multiple targets for each stream. 1417 Conference 2 (C2) has 3 participants and is also a fully meshed set of streams for each member of the conference. All ST2 Agents in the C2 illustration also have multiple ST2 neighbors, streams and interfaces to manage. 1418 Figure 19. , 1419 In addition, since both conferences are being conducted simultaneously, several Agents are managing streams from both conferences, which may be Grouped for Resource or Fatesharing characteristics. The dynamics of such internetwork topology and resource issues can become complex stream management issues. 1421 ST Working Group M. Rajagopal and Sharon Sergeant 1422 File: draft-ietf-st2-state-01.ps Expires: May, 1996 1424 ACKNOWLEDGEMENTS and AUTHORS: 1425 Many individuals have contributed to the work described in this memo. We thank the participants in the ST Working Group for their input, review, and constructive comments. 1426 We would also like to thank Luca Delgrossi and Louis Berger for allowing us to adopt the text from their [1] document. 1427 We would like to acknowledge inputs from Mark Pullen and his graduate students, Tim O'Malley, Eric Crowley, Muneyoshi Suzuki and many others. 1428 Murali Rajagopal EMail: murali@fbcs.com, Phone: 714-764-2952 1429 Sharon Sergeant EMail:sergeant@xylogics.com, Phone: 617-893-6142 1430 LIST OF REFERENCES: 1431 [1] L. Delgrossi and L. Berger: Internet STream Protocol Version 2 (ST2) - Protocol Specification- Version ST2+, RFC 1819 , August 1995. 1432 [2]D. Brand and P. Zafiropulo: On Communicating Finite-State Machines, J.ACM, 30, No.2, April 1983