IPS Working Group Charles Monia INTERNET DRAFT Josh Tseng Nishan Systems Expires May 2001 Category: Informational November 2000 An Architecture for a Fibre Channel Fabric on an IP Network Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026 [1]. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. For potential updates to the above required-text see: http://www.ietf.org/ietf/1id-guidelines.txt Comments Comments should be sent to the ips mailing list (ips@ece.cmu.edu) or to the author(s). Abstract This is the first installment of an informational document describing the architecture for implementing a fibre channel fabric on an IP network. The architecture is the basis for a protocol suite that provides the set of fibre channel fabric services required by FCP storage devices. Motivation The increase in demand for Internet bandwidth has propelled the wide deployment of gigabit networks. This demand has also driven the rapid growth of storage area networks and the development of high- volume IP components capable of meeting the needs of such networks. Monia, Tseng Category - Informational 1 FCP Architectural Framework November 2000 The purpose of the architecture and protocol suite is to provide the benefits of this IP networking technology to the considerable installed base of storage products and technologies built upon FCP. 1. Introduction This is an evolving informational document describing an architecture for the implementation of fibre channel fabric capabilities on public and local IP networks. The goals of the architecture are to: a) Enable the implementation of FC fabric capabilities using IP routing and switching elements in place of fibre channel components. b) Provide the fabric services required by storage devices, c) Produce implementations that run at the speed and latency of gigabit IP transports. d) Define the new interfaces to be specified in standards-track or informational documents. e) Identify interfaces defined by existing standards. The protocols and method of fibre channel frame mapping described in this document permit the transparent attachment of fibre channel storage devices to an IP-based fabric by means of lightweight gateways. This transparency permits implementations to run without modifications. The architecture achieves transparency through an address mapping process that allows normal frame traffic to pass through the gateway directly, with provisions for intercepting and emulating the fabric services required by an FCP device. This version of the document focuses on the IP transport mechanisms. These include layering, the IP Fabric network model, the method of mapping frame addresses and the process for servicing fibre channel link service requests. To simplify the description, UDP encapsulation in a private network is assumed. Subsequent versions will be expanded to discuss the interfaces to be standardized, compliance with existing standards. naming services and the public network implementation, including TCP/IP encapsulation, security and storage network management. 2. Standards and Informational Specifications The documents to be produced specify the fibre channel transport services and the name services required by fibre channel devices. Two variants of the storage transport protocol will be defined: a) mFCP for metro-area and local-area SANs based on UDP and, Monia, Tseng Category - Informational 2 FCP Architectural Framework November 2000 b) iFCP, a gateway-to-gateway protocol for interconnecting fibre channel and mFCP SANs via the TCP/IP public network. The protocols will be specified in the following documents: a) A standards track specification for iFCP; b) A standards track specification for the name service protocol, iSNS; c) An informational document for mFCP, the UDP variant of the storage protocol. iSNS, the companion name service protocol, is designed to meet the requirements of iFCP and mFCP and has been specially tailored to support iSCSI. iSNS can be used with either UDP or TCP transport protocols. 3. Definitions Terms needed to clarify the concepts presented in this document are presented here. Fibre Channel Network _ A fabric and all attached fibre channel devices. Fabric _ The part of a fibre channel network that transports frames between fibre channel devices. A fabric may be implemented in the IP framework by means of the architecture and protocols discussed in this document. FC-2 _ The fibre channel transport services layer described in the FC-FS specification. FCP Portal - An IP-addressable entity representing the point at which a logical or physical FCP device is attached to the IP network. N_PORT _ The interface to fibre channel device functionality. This interface implements the fibre channel N_PORT semantics specified in the FC-FS standard [FC-FS]. N_PORT fabric address _ The address of an N_PORT within the fibre channel fabric. N_PORT Network Address -- The address of an N_PORT in the IP fabric. As described in section 4.2, this address consists of the IP address of the FCP Portal and the N_PORT ID of the directly- attached fibre channel device. F_Port - The interface used by an N_PORT to access fibre channel fabric and fabric services functionality. Monia, Tseng Category - Informational 3 FCP Architectural Framework November 2000 iFCP _ A gateway-to-gateway protocol that provides the services of a fibre channel fabric on a TCP/IP network. mFCP _ A protocol that provides the services of a fibre channel fabric on a UDP/IP network in a metropolitan or local area network environment. Logical FCP Device _ An abstraction representing a fibre channel device as it appears on an IP network. Physical mFCP Device _ A device in which all fibre channel and mFCP protocol functionality is contained within the physical device. 4. IP Fabric Architecture This section describes the mechanisms for implementing a fibre channel fabric on an IP network. In what follows, the reader is assumed to have a basic knowledge of fibre channel technology. In that regard, the overview in section 8 may be helpful. Two kinds of IP fabric environments are discussed: Gateway implementations, through which legacy fibre channel devices are connected to the network and implementations in which a physical mFCP device, with all functionality self-contained, is directly attached to the IP network. 4.1 The IP Fabric The following diagram shows the topology of a fibre channel network. Fibre Channel Network +--------+ +--------+ +--------+ +--------+ | FC | | FC | | FC | | FC | | Device | | Device |<-------->| Device | | Device | |........| |........| |........| |........| | N_PORT | | N_PORT | | N_PORT | | N_PORT | +---+----+ +----+---+ +----+---+ +----+---+ | | | | +---+----+ +----+---+ +----+---+ +----+---+ | F_PORT | | F_PORT | | F_PORT | | F_PORT | +========+===+========+==========+========+==+========+==== | Fabric | | & | | Fabric Services | +-----------------------------------------------------+ The equivalent IP fabric implemented with gateways is shown below. Monia, Tseng Category - Informational 4 FCP Architectural Framework November 2000 An IP Fabric Implementation Logical FCP Device ^ | +--------+ +--------+ +--------+ +--------+ | | | | | | | | | | FC | | FC | | FC | | FC | | Device | | Device | Fibre | Device | | Device | Fibre |...|....| |........| Channel +........+ +........| Channel | N_PORT | | N_PORT |<--------->| N_PORT | | N_PORT | Device | | | | | | Traffic | | | | Domain +---+----+ +---+----+ +----+---+ +----+---+ | | | | +---+----+ +---+----+ +----+---+ +----+---+<----+ | F_PORT | | F_PORT | | F_PORT | | F_PORT | | =+========+==+====================+========+==+========+========== | | iFCP or mFCP | | iFCP or mFCP | | | | Layer |<--------->| Layer | Gateway |....................| Control |....................| | | | FCP Portal | Messages | FCP Portal | | +---|----+-----------+ +----------+---------+<----+ | | | V | | IP Fabric | | |<------Encapsulated Frames------->| | +------------------+ | | | | | +------+ IP Network +--------+ | | +------------------+ Here, fibre channel devices are connected to the fabric through F_PORTs implemented as part of the gateway (or equivalent edge switch). From the N_PORT interface, the fibre channel network in this example appears as a four-port fibre channel fabric. From the IP network, each fibre channel device appears as a logical FCP device, having an FCP portal, an iFCP or mFCP protocol layer, along with N_PORT and FC device functionality. The left-most vertical arrow in the above diagram relates the layers comprising a logical FCP device to the device's physical implementation in a gateway. N_PORT to N_PORT communications that traverse the IP network require the services of the iFCP or mFCP protocol layer within the gateway. This is done through the following operations on fibre channel frames: a) Map N_PORT addresses between the fibre channel device domain and the IP fabric. Monia, Tseng Category - Informational 5 FCP Architectural Framework November 2000 b) Service requests for fabric-supplied link services directed to one of the well-known fibre channel N_PORT addresses. c) Generate out-of-band frames in response to certain link service requests as described below. d) Encapsulate fibre channel frames for injection into the IP network and de-encapsulate frames received from the IP network. e) Direct de-encapsulated, in-band frames to the appropriate N_PORT. After address mapping on outbound frames, the protocol layer generates two types of traffic: a) FC frames to be passed between N_PORTs unchanged, except for the address field modifications described below. b) Control messages generated in response to certain N_PORT Link Service requests or protocol-specific connection management requests. These messages are passed between peer FCP portals for processing within the mFCP or iFCP protocol layer. For fibre channel link service messages, the protocol layer augments the FC payload with information needed to perform these service requests in the IP fabric domain. 4.2 A Physical mFCP Device Implementation The following diagram shows an mFCP fabric containing a physical mFCP device and two logical FCP devices. Monia, Tseng Category - Informational 6 FCP Architectural Framework November 2000 mFCP Fabric with Physical mFCP Device Logical FCP Device ^ | +------------+ +--------+ +--------+ | Physical | | FC | | FC| | | mFCP | | Device | | Device | Fibre | Device | | | | | | Channel |............| Fibre Channel |........| |........| Device | N_PORT |<------------->| N_PORT | | N_PORT | Domain |............| Traffic +----+---+ +----+---+ | | | | | | +----+---+ +----+---+<----+ | mFCP | | F_PORT | | F_PORT | | | Layer | Control ==+========+==+========+=========== | |<------------->| mFCP Layer | | Gateway |............| Messages |....................| | | FCP Portal | | FCP Portal | | | +-----+------+ +----------+---------+<----+ | Translated Frames | | |<---- with IP Encapsulation----->| V | +--------------------+ | | | | | +----+ IP Network +-------+ | | +--------------------+ For a physical mFCP device, the major difference is that, unlike a gateway-attached N_PORT, the N_PORT within the device is aware of the underlying IP environment. Hence mFCP address mapping is not required, as the device makes direct use of IP addresses. Within the IP network, the physical and logical mFCP devices are functionally equivalent. 4.3 Frame Address Mapping and IP Encapsulation As mentioned above, a gateway that supports either the iFCP or mFCP protocols is responsible for the mapping between N_PORT fibre channel addresses and N_PORT network addresses. In an IP fabric, the N_PORT network address has two components: the IP address of the FCP portal to which the fibre channel device is attached and an N_PORT ID that is unique within the scope of the FCP portal. When an FC frame is in transit, the IP components of the source and destination FCP portals are in the IP header. The source and destination N_PORT IDs are stored in the corresponding N_PORT S_ID and D_ID fields that are part of the FC frame. Monia, Tseng Category - Informational 7 FCP Architectural Framework November 2000 The gateway is responsible for assigning fibre channel N_PORT IDs to directly attached N_PORTs and maintaining a table of IP addresses and N_PORT IDs of all external N_PORTs to which directly attached devices have an established port login session. The gateway builds the store of N_PORT network addresses for external devices by: a) Intercepting name service requests issued by directly attached N_PORTs. These requests are transparently redirected to the iSNS name server. The name server returns the N_PORT network address, consisting of IP address of the FCP portal and the N_PORT ID. b) Intercepting incoming requests from external N_PORTs to initiate a fibre channel session with a directly attached device. In this case, the N_PORT network address of the originating device is extracted and saved. After saving this information in the store of N_PORT network addresses, the protocol layer then creates a 24-bit key that is returned to the directly-attached N_PORT as the N_PORT fibre channel address of the external device. The entry and key may be reused when other directly attached devices reference the same external N_PORT. For outbound frames, the table of external N_PORT network addresses is referenced to translate an N_PORT key to an IP address_N_PORT ID pair. The translation process for outbound frames is shown below. Monia, Tseng Category - Informational 8 FCP Architectural Framework November 2000 Raw Fibre Channel Frame +--------+-----------------------------------+ +--------------+ | | Destination N_PORT Key (D_ID) |---->| | | | | | IP fabric | +--------+-----------------------------------+ | address | | | Source N_PORT ID (S_ID) | | Lookup | +--------+-----------------------------------+ +--------------+ | | | Dest | Control information | | Address | and Payload | | & +--------------------------------------------+ | N_PORT | ID | Frame After Address Translation and IP Encapsulation | +--------------------------------------------+ | | IP Header | | | | | | IP Destination Address<-------------------------+ | IP Source Address | | +--------------------------------------------+ | | UDP Transport Header | | +--------------------------------------------+ | | mFCP Encapsulation Header | | +========+===================================+ <-----+ | | | Destination N_PORT ID (D_ID) |<------|----+ +--------+-----------------------------------+ | | | Source N_PORT ID (S_ID) | | +--------+-----------------------------------+ FC Frame | | | | Control information | | | and Payload | | +--------------------------------------------+ <-----+ For inbound frames, the store regenerates the internal address key from the source IP address and N_PORT ID contained in the encapsulated FC frame. The translation process for inbound frames is shown below. Monia, Tseng Category - Informational 9 FCP Architectural Framework November 2000 Network Format of Inbound Frame +--------------------------------------------+ | IP Header | | | +---------+ | IP Source Address------------------|------>| | | IP Destination Address | | Address | +--------------------------------------------+ | Key | | UDP Transport Header | | Lookup | +--------------------------------------------+ | | | mFCP Encapsulation Header | | | +========+===================================+ | | | | Destination N_PORT ID (D_ID) | | | +--------+-----------------------------------+ | | | | Source N_PORT ID (S_ID) |------>| | +--------+-----------------------------------+ +-----+---+ | | | Address | Control information | | Key | and Payload | | +--------------------------------------------+ | | | | Frame after Address Translation and De-encapsulation | +--------+-----------------------------------+ | | | Destination N_PORT ID (D_ID) | | +--------+-----------------------------------+ | | | Source N_PORT Key (S_ID) |<------------+ +--------+-----------------------------------+ | | | Control information | | and Payload | +--------------------------------------------+ 4.4 Control Traffic [TBD] 4.5 iFCP Session Model and TCP/IP Encapsulation [TBD] 5 The iSNS Name Server [TBD] 6 Compliance With Standards Controlled by T10 and T11 [TBD] 7 Security Considerations Monia, Tseng Category - Informational 10 FCP Architectural Framework November 2000 A subsequent draft of this document will address security issues related to deploying the technology in the public internet environment. 8 Fibre Channel Network Overview This section contains a brief discussion of the fibre channel concepts needed to understand the architecture described in this document. The reader is advised to consult the documents in section 9.4 for a thorough treatment of the technology. 8.1 The Fibre Channel Network The fundamental entity in fibre channel is the fibre channel network. As shown in the diagram below, a fibre channel network is comprised of the following elements: a) N_PORTs -- The end points for fibre channel traffic, b) FC Devices _ The fibre channel devices to which the N_PORTs provide access. c) F_PORTs -_ The ports within a fabric that provide fibre channel attachment for an N_PORT, d) The fabric infrastructure for carrying frame traffic between N_PORTs, e) Within the fabric, a set of auxiliary services and a name service for device discovery and address resolution. Fibre Channel Network +--------+ +--------+ +--------+ +--------+ | FC | | FC | | FC | | FC | | Device | | Device |<-------->| Device | | Device | |........| |........| |........| |........| | N_PORT | | N_PORT | | N_PORT | | N_PORT | +---+----+ +----+---+ +----+---+ +----+---+ | | | | +---+----+ +----+---+ +----+---+ +----+---+ | F_PORT | | F_PORT | | F_PORT | | F_PORT | +========+===+========+==========+========+==+========+==== | Fabric | | & | | Fabric Services | +-----------------------------------------------------+ The following sections describe the internals of a fibre channel fabric and give an overview of the fibre channel communications model. 8.1.1 Multi-Switch Fibre Channel Fabric The internals of a multi-switch fibre channel fabric are shown below. Monia, Tseng Category - Informational 11 FCP Architectural Framework November 2000 Fibre Channel Fabric Internals +----------+ +----------+ | FC | | FC | | Device | | Device | |..........| |..........| Fibre Channel | N_PORT |<-------->| N_PORT | Device Domain +----+-----+ +-----+----+ | | +----+-----+ +-----+----+ | F_PORT | | F_PORT | ==========+==========+==========+==========+============== | FC | | FC | | Switch | | Switch | +----------+ +----------+ Fibre Channel |Inter- | |Inter- | Fabric |Switch | |Switch | |Interface | |Interface | +-----+----+ +-----+----+ | | | | +-----+----+----------+-----+----+ |Inter- | |Inter- | |Switch | |Switch | |Interface | |Interface | +----------+ +----------+ | FC Switch | | | +--------------------------------+ The interface between switch elements is either proprietary or a standards-compliant E_PORT interface described by the FC-SW2 specification. 8.2 Fibre Channel Layers and Link Services Fibre channel consists of the following layers: FC0 -- The interface to the physical media, FC1 _- The encoding and decoding of data and out-of-band physical link control information for transmission over the physical media, FC2 _- The transfer of frames, sequences and exchanges comprising protocol information units. FC3 _- Common Services, FC4 _- Application protocols, such as FCP, the fibre channel SCSI protocol. In addition to the layers defined above, fibre channel defines a set of auxiliary operations, some of which are implemented in the transport layer, called link services. These are required to manage the fibre channel environment, establish communications with other devices, retrieve error information, perform error recovery and other similar services. Some link services are executed by the Monia, Tseng Category - Informational 12 FCP Architectural Framework November 2000 N_PORT. Others are implemented internally within the fabric. These internal services are described in the next section. 8.2.1 Fabric-Supplied Link Services Servers internal to the fabric handle certain classes of Link Service requests. The servers appear as N_PORTS located at well- known N_PORT fabric addresses. Service requests use the standard fibre channel mechanisms for N_PORT-to-N_PORT communications. The following services must be provided by all fabrics: Fabric F_PORT server _ Services an N_PORT request to access the fabric for communications. Fabric Controller -- Provides state change information to other N_PORTS. Used to inform other FC devices when an N_PORT exits or enters the fabric. Directory/Name Server _ Allows N_PORTs to register information in a database or retrieve information about other N_PORTs. The following optional services are defined: Broadcast Address/Server _- Transmits single-frame, class 3 sequences to all N_PORTs. Time Server _- Intended for the management of fabric-wide expiration timers or elapsed time values and is not intended for precise time synchronization. Management Server _ Collects and reports management information, such as link usage, error statistics, link quality and similar items. Quality of Service Facilitator _ For fabric-wide bandwidth and latency management. 8.3 Fibre Channel Devices A fibre channel device has one or more fabric-attached N_PORTs. The device and its N_PORTs have the following associated identifiers: a) A world-wide unique identifier for the device, b) A world-wide unique identifier for each N_PORT attached to the device, c) For each N_PORT, a fabric-assigned N_PORT fabric address provided when the device is granted fabric access. This address is unique within the scope of the fabric. Monia, Tseng Category - Informational 13 FCP Architectural Framework November 2000 Information about a fibre channel device, such as the fibre channel addresses and world wide names of its N_PORTs, can be discovered through the appropriate name service queries. 8.4 Fibre Channel Information Elements The fundamental element of information in fibre channel is the frame. A frame consists of a fixed header and up to 2112 bytes of payload having the structure described in section 4.4.1. The maximum frame size that may be transmitted between a pair of fibre channel devices is negotiable up to the payload limit, based on the size of the frame buffers in each fibre channel device and the MTU supported by the fabric. Operations involving the transfer of information between N_PORT pairs are performed through exchanges. In an exchange, information is transferred in one or more ordered series of frames referred to as sequences. Within a sequence, frames flow from the sequence originator to the sequence recipient. Control information within the frame: a) Delimits sequence boundaries, b) Identifies the position of a frame within a sequence, c) Provides for the initiation of a new sequence reversing the direction of data flow when required by the upper layer protocol. Within this framework, an upper layer protocol is defined in terms of transactions carried by exchanges. Each transaction, in turn, consists of protocol information units, each of which is carried by an individual sequence within an exchange. 8.4.1 Fibre Channel Frame Format A fibre channel frame consists of the payload and a header containing the control information necessary to route frames between N_PORTs and manage exchanges and sequences. The following diagram gives a highly simplified view of the frame, with non-relevant detail omitted. Monia, Tseng Category - Informational 14 FCP Architectural Framework November 2000 Fibre Channel Frame Format +-----+-----------------------+<----+ | | Destination N_PORT | | | | Fabric Address (D_ID) | | | | (24-bits) | | +-----+-----------------------+ 24-byte | | Source N_PORT | Frame | | Fabric Address (S_ID) | Header | | (24 bits) | | +-----+-----------------------+ | | Control information | | | for exchange management, | | | IU segmentation and | | | re-assembly | | +-----------------------------+<----+ | | | Frame payload | | (0 _ 2112 bytes) | | | | | | | +-----------------------------+ The source and destination N_PORT fabric addresses are embedded in the S_ID and D_ID fields respectively. 8.5 Fibre Channel Transport Services The fibre channel standard defines the following classes of service provided by a fabric implementation: Class 1 _ A dedicated physical circuit connecting two N_PORTs. Class 2 _ A frame-multiplexed connection with end-to-end flow control and delivery confirmation. Class 3 _ A frame-multiplexed connection with no provisions for end- to-end flow control or delivery confirmation. For class 2 or class 3 service, the fabric is not required to preserve frame ordering. Class 3 service is equivalent to UDP or IP datagram service. Fibre channel storage devices commonly use this class of service, relying on the ULP implementation to detect and recover from transient device and transport errors. In addition to the above services, fabrics may implement additional quality of service policies within the framework of class 2 or class 3 delivery mechanisms. Monia, Tseng Category - Informational 15 FCP Architectural Framework November 2000 8.6 N_PORT to N_PORT Communication An N_PORT joins the fabric and establishes a session with another N_PORT by invoking the following series of services: a) F_PORT login _- The device invokes the fabric login service to register its presence on the fabric and obtain an N_PORT fabric address. b) Name Service Lookup - The device obtains the N_PORT fabric address of another device through a name service query. c) N_PORT login - The device issues a port login request to establish an N_PORT-to-N_PORT session. d) Process Login _ The device performs a process login to establishes a ULP session with a peer process on the remote N_PORT. An N_PORT issues a corresponding set of logout requests to gracefully terminate the ULP and N_PORT sessions and fabric login. 9 References 9.1 Relevant SCSI (T10) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854- 7179 or (303) 792-2181, Fax: (303) 792-2192 SCSI-3 Architecture Model (SAM), ANSI X3.270-1996 SCSI Architecture Model-2 (SAM-2), Project 1157-D, revision 11 SCSI Primary Commands (SPC), ANSI X3.301-1997 SCSI Primary Commands-2 (SPC-2), Project 1236-D, revision 16 Fibre Channel Protocol for SCSI (FCP), ANSI X3.269-1996 Fibre Channel Protocol for SCSI, Second Revision (FCP-2), Project 1144D, revision 04 9.2 Relevant Fibre Channel (T11) Specifications The following documents are available from: Global Engineering, 15 Inverness Way East, Englewood, CO 80112-5704. Telephone (800) 854- 7179 or (303) 792-2181, Fax: (303) 792-2192 Fibre Channel Physical and Signaling Interface (FC-PH) Rev 4.3, ANSI X3.230:1994 Fibre Channel Physical and Signaling Interface (FC-PH-2) Rev 7.4, ANSI X3.297:1997 Monia, Tseng Category - Informational 16 FCP Architectural Framework November 2000 Fibre Channel Physical and Signaling Interface (FC-PH-3) Rev 9.4, ANSI X3.303:1998 Fibre Channel Generic Requirements (FC-FG) Rev 3.5, ANSI X3.289:1996 Fibre Channel Generic Services (FC-GS-2) Rev 5.2, ANSI NCITS 288 Fibre Channel Arbitrated Loop (FC-AL) Rev 4.5, ANSI X3.272:1996 Fibre Channel Arbitrated Loop (FC-AL-2) Rev 7.0, NCITS 332:1999 Fibre Channel Private Loop SCSI Direct Attachment (FC-PLDA), NCITS TR-19:1998 Fibre Channel Fabric Loop Attachment (FC-FLA), NCITS TR-20:1998 9.3 Relevant RFC Documents RFC768 User Datagram Protocol RFC791 Internet Protocol, DARPA Internet Program Protocol Specification RFC2460 Internet Protocol, Version 6 (IPv6) Specification 9.4 Other Reference Documents Fibre Channel, Gigabit Communications and I/O for Computer Networks, Alan F. Beener, McGraw-Hill, ISBN 0-07-005669-2 The Fibre Channel Consultant, A Comprehensive Introduction, Robert W. Kembel, Northwest Learning Associates, ISBN 0-931836-82-6 The Fibre Channel Consultant, Arbitrated Loop, Robert W. Kembel, Connectivity Solutions, a division of Northwest Learning Associates, ISBN 0-931836-84-0 What is Fibre Channel? (4th Ed.),J. Dedik, isbn 0-963743-99-6 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. 10. Author's Addresses Charles Monia Josh Tseng Nishan Systems 3850 North First Street San Jose, CA 95134 Phone: 408-519-3986 Email: cmonia@nishansystems.com Monia, Tseng Category - Informational 17 FCP Architectural Framework November 2000 Full Copyright Statement "Copyright (C) The Internet Society (date). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implmentation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into Monia, Tseng Category - Informational 18