TOC 
Internet Engineering Task ForceE. Haleplidis
Internet-DraftUniversity of Patras
Intended status: InformationalK. Ogawa
Expires: March 11, 2010NTT Corporation
 X. Wang
 Huawei Technologies Co., Ltd.
 C. Li
 Zhejiang Gongshang University
 September 07, 2009


ForCES Interoperability Draft
draft-ietf-forces-interoperability-04

Status of this Memo

This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as “work in progress.”

The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt.

The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html.

This Internet-Draft will expire on March 11, 2010.

Copyright Notice

Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.

Abstract

This document describes the details of the interoperability test of the Forward and Control Element Separation (ForCES) protocol that took place in the University of Patras in Rio, Greece, 15 and 16 July 2009. This informational draft provided necessary information, for all parties who wish to participate in the interoperability test.

This update also includes the results of the test.



Table of Contents

1.  Terminology and Conventions
    1.1.  Requirements Language
2.  Introduction
    2.1.  ForCES Protocol
    2.2.  ForCES Model
    2.3.  Transport mapping layer
3.  Definitions
4.  Date, Location and Access
    4.1.  Date
    4.2.  Location
    4.3.  Access
5.  Testbed architecture
    5.1.  Local configuration
    5.2.  Distributed configuration
6.  Scenarios
    6.1.  Scenario 1 - Pre-association Setup
    6.2.  Scenario 2 - TML priority channels connection
    6.3.  Scenario 3 - Association Setup - Association Complete
    6.4.  Scenario 4 - CE query
    6.5.  Scenario 5 - Heartbeat monitoring
    6.6.  Scenario 6 - Simple Config Command
    6.7.  Scenario 7 - Association Teardown
7.  Tested Features
    7.1.  ForCES Protocol Features
        7.1.1.  Protocol Messages
        7.1.2.  MainHeader Handling
        7.1.3.  TLV Handling
        7.1.4.  Operation Types Supported
        7.1.5.  ForCES Protocol Advanced Features
    7.2.  ForCES Model Features
        7.2.1.  Basic Atomic Types Supported
        7.2.2.  Compound Types Supported
        7.2.3.  LFBs Supported
            7.2.3.1.  FE Protocol LFB
            7.2.3.2.  FE Object LFB
    7.3.  ForCES SCTP-TML Features
        7.3.1.  TML Priority Ports
        7.3.2.  Message Handling at specific priorities
8.  Test details
9.  Results
10.  Acknowledgements
11.  IANA Considerations
12.  Security Considerations
13.  References
    13.1.  Normative References
    13.2.  Informative References
§  Authors' Addresses




 TOC 

1.  Terminology and Conventions



 TOC 

1.1.  Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] (Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” March 1997.).



 TOC 

2.  Introduction

Forwarding and Control Element Separation (ForCES) defines an architectural framework and associated protocols to standardize information exchange between the control plane and the forwarding plane in a ForCES Network Element (ForCES NE). [RFC3654] (Khosravi, H. and T. Anderson, “Requirements for Separation of IP Control and Forwarding,” November 2003.) has defined the ForCES requirements, and [RFC3746] (Yang, L., Dantu, R., Anderson, T., and R. Gopal, “Forwarding and Control Element Separation (ForCES) Framework,” April 2004.) has defined the ForCES framework.



 TOC 

2.1.  ForCES Protocol

The ForCES protocol works in a master-slave mode in which FEs are slaves and CEs are masters. The protocol includes commands for transport of Logical Function Block (LFB) configuration information, association setup, status, and event notifications, etc. The reader is encouraged to read FE-protocol (Dong, L., Doria, A., Gopal, R., HAAS, R., Salim, J., Khosravi, H., and W. Wang, “ForCES Protocol Specification,” March 2009.) [I‑D.ietf‑forces‑protocol] for further information.



 TOC 

2.2.  ForCES Model

The FE-MODEL (Halpern, J. and J. Salim, “ForCES Forwarding Element Model,” October 2008.) [I‑D.ietf‑forces‑model] presents a formal way to define FE Logical Function Blocks (LFBs) using XML. LFB configuration components, capabilities, and associated events are defined when the LFB is formally created. The LFBs within the FE are accordingly controlled in a standardized way by the ForCES protocol.



 TOC 

2.3.  Transport mapping layer

The TML transports the PL messages. The TML is where the issues of how to achieve transport level reliability, congestion control, multicast, ordering, etc. are handled. It is expected that more than one TML will be standardized. The various possible TMLs could vary their implementations based on the capabilities of underlying media and transport. However, since each TML is standardized, interoperability is guaranteed as long as both endpoints support the same TML. All ForCES Protocol Layer implementations MUST be portable across all TMLs. Although more than one TML may be standardized for the ForCES Protocol, for the purposes of the interoperability test, the mandated MUST IMPLEMENT SCTP TML [I‑D.ietf‑forces‑sctptml] (Salim, J. and K. Ogawa, “SCTP based TML (Transport Mapping Layer) for ForCES protocol,” January 2009.) will be used.



 TOC 

3.  Definitions

This document follows the terminology defined by the ForCES Requirements in [RFC3654] (Khosravi, H. and T. Anderson, “Requirements for Separation of IP Control and Forwarding,” November 2003.) and by the ForCES framework in [RFC3746] (Yang, L., Dantu, R., Anderson, T., and R. Gopal, “Forwarding and Control Element Separation (ForCES) Framework,” April 2004.). The definitions below are repeated below for clarity.



 TOC 

4.  Date, Location and Access



 TOC 

4.1.  Date

The date that the Interoperability test took place was 15-16/07/2009, one and a half week before IETF 75, in Stockholm.



 TOC 

4.2.  Location

Patras is a major harbor of Greece connecting it with Italy.

The University of Patras is located in Rio, 10km east out of Patras.

The following coordinates mark the Electrical and Computer Engineering building in the University.



 TOC 

4.3.  Access

The best way to come to Greece is by plane to the Athens International Airport.

From there there are three ways to arrive in the University of Patras.

  1. Renting a car and driving to the University. It is a maximum 2:30 hours drive from the aiport.
  2. Via coach station. Get from the airport to the coach station via X93 bus towards the Kifissos Coach Station. At the Coach Station there are buses to Patras every 30 minutes. The Bus to Patras may take about 2:30 - 3:00 hours, and the ride of the X93 bus may take about 30 mins - 1hour depending on the traffic, so it's about 3:30 - 4:30 hours away with the wait at the Coach Station.
  3. Via Train. It is recommended you already have booked your ticket beforehand as there are not many trains going to Patras, and mostly are booked in advanced. It is not recommended that you take the train to Patras, as you have to change at least 2 trains. In order to reach Patras from the Athens International Airport you need to take the Suburban Rail to Kiato. From Kiato you can catch a train to Patras. It will take you at least 5 hours to reach Patras.



 TOC 

5.  Testbed architecture

Most FEs and CEs were located locally at the University of Patras premises. One party participated connecting over the internet.

The test took place between FEs and CEs of different implementors with different permutations.

All protocol messages of each scenario were monitored using a protocol network analyzer that tested validity. Two tools were used:

All NE's in all the scenarios were comprised of one CE and one FE from different implementors.



 TOC 

5.1.  Local configuration

Hardware/Software (CEs and FEs) that were located within the University of Patras premises, were connected together using switches.

The scenarios were tested with only one CE associated with one or multiple FEs from different implementors. The CE and the FE(s) were connected in one LAN as shown in the following figure.

               +-----+
               | CE1 |
               |Impl1|
               +-----+
                  |
                  |
+------------------------------------+
|                LAN                 |
+------------------------------------+
   |       |         |          |
   |       |   ...   |          |
+-----+ +-----+   +-----+   +--------+
| FE1 | | FE2 |   | FEn |   |Protocol|
|Impl1| |Impl2|   |Impln|   |Analyzer|
+-----+ +-----+   +-----+   +--------+

All scenarios were tested more than once with permutation of the CE from different implementors. In the next permutation, the setup were as shown in the following figure.

               +-----+
               | CE2 |
               |Impl2|
               +-----+
                  |
                  |
+------------------------------------+
|                LAN                 |
+------------------------------------+
   |       |         |          |
   |       |   ...   |          |
+-----+ +-----+   +-----+   +--------+
| FE1 | | FE2 |   | FEn |   |Protocol|
|Impl1| |Impl2|   |Impln|   |Analyzer|
+-----+ +-----+   +-----+   +--------+


 TOC 

5.2.  Distributed configuration

For parties that cannot participate, public IPs can be provided and associations can be achieved over the internet as seen in the following figure.

+-----+   +------------+   /\/\/\/\/\   +----------+   +-----+
|FE/CE|   |Implementor |   \Internet/   |University|   |FE/CE|
|ImplX|---|   Router   |---/        \---|  Router  |---|ImplY|
+-----+   +------------+   \/\/\/\/\/   +----------+   +-----+

For interoperability issues, all CEs and FEs MUST implement no security even in the TML. For security, firewalls MUST be used that will allow only the specific IPs and the SCTP ports defined in the SCTP-TML draft (Salim, J. and K. Ogawa, “SCTP based TML (Transport Mapping Layer) for ForCES protocol,” January 2009.) [I‑D.ietf‑forces‑sctptml].



 TOC 

6.  Scenarios

Since the main goal of this interoperability test is to test the basic protocol functionality, we will limit the test parameters. Therefore:

  1. In the Association Setup Message, all report messages will be ignored.
  2. In the Association Setup Phase, the messages, FEO OperEnable Event (FE to CE), Config FEO Adminup (CE to FE) and FEO Config-Resp (FE to CE) will be ignored. The CE will assume that the FE is enabled once the LFBSelectors has been queried.
  3. Only FullDataTLVs are going to be used and not SparseData TLV's.
  4. There will be no transaction operations.
  5. Each message shall have only one LFBSelector TLV, one Operation TLV and one PathDataTLV per message when these are used.


 TOC 

6.1.  Scenario 1 - Pre-association Setup

While the Pre-association setup is not in the ForCES current scope it is an essential step before CEs and FEs communicate. As the first part in a successfull CE-FE connection the participating CEs and FEs should be able to be configured.

In the Pre-association Phase the following configuration items MUST be setup regarding the CEs:

In the Pre-association Phase the following configuration items MUST be setup regarding the FEs:

Once each element is setup and configured, Scenario 1 is successful.



 TOC 

6.2.  Scenario 2 - TML priority channels connection

For the current interoperability test, the SCTP will be used as TML. The TML connection with the associating element is needed for the scenario 2 to be successful.

The SCTP-TML draft (Salim, J. and K. Ogawa, “SCTP based TML (Transport Mapping Layer) for ForCES protocol,” January 2009.) [I‑D.ietf‑forces‑sctptml] defines 3 priority channels, with specific ports:

Once these channels have been established with each associated element, will the Scenario 2 be successful.



 TOC 

6.3.  Scenario 3 - Association Setup - Association Complete

Once the Pre-association phase has been complete in the previous 2 scenarios, CEs and FEs are ready to communicate using the ForCES protocol, and enter the Association Setup stage. In this stage the FEs attempts to join the NE. The following ForCES protocol messages will be exchanged for each CE-FE pair in the specified order:

Once the associations has been initialized scenario 3 will have been successful.



 TOC 

6.4.  Scenario 4 - CE query

Once the Association Phase stage has been complete, the FEs and CEs will enter the Established stage. In this stage the FE is continuously updated or queried. The CE should query the FE a specific value from the FE Object LFB and from the FE Protocol LFB. An example from the FE Protocol LFB is the HeartBeat Timer (FEHI) and from the FE Object LFB is the State of the LFB (FEState)

The following ForCES protocol messages will be exchanged:



 TOC 

6.5.  Scenario 5 - Heartbeat monitoring

The Heartbeat (HB) Message is used for one ForCES element (FE or CE) to asynchronously notify one or more other ForCES elements in the same ForCES NE on its liveness. The default configuration of the Heartbeat Policy of the FE is set to 0 which means, that the FE should not generate any Heartbeat messages. the CE is responsible for checking FE liveness by setting the PL header ACK flag of the message it sends to AlwaysACK. In this Scenario the CE should send a Heartbeat message with the ACK flag set to AlwaysACK and the FE should respond.

The following ForCES protocol messages will be exchanged:



 TOC 

6.6.  Scenario 6 - Simple Config Command

A config message is sent by the CE to the FE to configure LFB components in the FE. A simple config command easily visible and metered would be to change the Heartbeat configuration. This will be done in two steps:

  1. Change the FE Heartbeat Policy (FEHBPolicy) to value 1, to force the FE to send heartbeats.
  2. After some heartbeats from the FE, the FE Heartbeat Interval (FEHI) will be changed.

The following ForCES protocol messages will be exchanged:



 TOC 

6.7.  Scenario 7 - Association Teardown

In the end, the association must be terminated. There are two scenarios by which the association maybe terminated:

  1. Normal tear down by exchanging Association Teardown Message
  2. Irregular tear down by stopping heartbeats from a FE or a CE.
  3. Irregular tear down by externally shutting down/rebooting a FE or a CE.

All scenarios may be tested in the interoperability test.

The following ForCES protocol messages will be exchanged:



 TOC 

7.  Tested Features

The features that were tested are:



 TOC 

7.1.  ForCES Protocol Features



 TOC 

7.1.1.  Protocol Messages



Protocol Message
Association Setup
Association Setup Response
Association TearDown
Configuration
Configuration Response
Query
Query Response
HeartBeat

 ForCES Protocol Message 



 TOC 

7.1.2.  MainHeader Handling



Header Field
Correlator
Acknowledge Flag
Priority Flag

 MainHeader Handling 



 TOC 

7.1.3.  TLV Handling



TLV
Association Setup Result TLV
Association TearDown Reason TLV
LFBSelector TLV
Operation TLV
PathData TLV
FullData TLV
Result TLV

 TLVs Supported 



 TOC 

7.1.4.  Operation Types Supported



Operation
Set
Set Response
Get
Get Response
Report

 Operation Type Supported 



 TOC 

7.1.5.  ForCES Protocol Advanced Features



Feature
Batching
HeartBeats

 ForCES Protocol Advanced Features 

Although Batching was not initially designed to be tested, it was tested during the interoperability test.



 TOC 

7.2.  ForCES Model Features



 TOC 

7.2.1.  Basic Atomic Types Supported



Atomic Type
uchar
uint32

 Basic Atomic Types Supported 



 TOC 

7.2.2.  Compound Types Supported



Compound Type
structs
arrays

 Compound Types Supported 



 TOC 

7.2.3.  LFBs Supported



 TOC 

7.2.3.1.  FE Protocol LFB



Protocol DataTypes
CEHBPolicy
FEHIBPolicy

 FE Protocol LFB Datatypes 



Protocol Components
FEID
CEHBPolicy
CEHDI
FEHBPolicy
FEHI
CEID

 FE Protocol LFB Components 



 TOC 

7.2.3.2.  FE Object LFB



Object DataTypes
FEStateValues
LFBSelectorType

 FE Object LFB Datatypes 



Object Components
LFBSelectors
FEState

 FE Object LFB Components 



 TOC 

7.3.  ForCES SCTP-TML Features



 TOC 

7.3.1.  TML Priority Ports



Port
High priority (6700)
Medium priority (6701)
Low priority (6702)

 Priority Ports 



 TOC 

7.3.2.  Message Handling at specific priorities



ForCES Message
Association Setup
Association Setup Response
Association Teardown
Config
Config Response
Query
Query Response

 Message Handling at High priority (6700) Port 



ForCES Message
Heartbeats

 Message Handling at Low priority (6702) Port 



 TOC 

8.  Test details

The following tests occured:



Test#CEFE(s)Teardown OptionResultComment
1 Zhejiang Gongshang University NTT Teardown from FE Success  
2 Zhejiang Gongshang University NTT Teardown from CE Success  
3 Zhejiang Gongshang University NTT Cable disconnect Success Nobody saw the loss of cable. Everybody found out from loss of PL-heartbeats
4 Zhejiang Gongshang University NTT Loss of CE Heartbeats Success FE didn't send Teardown and closed connection
5 Zhejiang Gongshang University NTT Loss of FE Heartbeats Untestable  
6 NTT Zhejiang Gongshang University Teardown from CE Initial Failure CE couldn't handle Query Result for unknown LFBSelects.
7 Zhejiang Gongshang University University of Patras Teardown from FE Success Problems with retransmittion
8 Zhejiang Gongshang University University of Patras Teardown from CE Success Problems with retransmittion
9 Zhejiang Gongshang University University of Patras Cable disconnect Success Nobody saw the loss of cable. Everybody found out from loss of PL-heartbeats
10 Zhejiang Gongshang University University of Patras Loss of CE Heartbeats Success  
11 NTT Zhejiang Gongshang University Teardown from CE Success on Repeat Test# 6. Problems fixed
12 NTT Zhejiang Gongshang University Teardown from FE Success  
13 NTT Zhejiang Gongshang University Cable disconnect Success Nobody saw the loss of cable. Everybody found out from loss of PL-heartbeats.
14 NTT Zhejiang Gongshang University Loss of CE Heartbeats Success Problems with retransmittion
15 University of Patras Zhejiang Gongshang University Teardown from FE Success CE didn't terminat after sending Teardown. FE did
16 University of Patras Zhejiang Gongshang University Teardown from CE Success Problems with retransmittion
17 University of Patras Zhejiang Gongshang University Loss of CE Heartbeats Success FE didn't send Teardown and closed connection
18 Zhejiang Gongshang University NTT & University of Patrasx2 Teardown from CE Success  
19 NTT Zhejiang Gongshang University & University of Patrasx2 Teardown from CE Success  
20 University of Patras NTT & Zhejiang Gongshang University & University of Patrasx2 Teardown from CE Success  
21 University of Patras Zhejiang Gongshang University Batching Query and Config Success  
22 University of Patras NTT Teardown from FE Success  
23 University of Patras NTT Teardown from CE Success  
24 University of Patras NTT Loss of CE Heartbeats Success FE didn't send Teardown and closed connection
25 University of Patras NTT Cable disconnect Success Nobody saw the loss of cable. Everybody found out from loss of PL-heartbeats
26 NTT University of Patras Teardown from FE Success  
27 NTT University of Patras Teardown from CE Success  
28 NTT University of Patras Loss of CE Heartbeats Success FE didn't send Teardown and closed connection
29 NTT University of Patras Cable disconnect Success Nobody saw the loss of cable. Everybody found out from loss of PL-heartbeats

 Interoperability Tests 



 TOC 

9.  Results

All implementations were found to be interoperable with each other.

All scenarios were tested successfully.

The following issues were found and dealt with.

  1. Some messages were sent to the wrong priority channels. There was some ambiguities on the SCTP-TML draft that have been corrected.
  2. At some point, a CE sent a TearDown message to the FE. The CE expected the FE to shut down the connection, and the FE waited the CE to shut down the connection and were caught in a deadlock. This was a code bug and was fixed.
  3. Sometimes the association setup message, only on the remote connection test, although sent, was not received by the other end and made impossible the association. This was caused by network problems.
  4. An implementation did not take into account that the padding in TLVs MUST NOT be included in the lenght of the TLV. This was a code bug and was fixed.
  5. EM Flag was set to reserved by a CE and was not ignored by the FE. This was a code bug and was fixed.
  6. After the FEHBPolicy was set to 1 the FE didn't send any HeartBeats. This was a code bug and was fixed.
  7. Some FE's sent HeartBeats with the ACK flag with a value other than NoACK. The CE responded. This was a code bug and was fixed.
  8. When a cable was disconnected, the TML didn't realize that. The association was dropped due to heartbeats, this was a success, but this is an implementation issue implementers should keep in mind. This is a SCTP options issue. Nothing was needed to be done.
  9. A CE crashed due to unknown LFBSelector values. This was a code bug and was fixed.
  10. With the remote connection there were a lot of forces packet retransmittion. The problem is that packets like Heartbeats were retransmitted. This is a SCTP issue. Perhaps SCTP-PR is needed to be used.

The implementers went beyond the call of duty. The test was extended with another test for batching messages. This test was also done successfully.



 TOC 

10.  Acknowledgements

The authors of this draft would like to acknowledge and thank the chair of the ForCES working group Jamal Hadi Salim.

Also, the authors would like to acknowledge Professors Odysseas Koufopavlou and Spyros Denazis, as well as the Department of Electrical and Computer Engineering of the University of Patras for hosting the event.



 TOC 

11.  IANA Considerations

This memo includes no request to IANA.



 TOC 

12.  Security Considerations

Section 9 of the FE-protocol (Dong, L., Doria, A., Gopal, R., HAAS, R., Salim, J., Khosravi, H., and W. Wang, “ForCES Protocol Specification,” March 2009.) [I‑D.ietf‑forces‑protocol] specifies security considerations of the ForCES protocol. For this interoperability test, no security MUST be chosen even for the distributed architecture.



 TOC 

13.  References



 TOC 

13.1. Normative References

[I-D.ietf-forces-model] Halpern, J. and J. Salim, “ForCES Forwarding Element Model,” draft-ietf-forces-model-16 (work in progress), October 2008 (TXT).
[I-D.ietf-forces-protocol] Dong, L., Doria, A., Gopal, R., HAAS, R., Salim, J., Khosravi, H., and W. Wang, “ForCES Protocol Specification,” draft-ietf-forces-protocol-22 (work in progress), March 2009 (TXT).
[I-D.ietf-forces-sctptml] Salim, J. and K. Ogawa, “SCTP based TML (Transport Mapping Layer) for ForCES protocol,” draft-ietf-forces-sctptml-02 (work in progress), January 2009 (TXT).


 TOC 

13.2. Informative References

[RFC2119] Bradner, S., “Key words for use in RFCs to Indicate Requirement Levels,” BCP 14, RFC 2119, March 1997 (TXT, HTML, XML).
[RFC2629] Rose, M., “Writing I-Ds and RFCs using XML,” RFC 2629, June 1999 (TXT, HTML, XML).
[RFC3552] Rescorla, E. and B. Korver, “Guidelines for Writing RFC Text on Security Considerations,” BCP 72, RFC 3552, July 2003 (TXT).
[RFC3654] Khosravi, H. and T. Anderson, “Requirements for Separation of IP Control and Forwarding,” RFC 3654, November 2003 (TXT).
[RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, “Forwarding and Control Element Separation (ForCES) Framework,” RFC 3746, April 2004 (TXT).
[RFC5226] Narten, T. and H. Alvestrand, “Guidelines for Writing an IANA Considerations Section in RFCs,” BCP 26, RFC 5226, May 2008 (TXT).
[ethereal] Ethereal is a protocol analyzer. The specific ethereal that will be used is an updated Ethereal, by Fenggen Jia, that can analyze and decode the ForCES protocol messages..”
[tcpdump] Tcpdump is a linux protocol analyzer. The specific tcpdump that will be used is a modified tcpdump, by Jamal Hadi Salim, that can analyze and decode the ForCES protocol messages..”


 TOC 

Authors' Addresses

  Evangelos Haleplidis
  University of Patras
  Patras,
  Greece
Email:  ehalep@ece.upatras.gr
  
  Kentaro Ogawa
  NTT Corporation
  Tokyo,
  Japan
Email:  ogawa.kentaro@lab.ntt.co.jp
  
  Xin-ping Wang
  Huawei Technologies Co., Ltd.
  China
Email:  carly.wang@huawei.com
  
  Chuanhuang Li
  Zhejiang Gongshang University
  18, Xuezheng Str., Xiasha University Town
  Hangzhou, 310018
  P.R.China
Phone:  +86-571-28877751
Email:  chuanhuang_li@pop.zjgsu.edu.cn