idnits 2.17.1 draft-ietf-forces-interop-05.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 2 instances of lines with private range IPv4 addresses in the document. If these are generic example addresses, they should be changed to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x, 198.51.100.x or 203.0.113.x. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (January 3, 2013) is 4103 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'RFC2119' is mentioned on line 155, but not defined == Missing Reference: 'FORCES-LFBLIB' is mentioned on line 395, but not defined == Missing Reference: 'RFC2404' is mentioned on line 440, but not defined == Missing Reference: 'ForCES-CEHA' is mentioned on line 445, but not defined == Missing Reference: 'ForCES-LFBLIB' is mentioned on line 517, but not defined == Unused Reference: 'RFC5813' is defined on line 1232, but no explicit reference was found in the text == Unused Reference: 'RFC2629' is defined on line 1254, but no explicit reference was found in the text == Unused Reference: 'RFC3654' is defined on line 1257, but no explicit reference was found in the text == Outdated reference: A later version (-10) exists of draft-ietf-forces-ceha-04 == Outdated reference: A later version (-12) exists of draft-ietf-forces-lfb-lib-09 -- Obsolete informational reference (is this intentional?): RFC 2629 (Obsoleted by RFC 7749) Summary: 0 errors (**), 0 flaws (~~), 12 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force W. Wang 3 Internet-Draft Zhejiang Gongshang University 4 Intended status: Informational K. Ogawa 5 Expires: July 7, 2013 NTT Corporation 6 E. Haleplidis 7 University of Patras 8 M. Gao 9 Hangzhou BAUD Networks 10 J. Hadi Salim 11 Mojatatu Networks 12 January 3, 2013 14 Interoperability Report for Forwarding and Control Element Separation 15 (ForCES) 16 draft-ietf-forces-interop-05 18 Abstract 20 This document captures test results from the second Forwarding and 21 control Element Separation (ForCES) interoperability test which took 22 place on February 24-25, 2011 in the Internet Technology Lab (ITL) of 23 Zhejiang Gongshang University, China. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on July 7, 2013. 42 Copyright Notice 44 Copyright (c) 2013 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents 49 (http://trustee.ietf.org/license-info) in effect on the date of 50 publication of this document. Please review these documents 51 carefully, as they describe your rights and restrictions with respect 52 to this document. Code Components extracted from this document must 53 include Simplified BSD License text as described in Section 4.e of 54 the Trust Legal Provisions and are provided without warranty as 55 described in the Simplified BSD License. 57 Table of Contents 59 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 1.1. ForCES Protocol . . . . . . . . . . . . . . . . . . . . . 3 61 1.2. ForCES FE Model . . . . . . . . . . . . . . . . . . . . . 3 62 1.3. Transport Mapping Layer . . . . . . . . . . . . . . . . . 3 63 2. Terminology and Conventions . . . . . . . . . . . . . . . . . 5 64 2.1. Requirements Language . . . . . . . . . . . . . . . . . . 5 65 2.2. Definitions . . . . . . . . . . . . . . . . . . . . . . . 5 66 3. Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 67 3.1. Date, Location, and Participants . . . . . . . . . . . . . 7 68 3.2. Testbed Configuration . . . . . . . . . . . . . . . . . . 7 69 3.2.1. Participants Access . . . . . . . . . . . . . . . . . 7 70 3.2.2. Testbed Configuration . . . . . . . . . . . . . . . . 8 71 4. Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . 11 72 4.1. Scenario 1 - LFB Operation . . . . . . . . . . . . . . . . 11 73 4.2. Scenario 2 - TML with IPSec . . . . . . . . . . . . . . . 11 74 4.3. Scenario 3 - CE High Availability . . . . . . . . . . . . 12 75 4.4. Scenario 4 - Packet forwarding . . . . . . . . . . . . . . 14 76 5. Test Results . . . . . . . . . . . . . . . . . . . . . . . . . 17 77 5.1. LFB Operation Test . . . . . . . . . . . . . . . . . . . . 17 78 5.2. TML with IPSec Test . . . . . . . . . . . . . . . . . . . 23 79 5.3. CE High Availability Test . . . . . . . . . . . . . . . . 23 80 5.4. Packet Forwarding Test . . . . . . . . . . . . . . . . . . 24 81 6. Discussions . . . . . . . . . . . . . . . . . . . . . . . . . 27 82 6.1. On Data Encapsulation Format . . . . . . . . . . . . . . . 27 83 7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 30 84 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 31 85 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 86 10. Security Considerations . . . . . . . . . . . . . . . . . . . 33 87 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 88 11.1. Normative References . . . . . . . . . . . . . . . . . . . 34 89 11.2. Informative References . . . . . . . . . . . . . . . . . . 34 90 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36 92 1. Introduction 94 This document captures the results of the second interoperability 95 test of the Forwarding and control Element Separation (ForCES) 96 Framework which took place February 24-25, 2011 in the Internet 97 Technology Lab (ITL) of Zhejiang Gongshang University, China. The 98 test involved several documents namely: ForCES protocol [RFC5810] , 99 ForCES FE model [RFC5812] , ForCES TML [RFC5811] , ForCES LFB Library 100 [I-D.ietf-forces-lfb-lib] and ForCES CE HA specification 101 [I-D.ietf-forces-ceha]. Three independent ForCES implementations 102 participated in the test. 104 Scenarios of ForCES LFB Operation, TML with IPSec, CE High 105 Availability, and Packet Forwarding are constructed. Series of 106 testing items for every scenario are carried out and interoperability 107 results are achieved. Popular packet analyzers Ethereal/ 108 Wireshark[Ethereal] and Tcpdump[Tcpdump] are used to verify the wire 109 results. 111 The first interoperability test on ForCES was held in July 2008 at 112 the University of Patras, Greece. The test focused on validating the 113 basic semantics of the ForCES protocol and ForCES FE model. The test 114 results were captured by [RFC6053] . 116 1.1. ForCES Protocol 118 The ForCES protocol works in a master-slave mode in which FEs are 119 slaves and CEs are masters. The protocol includes commands for 120 transport of Logical Function Block (LFB) configuration information, 121 association setup, status, and event notifications, etc. The reader 122 is encouraged to read the ForCES protocol specification [RFC5810] for 123 further information. 125 1.2. ForCES FE Model 127 The ForCES FE model [RFC5812] presents a formal way to define FE 128 Logical Function Blocks (LFBs) using XML. LFB configuration 129 components, capabilities, and associated events are defined when the 130 LFB is formally created. The LFBs within the FE are accordingly 131 controlled in a standardized way by the ForCES protocol. 133 1.3. Transport Mapping Layer 135 The ForCES Transport Mapping Layer (TML) transports the ForCES 136 Protocol Layer (PL) messages. The TML is where the issues of how to 137 achieve transport level reliability, congestion control, multicast, 138 ordering, etc are handled. It is expected that more than one TML 139 will be standardized. The various possible TMLs could vary their 140 implementations based on the capabilities of underlying media and 141 transport. However, since each TML is standardized, interoperability 142 is guaranteed as long as both endpoints support the same TML. All 143 ForCES Protocol Layer implementations MUST be portable across all 144 TMLs. Although more than one TML may be standardized for the ForCES 145 Protocol, for the purposes of the interoperability test, the mandated 146 MUST IMPLEMENT SCTP TML [RFC5811] was be used. 148 2. Terminology and Conventions 150 2.1. Requirements Language 152 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 153 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 154 document are to be interpreted as described in [RFC2119]. 156 2.2. Definitions 158 This document follows the terminology defined by ForCES related 159 documents, including RFC3654, RFC3746, RFC5810, RFC5811, RFC5812, 160 RFC5813, etc. Some definitions are repeated below for clarity. 162 Control Element (CE) - A logical entity that implements the ForCES 163 protocol and uses it to instruct one or more FEs on how to process 164 packets. CEs handle functionality such as the execution of 165 control and signaling protocols. 167 Forwarding Element (FE) - A logical entity that implements the 168 ForCES protocol. FEs use the underlying hardware to provide per- 169 packet processing and handling as directed/controlled by one or 170 more CEs via the ForCES protocol. 172 LFB (Logical Functional Block) - The basic building block that is 173 operated on by the ForCES protocol. The LFB is a well defined, 174 logically separable functional block that resides in an FE and is 175 controlled by the CE via the ForCES protocol. The LFB may reside 176 at the FE's datapath and process packets or may be purely an FE 177 control or configuration entity that is operated on by the CE. 178 Note that the LFB is a functionally accurate abstraction of the 179 FE's processing capabilities, but not a hardware-accurate 180 representation of the FE implementation. 182 LFB Class and LFB Instance - LFBs are categorized by LFB Classes. 183 An LFB Instance represents an LFB Class (or Type) existence. 184 There may be multiple instances of the same LFB Class (or Type) in 185 an FE. An LFB Class is represented by an LFB Class ID, and an LFB 186 Instance is represented by an LFB Instance ID. As a result, an 187 LFB Class ID associated with an LFB Instance ID uniquely specifies 188 an LFB existence. 190 LFB Metadata - Metadata is used to communicate per-packet state 191 from one LFB to another, but is not sent across the network. The 192 FE model defines how such metadata is identified, produced, and 193 consumed by the LFBs. It defines the functionality but not how 194 metadata is encoded within an implementation. 196 LFB Components - Operational parameters of the LFBs that must be 197 visible to the CEs are conceptualized in the FE model as the LFB 198 components. The LFB components include, for example, flags, 199 single-parameter arguments, complex arguments, and tables that the 200 CE can read and/or write via the ForCES protocol. 202 ForCES Protocol - While there may be multiple protocols used 203 within the overall ForCES architecture, the term "ForCES protocol" 204 and "protocol" refer to the "Fp" reference points in the ForCES 205 framework in [RFC3746] . This protocol does not apply to CE-to-CE 206 communication, FE-to-FE communication, or to communication between 207 FE and CE managers. Basically, the ForCES protocol works in a 208 master-slave mode in which FEs are slaves and CEs are masters. 210 ForCES Protocol Transport Mapping Layer (ForCES TML) - A layer in 211 ForCES protocol architecture that uses the capabilities of 212 existing transport protocols to specifically address protocol 213 message transportation issues, such as how the protocol messages 214 are mapped to different transport media (like TCP, IP, ATM, 215 Ethernet, etc.), and how to achieve and implement reliability, 216 multicast, ordering, etc. The ForCES TML specifications are 217 detailed in separate ForCES documents, one for each TML. 219 3. Overview 221 3.1. Date, Location, and Participants 223 The second ForCES interoperability test meeting was held by IETF 224 ForCES Working Group on February 24-25, 2011, and was chaired by 225 Jamal Hadi Salim. Three independent ForCES implementations 226 participated in the test: 228 * Zhejiang Gongshang University/Hangzhou BAUD Corporation of 229 Information and Networks Technology (Hangzhou BAUD Networks), China. 230 This implementation is referred to as "China" or in some cases "C" in 231 the document for the sake of brevity. 232 * NTT Corporation, Japan. This implementation is referred to as 233 "Japan" or in some cases "J" in the document for the sake of brevity. 234 * The University of Patras, Greece. This implementation is referred 235 to as "Greece" or in some cases "G" in the document for the sake of 236 brevity. 238 Two other organizations, Mojatatu Networks and Hangzhou BAUD Networks 239 Corporation, which independently extended two different well known 240 public domain protocol analyzers, Ethereal/Wireshark [Ethereal] and 241 Tcpdump [Tcpdump], also participated in the interop test. During the 242 interoperability test, the two protocol analyzers were used to verify 243 the validity of ForCES protocol messages and in some cases semantics. 245 Some issues related to interoperability among implementations were 246 discovered. Most of the issues were solved on site during the test. 247 The most contentious issue found was on the format of encapsulation 248 for protocol TLV (Refer to Section 6.1 ). 250 Some errata related to ForCES document were found by the 251 interoperability test. The errata has been reported to related IETF 252 RFCs. 254 At times, interoperability testing was exercised between two instead 255 of all three representative implementations due to a third one 256 lacking a specific feature; however, in ensuing discussions, all 257 implementers mentioned they will be implementing any missing features 258 in the future. 260 3.2. Testbed Configuration 262 3.2.1. Participants Access 264 Japan and China physically attended on site at the Internet 265 Technology Lab (ITL) of Zhejiang Gongshang University in China. The 266 University of Patras implementation joined remotely from Greece. The 267 chair, Jamal Hadi Salim, joined remotely from Canada by using the 268 Teamviewer as the monitoring tool[Teamviewer]. The approach is as 269 shown in Figure 1. In the figure, FE/CE refers to FE or CE that the 270 implementer may act alternatively. 272 +---------+ +----+ +----------+ 273 | FE/CE | | | +---|Monitoring| 274 | China |-----| | /\/\/\/\/\ | |(TeamViewer) 275 +---------+ | | \Internet/ | | Canada | 276 |LAN |----/ \--| +----------+ 277 +---------+ | | \/\/\/\/\/ | +----------+ 278 | FE/CE |-----| | | | FE/CE | 279 | Japan | | | +---| Greece | 280 +---------+ +----+ +----------+ 282 Figure 1: Access for Participants 284 As specified in RFC 5811, all CEs and FEs SHALL implement IPSec 285 security in the TML. 287 On the internet boundary, gateways used MUST allow for IPSec, SCTP 288 protocol and SCTP ports as defined in the ForCES SCTP-TML [RFC5811] . 290 3.2.2. Testbed Configuration 292 CEs and FEs from China and Japan implementations were physically 293 located within the ITL Lab of Zhejiang Gongshang University and 294 connected together using Ethernet switches. The configuration can be 295 seen in Figure 2. In the figure, the SmartBits is a third-party 296 supplied routing protocol testing machine, which acts as a router 297 running OSPF and RIP and exchanges routing protocol messages with 298 ForCES routers in the network. The Internet is connected via an ADSL 299 channel. 301 /\/\/\/\/\ 302 \Internet/ 303 / \ 304 \/\/\/\/\/ 305 | 306 |124.90.146.218 (ADSL) 307 | 308 +------------------------------------------------------------------+ 309 | LAN (10.20.0.0/24) | 310 +------------------------------------------------------------------+ 311 | | | | | | 312 | | | | | | 313 |.222 |.230 |.221 |.179 |.231 |.220 314 +-----+ +-----+ +-----+ +-----+ +-----+ +---------+ 315 | CE | | CE | | | | | | | | Protocol| 316 |China| |Japan| | FE1 |.1 .2| FE |.1 .2| FE2 | | Analyzer| 317 +-----+ +-----+ |China|---------|Japan|----------|China| +---------+ 318 +---------| |192.169. | | 192.168. | |-------+ 319 | .2 +-----+ 20.0.24 +-----+ 30.0/24 +-----+ .2 | 320 | .12| |.12 | 321 | | | | 322 192.168.50.0/24 | | 192.168.60.0/24 323 | 192.168.10.0/24 192.168.40.0/24 | 324 .1 | |.11 |.11 |.1 325 +--------+ +---------------------------------------+ +--------+ 326 |Terminal| | Smartbits | |Terminal| 327 +--------+ +---------------------------------------+ +--------+ 329 Figure 2: Testbed Configuration Located in ITL Lab,China 331 Hardwares and Softwares (CE and FE) of Greece that were located 332 within the University of Patras, Greece, were connected together 333 using LAN as shown in Figure 3. The Internet is connected via a VPN 334 channel. 336 /\/\/\/\/\ 337 \Internet/ 338 / \ 339 \/\/\/\/\/ 340 | 341 |150.140.254.110(VPN) 342 | 343 +------------------------------------+ 344 | LAN | 345 +------------------------------------+ 346 | | | 347 | | | 348 +------+ +--------+ +------+ 349 | FE | |Protocol| | CE | 350 |Greece| |Analyzer| |Greece| 351 +------+ +--------+ +------+ 353 Figure 3: Testbed Configuration Located in the University of 354 Patras,Greece 356 All above testbed configurations can then satisfy requirements of all 357 the interoperability test scenarios that are mentioned in this 358 document. 360 4. Scenarios 362 4.1. Scenario 1 - LFB Operation 364 This scenario is to test the interoperability on LFB operations among 365 the participants. The connection diagram for the participants is as 366 shown in Figure 4. 368 +------+ +------+ +------+ +------+ +------+ +------+ 369 | CE | | CE | | CE | | CE | | CE | | CE | 370 | China| | Japan| | China| |Greece| | Japan| |Greece| 371 +------+ +------+ +------+ +------+ +------+ +------+ 372 | | | | | | 373 | | | | | | 374 +------+ +------+ +------+ +------+ +------+ +------+ 375 | FE | | FE | | FE | | FE | | FE | | FE | 376 |Japan | |China | |Greece| |China | |Greece| |Japan | 377 +------+ +------+ +------+ +------+ +------+ +------+ 379 Figure 4: Scenario for LFB Operation 381 In order to make interoperability more credible,the three 382 implementers are required to carry out the test in a way acting as CE 383 or FE alternatively. As a result, every LFB operation is combined 384 with 6 scenarios, as shown by Figure 4. 386 The test scenario is designed with the following purposes: 388 Firstly, the scenario is designed to verify all kinds of protocol 389 messages with their complex data formats, which are defined in RFC 390 5810. Specially, we try to verify the data format of a PATH-DATA 391 with nested PATH-DATAs, and the operation(SET, GET, DEL) of an array 392 or an array with a nested array. 394 Secondly,the scenario is designed to verify the definition of ForCES 395 LFB Library[FORCES-LFBLIB], which defines a base set of ForCES LFB 396 classes for typical router functions. Successful test under this 397 scenario also means the validity of the LFB definitions. 399 4.2. Scenario 2 - TML with IPSec 401 This scenario is designed to implement a TML with IPSec, which is the 402 requirement by RFC 5811. TML with IPSec was not implemented in the 403 first ForCES interoperability test as reported by RFC 6053. For this 404 reason, in the second interoperability test, we specifically designed 405 the test scenario to verify the TML over IPSec channel. 407 In this scenario, tests on LFB operations for Scenario 1 were 408 repeated with the difference that TML was secured via IPSec. This 409 setup scenario allows us to verify whether all interactions between 410 CE and FE can be made correctly under an IPSec TML environment. 412 The connection diagram for this scenario is shown as Figure 5. 413 Because of system deficiency to deploy IPSec over TML in Greece, the 414 text only took place between China and Japan. 416 +------+ +------+ 417 | CE | | CE | 418 | China| | Japan| 419 +------+ +------+ 420 | | 421 |TML over IPSec |TML over IPSec 422 +------+ +------+ 423 | FE | | FE | 424 |Japan | |China | 425 +------+ +------+ 427 Figure 5: Scenario for LFB Operation with TML over IPSec 429 In this scenario, ForCES TML was run over IPSec channel. 430 Implementers joined in this interoperability have used the same 431 third-party software 'racoon' to have established the IPSec channel. 433 China and Japan have made a successful test with the scenario, and 434 the following items have been realized: 436 o Internet Key Exchange (IKE) with certificates for endpoint 437 authentication. 439 o Transport Mode Encapsulating Security Payload (ESP). HMAC-SHA1-96 440 [RFC2404] for message integrity protection. 442 4.3. Scenario 3 - CE High Availability 444 CE High Availability (CEHA) was tested based on the ForCES CEHA 445 document [ForCES-CEHA]. 447 The design of the setup and the scenario for the CEHA were simplified 448 so as to focus mostly on the mechanics of the CEHA, which are: 450 o Associating with more than one CE. 452 o Switching to backup CE on master CE failure. 454 The connection diagram for the scenario is as shown in Figure 6. 456 master standby master standby 457 +------+ +------+ +------+ +------+ 458 | CE | | CE | | CE | | CE | 459 | China| |Greece| |Japan | |Greece| 460 +------+ +------+ +------+ +------+ 461 | | | | 462 +----------+ +-----------+ 463 | | 464 +------+ +------+ 465 | FE | | FE | 466 |Greece| |Greece| 467 +------+ +------+ 468 (a) (b) 470 Figure 6: Scenario for CE High Availability 472 In this scenario one FE is connected and associated to a master CE 473 and a backup CE. In the pre-association phase, the FE would be 474 configured to have China's or Japan's CE as master CE and Greece's CE 475 as standby CE. The CEFailoverPolicy component of the FE Protocol 476 Object LFB that specifies whether the FE is in High Availability mode 477 (value 2 or 3) would either be set in the pre-association phase by 478 the FEM interface or in post-association phase by the master CE. 480 If the CEFailoverPolicy value is set to 2 or 3, the FE (in the post- 481 association phase) will attempt to connect and associate with the 482 standby CE. 484 When the master CE is deemed disconnected, either by a TearDown, Loss 485 of Heartbeats or physically disconnected, the FE would assume that 486 the standby CE is now the master CE. The FE will then send an Event 487 Notification, Primary CE Down,to all associated CEs, only the standby 488 CE in this case, with the value of the new master CEID. The standby 489 CE will then respond by sending a configuration message to the CEID 490 component of the FE Protocol Object with its own ID to confirm that 491 the CE considers itself as the master as well. 493 The steps of the CEHA test scenario are as follows: 495 1. In the pre-association phase, setup of FE with master CE and 496 backup CE 498 2. FE connecting and associating with master CE. 500 3. When CEFailoverPolicy is set to 2 or 3, the FE will connect and 501 associate with backup CE. 503 4. Once the master CE is considered disconnected then the FE chooses 504 the first Associated backup CE. 506 5. It sends an Event Notification specifying that the master CE is 507 down and who is now the master CE. 509 6. The new master CE sends a SET Configuration message to the FE 510 setting the CEID value to who is now the new master CE completing 511 the switch. 513 4.4. Scenario 4 - Packet forwarding 515 This test scenario is to verify LFBs like RedirectIn, RedirectOut, 516 IPv4NextHop, IPv4UcastLPM defined by the ForCES LFB library 517 document[ForCES-LFBLIB], and more importantly, to verify the 518 combination of the LFBs to implement IP packet forwarding. 520 The connection diagram for this scenario is as Figure 7. 522 +------+ 523 | CE | 524 | Japan| 525 +------+ 526 | ^ 527 | | OSPF 528 | +-------> 529 +------+ +------+ 530 +--------+ | FE | | OSPF | +--------+ 531 |Terminal|------|China |-------|Router|------|Terminal| 532 +--------+ +------+ +------+ +--------+ 534 <--------------------------------------------> 535 Packet Forwarding 537 (a) 539 +------+ 540 | CE | 541 | China| 542 +------+ 543 ^ | ^ 544 OSPF | | | OSPF 545 <-----+ | +-----> 546 +-------+ +------+ +------+ 547 +--------+ | OSPF | | FE | | OSPF | +--------+ 548 |Terminal|----|Router |----|Japan |-----|Router|----|Terminal| 549 +--------+ +-------+ +------+ +------+ +--------+ 550 <--------------------------------------------> 551 Packet Forwarding 553 (b) 555 +------+ +------+ 556 | CE | | CE | 557 | Japan| | China| 558 +------+ +------+ 559 | ^ ^ | 560 | | OSPF | | 561 | +----------+ | 562 +------+ +------+ 563 +--------+ | FE | | FE | +--------+ 564 |Terminal|------|China |-------|Japan |------|Terminal| 565 +--------+ +------+ +------+ +--------+ 567 <--------------------------------------------> 568 Packet Forwarding 570 (c) 572 Figure 7: Scenario for IP Packet forwarding 574 In case (a), a CE by Japan is connected to an FE by China to form a 575 ForCES router. A Smartbits test machine with its routing protocol 576 software are used to simulate an OSPF router and are connected with 577 the ForCES router to try to exchange OSPF hello packets and LSA 578 packets among them. Terminals are simulated by Smartbits to send and 579 receive packets. As a result, the CE in the ForCES router need to be 580 configured to run and support OSPF routing protocol. 582 In case (b), a CE by China is connected to an FE by Japan to form a 583 ForCES router. Two routers running OSPF are simulated and connected 584 to the ForCES router to test if the ForCES router can support OSPF 585 protocol and support packet forwarding. 587 In case (c), two ForCES routers are constructed. One is with CE by 588 Japan and FE by China and the other is opposite. OSPF and packet 589 forwarding are tested in the environment. 591 Testing process for this scenario is as below: 593 1. Boot terminals and routers, and set IP addresses of their 594 interfaces. 596 2. Boot CE and FE. 598 3. Establish association between CE and FE, and set IP addresses of 599 FEs interfaces. 601 4. Start OSPF among CE and routers, and set FIB on FE. 603 5. Send packets between terminals. 605 5. Test Results 607 5.1. LFB Operation Test 609 The test result is as reported by Figure 8. For the convenience 610 sake, as mentioned earlier, abbreviations of 'C' in the table means 611 implementation from China,'J'Japan implementation, and 'G' Greece 612 implementation. 614 +-----+----+-----+-----+--------------+-------------------+---------+ 615 |Test#| CE |FE(s)|Oper | LFB | Component | Result | 616 | | | | | | /Capability | | 617 +-----+----+-----+-----+--------------+-------------------+---------+ 618 | 1 | C | J | GET | FEObject | LFBTopology | Success | 619 | | J | C | | | | Success | 620 | | G | C | | | | Success | 621 | | J | G | | | | Success | 622 | | G | J | | | | Success | 623 | | | | | | | | 624 | 2 | C | J | GET | FEObject | LFBSelector | Success | 625 | | J | C | | | | Success | 626 | | C | G | | | | Success | 627 | | G | C | | | | Success | 628 | | J | G | | | | Success | 629 | | G | J | | | | Success | 630 | | | | | | | | 631 | 3 | C | J | GET | EtherPHYCop | PHYPortID | Success | 632 | | J | C | | | | Success | 633 | | C | G | | | | Success | 634 | | G | C | | | | Success | 635 | | J | G | | | | Success | 636 | | G | J | | | | Success | 637 | | | | | | | | 638 | 4 | C | J | GET | EtherPHYCop | AdminStatus | Success | 639 | | J | C | | | | Success | 640 | | C | G | | | | Success | 641 | | G | C | | | | Success | 642 | | J | G | | | | Success | 643 | | G | J | | | | Success | 644 | | | | | | | | 645 | 5 | C | J | GET | EtherPHYCop | OperStatus | Success | 646 | | J | C | | | | Success | 647 | | C | G | | | | Success | 648 | | G | C | | | | Success | 649 | | J | G | | | | Success | 650 | | G | J | | | | Success | 651 | | | | | | | | 652 | 6 | C | J | GET | EtherPHYCop | AdminLinkSpeed | Success | 653 | | J | C | | | | Success | 654 | | C | G | | | | Success | 655 | | G | C | | | | Success | 656 | | J | G | | | | Success | 657 | | G | J | | | | Success | 658 | | | | | | | | 659 | 7 | C | J | GET | EtherPHYCop | OperLinkSpeed | Success | 660 | | J | C | | | | Success | 661 | | C | G | | | | Success | 662 | | G | C | | | | Success | 663 | | J | G | | | | Success | 664 | | G | J | | | | Success | 665 | | | | | | | | 666 | 8 | C | J | GET | EtherPHYCop | AdminDuplexSpeed | Success | 667 | | J | C | | | | Success | 668 | | C | G | | | | Success | 669 | | G | C | | | | Success | 670 | | J | G | | | | Success | 671 | | G | J | | | | Success | 672 | | | | | | | | 673 | 9 | C | J | GET | EtherPHYCop | OperDuplexSpeed | Success | 674 | | J | C | | | | Success | 675 | | C | G | | | | Success | 676 | | G | C | | | | Success | 677 | | J | G | | | | Success | 678 | | G | J | | | | Success | 679 | | | | | | | | 680 | 10 | C | J | GET | EtherPHYCop | CarrierStatus | Success | 681 | | J | C | | | | Success | 682 | | C | G | | | | Success | 683 | | G | C | | | | Success | 684 | | J | G | | | | Success | 685 | | G | J | | | | Success | 686 | | | | | | | | 687 | 11 | C | J | GET | EtherMACIn | AdminStatus | Success | 688 | | J | C | | | | Success | 689 | | C | G | | | | Success | 690 | | G | C | | | | Success | 691 | | J | G | | | | Success | 692 | | G | J | | | | Success | 693 | | | | | | | | 694 | 12 | C | J | GET | EtherMACIn | LocalMacAddresses | Success | 695 | | J | C | | | | Success | 696 | | C | G | | | | Success | 697 | | G | C | | | | Success | 698 | | J | G | | | | Success | 699 | | G | J | | | | Success | 700 | | | | | | | | 701 | 13 | C | J | GET | EtherMACIn | L2Bridging | Success | 702 | | J | C | | | PathEnable | Success | 703 | | C | G | | | | Success | 704 | | G | C | | | | Success | 705 | | J | G | | | | Success | 706 | | G | J | | | | Success | 707 | | | | | | | | 708 | 14 | C | J | GET | EtherMACIn | PromiscuousMode | Success | 709 | | J | C | | | | Success | 710 | | C | G | | | | Success | 711 | | G | C | | | | Success | 712 | | J | G | | | | Success | 713 | | G | J | | | | Success | 714 | | | | | | | | 715 | 15 | C | J | GET | EtherMACIn | TxFlowControl | Success | 716 | | J | C | | | | Success | 717 | | C | G | | | | Success | 718 | | G | C | | | | Success | 719 | | J | G | | | | Success | 720 | | G | J | | | | Success | 721 | | | | | | | | 722 | 16 | C | J | GET | EtherMACIn | RxFlowControl | Success | 723 | | J | C | | | | Success | 724 | | C | G | | | | Success | 725 | | G | C | | | | Success | 726 | | J | G | | | | Success | 727 | | G | J | | | | Success | 728 | | | | | | | | 729 | 17 | C | J | GET | EtherMACIn | MACInStats | Success | 730 | | J | C | | | | Success | 731 | | C | G | | | | Success | 732 | | G | C | | | | Success | 733 | | J | G | | | | Success | 734 | | G | J | | | | Success | 735 | | | | | | | | 736 | 18 | C | J | GET | EtherMACOut | AdminStatus | Success | 737 | | J | C | | | | Success | 738 | | C | G | | | | Success | 739 | | G | C | | | | Success | 740 | | J | G | | | | Success | 741 | | G | J | | | | Success | 742 | | | | | | | | 743 | 19 | C | J | GET | EtherMACOut | MTU | Success | 744 | | J | C | | | | Success | 745 | | C | G | | | | Success | 746 | | G | C | | | | Success | 747 | | J | G | | | | Success | 748 | | G | J | | | | Success | 749 | | | | | | | | 750 | 20 | C | J | GET | EtherMACOut | TxFlowControl | Success | 751 | | J | C | | | | Success | 752 | | C | G | | | | Success | 753 | | G | C | | | | Success | 754 | | J | G | | | | Success | 755 | | G | J | | | | Success | 756 | | | | | | | | 757 | 21 | C | J | GET | EtherMACOut | TxFlowControl | Success | 758 | | J | C | | | | Success | 759 | | C | G | | | | Success | 760 | | G | C | | | | Success | 761 | | J | G | | | | Success | 762 | | G | J | | | | Success | 763 | | | | | | | | 764 | 22 | C | J | GET | EtherMACOut | MACOutStats | Success | 765 | | J | C | | | | Success | 766 | | C | G | | | | Success | 767 | | G | C | | | | Success | 768 | | J | G | | | | Success | 769 | | G | J | | | | Success | 770 | | | | | | | | 771 | 23 | C | J | GET | ARP |PortV4AddrInfoTable| Success | 772 | | J | C | | | | Success | 773 | | C | G | | | | Success | 774 | | G | C | | | | Success | 775 | | J | G | | | | Success | 776 | | G | J | | | | Success | 777 | | | | | | | | 778 | 24 | C | J | SET | ARP |PortV4AddrInfoTable| Success | 779 | | J | C | | | | Success | 780 | | C | G | | | | Success | 781 | | G | C | | | | Success | 782 | | J | G | | | | Success | 783 | | G | J | | | | Success | 784 | | | | | | | | 785 | 25 | C | J | DEL | ARP |PortV4AddrInfoTable| Success | 786 | | J | C | | | | Success | 787 | | C | G | | | | Success | 788 | | G | C | | | | Success | 789 | | J | G | | | | Success | 790 | | G | J | | | | Success | 791 | | | | | | | | 792 | 26 | C | J | SET | EtherMACIn | LocalMACAddresses | Success | 793 | | J | C | | | | Success | 794 | | C | G | | | | Success | 795 | | G | C | | | | Success | 796 | | J | G | | | | Success | 797 | | G | J | | | | Success | 798 | | | | | | | | 799 | 27 | C | J | SET | EtherMACIn | MTU | Success | 800 | | J | C | | | | Success | 801 | | C | G | | | | Success | 802 | | G | C | | | | Success | 803 | | J | G | | | | Success | 804 | | G | J | | | | Success | 805 | | | | | | | | 806 | 28 | C | J | SET | IPv4NextHop | IPv4NextHopTable | Success | 807 | | J | C | | | | Success | 808 | | C | G | | | | Success | 809 | | G | C | | | | Success | 810 | | J | G | | | | Success | 811 | | G | J | | | | Success | 812 | | | | | | | | 813 | 29 | C | J | SET | IPv4UcastLPM | IPv4PrefixTable | Success | 814 | | J | C | | | | Success | 815 | | C | G | | | | Success | 816 | | G | C | | | | Success | 817 | | J | G | | | | Success | 818 | | G | J | | | | Success | 819 | | | | | | | | 820 | 30 | C | J | DEL | IPv4NextHop | IPv4NextHopTable | Success | 821 | | J | C | | | | Success | 822 | | C | G | | | | Success | 823 | | G | C | | | | Success | 824 | | J | G | | | | Success | 825 | | G | J | | | | Success | 826 | | | | | | | | 827 | 31 | C | J | DEL | IPv4UcastLPM | IPv4PrefixTable | Success | 828 | | J | C | | | | Success | 829 | | C | G | | | | Success | 830 | | G | C | | | | Success | 831 | | J | G | | | | Success | 832 | | G | J | | | | Success | 833 | | | | | | | | 834 | 32 | C | J | SET | EtherPHYCop | AdminStatus | Success | 835 | | J | C | | | | Success | 836 | | C | G | | | | Success | 837 | | G | C | | | | Success | 838 | | J | G | | | | Success | 839 | | G | J | | | | Success | 840 | | | | | | | | 841 | 33 | C | J | SET | Ether | VlanInputTable | Success | 842 | | J | C | | Classifier | | Success | 843 | | C | G | | | | Success | 844 | | G | C | | | | Success | 845 | | J | G | | | | Success | 846 | | G | J | | | | Success | 847 | | | | | | | | 848 | 34 | C | J | DEL | Ether | VlanInputTable | Success | 849 | | J | C | | Classifier | | Success | 850 | | C | G | | | | Success | 851 | | G | C | | | | Success | 852 | | J | G | | | | Success | 853 | | G | J | | | | Success | 854 | | | | | | | | 855 | 35 | C | J | SET | Ether | VlanOutputTable | Success | 856 | | J | C | | Encapsulator | | Success | 857 | | C | G | | | | Success | 858 | | G | C | | | | Success | 859 | | J | G | | | | Success | 860 | | G | J | | | | Success | 861 | | | | | | | | 862 | 36 | C | J | DEL | Ether | VlanOutputTable | Success | 863 | | J | C | | Encapsulator | | Success | 864 | | C | G | | | | Success | 865 | | G | C | | | | Success | 866 | | J | G | | | | Success | 867 | | G | J | | | | Success | 868 +-----+----+-----+-----+--------------+-------------------+---------+ 870 Figure 8: LFB Operation Test Results 872 Note on test 1#: 874 On the wire format of encapsulation on array, only the case of 875 FULLDATA-in-FULLDATA was tested. 877 In China's implementation,after test 2# CE have to get all LFBs' 878 instance data actively according to the queried component of 879 LFBSelectors. 881 Note on test 28# and 29#: 883 Only had new reachable network destination been set,can route entry 884 be added into system. 886 Note on test 30# and 31#: 888 Corresponding nexthop entry must be deleted before prefix entry which 889 is decided by FE's routing management. 891 5.2. TML with IPSec Test 893 In this scenario, the ForCES TML is run over IPSec. Implementers 894 joined this interoperability test use the same third-party tool 895 software 'racoon' to establish IPSec channel. Some typical LFB 896 operation tests as in Scenario 1 are repeated with the IPSec enabled 897 TML. 899 A note on this test is, because of the system difficulty to implement 900 IPSec over TML, Greece did not join in the test. Therefore, this 901 scenario only took place between C and J. 903 The TML with IPSec test results are reported by Figure 9. 905 +-----+----+-----+-----+--------------+-------------------+---------+ 906 |Test#| CE |FE(s)|Oper | LFB | Component/ | Result | 907 | | | | | | Capability | | 908 +-----+----+-----+-----+--------------+-------------------+---------+ 909 | 1 | C | J | GET | FEObject | LFBTopology | Success | 910 | | J | C | | | | Success | 911 | | | | | | | | 912 | 2 | C | J | GET | FEObject | LFBSelectors | Success | 913 | | J | C | | | | Success | 914 | | | | | | | | 915 | 3 | C | J | SET | Ether | VlanInputTable | Success | 916 | | J | C | | Classifier | | Success | 917 | | | | | | | | 918 | 4 | C | J | DEL | Ether | VlanInputTable | Success | 919 | | J | C | | Classifier | | Success | 920 +-----+----+-----+-----+--------------+-------------------+---------+ 922 Figure 9: TML with IPSec Test Results 924 5.3. CE High Availability Test 926 In this scenario one FE connects and associates with a master CE and 927 a backup CE. When the master CE is deemed disconnected the FE would 928 attempt to find another associated CE to become the master CE. 930 The CEHA scenario as is described in Scenario 3 was completed 931 successfully for both setups. 933 Due to a bug in one of the FEs, a interesting issue was caught: it 934 was observed that the buggy FE took up to a second to failover. It 935 was eventually found that the issue was due to the FE's 936 prioritization of the different CEs. All messages from the backup CE 937 were being ignored unless the master CE is disconnected. 939 While the bug was fixed and the CEHA scenario was completed 940 successfully, the authors feel it was important to capture the 941 implementation issue in this document. The recommended approach is 942 the following: 944 o The FE SHOULD receive and handle messages first from the master CE 945 on all priority channels to maintain proper functionality and then 946 receive and handle messages from the backup CEs. 948 o Only when the FE is attempting to associate with the backup CEs, 949 then the FE SHOULD receive and handle messages per priority 950 channel from all CEs. When all backup CEs are associated with or 951 deemed unreachable, then the FE SHOULD return to receiving and 952 handling messages first from the master CE. 954 5.4. Packet Forwarding Test 956 As described in the ForCES LFB library [I-D.ietf-forces-lfb-lib], 957 packet forwarding is implemented by a set of LFB classes that compose 958 a processing path for packets. In this test scenario, as shown in 959 Figure 7, a ForCES router running OSPF protocol was constructed. In 960 addition, a set of LFBs including RedirectIn, RedirectOut, 961 IPv4UcastLPM, and IPv4NextHop LFBs are used. RedirectIn and 962 RedirectOut LFBs redirect OSPF hello and LSA packets from and to CE. 963 A Smartbits test machine is used to simulate an OSPF router and 964 exchange the OSPF hello and LSA packets with CE in ForCES router. 966 Cases (a) and (b) in Figure 7 both need a RedirectIn LFB to send OSPF 967 packets generated by CE to FE by use of ForCES packet redirect 968 messages. The OSPF packets are further sent to an outside OSPF 969 Router by the FE via forwarding LFBs including IPv4NextHop and 970 IPv4UcastLPM LFBs. A RedirectOut LFB in the FE is used to send OSPF 971 packets received from outside OSPF Router to CE by ForCES packet 972 redirect messages. 974 By running OSPF, the CE in the ForCES router can generate new routes 975 and load them to routing table in FE. The FE is then able to forward 976 packets according to the routing table. 978 The test is reported with the results in Figure 10 980 +-----+----+-----+-------------------------+--------------+---------+ 981 |Test#| CE |FE(s)| Item | LFBs Related | Result | 982 +-----+----+-----+-------------------------+--------------+---------+ 983 | 1 | J | C | IPv4NextHopTable SET | IPv4NextHop | Success | 984 | | | | | | | 985 | 2 | J | C | IPv4PrefixTable SET | IPv4UcastLPM | Success | 986 | | | | | | | 987 | 3 | J | C |Redirect ospf packet from| RedirectIn | Success | 988 | | | | CE to SmartBits | | | 989 | | | | | | | 990 | 4 | J | C |Redirect ospf packet from| RedirectOut | Success | 991 | | | | SmartBits to CE | | | 992 | | | | | | | 993 | 5 | J | C | Metadata in | RedirectOut | Success | 994 | | | | redirect message | RedirectIn | | 995 | | | | | | | 996 | 6 | J | C |OSPF neighbor discovery | RedirectOut | Success | 997 | | | | | RedirectIn | | 998 | | | | | | | 999 | 7 | J | C | OSPF DD exchange | RedirectOut | Success | 1000 | | | | | RedirectIn | | 1001 | | | | | IPv4NextHop | | 1002 | | | | | | | 1003 | 8 | J | C | OSPF LSA exchange | RedirectOut | Success | 1004 | | | | | RedirectIn | | 1005 | | | | | IPv4NextHop | | 1006 | | | | | IPv4UcastLPM| | 1007 | | | | | | | 1008 | 9 | J | C | Data Forwarding | RedirectOut | Success | 1009 | | | | | RedirectIn | | 1010 | | | | | IPv4NextHop | | 1011 | | | | | IPv4UcastLPM| | 1012 | | | | | | | 1013 | 10 | C | J | IPv4NextHopTable SET | IPv4NextHop | Success | 1014 | | | | | | | 1015 | 11 | C | J | IPv4PrefixTable SET | IPv4UcastLPM| Success | 1016 | | | | | | | 1017 | 12 | C | J |Redirect ospf packet from| RedirectIn | Success | 1018 | | | | CE to other OSPF router | | | 1019 | | | | | | | 1020 | 13 | C | J |Redirect ospf packet from| RedirectOut | Success | 1021 | | | |other OSPF router to CE | | | 1022 | | | | | | | 1023 | 14 | C | J | Metadata in | RedirectOut | Success | 1024 | | | | redirect message | RedirectIn | | 1025 | | | | | | | 1026 | 15 | C | J |OSPF neighbor discovery | RedirectOut | Success | 1027 | | | | | RedirectIn | | 1028 | | | | | | | 1029 | 16 | C | J | OSPF DD exchange | RedirectOut | Failure | 1030 | | | | | RedirectIn | | 1031 | | | | | IPv4NextHop | | 1032 | | | | | | | 1033 | 17 | C | J | OSPF LSA exchange | RedirectOut | Failure | 1034 | | | | | RedirectIn | | 1035 | | | | | IPv4NextHop | | 1036 | | | | | IPv4UcastLPM| | 1037 +-----+----+-----+-------------------------+--------------+---------+ 1039 Figure 10: Packet Forwarding Test Results 1041 Note on test 1# and 2#: 1043 Before redirect channel working normally, multicast route pointed to 1044 localhost must be added manually firstly. 1046 Note on test 3# to 9#: 1048 During the tests, ospf packets received from CE were found by 1049 Ethereal/Wireshark with checksum errors. China's FE corrected the 1050 checksum in FE so that the Smartbits would not drop the packets and 1051 the neighbor discovery can continue. Such correcting action does not 1052 affect the test scenarios and the results. 1054 Comment on Test #16 and #17: 1056 The two test items failed. Note that Test #7 and #8 are exactly the 1057 same as these tests, only with CE and FE implementers are exchanged, 1058 and Test #12 and #13 show the redirect channel works well. As a 1059 result, it can be inferred that the problem caused the test failure 1060 was almost certainly from the implementation of the related LFBs 1061 rather than from the ForCES protocol design problem, therefore the 1062 failure does not lead to the interoperability problem on ForCES. 1064 6. Discussions 1066 6.1. On Data Encapsulation Format 1068 In the first day of the test, it was found that the LFB inter- 1069 operations about tables all failed. The reason is found to be the 1070 different ForCES protocol data encapsulation method among different 1071 implementations. The encapsulation issues are detailed as below: 1073 Assuming that an LFB has two components, one is a struct with ID 1 1074 and the other an array with ID 2, further with two components of u32 1075 both inside, as below: 1077 struct1: type struct, ID=1 1078 components are: 1079 a, type u32, ID=1 1080 b, type u32, ID=2 1082 table1: type array, ID=2 1083 components for each row are (a struct of): 1084 x, type u32, ID=1 1085 y, type u32, ID=2 1087 1. On response of PATH-DATA format 1089 When a CE sends a config/query ForCES protocol message to an FE from 1090 a different implementer, the CE probably receives response from the 1091 FE with different PATH-DATA encapsulation format. For example, if a 1092 CE sends a query message with a path of 1 to a third party FE to 1093 manipulate struct 1 as defined above, the FE is probable to generate 1094 response with two different PATH-DATA encapsulation format: one is 1095 the value with FULL/SPARSE-DATA and the other is the value with many 1096 parallel PATH-DATA TLV and nested PATH-DATA TLV, as below: 1098 format 1: 1099 OPER = GET-RESPONSE-TLV 1100 PATH-DATA-TLV: 1101 IDs=1 1102 FULLDATA-TLV containing valueof(a),valueof(b) 1103 format 2: 1104 OPER = GET-RESPONSE-TLV 1105 PATH-DATA-TLV: 1106 IDs=1 1107 PATH-DATA-TLV: 1108 IDs=1 1109 FULLDATA-TLV containing valueof(a) 1110 PATH-DATA-TLV: 1111 IDs=2 1112 FULLDATA-TLV containing valueof(b) 1114 The interoperability test witnessed that a ForCES element (CE or FE) 1115 sender is free to choose whatever data structure that IETF ForCES 1116 documents define and best suits the element, while a ForCES element 1117 (CE or FE) MUST be able to accept and process information (requests 1118 and responses) that use any legitimate structure defined by IETF 1119 ForCES documents. While in the case a ForCES element is free to 1120 choose any legitimate data structure as a response, it is preferred 1121 the ForCES element responds in the same format that the request was 1122 made, as it is most probably the data structure is the request sender 1123 looks forward to receive. 1125 2. On operation to array 1127 An array operation may also have several different data encapsulation 1128 formats. For instance, if a CE sends a config message to table 1 1129 with a path of (2.1), which refers to component with ID=2, which is 1130 an array, and the second ID is the row, so row 1, it may be 1131 encapsulated with three formats as below: 1133 format 1: 1134 OPER = SET-TLV 1135 PATH-DATA-TLV: 1136 IDs=2.1 1137 FULLDATA-TLV conaining valueof(x),valueof(y) 1138 format 2: 1139 OPER = SET-TLV 1140 PATH-DATA-TLV: 1141 IDs=2.1 1142 PATH-DATA-TLV: 1143 IDs=1 1144 FULLDATA-TLV containing valueof(x) 1145 PATH-DATA-TLV 1146 IDs=2 1147 FULLDATA-TLV containing valueof(y) 1149 Moreover, if CE is targeting the whole array, for example if the 1150 array is empty and CE wants to add the first row to the table, it 1151 could also adopt another format: 1153 format 3: 1154 OPER = SET-TLV 1155 PATH-DATA-TLV: 1156 IDs=2 1157 FULLDATA-TLV containing rowindex=1,valueof(x),valueof(y) 1159 The interoperability test experience shows that format 1 and format 1160 3, which take full advantage of multiple data elements description in 1161 one TLV of FULLDATA-TLV, get more efficiency, although format 2 can 1162 also get the same operating goal. 1164 7. Contributors 1166 Contributors who have made major contributions to the 1167 interoperability test are as below: 1169 Hirofumi Yamazaki 1170 NTT Corporation 1171 Tokyo 1172 Japan 1173 Email: yamazaki.horofumi@lab.ntt.co.jp 1175 Rong Jin 1176 Zhejiang Gongshang University 1177 Hangzhou 1178 P.R.China 1179 Email: jinrong@zjgsu.edu.cn 1181 Yuta Watanabe 1182 NTT Corporation 1183 Tokyo 1184 Japan 1185 Email: yuta.watanabe@ntt-at.co.jp 1187 Xiaochun Wu 1188 Zhejiang Gongshang University 1189 Hangzhou 1190 P.R.China 1191 Email: spring-403@zjgsu.edu.cn 1193 8. Acknowledgements 1195 The authors would also like thank the following test participants: 1197 Chuanhuang Li, Hangzhou BAUD Networks 1198 Ligang Dong, Zhejiang Gongshang University 1199 Jingjing Zhou, Zhejiang Gongshang Unviersity 1200 Liaoyuan Ke, Hangzhou BAUD Networks 1201 Kelei Jin,Hangzhou BAUD Networks 1203 9. IANA Considerations 1205 This memo includes no request to IANA. 1207 10. Security Considerations 1209 Developers of ForCES FEs and CEs must take the security 1210 considerations of the ForCES Framework [RFC3746] and the ForCES 1211 Protocol [RFC5810] into account. Also, as specified in the security 1212 considerations section of the SCTP-Based TML for the ForCES Protocol 1213 [RFC5811] the transport-level security, has to be ensured by IPsec. 1215 11. References 1217 11.1. Normative References 1219 [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, 1220 W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and 1221 Control Element Separation (ForCES) Protocol 1222 Specification", RFC 5810, March 2010. 1224 [RFC5811] Hadi Salim, J. and K. Ogawa, "SCTP-Based Transport Mapping 1225 Layer (TML) for the Forwarding and Control Element 1226 Separation (ForCES) Protocol", RFC 5811, March 2010. 1228 [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control 1229 Element Separation (ForCES) Forwarding Element Model", 1230 RFC 5812, March 2010. 1232 [RFC5813] Haas, R., "Forwarding and Control Element Separation 1233 (ForCES) MIB", RFC 5813, March 2010. 1235 11.2. Informative References 1237 [Ethereal] 1238 "Ethereal, also named Wireshark, is a protocol analyzer. 1239 The specific Ethereal that was used is an updated 1240 Ethereal, by Fenggen Jia, that can analyze and decode the 1241 ForCES protocol messages", http://www.ietf.org/ 1242 mail-archive/web/forces/current/msg03687.html . 1244 [I-D.ietf-forces-ceha] 1245 Ogawa, K., Wang, W., Haleplidis, E., and J. Salim, "ForCES 1246 Intra-NE High Availability", draft-ietf-forces-ceha-04 1247 (work in progress), July 2012. 1249 [I-D.ietf-forces-lfb-lib] 1250 Wang, W., Haleplidis, E., Ogawa, K., Li, C., and J. 1251 Halpern, "ForCES Logical Function Block (LFB) Library", 1252 draft-ietf-forces-lfb-lib-09 (work in progress), May 2012. 1254 [RFC2629] Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629, 1255 June 1999. 1257 [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation 1258 of IP Control and Forwarding", RFC 3654, November 2003. 1260 [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, 1261 "Forwarding and Control Element Separation (ForCES) 1262 Framework", RFC 3746, April 2004. 1264 [RFC6053] Haleplidis, E., Ogawa, K., Wang, W., and J. Hadi Salim, 1265 "Implementation Report for Forwarding and Control Element 1266 Separation (ForCES)", RFC 6053, November 2010. 1268 [Tcpdump] "Tcpdump is a Linux protocol analyzer. The specific 1269 tcpdump that was used is a modified tcpdump, by Jamal Hadi 1270 Salim, that can analyze and decode the ForCES protocol 1271 messages", http://www.ietf.org/mail-archive/web/forces/ 1272 current/msg03811.html . 1274 [Teamviewer] 1275 "TeamViewer connects to any PC or server around the world 1276 within a few seconds.", http://www.teamviewer.com/ . 1278 Authors' Addresses 1280 Weiming Wang 1281 Zhejiang Gongshang University 1282 18 Xuezheng Str., Xiasha University Town 1283 Hangzhou, 310018 1284 P.R.China 1286 Phone: +86-571-28877721 1287 Email: wmwang@zjsu.edu.cn 1289 Kentaro Ogawa 1290 NTT Corporation 1291 Tokyo, 1292 Japan 1294 Email: ogawa.kentaro@lab.ntt.co.jp 1296 Evangelos Haleplidis 1297 University of Patras 1298 Patras, 1299 Greece 1301 Email: ehalep@ece.upatras.gr 1303 Ming Gao 1304 Hangzhou BAUD Networks 1305 408 Wen-San Road 1306 Hangzhou, 310012 1307 P.R.China 1309 Phone: +86-571-28877751 1310 Email: gmyyqno1@pop.zjgsu.edu.cn 1312 Jamal Hadi Salim 1313 Mojatatu Networks 1314 Ottawa 1315 Canada 1317 Email: hadi@mojatatu.com