2.5.3 Forwarding and Control Element Separation (forces)

NOTE: This charter is a snapshot of the 63rd IETF Meeting in Paris, France. It may now be out-of-date.
In addition to this official charter maintained by the IETF Secretariat, there is additional information about this working group on the Web at:

       Additional FORCES Web Page

Last Modified: 2005-05-19


Patrick Droz <dro@zurich.ibm.com>
David Putzolu <David.Putzolu@intel.com>

Routing Area Director(s):

Bill Fenner <fenner@research.att.com>
Alex Zinin <zinin@psg.com>

Routing Area Advisor:

Alex Zinin <zinin@psg.com>

Mailing Lists:

General Discussion: forces@peach.ease.lsoft.com
To Subscribe: listserv@peach.ease.lsoft.com
In Body: (un)subscribe forces
Archive: ftp://ftp.ietf.org/ietf-mail-archive/forces

Description of Working Group:

The emergence of off-the-shelf network processor devices that
the fast path or forwarding plane in network devices such as routers,
along with the appearance of a new generation of third party
routing, and other router control plane software, has created the need
for standard mechanisms to allow these components to be combined into
functional wholes. ForCES aims to define a framework and associated
mechanisms for standardizing the exchange of information between the
logically separate functionality of the control plane, including
entities such as routing protocols, admission control, and signaling,
and the forwarding plane, where per-packet activities such as packet
forwarding, queuing, and header editing occur. By defining a set
of standard mechanisms for control and forwarding separation, ForCES
will enable rapid innovation in both the control and forwarding
A standard separation mechanism allows the control and forwarding
planes to innovate in parallel while maintaining interoperability.

The products of this working group will be:

o A set of requirements for mechanisms to logically
  separate the control and data forwarding planes of
  an IP network element (NE)

o An applicability statement for the ForCES model
  and protocol

o Informational RFCs as necessary documenting current
  approaches to the functional model and controlled
  objects therein

o An architectural framework defining the entities
  comprising a ForCES network element and identifying
  the interactions between them.

o A description of the functional model of a
  Forwarding Element

o A formal definition of the controlled objects in the
  functional model of a forwarding element. This
  includes IP forwarding, IntServ and DiffServ QoS. An
  existing specification language shall be used for
  this task.

o Specification of IP-based protocol for transport of the
  controlled objects. When the control and forwarding devices
  are separated beyond a single hop, ForCES will make use of an
  existing  RFC2914 compliant L4 protocol with adequate reliability,
  security and congestion control (e.g. TCP, SCTP) for transport

The main focus area of the working group will be control and
forwarding separation for IP forwarding devices where the
control and forwarding elements are in close (same room/small
number of hops) or very close (same box/one hop) proximity. Other
scenarios will be considered but at not the main focus of the
work. The functional model of the forwarding element will include
QoS (DiffServ and IntServ) capabilities of modern networking
devices such as routers.  In order to minimize the effort to
integrate forwarding elements and control elements, a mechanism
for auto discovery and capability information exchange will form
an integral part of the standardized interface.

ForCES will coordinate with other standards bodies and working
groups as appropriate. Examples of such bodies include IETF/GSMP,
IETF/Megaco, the Network Processing Forum (NPF), the Multiservice
Switching Forum (MSF), IEEE P1520, and SoftSwitch. ForCES will
review relevant protocol efforts such as GSMP and Megaco and will
extend or reuse them if appropriate. If protocol reuse is
accepted as satisfactory for fulfilling the ForCES requirements
then ForCES may recharter to adopt specific deliverables around
the selected protocol.

Goals and Milestones:

Done  Submit requirements document to IESG
Done  Submit framework document to IESG
Nov 04  Submit forwarding element functional model document to IESG
Mar 05  Submit formal definition of controlled objects in functional model
Mar 05  Submit protocol selection/definition document to IESG
Mar 05  Submit applicability statement to IESG


  • draft-ietf-forces-model-05.txt
  • draft-ietf-forces-protocol-04.txt
  • draft-ietf-forces-tcptml-01.txt
  • draft-ietf-forces-discovery-00.txt

    Request For Comments:

    RFC3549 I Linux Netlink as an IP Services Protocol
    RFC3654 I Requirements for Separation of IP Control and Forwarding
    RFC3746 I Forwarding and Control Element Separation (ForCES) Framework

    Current Meeting Report

    IETF ForCES wg meeting, ietf63, Paris, Aug 1, 2005, 10h30-12h30.

    Robert Haas and Alex Zinin chairing the meeting.

    1) administratrivia (Robert Haas, 4 min)

    Presentation: forces-ietf63-chairing-haas.ppt

    Consider last call for protocol and model drafts before ietf-64. Attendance/list is asked to participate in preparing the MIB draft(s) and LFB library draft(s). Also, existing or in-progress implementations should be reported to the list.

    2) Flexinet presentation (Robert Haas, 6 min)

    Presentation: forces-ietf63-flexinet-haas.ppt

    Description of the basic and extendable function blocks in the Hitachi modular router. Packets received by the NE are directed to one or more other FEs for packet treatment, which requires metadata exchange among FEs (future scope). Description of the ForCEG (ForCES CE Gateway) used to gateway between services such as AAA Proxy and QoS and the FEs, ensuring consistency and appropriate demultiplexing to those services. Implementation status.

    3) ForCES protocol updates presentation (Robert Haas, 33 min).

    Presentation: forces-ietf63-protocol-updates-haas.ppt

    Review of all technical issues in the tracker:

    Issue 6: DATARAW and PACKED-DATARAW. Necessary due to existence of optional fields. Need to introduce to ILVs (Index-Lengh-Value). Question by Alex Zinin to verify if the consensus was indeed agreed on the list. Accept.

    Issue 11: Heartbeat piggybacking used to skip sending heartbeat during protocol activity (shows that both FE and CE are alive), need to add a flag in the FE Protocol LFB. Accept.

    Issue 12: Message obsoleting at TML. Not supported by most transports. Reject.

    Issue 22: Response formatting. Use of RESULT-TLV. Accept.

    Joel comment: depending on the execution mode (do all, up to error), each request in the config message can report its result, or a single result can be reported if the execution is stopped at the first error. Accept.

    Issue 23: Common Header: no windowing at ForCES level. CE-FE is master-slave relationship. Atomicity and batch flags remain in the common header. Accept.

    Issue 24: Abandon the CE LFB. Accept.

    Issue 25: Association setup with FE ID left to 0 for CE to assign dynamically does not cause issues with mcast IDs. Accept.

    Issue 26: Teardown reason TLV. Accept.

    Issue 28: Multiple LFB instances select (under the same LFB-select TLV). Reject.

    Joel comment: not needed at this time.

    Jamal comment: corresponds to xcast. Can use multiple LFB-select TLVs instead.

    Issue 30: All event are subscribable. Accept.

    Issue 31: special Notification Response Message (from CE to FE). Can use ACK flag instead. Reject.

    Issue 32: Packet redirect with two LFBs: sink and tap. Encapsulation of redirected packet in a Redirect Message and TLVs. Static metadata assignment, with metada transcoding. Accept.

    Issue 33: Command pipelining is inherently permitted ny the protocol. Correlators may be used to match responses with requests. Accept.

    Issue 34: Header flags: text will be provided to set the precise meaning of the ACK, priority flags. Throttle flag will be dropped. Pending.

    Issue 36: FE/CE failover. CE failover policy determines if the FE should continue or stop forwarding packets after an association is lost. FE failover policy determines how the FE should behave after it regains an association, currently: start from default state again (no fast recovery). Accept.

    Issue 37: Remove requirement for the FE Protocol LFB. Instead, global defaults must be used. Heartbeat at 300 ms, with a Association Expiry timer of 900 ms. Accept.

    Issue 40: BLOCK operations. May include in a future release of the protocol. Reject.

    Issue 47: PL sequence number and correlator. Convert the sequence number and the correlator into a 64-bit correlator. Correlator used in Config and Query messages. Must be set to 0 for Notifications.

    Joel comment: use a 64-bit correlator field, left to the CE to split it how it wants.

    Accept the introduction of correlator. Accept the 64-bit correlator to replace sequence number and correlator.

    Issue 52: Association Setup. Unsolicited information can be transmitted in the Association Setup message from the FE, using GET-RESPONSE. CE can set attributes with the Association Setup Response using SET. Accept.

    Issue 55: Recovery of transactions in intermediate state after association comes back up. Relates to issue 36. Reject.

    Issue 56: Association Setup Result TLV. Accept.

    Issue 57: Operation types.

    Joel comment: No Event Notification Response message needed.

    Avri comment: some of the operations are historically here still.

    Joel coment: Need to have set and delete. The rest of the operations (add, update, replace, del all, cancel are not needed).


    4) Model updates (Joel Halpern 11 min). No slides.

    Comments about the draft from the working group are awaited. LFB definitions are needed, contributions from the working group are awaited.

    Three noticeable changes:

    a) addition of properties information (information about the elements in the LFB): there is a flag in the protocol to reference properties instead of data itself. Model includes property information such as is the element readable/writable, array information (how many elements exist, highest/lowest subscript). Properties allow the capabilities information to be linked to the elements themselves.

    b) aliases: for a given LFB instance to be able to reference information in another LFB instance, so that data does not have to be duplicated. For instance, the ARP LFB needs the MAC address field from the Ethernet-port LFB. Properties information tell which field of which LFB is referenced using PATH notation. Most LFBs won’t need aliases.

    c) events: event declaration added to the model. Declares an ID for the path to the base of all events in an LFB, and all events declared within that. Allows a subscribe-to-all events. Event declaration contains: event ID, target variable (to indicate which element in the LFB is causing the event), condition, information to report (value that change, subscript, etc). Base on a nested XML structure with field names or variable names that can be used to represent array subscripts. Event properties contain a threshold value, if the event is subscribed, and an hysterisis (FE not required to support hysterisis, but important to suppress oscillations and DoS’ing the CE).

    Key and Keyref parts of the XML should be validated.

    No questions.

    5) LFB examples (Jamal Hadi Salim, 25 min).

    Presentation: forces-ietf63-LFB-examples-jamal.pdf

    Netdevice LFB (aka Port LFB). Abstracts layer 1 processing. Example of Netdevice with ingress/egress parts, each with wire/upstream/downstream LFBs ports.

    Joel question: why does ingress has an upstream LFB port ?

    Jamal: Used to route back packets to other ports, applicable if netdevices are virtual ports.

    Joel question: need of a loopback device in the model ?

    Jamal: loopback is a bad example. Tunnel device that validate headers.

    Description of the Netdevice LFB: capabilities, topology constraints, MIB-derived info, etc. More specific description of the attributes of the Ethernet Netdevice.

    Robert question, Joel answer: events are created whenever an LFB is instantiated. So the CE is aware of LFBs that are autonomously created by the FE.

    Sample topologies of Netdevices, with Ethernet, bridge, ipv4/v6, forwarding LFBs.

    XML model for the Netdevice LFB with capabilities, events, and attributes. The LFB contains a table of netdevices. Existing implementation of Ethernet netdevices.

    Joel comment: change name from IPv4 LFB to IPv4 validation LFB (TTL decrement, checksum, header validation. Only output is either valid or non-valid) LFB.

    Joel comment: The IPv4 LFB should not own the locally-terminated IPv4 addresses (the router’s IPv4 addresses). IPv4 addresses should not be attributes of the IPv4 validation LFB. They should be handled by a separate classifier LFB (model rule is to not fold a classifier into the IPv4 validation LFB, use more LFBs instead). On the other hand, the netdevice may include the classification based on the Ethertype, as it is a desperately common case (alternatively, the netdevice may pull out the Ethertype as metadata and give it to a classifier LFB). Use “thin” LFBs.

    Short overview of the IPv6 LFB.

    6) Updates to TCP TML (Jon Maloy, 19 min)

    Presentation: forces-ietf63-tcp-tml-maloy.ppt

    Jamal, Joel comment: multicast model for TCP is multi-unicast.

    TCP no longer specified for the data channel, issues with transporting TCP fragments over TCP. Possibility of using DCCP or GRE tunnels.

    PL layer knows only FE and CE IDs, does not see channel descriptors anymore.

    In the High Availability case, FE TML not responsible to decide which CE to open channels to.

    Complete removal of TML messaging and modifications to channel setup.

    Multicast model: TML maintains mapping between mcast IDs and channels.

    Description of the TML Service Interface used by the PL.

    Robert question: Is the FE Protocol LFB used to set in the FE the mcast IDs the FE belongs to ? Jon: a specific PL-message MC Grp Join Req is to be used.

    Examples of unicast channel setup, and mcast group join and leave.

    Robert question: McId shown on slide 13 is a regular PL mcast ID ?

    Jamal: This document contains more PL-TML APIs than before. Was the PL-TML API not supposed to be published in a separate document, according to consensus from previous ietf meeting ?

    Joel: there is no other wg document now that contains this.

    Jon: the PL-TML API can be moved to a separate document.

    Open issues: which protocol to select for data channel ? Security: IPSec, and/or TLS ?

    Question by Toshiaki Suzuki: to ensure HA, one FE may be connected to two CEs.

    Joel: currently out of the scope of the charter to support multiple CEs for load-balancing reasons.

    Robert: mcast IDs at PL level can be used for such purposes. Failover to a CE in standby mode is in the scope of the charter.

    Jon: There is a Service Availability Forum model to make HA clusters.

    7) TIPC TML (Jon Maloy, 14 min)

    Presentation: forces-ietf63-tipc-tml-maloy.ppt

    TIPC is a lightweight protocol, potentially better performance than TCP.
    TIPC sses the same addressing as PL IDs.

    A reliable connection for control and a best effort connection for data.

    Description of the address mapping for unicast and mcast.

    No security, only for closed LANs. Congestion controlled.

    No Nagle (immediate delivery).

    TIPC support for failover over parallel links.

    4 priorities only in TIPC. Need to map from the up to 8 priorities.

    Joel question: in case of congestion on the data channel, messages are dropped by TIPC.


    Agenda and Status
    Architecture of the Flexinet ForCES-based Control Point
    ForCES Protocol Updates
    Examples of ForCES Logical Function Blocks
    ForCES TCP-based TML
    ForCES TIPC-based TML