ForCES Working Group, MONDAY, March 26, 2012 CHAIR: Jamal Hadi Salim Minutes: Kentaro Ogawa and Evangelos Haleplidis Attendees: About 70 people attended 1) Meeting started with the chair presenting the agenda and pointing details to remote users. 2) Chair presented WG status - LFB library draft. WG last call happened and feedback received. The authors have responded to the feedback but the originator has not responded yet. That's what holding us back. May have to repeat WG call. - CEHA update. New implementation experience that suggests tweaks. Demo today. Going to publish a new version of the draft and do a WG last call before next meeting. 3) LFBlib Update (Weiming Wang): -------------------------------- version 8 released. Compared with last version changes mainly on the definitions and resolved some issues. We've also received valuable comments for which we have responded to. - Brief intro on updates. Update mainly on definitions, name of datatype. More importantly changed definitions of datatypes, and we added some reserved fields for datatypes, for ease of implementation. - comments from Edward Crabbe. *First issue about metadata. Question is, if it's not used, it's effectively invisible, and implementations in a HAL shouldn't pass through. When we design we just say that if... Joel: this was very explicit protocol design decision. When an LFB produces a metadata that will be used downstream, you don't update all LFBs definitions specifying this passes through. You consume only what is defined, the rest is passed through. The library document should not be messing with this at all. We need to have more discussion continued in the list. *Next issue about flow control. Question: Isn't the ability to send pause frames a capability? We currently define flow control as a configurable element rather than a capability. We are considering to add a FlowControl capability for the LFB. *Question raised on Ethernet Classifier Incoming port identification. We have single metadata describing incoming port. Worry that it may be insufficient. Discuss later. *On EtherEncap LFB Media Encapsulation Index question: The current approach is so that if someone requires address resolution it will not be able to find the Media Encapsulation Index. Downstream LFBs use this information as a trigger to ARP if they need to. This allows us to support both a model where ARP sits in the FE or CE. *Left some editorial issues, not so important to spend time. Continue discussion in the list. All these issues not in the list. These slides to put it in the list and to the proceedings. Jamal: Slides online. Please provide comments. Need to resolve any outstanding issues and address them. Are there any big changes? Weiming: Some technical issues, not very big. Jamal: Worried about need for last call. Discuss later. 4) Jamal: CEHA document ----------------------- Published, version 3. Next update to be based on the slides. More implementation experience on the CEHA draft. *FEPO BackupCEs table left untouched (for backwards compatibility), and add a new table AllCEs (read-only) advertises what CEs exists and state. For old versions of FEPO use only BackupCEs. *RFC5810 has only cold-standby. No way to turn it on/off. update introducing a new knob that can be set by the CE or the FEM interface to select either hot/cold standby or no HA. *The previous draft version added two new fields, it is unecessary. The CEFailover Policy defines what to do when you lose state. Administratively disabled, or some form of graceful restart. *The new draft will introduce the capability of Hot-standby, cold-standby. *Shows a typical RFC5810 cold standby setup with 3 CEs used in the demo. The FE will associate with only the master, CE03 (and not backups CE05, CE07). *Shows what happens in master CE failure. Could be introduced by either connectivity failure, heartbeat dead interval, or the CE may issue a failover by setting CEID. *Demonstrated the cold-standby described in the slides *In case of hot-standby, the draft suggests to have association with all CEs. On startup tries to associates with the master then on best-effort basis with the rest. Events, HBs get sent to all CEs. Configs only from master CE. *In the case of failure or a planned migration (change of connection), there is a difference. In cold-standby only one association, now multiple; therefore failover would be much faster. *Shows 2nd Demo with hot-standby by a) killing the master CE and b) by running an application which forced a mastership switch. * During implementation found it easier to have backup CE discover by doing GETs to the FE. Previous draft says: no messages from backup CEs. Only Events and HBs. Seems valuable to allow GETs as they don't affect the consistency. Still won't accept any other messages from backup CEs for simplicity. * Questions: (Didn't catch the name): I wonder as to the historical nature why you considered that the FE updates the CE of the master change. Why it is done from the FE and not from the CE? Jamal: It is a mix. We assume multiple CEs and some application that synchronises them. The event emmited from the FE, that happens on a failure, is just to advertise which CE we think is our master. (Didn't catch the name): In that case you must take into consideration scenarios that FE now thinks that it's pointing to one master, but the application points to another. Jamal: Yes. There is a confirmation state the FE must undergo. The FE says "i think the CE# is our master". The CE confirms or denies that. The successful confirmation is advertised to all CEs. (Samita Chakrabarti): You mentioned two implementation available for HA? Jamal: There is another implementation which was used on previous interop. (Samita): Possible to know who? Jamal: University of Patras. Based on previous version of this draft. (Samita): This is implemented by you (Jamal)? Jamal: Yes, implemented by my company. Cold-standby is defined by RFC5810. I suspect all existing implementations have it. Kentaro did you implement cold-standby? (Kentaro says yes) 5) Sue Hares Openflow comparison to ForCES * stated that her company is looking at implementation of ONF and she has a background of ForCES. Had discussion with Edward Crabbe who inspired her to look. *plans to post a draft but needs input from people on this presentation. * Major question was how ONF different from ForCES. Shows a slide on what the latest view of what ONF is based on public info. * The real question is how does this help us to get to devices that operate in the Cloud or S"D"N. * Edward had commented that ONF 0 is diff-serv Tspec, OpenFlow 1.0 is ForCES-- and 1.1 is ForCES++. * Sue summarizes that the ONF approach as incomplete repackaging of ForCES stuff. Implementation may differ. *Historical: ForCES started with industry forum (NPF) trying to comoditize hardware chips. Went to IETF to standardize protocol. OF, started trying to do research (GENI was NSF funded) then formed an Industry Forum (ONF). *Architecture: OF Controllers look like CEs and datapath looks like FEs. What the OFS don't have is the managers. The OF switching do not define initialization in their specs at the moment. * The flow logic: -Staring with OF1.0. you see a flow go through a series of tables. no match may drop, redirect or send to the controller. How does this look at the ForCES? OF1.0 will be considered a very static definition of a set of ForCES LFBs. i.e in ForCES one defines the different functional blocks and how they are connected in a directed graph. -Then OF 1.1 and 1.2. Introduced is metadata, frame packets, action sets. Action sets look like metadata to me. I have not implemented the group table. Like to talk to implementers of Group Table, seems like LFB logic but that's probably what Edward meant by ForCES++. If you've done an implmenetation please enlighten me. How does this look at the ForCES? again, OF1.1/2 will be considered a very static definition of ForCES LFBs. * Modelling: -If you look at the modelling and libraries, OF1.0-1.2 all have a static model assumptions of how a datapath looks like. It seems there is realization now at the ONF and there is a "future buzz" that modelling is important. Not sure if this is possible without a total revamp. *Protocol. - ForCES protocol definition is static which is a nice feature (few simple verbs working on many defined nouns); whereas OF is pretty dynamic with nouns built into the protocol (meaning every time a new data path component is defined a new protocol message is neeeded). - Both have an approach to securing transport. - ForCES underlying transport is "pluggable" depending on environment whereas OF is strictly TCP. -ForCES goes to great pain to treat events, config, queries and redirects differently for scaling and robustness. OF doesnt. * Discussion Sue: Look at standardization of two facets, pros/cons. If ONF continues to just take ForCES concepts (as they seem to be doing), why don't you drink the whole drink rather than pieces. If something's different then i'd like to figure out what it is and resolve it. Jamal: Anyone from ONF here? Somebody gotta ask questions or make comments. Adrian Farrel (speaking as an individual not AD): It is useful to have this discussion, i don't know where it goes in IETF. Susan Hares: Presentation and followup draft is informational, if it's useful we'll continue publishing. It is useful to continue to understand in the world of marketting buzz of ONF what is ForCES' role. Jamal: Lot of confussion due to the ONF marketing noise because people seem to be equating OF to ForCES. Glad for this presentation. Boil it down to differences. People think that they can build a router with OpenFlow, i am unsure how one goes building a BGP based router with an ACL table that holds 256 entries. (Samita): Like the work, seems important to figure out differences, a lot of us have the same problem wadding through the marketting confusion. It looks like Openflow is slowly morphing into ForCES. Could be nice if IETF organization could come (Rotem Salomonovitch): Does CE1 to CE2 have some sort of interface? Susan Hares: Directly from ForCES draft, no protocol currently, not defined. (Omar Cherkaoui): Terms in granularity of changing the behaviour of the LFB. Not so familiar with ForCES and how you can change the behaviour of LFB or the switches. Policy based approach? Susan Hares(with interjection from Joel Halpern): In ForCES, programming of the LFB instances defines how the FE behaves. This is done by the CE as a result of service configuration by a protocol or policy; it may be as a result of arriving packets or events. Additionaly an LFB may have state which defines future treatment of subsequent packets. ForCES services is driven by a directed graph of LFB instances. They can change the packet, can send out multiple packets, can do the logic, you may even loop. I dont know how you do that with OF. Trying to get clarity please let me know. Use the wg for discussion. Jamal: We are already 10 minutes past so this is going to be the last question. (Marc Lasserre): How do you see OpenFlow evolving, we see 1.0, 1.1, 1.2; i hear there is 1.3 and i think there will be 1.4 maybe 2.0? Jamal: Well, it seems that whoever is implementing OF will eventually reach the same conclusion we already have reached with ForCES. This can be observed from some of the evolution Sue mentioned. We welcome the ONF to use ForCES. Marc asked a secondary question that we lost. The meeting was adjourned at this point. The discussion went on for a while longer both in the room and in the hallways.