DetNet 101 Minutes NOTE TAKERS included (not required): Pat Thaler Jouni korhonen (colors do not match minute takers properly, except for Ethan) Ethan Grossman Andy Malis > DetNet Agenda IETF101 (London) > > > Chairs: Pat Thaler > Janos Farkas > Lou Berger > > Friday, March 23, 2018 (GMT) > 0930-1130 Morning Session I > 11:50-14:30 Friday Afternoon session I > > Blenheim > > Slides: https://datatracker.ietf.org/meeting/101/session/detnet > Etherpad: http://etherpad.tools.ietf.org:9000/p/notes-ietf-101-detnet?useMonospaceFont=true > Meetecho: http://www.meetecho.com/ietf101/detnet > Audio stream: http://ietf101streaming.dnsalias.net/ietf/ietf1012.m3u > Jabber: xmpp:detnet@jabber.ietf.org?join > > Available post session: > Recording: https://www.ietf.org/audio/ietf101/ > YouTube: https://www.youtube.com/watch?v=9MXy3OdCxp8 > > # Start Time Information > 0 9:30 15 Title: Intro, WG Status, Draft Status > Presenter: Chairs Deborah Brungard: Appreciation for Pat Thaler. Janos Farkas: IETF and IEEE 802 are meeting consecutive weeks in the same hotel. IETF 103 in Bangkok 3-9Nov18 and IEEE 802.1 TSN starts 9Nov18. Pat Thaler: Likely joint meeting with 802.1 TSN the Saturday after IETF103. Norm Finn: There is an 802.1 joint effort with IEC SC65 group - on industrial profile for TSN. If we can get them to attend IETF 103, they would be a third group who could profit from DetNet since DetNet would be relevant for them as they get to larger networks. Pat Thaler: We discussed this this morning, but Glenn thought they might not be available then. Not clear if they would attend, but we can invite them. > 1 9:45 5 Title: Use Cases > Presenter: Ethan Grossman > Draft: https://tools.ietf.org/html/draft-ietf-detnet-use-cases-14 Ethan will update the draft in the next week with IPv4 and slicing content. Then last call will begin. Stewart Bryant: There will be more revisions afterwards, but best to start with the best we have. Ethan Grossman: Lou, any opinion on IPv4 mention in Use Cases draft? Lou Berger: IPv4 is in our charter. Must deliver IPv4 solution, reasonable to mention in Use Case draft. Ethan to add brief mention. Reviewers should pay attention to whether solution is effective for IPv4. > 2 9:50 5 Title: Security Considerations > Presenter: Ethan Grossman > Draft: https://tools.ietf.org/html/draft-ietf-detnet-security-01 (No comments) > 3 9:55 15 Title: DetNet Flow Information Model > Presenter: Balázs Varga > Draft: https://tools.ietf.org/html/draft-ietf-detnet-flow-information-model-01 Toerless Eckert: Is there support for a model where the end node performs some DetNet function e.g. PREF. For example "video applications" where end hosts are doing "PREF" type things is a well- standardised, e.g., SMPTE 2022, Which is done over RTP/UDP. Janos Farkas: Agree important, but let's defer discussion to Data Plane discussion. Ethan Grossman: Note that this service is done at a layer above DetNet, i.e. not an "inherently highly reliable service" which is delivered transparently to the DetNet user. SMPTE standards are not (at least in past) publicly available. Lou Berger: Is the host described in the information flow models? Janos Farkas: yes Toerless Eckert: Specifically for PREF, current UNI subsumes both network and user side. Lou Berger: Agree it would be important to cover this, please suggest text for the document. Janos Farkas: Also need new reference points for the discussion. Lou Berger: Model should be independent of any first steps, which may have compromises, but model should support even w/o those compromises. Janos Farkas: Target is to make the model general, and service provider-independent. > 4 10:10 15 Title: DetNet Configuration YANG Model > Presenter: Xuesong Geng > Draft: https://tools.ietf.org/html/draft-geng-detnet-conf-yang-01 Xuesong Geng: Should we have DetNet traffic class to distinguish DetNet flows by priority? Should DetNet have its own queuing algorithm(s)? Greg Mirsky: Link delay and usefullness of these metrics. These RFCs exclude queueing delays, which we are interested in. These introduce jitter, the rest is physics. So link delay can be calculated based on distance. What we need is residence time which is measured by ingress/egress, how traverse the node. An ordered, directional pair. That would be more useful information, already included in SFC document. Preparing doc on more general use of residence time in calculating "t" (time?). Lou Berger: This is a comment on the info model not the Yang model. If you're saying this is wrong, check there first. Greg Mirsky: Not saying it is wrong, just that may not be useful. Lou Berger: If something else is more useful, should be in Info model first, then flow down to Yang. Xuesong Geng: Point of DetNet is that delay is deterministic, i.e. from queueing management, and thus can be calculated - doesn't need to be measured, so it is different from other WG work. Norm Finn: New draft - please participate - on bounded latency and network calculus. Diff't ways to compute, and we'll pick one. Comment on "More than one DetNet class of service" - Many won't need that, but some may need it, e.g.due to different trade-offs, so don't exclude more than one detnet class. There is parallel Yang work ongoing in IEEE 802.1TSN. It would be great to keep these two activities in sync. Greg Mirsky: Should this Yang model be normative for our info model? Usually there is a mechanical way to derive a Yang model from an info model, which guarantees that they will stay in sync. What guarantees that this "independent" Yang model will be identical to a Yang model mechanically generated from the info model? Xuesong Geng: Look at first slide. Those two Yang models cover different data models, like northbound vs southbound interfaces. Flow DM,Service DM - from User to Controller - convered by Info mode. Configuration model covers different parts. Greg Mirsky: What guarantees that there will be no overlap? Between Yang model derived from Info model vs these Yang models. Xuesong Geng: Agree, we should consider the interface between these models. Toerless Eckert: We would like to remove the human translation from Info model to the Data models, it is another place where we may create bugs. Lou Berger: What is your proposal? Toerless Eckert: There are many abstract things in the Info model that are not captured in Yang, that's a problem. If every thing in Info model is meant to be represented in Yang, why aren't they equivalent? The proposal is to have only one normative place, that should be the Info model. Then the associated mechanically generated Yang models would by definition cover all of the Info model. Mach Chen: The Topology model is not concurrent with Info model. It is to define a model to discover the capability of the network. Toerless Eckert: Not saying we need Info model for every Data model. But if we have a Data model for every Info model then what is the added value of the Info model other than to create bugs. Greg Mirsky: Organizations usually do either info model or data model. How to guarantee consistency between the two if DM is not mechanically generated by IM? Lifecycle orchestration issue of service model. Info layer vs Service layer are diff't layers - how to guarantee consistency between these two views? Lou Berger: Do we need to guarantee that? Greg Mirsky: Yes. Lou Berger: Have info model to inform yang model. Yang model drives implementations. Greg Mirsky: Orgs use info model to generate data model. Lou Berger: But they are using a formal tool like UML, but we are documenting in text form, so we need a human step not a button push. Greg Mirsky: I'm not saying that is wrong, but maybe rename from info model to info flow or something, so can differentiate from the way orgs usually call them. Lou Berger: Some in IETF are done with UML, some narrative. But not aligned with ITU-T. Diff't approaches. Alex: There is an RFC3444 explains relation of info to data model - should clear up some confusion here. Balasz: Flow Info model is about flow. Yang model is for configuration details. Many details are not included in the Flow Info models. That is why we have separate documents for Flow Info and Yang models. Lou Berger: Flow definition: A specific series of packets vs broader context of how our charter definesit,which is all the info needed for flow establishment and control. Greg Mirsky: When you say "develop an info model" that means a thing you can derive Yang from. Not just ITU-T, also other SDOs. Toerless Eckert: My original goal in this topic was to simplify our life, i.e. any place where there IS a one-to-one correspondence we should have only one normative place for that info. Lou Berger: Always need some narrative form and then a module description. One is syntax, other is concept. Mostly concept in info model. Syntax is in Yang. Toerless Eckert: Yang model where logic Lou Berger: Normally one would reference other normative documents, but we don't have previous things to refer to. Lou Berger: So maybe anything in Yang model doesn't have to be in info model? Point is to be efficient. Xuesong Geng: If we have anything we can reference from Info model, we would reference it. Lou Berger: Good to have narrative in one place, syntax in another, but currently maybe too spread out. Relay Node Config slide: Greg Mirsky: Term "node" - Can some nodes do more than one function? Elim could be part of edge node? Maybe use a different term? Xuesong Geng: Yes, we could. Greg Mirsky: Maybe "Config function". Lou Berger: This is an issue for the Arch draft - please address it in that context. It is about to be last called, please comment promptly. Lou Berger: Have charter item to work on Yang models. Need to follow up to make sure doc is appropriately aligned with other docs. If you find you are repeating text, instead just point to other doc. May need to propose text for other doc so that we can look in the minimum number of docs for a given bit of info. Please do this quickly so we can poll the WG before the next meeting. Narrative parts are the sticking point now. Lou Berger: How many have read this doc? "an OK number" have read the doc, could use more. Lou Berger: How many think it is ready for WG adoption - about the same number. It's a good plan, please review and send comments to authors. > 5 10:25 10 Title: DetNet Data Plane Encapsulation -- Recent updates and plan > Presenter: Jouni Korhonen > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-03 Lou Berger: IP encapsulation is an important topic for this afternoon. Please come to second session. > 6 10:35 10 Title: Operation of Deterministic Networks over MPLS > Presenter: Stewart Bryant > Draft: https://tools.ietf.org/html/draft-bryant-detnet-mpls-dp-00 'Payload type' slide: (3 types of packet, IPv4, IPv6 and Ethernet) type? Stewart Bryant: MPLS doesn't use IP version field to distinguish payload type. Need 3 types. So need diff't flow group for each? Greg Mirsky: Restate as MPLS packet type recognition relied on first nibble. Stewart Bryant: MPLS does OAM recognition on first nibble. Greg Mirsky: If have MPLS LSP, IP is native payload over Ethernet... Assumed that label element with S bit set, first nibble will indicate payload type. Stewart Bryant: MPLS doesn't work like that - may do an OAM descrimination (PW thing). But no LSP relies just on the nibble. E.g. gets packets on diff't Ethertype. Greg Mirsky: For MPLS over Ethernet have MPLS encapsulation. If have PW use control word. If have GAL label then know you have OAM. Stewart Bryant: For payload other than OAM, have one different LSP for each payload type. Lou Berger: Both correct to some degree. A combination of control plane. Some info from there (e.g. is IP) then look at nibble to get version. Greg Mirsky: Differentiated by nibble value - must be dif't than 0, 1, 4, 6 - Bier was 5. Lou Berger: But 5 was also used. Need to take that up with IANA. Stewart Bryant: Want to align with MPLS separate vector for v4 and v6. Lou Berger: Not stated as a requirement to be that way, but some implementations did that. Need to look that up. Greg Mirsky: From standpoint of FAQ (?) will be diff't FAQ for v4, v6. Lou Berger: Yes, but label could be the same. Do we need a new identifier for type? Janos Farkas: There is an RFC 7813 that is applicable for setting up a graph with ISIS Stewart Bryant: Yes, need to follow up on that. Need to make sure this is an accurate reflection of our plan, then get it reviewed by more MPLS people. Lou Berger: Data plane discussion - decide how to incorporate this material into dp-sol draft or keep independent. Balázs Varga: on Flow aggregation slide: There are limitations on PREF in that situation, for aggregate just have single control word - so not end to end? Lou Berger: Defer to data plane discussion. > 7 10:45 10 Title: DetNet IP Encapsulation > Presenter: Andy Malis > Draft: https://tools.ietf.org/html/draft-malis-detnet-ip-dp-00 Lou Berger: This is documenting something presented in Prague in soln doc. Feedback at the time was didn't want to go this way, a discarded option. Stewart said in Prague that this was never implemented, shouldn't head this way (maybe his statement was misinterpreted?). Did not align well with hosts, would not be able to deliver end to end detnet service with this approach. That led us to the simplified approach in the interims. Need to articulate why we would want to bring this back. Andy Malis: Result of 3rd interim discussion. Addresses comments that DetNet is a nonstarter if can't use IPv4. This is to allow DetNet to be used in cases where must have IPv4. "Host" is DetNet unaware end system, no DetNet header. Stewart Bryant: To clarify: Seems like could use the same encap stream in host as edge: from a host point of view, from the UDP piece down this looks like a socket call, while the part above is just some magic numbers in the host payload. Why can't we use same in host as in edge? Lou Berger: Discussion was inserting bet IP xport (UDP, TCP) and IP, which is not easy to do in most host stacks. Stewart Bryant: Since host has already put UDP on it in before we get it? Lou Berger: No, the app makes a posix socket call. Then we have to insert our header in the middle of the stack. Stewart Bryant: Couldn't we convince them to put magic bytes in front of its output then we'd be done? Lou Berger: Concensus was that this wasn't a reasonable ask. Would have to work with Transport area. Doing MPLS over IP PSN is well known on edge nodes, but for host side there was a lot of push back. Lou Berger: Also there are many devices where it is hard to change the stack, e.g. embedded, want to support those. Andy Malis: The DetNet encapsulation is added in the edge node, not in the host, to conform with the simplified model. Taking it to the host is not part of this draft. Greg Mirsky: So this is a derivative of unified segment routing. Andy Malis: Yes, similar to that. Greg Mirsky: So when discuss host, need to extend IGP SR to support host, to participate in this. Andy Malis: Yes if we go to the host, which we don't know. Greg Mirsky: Whatever edge of DetNet domain, has to use IGP SR protocol extenstions. Andy Malis: If we want to do SR at the host, which we don't have to. Jouni Korhonen: It is do-able, one can view as tunneling. Stripping away additional away IP header was the hard part for the host. This refers to approach where there is only one IP header in the packet. Assume the end system send X over IP, and the stack is then supposed to output X over "detnet shim" over IP. It is relative simple to do X over IP over "detnet shim" over "transport IP". Jakob ???: About inserting header in the middle - if host generates IP packets, don't see way today allowed by IETF to not have basicaly two IP headers, one IP header for DetNet, then payload packet would have to have its own IP header. Look at Rohl(?) and others who have tried to insert things in the middle and have failed on constraints of IPv6 arch. Not allowed to do before, not allowed to change IPv6. Unless have a model where DetNet goes all the way to the host then the whole stack can do whatever it wants and can get away with just one IP header. Possible to fight this battle but seems very difficult. Cha hoo (?) from Ali Baba: This is almost same as MPLS over IP with slight control word label differences. Like original MPLS over IP encapsulation draft. No need to re-describe this. Andy Malis: Yes, I referred to that RFC 7510 in my document. MPLS over IP approaches were proposed by the DT in IETFs 98 and 97 > 8 10:55 10 Title: BIER-TE Overview > Presenter: Toerless Eckert > Draft: https://tools.ietf.org/html/draft-ietf-bier-te-arch-00 > Draft: https://tools.ietf.org/html/draft-huang-bier-te-encapsulation-00 > Draft: https://tools.ietf.org/html/draft-thubert-bier-replication-elimination-03 Greg Mirsky: You realize that the proposed format for BIER-TE DetNet is not compatible with existing BIER encapsulation? Toerless Eckert: This is being proposed in BIER, adding control word sequence number - this is not needed for BIER forwarding, but is used for other overlay service e.g. DetNet in this case (there are also other cases). Greg Mirsky: But is it same control word? Would not be BIER. Toerless Eckert: Could assign new Ethertype. Lou Berger: We look forward to hearing what the BIER group comes up with. DetNet over BIER is not exactly out of scope but probably not what we are doing now, unless someone wants to work more on it. > 9 11:05 15 Title: DetNet Bounded Latency > Presenter: Norm Finn > Draft: https://tools.ietf.org/html/draft-finn-detnet-bounded-latency-00 Kirin ???: When you compute these queueingd the outcome will be bounded latency or queueing model then compute latency on top of that? Norm Finn: Expectataion is that a give flow has a req't for end to end latency, and that you must reserve resources along path to achieve that. We haven't gotten into control plane yet but assuming Yang models, central path controller will be able to compute the latency that can be achieved for this flow. Whether we have a protocol that goes go hop by hop along the path or via a central controller is TBD. Kirin: Queuing models are on a per-node basis - i.e. specific to a node, not the whole network. Lou Berger: Expectation is a timing model that includes queueing delay. Could schedule an interim to discuss this further. > 10 11:20 10 Title: Deterministic Networking Application in Ring Topologies > Presenter: Yuanlong Jiang > Draft: https://tools.ietf.org/html/draft-jiang-detnet-ring-00 Greg Mirsky: Not efficient since not using physically disjoint network, but are cutting your bandwidth in half. Lou Berger: Time is up, please continue discussion on the list. > Adjourn 11:30 > 11:50-14:30 Friday Afternoon session I WebEx: https://www.youtube.com/watch?v=yUtlUHHVOkM > > # Start Time Information > 0 11:50 5 Title: Intro, WG Status, Draft Status > Presenter: Chairs > Draft: Toerless Eckert: Question whether multicast is part of this round of documentation. Lou Berger: I would expect yes it is part of the data plane, but that is my assumption, not consensus. Toerless Eckert: would be great if mentioned explicitly.. also whether multicast dest addresses are in scope.. If not enough review and support on this, may not want to say we support multicast. Lou Berger: has concerns multicast of PR+EF. We should consider for each key question (1, 2) "does it work for multicast - are we all on board for multicast". For MPLS I'm saying point to multipoint, for IP I'm saying multicast addressing. Toerless Eckert: High level will be if reviewers feel confident about supporting multicast. Many potentially hard cases e.g. differential delay for Norm's bounded latency doc, doesn't exist in unicast, many others. Stewart Bryant: if we get PR+EF right, multicast will come almost for free. PR without EF. Lou Berger: Normally think of 1+1 PREF case, but may now have 1+N case, i.e. N packets to eliminate. Probably will work if we get PREF right. Norm Finn: echoes that multicast comes "easily" if/when PR+EF done correctly, based on 802.1 experience. > 1 11:55 10 Title: Intro: DetNet Data Plane Encapsulation -- Resolving open issues > Presenter: Chairs > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-03 > 2 12:05 130 Title: WG Dicussion: DetNet Data Plane Encapsulation -- Resolving open issues > Led by: Balázs Varga/Jouni Korhonen/Norm Finn > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-03 Toerles?? The context for d_CW is S-Label not T-Label? Balázs Varga: yes, in the context of S-Label. Toerless Eckert: That would be a scaling issue. Maybe context needs to be defined for T-Label. For aggregation. Balázs Varga: Aggregation is separate discussion. Stewart Bryant: Need aggregartion discussion. Encap looks same with or without PR+EF, except seq num in control word, else identical. But the CW should be there in all cases - in PWE3 had problems with shipping a lot of Ethernets w/o CW and had a lot of work to retrofit. Lou Berger: This is the generic MPLS encap, i.e. applies to Ethernet and IP. Doesn't counter whether always have CW, still may, but this is equally applicable for L2 VPN or IP over MPLS in any usage context. Stewart Bryant: For Ethernet must have CW even if just zeros. Toerless Eckert: Good to figure out what flexibility we have to define context for CW. Fixed to S-Label is simplest, but concern about scalability i.e. number of flows we can support. E.g. memory for sequence numbers of packets recieved with a given time window. For aggregates, 28 bits may be enough. All traffic for one ingress node to one egress nodes. If context of these could be defined more flexibly so could use for aggregates, would help optimize HW resources. Toerless Eckert: worried about the amount of state maintained for the seqnum. I.e. how much memory is required for each flow. Andy Malis: scalability is exactly the same as for PW and L3VPNs. We have a lot of field experience that routers work fine in terms of scalability with current def of that label. Toerless Eckert: Not aware of implementations of L3VPN equipment where SeqNums are used for per-packet de-duplication, which is what brings in the state requirements. So if there is any real evidence of this from the field that it supports large numbers we'd like to hear it (we don't think it exists). Lou Berger: There are multiple T-Labels so an implementation can add more labels if it needs to bring more context. At Control or Mgmt plane want to ensure we don't preclude that. Toerless Eckert: MPLS experts might say it is a change to the control plane, imply that kind of semantic that the context would be the T-Word, or no you can't even do it in the forwarding plane. Lou Berger: Existing control planes are pretty flexible like that. Stewart Bryant: This is the right way to go on. We could hit a scaling limit earlier than with PW, but if so we can add a context label and re-spin the design, but it isn't a fundamental issue. Lou Berger: Is the number of bits in CW sufficient? Created a spreadsheet. Only ran into issues when have flows of 400GB - eventually might need to add longer CW, probably not soon. discusison jumping to CW SeqNum size.. norm: reminds about the book keeping. You need to keep history of past packets for PR+EF purposes. Speaks furiously for 16 bits ;) Norm Finn: Can allow for more bits of SeqNum, but need an option for exactly 16 bits to maintain compatibility with existing equipment. Lou Berger: Assumed sliding window approach. Norm Finn: Agrees. Stewart Bryant: Agrees. Size of window and size of seq num are parameters that we establish. Each relay node needs to establish params of d-CW. Jouni Korhonen: Agree with Norm and Stewart about SeqNum - needs to be parameterized. real recent hw I know that do 802.1CB remembers just few K past packets.. sliding window can be any size. Lou(?) Just because 802 does something doesn't mean we have to do it that way. Jouni Korhonen: current DP draft already documents parts of this. Norm Finn: Advocates for interoperability with 802.1CB Stewart Bryant: No gratuitious differences, but allow differences where needed. No PREF case: (do we still need a CW?) Andy Malis: control word is must. refers to parallel work in PALS. Stewart and Jouni acho the same. Stewart Bryant: Do you wish to control whether the underlay does ECMP on you. Using CW we got a full control of the flows.. Andy Malis: there's a lot of bad hardware out there that make funny assumptions about what it can do regarding ECMP. The only way for us to be in control is to include CW even for IP. Stewart Bryant: Only gets us partial control. Some equipment says if I see zeros here I assume it is IP and ECMP deep into the packet. Bad. Lou Berger: If we go this way on the data plane, we need to talk to MPLS WG on this, get broad buy-in. Andy Malis: Not a change: it is IP in MPLS over IP Lou Berger: IP over MPLS is done 2 diff't ways: 1) directly mapped FAQ to label, 2) L3VPN, no CW there. Stewart Bryant: But they don't do in-order delivery, expect packets to be ECMP'd - a different world. Lou Berger: Enhanced VPN is good topic for TEAS. Toerless Eckert: Maybe we don't care about broken equipment, we just document? Lou Berger: If we want to do something different than usual with the MPLS data plane, we need to talk to MPLS WG. Lou Berger: We heard: w/o PREF no brainer to alwyas include d-CW. No objections. With IP, there's a preference from some people to use CW also. Two questions: Lou Berger: 1) do we want to use d-CW for L3VPNs? (i.e. do we support IP over MPLS only over VPNs, or both ways it is supported today). 2) do we want to use d-CW for non-L3VPNs Lou Berger: So how are we going to do IP over MPLS. Toerless Eckert: Would always like a CW even if don't need PREF, likes the OAM aspect of it, would help a lot. Lou Berger: There are other ways to do OAM on MPLS Toerless Eckert: sure, there are subnet specific ways, but they all have issues, and CW is strong operational feature, not just for PREF. Lou Berger: this is DetNet over MPLS, not DetNet over PW. So have a CW, that's part of it, but so is MPLS. OK to use MPLS mechanisms, and at T level will need to use them. Toerless Eckert: If underlying mechanism provides equal or better CW than our d-CW, happy to use it. Stewart Bryant: almost certainly VPN type approach needed. Probably have PR+EF in there as well. Want to define VPN with CW. If specific cases can take bytes out of the header. But if have CW in there will solve all of the cases. Andy Malis: Keep a single format if possible. Simplifies implementation. Jouni Korhonen: single format. Keep c-CW as it is easier for HW. Even if not using it. Stewart Bryant: Can tell HW to skip over it. Lou Berger: Sounds like a strong preference to always use same format, i.e. always keep d-CW present. Lou Berger: for IP over MPLS.. do we want to support an enhanced L3VPN approach first? Stewart Bryant: That's what I was trying to propose before. If need PREF with IP, will need SeqNum, so need CW, need context for it. So always assume VPN which provides context, CW which provides context, and then IP. Don't optimize out those 8 bits. Lou Berger: For IP over MPLS do we always use L3VPN technique? First? Only? Stewart Bryant: Start with general sol'n which will cover most of our solutions. Andy Malis: without VPN approach i.e., without S-Label, flow ID becomes more difficult. Lou Berger: any objections with having initial approach to IP over MPLS be within context of L3VPN? No objections. Anyone wanting to see our initial definition include IP over native MPLS (like core instance. If you running in a router, an LER, not running VPNs, just map IP to MPLS) as an initial detnet IP over MPLS definition? no hands. Not precluding it in the future. Janos Farkas: Conclusion: MPLS finished. with or without PR+EF there's going to be d-CW, even for IP over MPLS. There is a preference to use d-cw in all cases, even when PREF is not used -- both for L2VPN and IP over MPLS [need to raise CW+MPLS with MPLS WG] Andy Malis: speaks for his draft, which allows seqnum (RFC4023/7510 approach) allows PREF and conforms to simplified model. Lou Berger: Let's take perspective of host first, then other cases where we would use IP. You explicitly excluded host, right? Andy Malis: OK. Discussion about the red circles in the slide 9. Host and network level circles do not necessarily have the same encapsulation.. Discussion on DSCP.. Jouni Korhonen: If have diff't encap in each segment, you can do service protection until the end of the segment. Segments are independent, so there is a single point where you can go between segments. Don't share any state. More than just ACLs. Lou Berger: Endpoint is limited to subnet alone. E.g. TSN. Elsewhere have say IP over MPLS, or IP over MPLS over IP. Andy Malis: Should make circles diff't color since they are not the same encap, or same treatment. Lou Berger: At IP layer get same treatment, at subnet layer diff't treatment. Toerless Eckert: DSCP previously was used to indicate path through the network, which it wasn't designed for. How is this any different? Lou Berger: Could have a few DSCP values, not just one. Toerless Eckert: Is DSCP treated as it always was, or is it different for DetNet? Norm Finn: Don't know how to treat multiple priorities within a single DetNet flow. Don't know how to guarantee end to end latency. Could have diff't code points. In 802 have diff't priorities on diff't packets. Needs flow ID and right DSCP in order to get same DetNet service. If it has the wrong DSCP value it wouldn't get same treatment. So that shouldn't be a problem. Balázs Varga: Flow rank can be encoded in DSCP. Toerless Eckert: ID flow by 5-tuple. Lou Berger: For DetNet flow classification, i.e. map to diff't services. Stewart Bryant: ID the flow with 5-tuple, then map to service based on DSCP. Pascal Thubert: App flow is 5-tuple, but DetNet cares about DSCP also. Decides what goes through the DetNet pipe based on DSCP. DetNet doesn't care about other packets. Toerless Eckert: Concerned that packets with same 5-tuple will take different paths through the network. Lou Berger: Maybe not diff't path, but diff't reliability, e.g. PREF or not. Toerless Eckert: But in current model there are bad things that can happen with 6-tuple. Should have TSV review. Don't want different behavior for packets in same flow. Norm Finn: to decide how to treat in terms of class of service, can use DSCP to id flow, but don't have to. Lou Berger: different DSCP does not imply the packets take different route. Toerless Eckert: points out that TSVWG should review this approach (DSCP usage). Toerless Eckert: worried about what bad stuff can be done with 6-tuple Lou Berger: 5-tuple instead without DSCP? Toerless Eckert: no.. per hop behavioir. You express that with DSCP? Lou Berger: no that is done with DetNet flow id. Toerless Eckert: worried about redefining the use fo DSCP. Another good reason to have TSVWG to review this. Lou Berger: in 6-tuple any of those can be wildcarded. Routing based on DetNEt service to which the flow id maps to (= 6-tuple) Toerless Eckert: do we need to map multiple DSCP codes to the same DetNet flow? Lou Berger: from a toolbox point of view, yes. Example of concrete use case.. Stewart Bryant: If we have an application that has various requirements, we would differentiate them with DSCP, as opposed to setting up separate ports to distinguish. Stewart Bryant: Trying to build a mental model why you want to do this? What is here is not a standard method. Why not using different UDP ports, for example. Norm Finn: Example brought up from video domain: I frames always got through, P frames not so important.. Lou Berger: do 5-tuple or 6-tuple? Get the sense of the room. Fabian Schneider: there are implementations that just look at DSCP value at the beginning of the flow and then later use just 5-tuple without checking DSCP (caching, queuing, etc). Kiran: How do we identify a detnet flöow if we just look into 5-tuple. Norm Finn: Many ways to ID a flow. For some queueing technologies, don't have to identify each flow. At the edge you guarantee that the packets come from people who know what they are doing, then in the network just look at DSCP and assign to right class of service to get bounded latency and zero congestion. There are other cases where you do have to look at it e.g. to do per-flow shaping. So it depends which technology you are using to provide DetNet Service. Steward: think there is not enough knowledge at this point to make a decision. Document and then decide later. Norm Finn: Reason it is complicated is that the idea of a detnet flow is not the same at every hop in the network. Some devices look at flows that have different reservations but they count as one DetNet flow as far as they are concerned. So not every device has the same idea of whether two packets are in the same flow or a different flow. Lou Berger: Need to capture that idea in the aggregation section of IP. We do aggregation like that very well in MPLS, but not as well understood in IP space. Janos Farkas: how to do the I & P frame example just using 5-tuple (guestion directed to Toerles). Toerless Eckert: this is probably a good example (and justification) for the use of DSCP within a single flow. Concern about what tsv does and what can we do to prevent abuse, e.g. DSCP based routing. Ling Hao from Huawei: Should remove DSCP, to distinguish for diff't service could use 5-tuple or could use port number instead. DSCP is independent of flow ID. Some implementations check DSCP first. Norm Finn: could the chair clarify what we are arguing about? Lou Berger: we have 3 options on the table for ID'ing flows: 1) 5-tuple and value of DSCP - maps to a single detnet flow which maps to a detnet service. 2) ignore DSCP -> just use 5-tuple (all map to a single detnet service) 3) we specify a specific value for DSCP that must always be used with a detnet flow/service (else undefined or not part of DetNet service). Norm Finn: Of those 3, talking about a certain QOS. To require that we ignore DSCP is silly, reject out of hand. Don't think we should ignore current convention e.g. Iframe at one DSCP value, Pframe at another value. Apart from that don't care. Make DSCP one of the wild-cardable fields. Toerless Eckert: Have 5-tuple, then the perhop behavior is defined by DSCP. Not used for routing, only perhop behavior. As defined in RFCs today. Lou Berger: Do you mean "Option 1 with restriction that they must follow the same path, i.e. be routed co-routed"? Toerless Eckert: Yes but are there other implications that aren't appropriate for DSCP. For per-hop behavior but not sure if other things in flow definition. Janos Farkas: If use DSCP to determine PREF or not, that would be bad because use two paths or one path. Norm Finn: We are confusing QOS and routing. Some mask and match and fields mean nothing (the SDN model), won't argue with that. Diff't reason to look at flow is to route it, another is to assign to the right queue. Toerles concerned that we would route based on DSCP. Don't want to rule that out right now - to give you the service, I may need to pass you through a nailed down path, which would be difficult. Janos Farkas: Propose: For now keep 6-tuple as on slide. If we can scale down to lose DSCP then we can do that later. Lou Berger: Path selection vs per hop behavior which is local traffic treatment. Toerless Eckert: Good to see PREF as an example of why you would want to do this. Lou Berger: We have accomplished our goals for the meeting! Still 2 major open issues: 1) With MPLS - have a major change to use CW on L3VPNs - not done today - this is a potential issue. Always do that so we can support PREF and non-PREF seamlessly. Show of hands for got that wrong: none. Show for those who thought I got it right: some, not the whole room. Loa Anderson: Did not raise hand because waiting for MPLS discussion before committing. Item 2: Discussion is about host, not middle of network. Going down path of 6-tuple, but 5-tuple is for identification, DSCP is for differentiating service. No one disagrees. Pascal Thubert: Concerned about nailed down path if do that then rest of the flow would have to follow that path. It is a perhop behavior but may have to create circuit controlled by a controller, not by IP routing. Need to make this distinction otherwise seems like everything has to go over the same path through the tunnel. > 3 14:15 15 Title: Wrap up: DetNet Data Plane Encapsulation -- Resolving open issues > Presenter: Chairs > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-03 Pascal Thubert: concerned about nailed down paths. Detnet service may be a "circuit" such as an IP in IP tunnel programmed by a controller. The encapsulation for the IP in IP tunnel may include a sequence number in a destination option for PREF. The nailed-down path may be a strict source routed path. It is NOT necessarily congruent with the path that is obtained by a distributed IGP and that is followed by the best effort routed portion of the flow. We must allow two packets for the same flow take different paths. Note that the 6-tuple detnet flow that is "extracted" from the 5-tuple application flow based on DSCP will be encapsulated IP in IP, and that the actual routing will be done on the packet destination in th eouter header not on the DSCP. In that regard, we can claim that we are not "routing on DSCP" but we are making a per-hop decision of either to place the packet on the tunnel interface or pass it to the forading plane straught away. Lou Berger: So packets with different DSCP values within the same flow can take different paths? Pascal Thubert: yes. One of them gets encapsulated and the path is determined by that encapsulation, not by the encapsulated path. Lou Berger: So the doc will start out talking about that 5-tuple will select path. Then need to bring in this example if it is not properly covered. Toerless Eckert: asks for a concrete example where packets of the same flow take different path depending on DSCP. Can't guarantee that paths won't be different, but if out of our control we can't do anything about that. Pascal Thubert: doesn't like terminology: "routed over diff't path". Because once packet is past detnet ingress, it is not routed anymore, it is following its encapsulation, which may or may not have happened. If it happens, it won't be routed because of DSCP, it will be routed based on the encapsulation. Pascal Thubert: (offline) gave an example where the multilingual closed caption may be sent on best effort whereas the movie is sent over the tunnel. Inside the tunnel, I frames may get PREF whereas delta would be sent on either path in an ECM fashion. Mentioned SCTP as a potential transport for that. Padma Pillay-Esnault: how does this work with ECMP. Janos Farkas: bring into the mailing list and continue the discussion there.. Lou Berger: One more conclusion from the group: We can provide DetNet service over IP in the middle of the network as a case of MPLS over IP building on standard techniques for running MPLS over IP that exist as RFCs - covers what Andy was saying. Not native IP support, rather our MPLS mechanisms (including PREF) over an IP PSN. Any objections? No objections Anyone agrees? Small but reasonable number. Lou Berger: Stewart did a good job starting text on aggregates for MPLS. Also great that Norm described what happens in IP networks - want that text captured and sent to the list or authors. Lots of details to go though. Stewart Bryant: Need regular interims. Lou Berger: Need to capture result of these discussions - who will write them up? Lou Berger: It is time to split dp-sol into MPLS and IP into separate documents. Does anyone disagree. No one disagrees. Loa Anderson: Should split, but at least on MPLS side have good text from Stewart. Stewart Bryant: the authors of both documents should continue to work on it. Lou Berger: Authors of the old document and new authors should work together on the split. Stewart Bryant: Original team should contribute to new version. Next version should be split, and include conclusions from todays discussion. Stewart Bryant: need new draft name Lou Berger: Agree. File names will be dp-sol-mpls and dp-sol-ip - will start as WG docs, not individual drafts. Jouni from chat: I would love to have contributions from Andy & Stewart. their text is great to add. Lou Berger: need to accellerate our work. People involved should self-organize. Have IETF Webex available, though some limitations on number of dial-ins. Can be weekly, semi-weekly, whatever, as long as it is reported. Note that selecting a fixed time eliminates people from diff't time zones - consider shifting time zones. Discussed whether to have a f2f interim. No support for doing that (except Stewart) - budgets and travel are issues. Should continue virtual interims which have been productive. Also data plan authors can also have informal meetings to work on the draft content. They should be open, announced on the list. Reports should be generated for each meeting. Jouni Korhonen: Good if people prepare in advance for an interim, as they would for a F2F. Lou Berger: Schedule an interim as soon as we have one of the docs published. Say get MPLS first, then once we have an interim on that doc, then can go to working mode. Stewart Bryant: Who is writing the IP version? Lou Berger: The sum of everyone who has written so far, and some ones who haven't... Loa Anderson: Must have good reporting to the list on each and every meeting, each and every thing we want to change. Lou Berger: Agree. Part of openness and consensus building. Yuanlong Jiang: What about control plane? Lou Berger: Deliverables don't include control plane. And particularly not yet since we are behind everythign else. Once we get more progress on existing work we can take control plane in. Requirements for control plane are also welcome. Lou Berger: keep in mind that we are not the owners of the control plane protocols. We are OK to talk requirements. > Enhanced VPN (VPN+) > draft-bryant-rtgwg-enhanced-vpn > Stewart Bryant https://datatracker.ietf.org/doc/draft-bryant-rtgwg-enhanced-vpn/ Stewart Bryant: This is underpinning for Net Slicing. Noting that it is useful for other things as well, e.g. DetNet. DetNEt is useful to implement VPN+. > 4 14:30 10 Title: Time permitting: IGP-TE Extensions for DetNet Information Distribution > Presenter: Xuesong Geng, Mach Chen > Draft: https://tools.ietf.org/html/draft-geng-detnet-info-distribution-02 Greg Mirsky: Concern about accurate measurement of delay.. Prefers residence time Xuesong reply: This parameter is to allow for configuration Greg Mirsky: If it is used for configuration it should also be measurable for verification. Mentioned RFC7813 IS-IS extension. Norm Finn: should provide for both configuration and measurement. Norm Finn: What is min-max queueing delay Xuesong Geng: Based on queueing mechanism, per port. Time for packet to go through device, using detnet queueing algorithm. Norm Finn: Network calculus says that per port delay may not be useful because adding them along the path overestimates end-to-end latency. Worst case delay at each hop depends on what happened at previous hop. So will get worst case answer of say 18 usec but need 16usec but actual is 14usec. Should wait until more work on network calculus to decide the parameters to use. Toerless Eckert: Are you using this for path calculation? Xuesong Geng: Only using this for OAM collection, not path calculation. Toerless Eckert: Maybe too soon to define this level of detail? Need to settle framework before getting into parameters. Greg Mirsky: Don't need IGP extensions if it is centralized controller, i.e. not distributed? Janos Farkas: Never know, may have approach where centralized controller collects info from IGP. So hybrid approach is already in the field. Pascal Thubert: Disagrees with Greg. Arch supports both deterministic and non-deterministic. Some central controller will do the DetNet part, but there is also background traffic that may not be centrally routed. > Adjourn 14:30