NOTE TAKERS ADD YOUR NAMES HERE (not required): Ethan Grossman Janos Farkas Andy Malis > DetNet Agenda IETF102 (Montreal) > Version: Jul 03, 2018 > Monday, July 16, 2018 (EDT) > 9:30-12:00 Monday Morning session I > Place du Canada > Slides: https://datatracker.ietf.org/meeting/102/session/detnet > Etherpad: http://etherpad.tools.ietf.org/p/notes-ietf-102-detnet?useMonospaceFont=true > Meetecho: http://www.meetecho.com/ietf102/detnet/ > Audio stream: http://ietf102streaming.dnsalias.net/ietf/ietf1025.m3u > Jabber: xmpp:detnet@jabber.ietf.org?join > Available post session: > Recording: https://www.ietf.org/audio/ietf102/ietf102-placeducanada-20180716-0930.mp3 > YouTube: https://www.youtube.com/watch?v=jPL0ifMMjiY > Presentation Start Time Duration Information > 0 9:30 10 Title: Intro, WG Status, Draft Status > Presenter: Chairs > Draft: N/A Jouni Korhonen has stepped down from being DetNet WG Secretary, and Ethan Grossman is taking on that role. Please use the list for DetNet discussion, and please respond promptly to queries from the list, e.g. IPR polls. Use cases draft is waiting for submission until finalization of arch doc (which is in extended last call, which ends Friday). Problem statement draft has only minor changes, to be fixed soon. Dataplane solutions are 00 because were split from original draft, so are more mature than 00, and we did not do a poll to adopt them. Security draft is the only WG draft not presented today; an update was posted on the list. Work on that draft is waiting on finalization of the solution documents; we don't want to publish it and then have to rev it as a result of design changes. We do expect the security draft to go forward and to influence the solution documents. Core deliverables: Good progress but no YANG model WG document yet. We have an individual contribution draft that we will discuss today, and which we could adopt as a starting place if and when we consider it ready. There are already regular coordination meetings between IEEE802.1TSN (Time Sensitive Networking)and DetNet. There will be a joint DetNet/IEEE802.1TSN meeting following IETF 103 in Bangkok, due to the coincidence that the two groups are meeting there on adjacent weeks. As a result of a poll regarding preferred dates for the meeting, it was decided to hold the meeting on the Sunday after IETF, i.e. just before the IEEE Plenary meeting. The IEEE are to arrange a meeting room, presumably without charge to us. So far about 40 people expressed interest. It is envisioned to be free, there will be a registration website. Plan is not confirmed yet; more details to follow. This is part of the effort to synchronize DetNet with other relevant standards organizations including IEEE and IEC. > 1 9:40 15 Title: Deterministic Networking Architecture > Presenter: Janos Farkas > Draft: https://tools.ietf.org/html/draft-ietf-detnet-architecture-06 Janos Farkas: Extened LC closes this Fri. Lou Berger: Acronym CPE used in the architecture draft for Control Plane Entity, but it is well known for Customer Premises Equipment. We could just change the acronym, e.g. to CPENT or CPENTITY, or we could change the name and acronym to something else. Andy Malis: It is used in only 8 places in doc. How about Control Plane Device (CPD)? Lou Berger: From the discussion on the list, that implies a physical box to many, but the architecture allows for fully centralized, fully distributed or any hybrid combination. Andy Malis: OK with Entity. Pascal Thubert: Concerned about "device" since it sounds like a single function, even "entity" sounds like a single thing. Lou: Maybe "function"? CPF? Norm Finn: Likes control plane function. Lou Berger: Does anyone want to object to "Control Plane Function (CPF)"? Pascal Thubert: Maybe "functions" plural? (No other objection from the room) Lou Berger: Great. We will confirm on the list. > 2 9:55 15 Title: Flow Information Model > Presenter: Janos Farkas > Draft: https://tools.ietf.org/html/draft-ietf-detnet-flow-information-model-01 Janos Farkas: Zero is a valid value for max out-of-order packets - means application cannot tolerate any mis-ordering. Hannu Flinck : What is relationship of this information model to ACTN? (Abstraction and Control of Traffic Engineered Networks) Janos Farkas: This is specific to DetNet, so we need to capture the DetNet-specific service attributes. Hannu Flinck: Is this not a traffic engineered network? Janos Farkas: Yes but even though the basis is a TE topology, on top of that we need the specific functions such as PREOF (packet replication, elimination and ordering functions), and we need to know which nodes are capable of which DetNet functionality. So we need to add some addition information to what currently exists for TE. Lou Berger: Also ACTN is about the control plane, whereas DetNet is currently focused on the data plane. Once we have sufficient progress on current core deliverables such as YANG models we can consider how control DetNet in relation to the existing toolkit, and ACTN certainly fits well. Greg Mirsky: (From Jabber) We may discuss the Out-of-Order Performance metric with IPPM WG. Janos Farkas: Yes. > 3/4 10:10 20 Title: DetNet IP Data Plane Encapsulation > Presenter: Bala'zs Varga (remote) > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-ip-00 (Immediately go to MPLS DP draft without break for discussion) > 3 10:30 20 Title: DetNet MPLS Data Plane Encapsulation > Draft: https://tools.ietf.org/html/draft-ietf-detnet-dp-sol-mpls-00 (10:24:17 AM) Yaakov Stein (from Jabber): So, the S-label is the PW label and the T-label(s) is the MPLS label stack. Why revert to old names? Lou Berger: Maybe there is some confusion between the DetNet Control Word (d-CW) and the S-Label. Janos Farkas: For a DetNet flow you always need two identifiers (or parameters); one is to identify the flow, which is the S-label. The second is the sequence number, which is in the d-CW. (10:27:13 AM) Yaakov Stein: I understand what a control word is. This is precisely the PW stack. Lou Berger: The normal PW label would be the d-CW. The S-label would be a context MPLS label, not a PW label. (10:27:17 AM) Gregory Mirsky: No, the S-Label, in PWE3 terminology, is PW-Label (10:27:57 AM) Yaakov Stein: No. the CW is not a LABEL (10:28:04 AM) Gregory Mirsky: PW CW is d-CW Stewart Bryant: This is just names, not functionality, we can sort out names later if we want. There are differences betweeen the way PWs work and the way DetNet works, in terms of the added functionality of DetNet, so I'm OK with changing the names or merging them, but I'm fine with this set of names. Lou Berger: The key is to define the way they work. Janos Farkas: I wanted to highlight the differences so this is why we have these names, but we can have some further discussion. Yaakov Stein (from Jabber): If we call it DN PW, we should use PW terminology. Lou Berger: We decided to avoid the word PW to avoid constraining how the d-CW and services are managed. For example it could be done using EVPN control techniques which use a control word but are not PW. that is some peoples' perspective, others see it as exactly a PW. The authors could comment, but that's the current answer to Yaakov's point. Stewart Bryant: Depends on whether you started at top with arch naming, or go bottom up, in this one particular case, we will use PW technology. We chose to start at the top in this particular document and propagate the terminology down. I don't have any issue which terminology we use, but we decided to use top down. Andy Malis: While this is not the preferred control word, it does conform to RFC 4385. Which defines the PWE3 control word. Yaakov Stein (from Jabber): Sent mail to list to continue discussion. Andy sent a reply to Yaakov on the list. (10:36:31 AM) Gregory Mirsky (Jabber): Use of d-CW has some consequences on OAM. Lou Berger: Greg has a preso on this today, let's defer this discussion until that slot. Lou Berger: But to Greg's point, where do we think the OAM info related to the MPLS data plane belongs? In a data plane solution doc or separate OAM doc? Per Greg's point, there may be implications on definition of encapsulation if you take into account OAM. Stewart Bryant: Don't understand Greg's point - the design is consistent with most basic version of the PW CW, and if the OAM is done using the ACh (MPLS Generic Associated Channel (G-ACh)) mechanismit is still consistent with that, where you set the four zero's to three zero's and a one. Lou Berger: Greg has a preso on that later in the session, let's leave this discussion until then. For now, do you have any comment on where the OAM info belongs? Stewart Bryant: OAM should be in companion doc with enough hooks in DP doc to specify the OAM indicator. That is we need to specify in the DP how it is known that a packet is OAM, and the characteristics of the OAM following the same path, etc, but the detailed OAM design will be a big piece of work so it should be a companion doc. Greg and Yaakov agree on companion doc. Lou: Comment from earlier via Jabber: Yaakov Stein: (on jabber) Andy, it does not conform since the Ethernet's CW avoids the zero value and the TDM control word has a shorter SN. Stewart Bryant: The particular CW design with the zero indicator only applied to a certain class of PW. You can have whatever design you want, the only thing that is important in the basic design is that you have the first nibble set. So it sounds like we need a longer discussion on the list. Lou Berger: That would be great. And I know you are the PALS chair but we should also include the other PALS WG people. Lou Berger: Both the IP and MPLS solution drafts have a lot of good narrative description of how the DP is supposed to operate. Now is a good time to read the solution documents if you haven't read them recently to make sure it is clear to you how the data plane is to operate, and to be sure you agree with it, and to provide comments, before we put in the conformance statements, since they will just refer back to the narrative text. > 5 10:50 15 Title: DetNet Bounded Latency > Presenter: Jean-Yves Le Boudec/Norm Finn > Draft: https://tools.ietf.org/html/draft-finn-detnet-bounded-latency-01 Norm Finn: Will cover new stuff not whole draft. Mathematically assured latency and loss values. IEEE work 802.1 Asynchronous Traffic Shaping, ATS. Talks about second layer of queues. Per flow queues can give good QOS. Can share any number of flows per queue if come in on same port and out on same port (same path through router) but you still need separate shaper per flow, but then can get by with fewer queues in the box, an optimization. Extra layer of per flow queues is called "interleaved flow regulators". If use "flow regulators" as proposed, then you can treat interference from other traffic as aggregate, simplifies math to make contribution of each hop linear with number of hops. Makes it much easier to make reservations. No penalty for this behavior if would be delayed at next hop anyway. Adding worst case delays gives pessimistic result. Worst case can only occur if drain from previous hops to fill queue at this hop. So you can't have worst case at every hop. Pascal Thubert: Question on regulators - works as long as all flows out the same Q are regulated? Norm: All flows sharing a low level Q must be regulated. So you reserve certain low level Qs for deterministic flows, others for best effort traffic. Toerless eckert: Cool. But there is more to what happens in large routers internally, in terms of assuming a non-blocking fabric? Norm: No assumption of non-blocking, but does assume have the ability to refuse a reservation. Toerless: Issue is what is the worst case of impact of other traffic going through the router, which is delaying your traffic. Norm: only worrying about DN traffic that has reserved BW. Toerless Eckert: But you have a fabric in which other traffic may impact your delay of traffic from input A to output B in your box. Norm Finn: Yes but it is based on the same analysis. Toerless Eckert: Larger routers have more complexity than TSN model. Norm Finn: the assumption is that your model includes the effect of other traffic. DetNet flows must get suitable priority or else you have no guarantees. Toerless Eckert: But this looks more like a solution not a req't? Would it be fair to put this into a separate document? Norm Finn: There are two drafts, one for requirements, this is not requirements. Lou Berger: Do you intend this to be normative or informational? I.e. a proposed standard, part of the implementation. Norm Finn: I intended it to be normative, but I am open to discussion about considering it informational. Lou Berger: The service a box provides will be part of the standard, but not how it is implemented. Lou Berger: How many people would be interested in prescribing a specific implementation defining Q'ing behavior? (Not much response.) Maybe nobody understood the question? Toerless Eckert: This proposed solution can be some internal behavior that doesn't introduce anything new on the wire? Lou Berger: I want to ask a couple of questions then we can continue the discussion. Lou Berger: Again, how many people in the room would be interested in prescribing a specific implementation defining Q'ing behavior inside a piece of equipment? (Very few). Lou Berger: How many think it would be useful to have an informational document describing one possible such implementation? (somewhat more, at least twice as many, but not half the room). Lou Berger: How many people are not interested in this topic at all? (nobody). Toerless Eckert: The problem is that if we all want the requirements document for bounded latency to be the core of the working group but we don't specify how to do it, then it is just an escape without answer. For customers that want to build a DetNet need guidance; we are putting lots of requirements and saying DetNet can be done, but then we are just hand waving about how to do it. We need to be a lot stronger if we agree that bounded latency is at the core of what we want to do here. Lou: So you would like a standard Q'ing implementation? Toerless: We have such a range of requirements from large scale to small, one solution may not fit all. But we need more than just requirements, informational is the very least we can do. Paul Congen: I would be in favor of standardizing in 802.1 and referencing as informational from ietf. Bing from Huawei: Key is interleaving between packets, so time controlled scheduling is key point. If it is accurate or not determines if we will make our deadline. Is there any analysis on the accuracy? routers and switches are not always accurate. Norm Finn: The techniques we presented treat inaccuracies like that as uncertainties in forwarding or queueing delays, which are factored into the worst case latency. So the number you get from the calculation is somewhat pessimistic assuming the implementer's worst case currently known innaccuracies. Bing: I didn't see that in the paper, but I will check it again. Bing: Second question is about multiplexing. You have to mix DN traffic with best effort traffic. If you want to reduce the resources of the interface of the interface, you have to control the interleaving between the DN packets. So you have to send a best effort packet between DN packets. If you don't want to miss the DN packet deadline, you can't use the resource for best effort traffic. Norm: At every hop, if no DN packets are ready to be transmitted at the instant, i.e. all are held up in lower queues, then you xmit a best effort packet. One possibility is that if very soon after start of transmit of a best effort packet you get a DN packet to xmit, but you have to wait until completion of sending the best effort packet, then that contributes to worst case latency. The other possibility is to preempt, which results in lower latency. If you just finished transmitting one DN packet with another DN packet ready, then you xmit the next DN packet - you are not required to interleave best effort with DN traffic. If you are using time scheduling, then you can enable both in the same window, but you give higher priority to DN traffic, so either way it is factored in. Bing: If time between two DN packets is too short to schedule a BE packet, time is wasted. Norm Finn: Yes, if it is smaller than the minimum size of a preemption packet fragment, then this is true. Biggest reason for preemption (a new 802.3 standard that allows you to interrupt transmission of a low priority packet to transmit one or more high priority packets then resume transmission of the low priority packet) is to be able to fit BE traffic in intervals between DN packets when the interval is smaller than a full packet so don't waste so much bandwidth. Janos Farkas: Let's take this offline. Lou Berger: It would be great to have the details discussed on the list. Lou Berger: two comments in from Jabber: Yaakov Stein: An informational implementation doc would be very useful. Greg: Agrees. If we do want to standardize a queueing implementation, there is another IETF WG which standardizes Q'ing techniques - should consider specifying it there. Lou Berger: That group is Active Queue Management WG (AQM). Focus is a little different but they do standardize queueuing techniques. Norm Finn: My leaning now is that we need a standards track RFC to specify characteristics for low level queueing to meet our the goals. And an informational RFC that says by the way here are some techniques that would meet the goals. Lou Berger: So that is informational. Norm Finn: Exactly. Xuesong Geng: Given the bounded latency requirement from transport layer, will different queueing mechanisms influence the encapsulation defined in the data plane? Norm Finn: To my knowledge the encapsulation doesn't affect which queueing mechanism you use. Xuesong Geng: For example in the slides you just introduced, a per-flow queue will require flow identification at transport layer. Norm: Yes, you can't do per-flow queues if you can't identify the flow. I would take that as a correction to what I said. You are right - it must be possible to identify the flow, and in that sense the assumption of the documents (of 802.1) so far is that you have a choice between deep packet inspection and ... Lou Berger: Need to take this offline. Please look at solution docs and see if there is sufficient description on how to do flow identification at least for unaggregated case. In next steps it says we need more work on aggregation case, but un-aggregated case should be covered. > 6 11:05 10 Title: DetNet Bounded Latency - Requirements > Presenter: Liang Geng (remote) > Draft: https://tools.ietf.org/html/draft-geng-detnet-requirements-bounded-latency-00 Liang Geng: this is about bounded latency requirements specifically for large scale DetNet. Janos: TSN is just one of the cases, just one subnet that can be used to meet requirements. Also there is also asynch solution for TSN, e.g. asynch shaper doesn't require time sync. The chairs decided to move this preso up in the schedule (was #10, is now after #6). > 10 11:50 10 Title: Large Scale DetNet > Presenter: Toerless Eckhert > Reference: https://tools.ietf.org/html/draft-qiang-detnet-large-scale-detnet-01 Norm Finn: There is some characterizing here of 802.1QCH cyclic Q'ing and forwarding that is in a very narrow sense. Toerless is right about 2 vs 3 buffers, will send pointer to list. Is supported in QCH. Trick is to identify which cycle an incoming packet belongs. Lou Berger: Defer detailed discusssion please. Toerless Eckert: Key part is if we do something different on the wire, how do we get that formalized? Lou Berger: What you're aiming for is to describe the Q'ing mechanism we'll need, once we have marking of flows and thus can aggregate, then we need Q'ing discussion on dealing with those aggregated flows. Isn't that where this work is headed? Toerless Eckert: Yes but you can come up with any kind of aggregation method, such as what Norm described, which has an in-node aggregation which isn't visible on the wire. Norm Finn: This is independent of aggregation. We have been trying not to have to include some bits to indicate which cycle a flow belongs to, that's tough. Lou Berger: Authors please work offline to come up with a consolidated document or other way to move it forward? More interesting in the informational approach. > 7 11:25 (11:33) 15 Title: DetNet Configuration Yang Model > Presenter: Xuesong Geng > Draft: https://tools.ietf.org/html/draft-geng-detnet-conf-yang-02 Lou Berger: How many people have read this draft? (A few). Lou Berger: This is currently the only candidate for a YANG draft, which is required by our charter. The document is maturing, and though I have some technical questions about this work, it isn't a question of whether to adopt it, but when to adopt it. So do we adopt now or wait til more mature? Toerless Eckhert: First question would be whether we have critical mass of authors, that is when we would adopt. Lou Berger: How many think we should wait before adopting? (Couple of weak hands) Lou Berger: How many think we should adopt now and mature the draft within the WG? (Not a huge number but more interest). We will do adoption poll on the list. Lou Berger: Regarding enough authors we can bring that to adoption poll. ToerlessEckhert: I would like to give potential authors confidence that they are working on something we really need, that is why we would adopt sooner. I also have technical comments... Lou Berger: For technical comments, please send them to the list. > 8 11:30 10 Title: OAM in DetNet > Presenter: Greg Mirsky > Draft: https://tools.ietf.org/html/draft-mirsky-detnet-oam-00 Janos Farkas: We want to have a separate OAM doc, this seems like a good start, but need to develop it further, would like more contribution, then we can consider adoption. Greg Mirsky: The main point is that the proposed MPLS encapsulation limits active OAM - so we need to make some adjustments to allow active OAM. Lou Berger: This fits well with our request to review the solution documents so please send these comments to the list. > 9 11:40 10 Title: Deterministic Networking Application in Ring Topologies > Presenter: Yuanlong Jiang > Draft: https://tools.ietf.org/html/draft-jiang-detnet-ring-01 Lou Berger: How many are interested in this work? (few people) Lou Berger: How many have read the doc? (notably more). OK. We will look forward to hearing how it develops relative to the actual data plane solutions. Lou Berger: David Black our technical advisor from the transport are is here, we appreciate his contribution. David Black: I am happy to be here and hope to do something useful. Lou Berger: Please read the IP solution document, and tell us what you think! Lou Berger: Joint meeting with IEEE802 at IETF 103 - must be Sunday (after IETF, before IEEE), will end by 5pm - please make travel arrangements accordingly. > Adjourn 12:00