IETF106 NVO3 WG agenda ---------------------- Network Virtualisation Overlays WG Monday - 18:10-19:10 - Hullet 1. WG Status update (WG Chairs, 10 min) Agenda bashing... Standard track document Geneve just completed IETF last call Encap draft, who read the latest version? 1 Hope more reviews during WG last call We had some informational encapsulation drafts, so these were kind of some of the other options that were considered by the working group. The only one that's actually alive at the moment is VXLAN-GPE and that's on the agenda today. What we originally said was once we had sent Geneve to the IESG, we would consider last calling as informational documents the other encapsulations. Distributed control plane so we have an eVPN draft that describes how to apply evpn with Geneve. That's now with Martin. VMM has some editorial nits, with shepherd. We have a lot of security drafts that Daniel was editing. They are currently individual drafts that some of them have been presented a few times, particularly the Geneve security requirements. We did try to adopt the Geneve security requirements a couple of times. Unfortunate it has been failed to reach consensus for working group adoption. I think we need to have a discussion about how to proceed with these drafts. I think one of the concerns for the cause the problem with adoption was to put unrealistic requirements onto an encapsulation, security requirements weren't necessarily implementable. That was at least one of the issues. Does anyone have any comments on how to proceed with these drafts? I think we know that security is a major requirement and it's something that we do need to address properly and one of the reasons we went into this exercise of developing a set of security requirements and architecture and some proposals for protocol enhancements for that was because the security ADs do require us to come up with some security solutions to address all the potential security issues with our encapsulations. Any comments? Should we just let this die then? Are there any operators in the room who are concerned about security in their data centers that would like to see this addressed? Martin: No, I would prefer not to let this die. I know that's easy to say, not necessarily easy to make it to reality, pity that Daniel is not here, but I think we need to find a solution. That's part of our the commitments, and Geneve is about to IESG and if I promise the IESG that there will be some security aspects of Geneve coming after and nothing comes, I'm kind of lying. So maybe ways to loosen some of those requirements, maybe to having them not optional but optional to implement, I don't know¡­ I don't know the solution right now, but I would really like the working group to give it another try. Sam: Martin, this is a second time we went through the requirements security requirements effort. I think one of the major concerns is that if the folks who wanted the security requirements or security aspects are very few in this working group. I believe it's across the routing area itself, it's not just this working group. So it's getting hard to really get the expertise of the security experts to really come up with good requirements and solutions related to that. It's a problem at large, but I don't know how to really tackle this. Martin: so maybe I can reach out to my fellow security ADs to bring more broad¡­ I don't know if Daniel has expressed any opinion on whether he would continue or leave that as is. Matthew: No, he tried twice to get this through¡­ Martin: Let's take it offline. I have an action point if we can find some someone on top of Daniel to push that. Matthew: OK. NVO3 OAM drafts. None of them is adopted so far. We have one draft on the agenda today, the BFD Geneve, and I think the authors are probably going to ask for adoption for that draft. I think a number of encapsulations proposed in it. Greg: No. I just want to point is that there is a draft that were presented in Montreal. It's a result of collaboration of four authors and it lists even four possible encapsulations. And the purpose of this draft is not to be published, but to initiate the discussion and the working group can select preferably single encapsulation. Because multiplicity of encapsulations in OAM is a headache and we should avoid it. We should agree on one encapsulation for active oam in Geneve. Matthew: okay and we have a common YANG model on the agenda for this meeting. 2. GPE update (Fabio Maino, 15 min) draft-ietf-nvo3-vxlan-gpe-08 Fabio presenting¡­ Greg: Two questions: first is that how active oam to be encapsulated in VXLAN-GPE because I think that it's a similar situation that there might be multiple ways to do it. You can use IP and then use destination port number two the multiplex known active protocols it could be MPLS and use Gal label and then have ACH channel type to the multiplex. So you want to be a multiplicity or you will basically instruct implementers that you should use particular encapsulation method? Fabio: What we define here is just the capability to identify your next protocol and then we define these she matters with a fixed structure so when you pass the packet you will find the type, the length and basically the next protocol. So if you know about a particular protocol, you can go and pass further deeper; if not, you just skip to the next header and do it. For encapsulation of ioam in particular, from the VXLAN-GPE point of view, it's just one header. As long as the shim headers respect this format of having a type, a length and the next protocol field in the right position, then it's up to the document¡­ Greg: My question was different. My question was not about ioam but active oam. So it's protocols like BFD, ping, traceroute, Fabio: those are oam protocols Greg: Yes Fabio: I think that there is a bit used to identify that the payload of the actual VXLAN packet is an oam packet Greg: What's the value of this bit? What it helps with? If for example the multiplexing is done by the UDP port or channel type in ACH, I really don't understand the value of this o bit for active oam use case. Fabio: I think it was supposed to leave it to this layer the capability to use this for OAM capabilities. I don't think that has been particularly developed. Greg: I think that the document will benefit if it will be more deterministic and clear about what's the use case for o bit, because otherwise it's kind of confusing. Another thing is that you have a protocol type vBNG, I can assume that it's for control and user plane separation use case. So I would probably name it for CUPS or something like that, because control user plane separation not necessarily specific to vBNG and vBNG is more accurately used for disaggregated BNG. CUPS is more generic. Fabio: In general what VXLAN-GPE does defines the wrap-up. Then what goes inside, it depends on the particular extension that one wants to define. So as I say probably all those reference will disappear, they should be defined in the actual draft that is specifying what vBNG is or what GBP is or whatever particular extension one wants to define. Sam: How do you secure the payload, like how exactly carry the keys/ Fabio: Traditionally what is done in practice is using some form of IPSec whenever VXLAN-GPE is carried typically outside of the data center. If you want to secure the VXLAN-GPE layer what you could do is define one extension. It is probably similar to the work you guys were doing for Geneve that could provide authentication or could provide encryption of the rest of the payload. Sam: Some text will actually help to understand Fabio: yes, thanks, good idea. Frank: I just want to go and echo, it would be a big help for implementers to go and see us allocating code points in some shape and form so that we can go and have at least harmonization across the current implementations, whether we have this as an IETF work or not, I think if we have harmonization in some means and the way to go and progress this either as a kind of working group blessed informational or an AD blessed informational forward, maybe Ignas can go on and give us some guidance from of his IESG perspective, I think it would be of benefit to us so that we at least get the implementations that are out there to use the same code points and help harmonization and interoperability, so I think it's a good approach. It helps my document for sure. Remy: you mean that the tunnel endpoints which is declared to support VXLAN-GPE at least it should know there's a type and length of the shim header so that it can skip it. Fabio: yes. Remy: so if a tunnel endpoint which does not recognize the type or length, it should not be declared to support a VXLAN-GPE at all. Fabio: yes. Format is fixed. So basically even if you don't know what the actual shim header contains, the first byte is the type, the second is the length and the fourth byte is the next protocol, so you can skip. Remy: The second question is do you think the order of the shim headers matters. There is one draft I think Sami wrote it. I don't know its current status. It is basically the extension of evpn which is used to negotiate the order of the different TLVs in Geneve. Do think it is a firm requirement to VXLAN-GPE or not? Fabio: I think there is more flexibility today in the way hardware is working in terms of having some more flexibility in terms of parsing a header like this one, the more structure you can give to the header the better it is for sure. It's the same conversation we're having before. When you go and design some ASIC, you allocate some cost to support some of this feature, which one do you support. That is really the aspect that is relevant here. That's why having the shim matters in front is one of the motivations, it helps designing and locating buffer to handle those use cases. Ordering might be helpful. The problem is which is the order and what do you do when you don't support in a certain order. If you have suggestion, that can help. It's hard to say that A B C and D should happen. Frank: It would be beneficial if we would be able to agree on an order. I don't think that we are ready to agree on an order. If you look at the lengthy discussions we¡¯re having for roughly 20 years on extension header ordering in v6, and where it's still like, could I go priority to your header please? Not sure that's a useful discussion to go and do it again here. Fabio: agree. Greg: My personal comment would be that it used to be very nice light-weight fixed header protocol and now you are like Geneve and GUE, that hurts. Fabio: I know.. thanks.. 3. NVO3 Yang Model (Remy Liu, 5 min) draft-ietf-nvo3-yang-cfg-01 Remy presenting¡­ Remy: reference to bgp-l3vpn, not sure if the authors will keep working on the draft. Our draft is not easy to couple from it. If we go faster than this ietf-bgp-l3vpn, will this cause a misref situation or something like? Matthew: It's normative reference, by definition it's going to cause a misref. We can hold the draft and wait for the l3VPN one before sending it to IESG. It is usually the best way to go. Matthew: raise your hand if read this latest version of the draft. [no] I think it would be really useful to have some more review of this YANG model. Please if you could review the draft and send comments to list it'll be great. 4. LOOPS (Local Optimizations on Path Segments) and its Geneve binding (Yizhou Li, 10 min) draft-bormann-loops-geneve-binding-00 draft-li-tsvwg-loops-problem-opportunities-03 draft-welzl-loops-gen-info-02 Yizhou presenting¡­ Informational briefing about LOOPS and relationship to Geneve. 5. BFD for Geneve (Min Xiao,10 min) draft-xiao-nvo3-bfd-geneve-01 Min presenting¡­ Sam: Why do you need those three types? This is BFD for Geneve, right? The second option is good enough. Why do you have to come up with three? Either you're not clear what is the right one or is it something¡­ Min: Geneve can carry multi-protocol payload. Sam: Sure, but you're talking about BFD for Geneve. It's not about payload. It is about for Geneve. Min: BFD sessions are originated and terminated via VAP. VAP is the NVE side of the interface between NVE and the TS. So for different payload, the VAP we use different encapsulation for the BFD. There's a different kinds of VAP¡­ Greg: I'm co-author of this draft. I think it might be that some implementations of Geneve will do balancing based on the payload though I think it's not likely. When we started this work, there was no draft on possible encapsulations of active oam. I think that probably the better outcome would be is that if working group decides that active oam should use one particular encapsulation across all active protocols. As I mentioned earlier at the mic, it is the purpose of draft that we have now listing all possible oam encapsulations. Min: But my personal opinion is that for different kinds of VAP, the different payload will be carried in Geneve tunnel. So for BFD we also need to test different kinds of payload of Geneve tunnel. Greg: BFD specifies RFC5880 that it monitors the path between two systems and to certain extend the forwarding engine on the systems that host BFD application. So yes if we use different encapsulation we will have some benefit of verifying that this particular encapsulation is working somewhat, but at the same time the problem would be is that how two endpoints agree on which encapsulation use if multiple encapsulations can be supported. So there are something to discuss Matthew: I'm not aware of any dependency between the encapsulation for BFD. In other areas where we use BFD over some kind of tunnel, like MPLS or pseudowire, are not aware of a dependency between how you encapsulate a BFD and how you encapsulate the payload. Sam: In the interest of time, what we need to do, for example take the solution number two, if you are saying that that is not going to solve the problem irrespective of the payload type, I think that is fundamentally something wrong in how you are actually designing. I think that the solution, so we need to really think through¡­ it's not going to be payload type. There's a protocol for the data plane header. Min: I want to suggest one point. BFD over Geneve is different from the underlay BFD. So if we run BFD between the NVE just for underlay test, we can use just a one kind of encapsulation. But if we want to use BFD for Geneve, actually we want to test the overlay. It's a tunnel so the different tunnel payloads should use different BFD encapsulation, so that's the point. Sam: I think you should really come up clearly like why do you need different types of BFD in different use cases. I think we're talking purely about the overlay and we're not talking about underlay. Why do we need that? I think if you can make really good call on that, then we can discuss further on. Matthew: I think another case where we have another VPN technology is pseudowires. If you have an Ethernet pseudowire or if you have an ATM pseudowire you don't have a different BFD encapsulation for VCCV BFD for those different types. But they're completely different technologies for the virtual connectivity. Greg: Right, basically because we are monitoring the pseudowire itself not the service. Matthew: correct Greg: Right. I agree. If we are monitoring Geneve tunnel then we need to be clear that we are monitoring Geneve tunnel and not any extension to their tenant. Matthew: We are we monitoring the Geneve tunnel or we are monitoring the VNI, the connectivity for the VNI. I think one of the issues here is maybe that the layering here is not very clear, the OAM architecture is not really that clear and maybe that needs to be clarified somewhere either this draft or and in the OAM requirements. We've got some sort of oam architecture draft. So when we say we're monitoring Geneve, what we mean and which layers do we inject OAM? Greg: We had very good discussion about what we're monitoring in course of working through BFD over VXLAN, and actually what we agreed is that in most cases monitoring VXLAN on one of VNIs may be sufficient, but at the same time there is an clear interest of being able to monitor separate VNIs. But again at the same time it's orthogonal to the question of how we encapsulate BFD packet into their overlay tunnel. So it¡¯s only the discussion of whether we allow multiple BFD sessions between two endpoints. Min: but the difference is that for VXLAN we have just Ethernet frames, but for Geneve we have different kinds of payload. Greg: yes we may have different kinds of payload, but again unless we know of the cases where some implementations will look at their Geneve payload to do ECMP, I think that there might be little value of documenting and standardizing multiple encapsulations for active OAM. Because like in IP, there will be done on outer encapsulation, then oam will be sharing fate with their payload with the data traffic, regardless of how active oam is encapsulated. Again validation of their decapsulating side of their tunnel is really limited if we use specific transport encapsulation for BFD packet. Matthew: Out of time now, move on to the next, take this one to the list. 6. Performance Measurement for Geneve (Min Xiao,5 min) draft-xiao-nvo3-pm-geneve-00 Min presenting¡­ Matthew: Question for Martin on charter and scope of the working group. It seems that in the context of these oam drafts, you're trying to monitor the service, almost tenant system to tenant system connectivity. No or yes? Otherwise why do you care about how you're enca.. Min: test the tunnel.. Matthew: No, you said you're monitoring a service. Greg: No, in this case we're not monitoring the service in this case. We're just monitoring the tunnel, and it helps to basically understand how ¡­ For operators it's useful because then they can understand which part of overall path to the tenant system contributes most if they experience service degradation or something that they can separate what's contributing by the performance in a tenant system versus their tunnel. Performance is clearly for the tunnel. Matthew: ok so the previous draft was service monitoring. Greg: No it's not service per se. Because it's not BFD session between tenant systems. BFD session between tenant systems is undistinguishable for their Geneve endpoints. For geneve it doesn't differentiate between tenant data and BFD session between tenant systems. The previous draft, it monitors their tunnel, and if there is a multi-path in the underlay, then they can run, and if they know how the VNI is being placed on different underlay paths, they can use some intelligence to monitor particular paths through VNIs. But idea is that BFD monitors continuity of Geneve tunnel between endpoints of the tunnel. Sam: Maybe I'm not clear. If you're monitoring the NVEs in this case, the edge where the inner encap is going to happen and decap is going to happen, so why do you care about what kind of payload it is carrying after Geneve header? If you can pop the BFD packet, why do you really care about what the payload is gonna be? Greg: I think it's a legitimate question. As I mentioned and probably that's something that we need to talk with ???. If we know that underlay network doesn't use the payload to do hashing and load balancing, then it's absolutely irrelevant; if it does use payload for the load balancing, then there is some value in using the same encapsulation because then you can use certain algorithms and certain methods to try to monitor a particular path. It's similar as how you do BFD over IP MPLS Network. If you use IP information from the LSP payload to do load balancing, then you can use the combination of methods to run your BFD session and monitor the particular path in your multipath environment. Sam: At least in the virtual world, none of the transit devices know anything about the inner payload, so the load balancing issue doesn't arise based on the payload. I think that is given at least NVO3 architecture document if you can look at it. Greg: If we use definite normative language saying that you must use this information for the load balancing in underlay and it will be outer encapsulation I think that will be very beneficial because it will make certain that inner encapsulation has no any impact on choosing their path in the multipath environment. Sam: At least the architecture document doesn't talk anything about the transit devices. I think at least from what we have is that the transit devices do not play a role in terms of what the inner payload is gonna be. As simple as this, if I secure my inner payload, there is no way they can actually look it up. It doesn't matter what kind of payload it is. And by the way like by default everyone actually secures the packet in the back. Min: I think the benefit is that we can use this to test the encapsulation and decapsulation of different kinds of payload of Geneve tunnel. Sam: We can do that without looking at the payload. Min: As I said the underlay BFD is different from the BFD for tunnel. Sam: All I'm saying is you're running BFD session from tunnel endpoints. Min: you mean you just need underlay BFD Sam: No, I am talking about the tunnel endpoints, that¡¯s overlay BFD Min: That¡¯s what the draft describes. Sam: Correct. What I'm saying is you don't need those three different types, rather you can achieve with just one type, which is IP, which is type two. Min: you know in previous meeting we also have suggestion to just use MPLS encap. Greg: In the meeting for Montreal we had a discussion and use of IP is probably sub-optimal for encapsulation, because it creates a significant overhead and moves the BFD packet out and possibly out of the fast memory, so that's why effectively probably MPLS encapsulation is most efficient for the Geneve. Matthew: a bit of a generic statement to say moves out of the fast path when saw somewhere the vast majority of LSP BFD implementations are just encapsulating it in a UDP header Greg: LSP doesn't have their overhead. Imagine that we have ipv6, then we have Geneve and then we use ipv6 in the encapsulation for BFD, LSP doesn't have any of it. Sam: Not all virtual stacks speak MPLS, but they all speak IP. Greg: that's a good point and that's why we have the draft which lists of options and we really appreciate their discussion about it. Matthew: Good to have some more discussion on the list, and get some feedback from the working group in general ¡¡¡¡ ¡¡¡¡ ¡¡¡¡ ¡¡¡¡