< draft-ietf-sfc-nsh-ecn-support-05.txt		draft-ietf-sfc-nsh-ecn-support-06.txt >
	INTERNET-DRAFT D. Eastlake		INTERNET-DRAFT D. Eastlake
	Intended status: Proposed Standard Futurewei Technologies		Intended status: Proposed Standard Futurewei Technologies
	B. Briscoe		B. Briscoe
	Independent		Independent
	Y. Li		Y. Li
	Huawei Technologies		Huawei Technologies
	A Malis		A. Malis
	Malis Consulting		Malis Consulting
	X. Wei		X. Wei
	Huawei Technologies		Huawei Technologies
	Expires: October 1, 2021 April 2, 2021		Expires: March 12, 2022 September 13, 2021
	Explicit Congestion Notification (ECN) and Congestion Feedback		Explicit Congestion Notification (ECN) and Congestion Feedback
	Using the Network Service Header (NSH) and IPFIX		Using the Network Service Header (NSH) and IPFIX
	<draft-ietf-sfc-nsh-ecn-support-05.txt>		<draft-ietf-sfc-nsh-ecn-support-06.txt>
	Abstract		Abstract
	Explicit congestion notification (ECN) allows a forwarding element to		Explicit congestion notification (ECN) allows a forwarding element to
	notify downstream devices of the onset of congestion without having		notify downstream devices of the onset of congestion without having
	to drop packets. Coupled with a means to feed information about		to drop packets. Coupled with a means to feed information about
	congestion back to upstream nodes, this can improve network		congestion back to upstream nodes, this can improve network
	efficiency through better congestion control, frequently without		efficiency through better congestion control, frequently without
	packet drops. This document specifies ECN and congestion feedback		packet drops. This document specifies ECN and congestion feedback
	support within a Service Function Chaining (SFC) architecture domain		support within a Service Function Chaining (SFC) architecture domain
	skipping to change at page 2, line 4 ¶		skipping to change at page 2, line 4 ¶
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	Task Force (IETF), its areas, and its working groups. Note that		Task Force (IETF), its areas, and its working groups. Note that
	other groups may also distribute working documents as Internet-		other groups may also distribute working documents as Internet-
	Drafts.		Drafts.
	Internet-Drafts are draft documents valid for a maximum of six months		Internet-Drafts are draft documents valid for a maximum of six months
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	The list of current Internet-Drafts can be accessed at		The list of current Internet-Drafts can be accessed at
	http://www.ietf.org/1id-abstracts.html. The list of Internet-Draft		https://www.ietf.org/1id-abstracts.html. The list of Internet-Draft
	Shadow Directories can be accessed at		Shadow Directories can be accessed at
	http://www.ietf.org/shadow.html.		https://www.ietf.org/shadow.html.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Table of Contents		Table of Contents
	1. Introduction............................................4		1. Introduction............................................4
	1.1 NSH Background.........................................4		1.1 NSH Background.........................................4
	1.2 ECN Background.........................................6		1.2 ECN Background.........................................6
	1.3 Tunnel Congestion Feedback Background..................6		1.3 Tunnel Congestion Feedback Background..................6
	1.4 Conventions Used in This Document......................8		1.4 Conventions Used in This Document......................8
	2. The NSH ECN Field......................................10		2. The NSH ECN Field......................................10
	skipping to change at page 4, line 5 ¶		skipping to change at page 4, line 5 ¶
	6.2 IPFIX Information Element IDs.........................27		6.2 IPFIX Information Element IDs.........................27
	7. Security Considerations................................29		7. Security Considerations................................29
	8. Acknowledgements.......................................29		8. Acknowledgements.......................................29
	Normative References......................................30		Normative References......................................30
	Informative References....................................31		Informative References....................................31
	Authors' Addresses........................................32		Authors' Addresses........................................32
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	1. Introduction		1. Introduction
	Explicit Congestion Notification (ECN [RFC3168]) allows a forwarding		Explicit Congestion Notification (ECN [RFC3168]) allows a forwarding
	element to notify downstream devices of the onset of congestion		element to notify downstream devices of the onset of congestion
	without having to drop packets. Coupled with a means to feed		without having to drop packets. Coupled with a means to feed
	information about congestion back to upstream nodes, this can improve		information about congestion back to upstream nodes, this can improve
	network efficiency through better congestion control, frequently		network efficiency through better congestion control, frequently
	without packet drops. This document specifies ECN and congestion		without packet drops. This document specifies ECN and congestion
	feedback support within a Service Function Chaining (SFC [RFC7665])		feedback support within a Service Function Chaining (SFC [RFC7665])
	architecture domain through use of the Network Service Header (NSH		architecture domain through use of the Network Service Header (NSH
	skipping to change at page 5, line 5 ¶		skipping to change at page 5, line 5 ¶
	The Service Function Chaining (SFC [RFC7665]) architecture calls for		The Service Function Chaining (SFC [RFC7665]) architecture calls for
	the encapsulation of traffic within a service function chaining		the encapsulation of traffic within a service function chaining
	domain with a Network Service Header (NSH [RFC8300]) added by the		domain with a Network Service Header (NSH [RFC8300]) added by the
	"Classifier" (ingress node) on entry to the domain and the NSH being		"Classifier" (ingress node) on entry to the domain and the NSH being
	removed on exit from the domain at the egress node. The NSH is used		removed on exit from the domain at the egress node. The NSH is used
	to control the path of a packet in an SFC domain. The NSH is a		to control the path of a packet in an SFC domain. The NSH is a
	natural place, in a domain where traffic is NSH encapsulated, to note		natural place, in a domain where traffic is NSH encapsulated, to note
	congestion, avoiding possible confusion due, for example, to changes		congestion, avoiding possible confusion due, for example, to changes
	in the outer transport header in different parts of the domain.		in the outer transport header in different parts of the domain.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	|		|
	v		v
	+----------+		+----------+
	. .|Classifier|. . . . . . . . . . . . . .		. .|Classifier|. . . . . . . . . . . . . .
	. +----------+ .		. +----------+ .
	. | +----+ .		. | +----+ .
	. | --+ SF | Service .		. | --+ SF | Service .
	. | / +----+ Function .		. | / +----+ Function .
	. v --- Chaining .		. v --- Chaining .
	. +-----+/ +----+ domain .		. +-----+/ +----+ domain .
	skipping to change at page 6, line 4 ¶		skipping to change at page 6, line 4 ¶
	of the NSH. Traffic passes through a sequence of Service Function		of the NSH. Traffic passes through a sequence of Service Function
	Forwarders (SFFs) each of which sends the traffic to one or more		Forwarders (SFFs) each of which sends the traffic to one or more
	Service Functions (SFs). Each SF performs some operation on the		Service Functions (SFs). Each SF performs some operation on the
	traffic, for example firewall or Network Address Translation (NAT) or		traffic, for example firewall or Network Address Translation (NAT) or
	load balancer, and then returns it to the SFF from which it was		load balancer, and then returns it to the SFF from which it was
	received.		received.
	Logically, during the transit of each SFF, the outer transport header		Logically, during the transit of each SFF, the outer transport header
	that got the packet to the SFF is stripped (see Figure 3), the SFF		that got the packet to the SFF is stripped (see Figure 3), the SFF
	decides on the next forwarding step, either adding a new transport		decides on the next forwarding step, either adding a new transport
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	header or, if the SFF is the exit/egress, removing the NSH header.		header or, if the SFF is the exit/egress, removing the NSH header.
	The transport headers added may be different in different regions of		The transport headers added may be different in different regions of
	the SFC domain. For example, IP could be used for some SFF-to-SFF		the SFC domain. For example, IP could be used for some SFF-to-SFF
	communication and MPLS used for other such communication.		communication and MPLS used for other such communication.
	1.2 ECN Background		1.2 ECN Background
	Explicit congestion notification (ECN [RFC3168]) allows a forwarding		Explicit congestion notification (ECN [RFC3168]) allows a forwarding
	element (such as a router or a Service Function Forwarder (SFF) or		element (such as a router or a Service Function Forwarder (SFF) or
	Service Function (SF)) to notify downstream devices of the onset of		Service Function (SF)) to notify downstream devices of the onset of
	skipping to change at page 7, line 5 ¶		skipping to change at page 7, line 5 ¶
	first, otherwise the system could go unstable. For instance by the		first, otherwise the system could go unstable. For instance by the
	ingress taking a smoothed average of the level of congestion signaled		ingress taking a smoothed average of the level of congestion signaled
	by feedback from the tunnel egress or delaying any action for at		by feedback from the tunnel egress or delaying any action for at
	least the worst case global round trip time (for example 100		least the worst case global round trip time (for example 100
	milliseconds).		milliseconds).
	(1) Traffic throttling (policing), where the downstream traffic		(1) Traffic throttling (policing), where the downstream traffic
	flowing out of the ingress node is limited to reduce or eliminate		flowing out of the ingress node is limited to reduce or eliminate
	congestion.		congestion.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	(2) Upstream congestion feedback, where the ingress node sends		(2) Upstream congestion feedback, where the ingress node sends
	messages upstream to or towards the ultimate traffic source, a		messages upstream to or towards the ultimate traffic source, a
	function that can throttle traffic generation/transmission.		function that can throttle traffic generation/transmission.
	(3) Traffic re-direction, where the ingress node configures the NSH		(3) Traffic re-direction, where the ingress node configures the NSH
	of some future traffic so that it avoids congested paths. Great		of some future traffic so that it avoids congested paths. Great
	care must be taken with this option to avoid (a) significant re-		care must be taken with this option to avoid (a) significant re-
	ordering of traffic in flows that it is desirable to keep in		ordering of traffic in flows that it is desirable to keep in
	order and (b) oscillation/instability in traffic paths due to		order and (b) oscillation/instability in traffic paths due to
	alternate congestion of previously idle paths and the idling of		alternate congestion of previously idle paths and the idling of
	skipping to change at page 8, line 5 ¶		skipping to change at page 8, line 5 ¶
	(not shown) before entering the SFC domain and after leaving the SFC		(not shown) before entering the SFC domain and after leaving the SFC
	domain.		domain.
	The figure shows typical congestion feedback that would be expected		The figure shows typical congestion feedback that would be expected
	from the final receiver to the origin sender, which controls the load		from the final receiver to the origin sender, which controls the load
	the origin sender applies to all elements on the path. The figure		the origin sender applies to all elements on the path. The figure
	also shows the congestion feedback from the egress to the ingress of		also shows the congestion feedback from the egress to the ingress of
	the SFC domain that is described in this document, to control or		the SFC domain that is described in this document, to control or
	balance load within the SFC domain.		balance load within the SFC domain.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	.:= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = :.		.:= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = :.
	_||_ End-to-End Congestion Feedback ||		_||_ End-to-End Congestion Feedback ||
	\ / ||		\ / ||
	\/ ||		\/ ||
	__ Inner Transport Header and Payload __		__ Inner Transport Header and Payload __
	| | ->- - - - - - - - - - - - - - ->- - - - - -- - - - - - ->- | |		| | ->- - - - - - - - - - - - - - ->- - - - - -- - - - - - ->- | |
	| | | |		| | | |
	| | .:= = = = = = = = = = = = = = = = = = = = = =:. | |		| | .:= = = = = = = = = = = = = = = = = = = = = =:. | |
	| | _||_ Tunnel Congestion Feedback || | |		| | _||_ Tunnel Congestion Feedback || | |
	| | \ / || | |		| | \ / || | |
	skipping to change at page 9, line 4 ¶		skipping to change at page 9, line 4 ¶
	14 [RFC2119] [RFC8174] when, and only when, they appear in all		14 [RFC2119] [RFC8174] when, and only when, they appear in all
	capitals, as shown here.		capitals, as shown here.
	Acronyms:		Acronyms:
	AQM - Active Queue Management [RFC7567]		AQM - Active Queue Management [RFC7567]
	CE - Congestion Experienced [RFC3168]		CE - Congestion Experienced [RFC3168]
	downstream - The direction from ingress to egress		downstream - The direction from ingress to egress
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	ECN - Explicit Congestion Notification [RFC3168]		ECN - Explicit Congestion Notification [RFC3168]
	ECT - ECN Capable Transport [RFC3168]		ECT - ECN Capable Transport [RFC3168]
	IPFIX - IP Flow Information Export [RFC7011]		IPFIX - IP Flow Information Export [RFC7011]
	Not-ECT - Not ECN-Capable Transport [RFC3168]		Not-ECT - Not ECN-Capable Transport [RFC3168]
	NSH - Network Service Header [RFC8300]		NSH - Network Service Header [RFC8300]
	skipping to change at page 10, line 5 ¶		skipping to change at page 10, line 5 ¶
	SFC - Service Function Chaining [RFC7665]		SFC - Service Function Chaining [RFC7665]
	SFF - Service Function Forwarder [RFC7665] - A type of node that		SFF - Service Function Forwarder [RFC7665] - A type of node that
	forwards based on the NSH.		forwards based on the NSH.
	TLV - Type Length Value		TLV - Type Length Value
	upstream - The direction from egress to ingress		upstream - The direction from egress to ingress
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	2. The NSH ECN Field		2. The NSH ECN Field
	The NSH header is used to encapsulate and control the subsequent path		The NSH header is used to encapsulate traffic and control its
	of traffic (see Section 2 of [RFC8300]). The NSH also provides for		subsequent path (see Section 2 of [RFC8300]). The NSH also provides
	optional metadata inclusion, as shown in Figure 3.		for optional metadata inclusion, as shown in Figure 3.
	+-----------------------------------+		+-----------------------------------+
	| Outer Transport Header |		| Outer Transport Header |
	+-----------------------------------+		+-----------------------------------+
	| Network Service Header (NSH) |		| Network Service Header (NSH) |
	| +------------------------------+ |		| +------------------------------+ |
	| | Base Header | |		| | Base Header | |
	| +------------------------------+ |		| +------------------------------+ |
	| | Service Path Header | |		| | Service Path Header | |
	| +------------------------------+ |		| +------------------------------+ |
	skipping to change at page 11, line 5 ¶		skipping to change at page 11, line 5 ¶
	Figure 4. NSH Base Header		Figure 4. NSH Base Header
	Note to RFC Editor: The above figure should be adjusted based on the		Note to RFC Editor: The above figure should be adjusted based on the
	bits assigned by IANA (see Section 5) and this note deleted.		bits assigned by IANA (see Section 5) and this note deleted.
	Table 1 shows the meaning of the code points in the NSH ECN field.		Table 1 shows the meaning of the code points in the NSH ECN field.
	These have the same meaning as the ECN field code points in the IPv4		These have the same meaning as the ECN field code points in the IPv4
	or IPv6 header as defined in [RFC3168].		or IPv6 header as defined in [RFC3168].
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Binary Name Meaning		Binary Name Meaning
	------ ------- --------------------------------		------ ------- --------------------------------
	00 Not-ECT Not ECN-Capable Transport		00 Not-ECT Not ECN-Capable Transport
	01 ECT(1) ECN-Capable Transport		01 ECT(1) ECN-Capable Transport
	10 ECT(0) ECN-Capable Transport		10 ECT(0) ECN-Capable Transport
	11 CE Congestion Experienced		11 CE Congestion Experienced
	Table 1. ECN Field Code Points		Table 1. ECN Field Code Points
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	3. ECN Support in the NSH		3. ECN Support in the NSH
	This section describes the required behavior to support ECN using the		This section describes the required behavior to support ECN using the
	NSH. There are two aspects to ECN support:		NSH. There are two aspects to ECN support:
	1. ECN propagation during encapsulation or decapsulation		1. ECN propagation during encapsulation or decapsulation
	2. ECN marking during congestion at bottlenecks.		2. ECN marking during congestion at bottlenecks.
	While this section covers all combinations of ECN-aware and ECN-		While this section covers all combinations of ECN-aware and ECN-
	unaware, it is expected that in most cases the NSH domain will be		unaware, it is expected that in most cases the NSH domain will be
	uniform so that, if this document is applicable, all SFFs will		uniform so that, if this document is applicable, all SFFs will
	skipping to change at page 13, line 5 ¶		skipping to change at page 13, line 5 ¶
	Detection of congestion will be most effective if ECN marking is		Detection of congestion will be most effective if ECN marking is
	supported by all potential bottlenecks inside the domain in which		supported by all potential bottlenecks inside the domain in which
	NSH is being used to route traffic as well as at the ingress and		NSH is being used to route traffic as well as at the ingress and
	egress. Nodes that do not support ECN marking, or that support		egress. Nodes that do not support ECN marking, or that support
	AQM but not ECN, will naturally use drop to relieve congestion.		AQM but not ECN, will naturally use drop to relieve congestion.
	The gap in the end-to-end packet sequence will be detected as		The gap in the end-to-end packet sequence will be detected as
	congestion by the final receiving endpoint, but not by the NSH		congestion by the final receiving endpoint, but not by the NSH
	egress (see Figure 2).		egress (see Figure 2).
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	3.1 At The Ingress		3.1 At The Ingress
	When the ingress/Classifier encapsulates an incoming IP packet with		When the ingress/Classifier encapsulates an incoming IP packet with
	an NSH, it MUST set the NSH ECN field using the "Normal mode"		an NSH, it MUST set the NSH ECN field using the "Normal mode"
	specified in [RFC6040] (i.e., copied from the incoming IP header).		specified in [RFC6040] (i.e., copied from the incoming IP header).
	Then, if the resulting NSH ECN field is Not-ECT, the ingress SHOULD		Then, if the resulting NSH ECN field is Not-ECT, the ingress SHOULD
	set it to ECT(0). This indicates that, even though the end-to-end		set it to ECT(0). This indicates that, even though the end-to-end
	transport is not ECN-capable, the egress and ingress of the SFC		transport is not ECN-capable, the egress and ingress of the SFC
	domain are acting as an ECN-capable transport. This approach will		domain are acting as an ECN-capable transport. This approach will
	skipping to change at page 14, line 5 ¶		skipping to change at page 14, line 5 ¶
	| ECT(0) | ECT(0) |		| ECT(0) | ECT(0) |
	| ECT(1) | ECT(1) |		| ECT(1) | ECT(1) |
	| CE | CE |		| CE | CE |
	+-----------------+---------------+		+-----------------+---------------+
	Table 2. Setting of ECN fields by an ingress/Classifier		Table 2. Setting of ECN fields by an ingress/Classifier
	The requirements in this section apply to all ingress nodes for the		The requirements in this section apply to all ingress nodes for the
	domain in which NSH is being used to route traffic.		domain in which NSH is being used to route traffic.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	3.2 At Transit Nodes		3.2 At Transit Nodes
	This section described behavior at nodes that forward based on the		This section described behavior at nodes that forward based on the
	NSH such as SFF and other forwarding nodes such as IP routers. Figure		NSH such as SFF and other forwarding nodes such as IP routers. Figure
	5 shows a packet on the wire between forwarding nodes.		5 shows a packet on the wire between forwarding nodes.
	+-----------------+		+-----------------+
	| Outer Header |		| Outer Header |
	+-----------------+		+-----------------+
	| NSH |		| NSH |
	skipping to change at page 15, line 5 ¶		skipping to change at page 15, line 5 ¶
	When the NSH encapsulated packet is further encapsulated for		When the NSH encapsulated packet is further encapsulated for
	transmission to the next SFF or SF, ECN marking behavior depends on		transmission to the next SFF or SF, ECN marking behavior depends on
	whether or not the node that will decapsulate the outer header		whether or not the node that will decapsulate the outer header
	supports Compliant ECN Decapsulation (see Section 3). If it does,		supports Compliant ECN Decapsulation (see Section 3). If it does,
	then the encapsulating node propagates the NSH ECN field to this		then the encapsulating node propagates the NSH ECN field to this
	outer encapsulation using the "Normal Mode" of ECN encapsulation		outer encapsulation using the "Normal Mode" of ECN encapsulation
	[RFC6040] (the ECN field is copied). If it does not, then the		[RFC6040] (the ECN field is copied). If it does not, then the
	encapsulating node MUST clear ECN in the outer encapsulation to non-		encapsulating node MUST clear ECN in the outer encapsulation to non-
	ECT (the "Compatibility Mode" of [RFC6040]).		ECT (the "Compatibility Mode" of [RFC6040]).
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	3.2.2 At an SF/Proxy		3.2.2 At an SF/Proxy
	If the SF is NSH and ECN-aware, the processing is essentially the		If the SF is NSH and ECN-aware, the processing is essentially the
	same at the SF as at an SFF as discussed in Section 3.2.1.		same at the SF as at an SFF as discussed in Section 3.2.1.
	If the SF is NSH-aware but ECN-unaware, then the SFF transmitting the		If the SF is NSH-aware but ECN-unaware, then the SFF transmitting the
	packet to the SF will use Compatibility Mode. Congestion encountered		packet to the SF will use Compatibility Mode. Congestion encountered
	in the SFF to SF and SF to SFF paths will be unmanaged.		in the SFF to SF and SF to SFF paths will be unmanaged.
	If the SF is not NSH-aware, then an NSH proxy will be between the SFF		If the SF is not NSH-aware, then an NSH proxy will be between the SFF
	skipping to change at page 15, line 44 ¶		skipping to change at page 15, line 42 ¶
	|Forwarder)| +-------+ |Function)|		|Forwarder)| +-------+ |Function)|
	+----------+ +---------+		+----------+ +---------+
	|		|
	v		v
	Figure 6. Proxy for NSH Un-aware SFF		Figure 6. Proxy for NSH Un-aware SFF
	If the proxy is ECN-aware, the proxy uses an AQM to indicate		If the proxy is ECN-aware, the proxy uses an AQM to indicate
	congestion within the proxy in the NSH that it returns to the SFF.		congestion within the proxy in the NSH that it returns to the SFF.
	The outer header used for the proxy-to-SF path uses Normal Mode.		The outer header used for the proxy-to-SF path uses Normal Mode.
	The outer head used for the proxy-to-SFF path uses Normal Mode		The outer header used for the proxy-to-SFF path uses Normal Mode
	based copying of the NSH ECN field to the outer header. Thus		based copying of the NSH ECN field to the outer header. Thus
	congestion in the proxy will be managed. Congestion in the SF will		congestion in the proxy will be managed. Congestion in the SF will
	be managed only if the SF is ECN-aware and implements an AQM.		be managed only if the SF is ECN-aware and implements an AQM.
	3.2.3 At Other Forwarding Nodes		3.2.3 At Other Forwarding Nodes
	Other forwarding nodes, that is non-NSH forwarding nodes between NSH		Other forwarding nodes, that is non-NSH forwarding nodes between NSH
	forwarding nodes, such as IP or label switched routers, might also		forwarding nodes, such as IP or label switched routers, might also
	contain potential bottlenecks. If so, they SHOULD implement an AQM		contain potential bottlenecks. If so, they SHOULD implement an AQM
	algorithm to update the ECN marking in the outer transport header as		algorithm to update the ECN marking in the outer transport header as
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	specified in [RFC3168].		specified in [RFC3168].
	3.3 At Exit/Egress		3.3 At Exit/Egress
	At the SFC domain egress node, first any actions are taken based on		At the SFC domain egress node, first any actions are taken based on
	Congestion Experienced or other values of ECN marking, such as		Congestion Experienced or other values of ECN marking, such as
	accumulating statistics to send back to the ingress (see Section 4)		accumulating statistics to send back to the ingress (see Section 4)
	or for other uses. If the packet being carried inside the NSH is IP,		or for other uses. If the packet being carried inside the NSH is IP,
	when the NSH is removed the NSH ECN field MUST be combined with the		when the NSH is removed the NSH ECN field MUST be combined with the
	IP ECN field as specified in Table 3 that was extracted from		IP ECN field as specified in Table 3 that was extracted from
	skipping to change at page 17, line 5 ¶		skipping to change at page 17, line 5 ¶
	it receives. Such actions might appear to be packet loss due to		it receives. Such actions might appear to be packet loss due to
	congestion or might mask the loss of packets by generating additional		congestion or might mask the loss of packets by generating additional
	packets.		packets.
	The tunnel congestion feedback approach (Section 4) detects loss by		The tunnel congestion feedback approach (Section 4) detects loss by
	counting payload bytes in at the ingress and counting them out at the		counting payload bytes in at the ingress and counting them out at the
	egress. This does not work unless nodes conserve the amount of		egress. This does not work unless nodes conserve the amount of
	payload bytes. Therefore, it will not be possible to detect loss		payload bytes. Therefore, it will not be possible to detect loss
	using this technique if they are not conserved.		using this technique if they are not conserved.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Nonetheless, if a bottleneck supports ECN marking, it will be		Nonetheless, if a bottleneck supports ECN marking, it will be
	possible to detect the very high level of CE markings that are		possible to detect the very high level of CE markings that are
	associated with congestion that is so excessive that it leads to		associated with congestion that is so excessive that it leads to
	loss. However, it will not be possible for the tunnel congestion		loss. However, it will not be possible for the tunnel congestion
	feedback approach to detect any congestion, whether slight or severe,		feedback approach to detect any congestion, whether slight or severe,
	if it occurs at a bottleneck that does not support ECN marking.		if it occurs at a bottleneck that does not support ECN marking.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	4. Tunnel Congestion Feedback Support		4. Tunnel Congestion Feedback Support
	The collection and storage of congestion information at the egress		The collection and storage of congestion information at the egress
	may be useful for later analysis but, unless it can be fed back to a		may be useful for later analysis but, unless it can be fed back to a
	point which can take action to reduce congestion, it will not be		point which can take action to reduce congestion, it will not be
	useful in real time. Such congestion feedback to the ingress enables		useful in real time. Such congestion feedback to the ingress enables
	it to take actions such as those listed in Section 1.3.		it to take actions such as those listed in Section 1.3.
	IP Flow Information Export (IPFIX [RFC7011]) provides a standard for		IP Flow Information Export (IPFIX [RFC7011]) provides a standard for
	communicating traffic flow statistics. As extended by this document,		communicating traffic flow statistics. As extended by this document,
	IPFIX messages from the egress to the ingress are used to communicate		IPFIX messages from the egress to the ingress are used to communicate
	the extent of congestion between an ingress and egress based on ECN		the extent of congestion between an ingress and egress based on ECN
	marking in the NSH.		marking in the NSH.
	4.1 Congestion Level Measurement		4.1 Congestion Level Measurement
	The congestion level measurement is based on ECN marking in the NSH		The congestion level measurement is based on ECN marking in the NSH
	and packet drop, particularly the fraction of packets that are CE-		and packet drop. In particular the congestion information includes
	marked packet and fraction that are dropped. If the congestion level		the ratio of CE-marked packets to all packets and the ratio of
	is not high enough, the packets are marked as CE instead of being		dropped packets to all packets.
	dropped, and then it is easy to calculate congestion level according
	to the ratio of CE-marked packets. If the congestion level is so high		If the congestion level is not high enough, the packets are marked as
	that ECT packets will be dropped, then the packet loss ratio could be		CE instead of being dropped, and then it is easy to calculate
	calculated by comparing total packets entering ingress and total		congestion level according to the ratio of CE-marked packets. If the
	packets arriving at egress over the same span of packets. If packet		congestion level is so high that ECT packets will be dropped, then
	loss is detected, it could be assumed that severe congestion has		the packet loss ratio could be calculated by comparing total packets
	occurred in the tunnel.		entering ingress and total packets arriving at egress over the same
			span of packets. If packet loss is detected, it can be assumed that
			severe congestion has occurred in the tunnel.
	The egress calculates the CE-marked packet ratio by counting packets		The egress calculates the CE-marked packet ratio by counting packets
	with different ECN markings. The CE-marked packet ratio will be used		with different ECN markings. The CE-marked packet ratio will be used
	as an indication of tunnel load level. It is assumed that nodes		as an indication of tunnel load level. It is assumed that nodes
	between the ingress and egress will not drop packets biased towards		between the ingress and egress will not drop packets biased towards
	certain ECN codepoints, so calculating of CE-marked packet ratio is		certain ECN codepoints, so calculating of CE-marked packet ratio is
	not affect by packet drop.		not affect by packet drop.
	The calculation of volumes of packet drop is by comparing the traffic		The calculation of volumes of packet drop is by comparing the traffic
	volumes between ingress and egress.		volumes between ingress and egress.
	Faked ECN-Capable Transport (ECT) is used at the ingress to defer		Faked ECN-Capable Transport (ECT) is used at the ingress to defer
	packet loss to the egress. The basic idea of faked ECT is that, when		packet loss to the egress. The basic idea of faked ECT is that, when
	encapsulating packets, the ingress first marks the tunnel outer		encapsulating packets, the ingress first marks the tunnel outer
	header (NSH for an SFC domain) according to [RFC6040], and then		header (NSH for an SFC domain) according to [RFC6040], and then
	remarks the outer header of Not-ECT packets as ECT. (ECT(0) and		remarks the outer header of Not-ECT packets as ECT. (ECT(0) and
	ECT(1) are treated as the same.) Thus, as transmitted by the ingress		ECT(1) are treated as the same.) Thus, as transmitted by the ingress
	node, there will be one of three combinations of outer header ECN		node, there will be one of three combinations of outer header ECN
	field and inner header ECN field: CE|CE, ECT|N-ECT, and ECT|ECT (in		field and inner header ECN field as follows: CE|CE, ECT|N-ECT, and
	the format of outer-ECN|inner-ECN); when decapsulating packets at the		ECT|ECT (in the format of outer-ECN|inner-ECN); when decapsulating
	egress, [RFC6040] defined decapsulation behavior is used, and		packets at the egress, [RFC6040] defined decapsulation behavior is
			used, and according to [RFC6040], the packets marked as CE|N-ECT will
	INTERNET-DRAFT NSH ECN & Congestion Feedback		be dropped. Faked-ECT is used to shift some drops to the egress in
			order to allow the egress to calculate the CE-marked packet ratio
	according to [RFC6040], the packets marked as CE|N-ECT will be		more precisely.
	dropped. Faked-ECT is used to shift some drops to the egress in order
	to allow the egress to calculate the CE-marked packet ratio more
	precisely.
	The ingress encapsulates packets and marks their outer header		The ingress encapsulates packets and marks their outer header
	according to faked ECT as described above. The ingress cumulatively		according to faked ECT as described above. The ingress cumulatively
	counts packet bytes for three types of ECN combination (CE|CE, ECT|N-		counts packet bytes for three types of ECN combination (CE|CE, ECT|N-
	ECT, and ECT|ECT) and then the ingress regularly sends cumulative		ECT, and ECT|ECT) and then the ingress regularly sends cumulative
	bytes counts message of each type of ECN combination to the egress.		bytes counts message of each type of ECN combination to the egress.
	When each message arrives at the egress, (1) egress calculates the		When each message arrives at the egress, (1) the egress calculates
	ratio of CE-marked packet; (2) the egress cumulatively counts packet		the ratio of CE-marked packets; (2) the egress cumulatively counts
	bytes coming from the ingress and adds its own bytes counts of each		packet bytes coming from the ingress and adds its own bytes counts of
	type of ECN combination (CE|CE, ECT|N-ECT, CE|N-ECT, CE|ECT, and		each type of ECN combination (CE|CE, ECT|N-ECT, CE|N-ECT, CE|ECT, and
	ECT|ECT) to the message for ingress to calculate packet loss. The		ECT|ECT) to the message for ingress to calculate packet loss. The
	egress feeds back CE-marked packet ratio and bytes counts information		egress feeds back the CE-marked packet ratio and bytes counts
	to the ingress for evaluating congestion level in the tunnel.		information to the ingress for evaluating congestion level in the
			tunnel.
	The counting of bytes can be at the granularity of all traffic from		The counting of bytes can be at the granularity of all traffic from
	the ingress to the egress to learn about the overall congestion		the ingress to the egress to learn about the overall congestion
	status of the path between the ingress and the egress. The counting		status of the path between the ingress and the egress. The counting
	can also be at the granularity of individual customer's traffic or a		can also be at the granularity of individual customer's traffic or a
	specific set of flows to learn about their congestion contribution.		specific set of flows to learn about their congestion contribution.
	For example, the tunnelEcnCEMarkedRatio field (specified below)		For example, the tunnelEcnCEMarkedRatio field (specified below)
	indicates the fraction of traffic that has been marked in the ECN		indicates the fraction of traffic that has been marked in the ECN
	field of the NSH as Congestion Experienced (CE).		field of the NSH as Congestion Experienced (CE).
	4.3 Congestion Information Delivery		4.3 Congestion Information Delivery
	As described above, the tunnel ingress needs to send a messages		As described above, the tunnel ingress needs to send a messages
	containing cumulative bytes counts of packets of each type of ECN		containing cumulative bytes counts of packets of each type of ECN
	combination to the tunnel egress, and the tunnel egress also needs to		combination to the tunnel egress, and the tunnel egress also needs to
	feed back messages with cumulative bytes counts of packets of each		feed back messages with cumulative bytes counts of packets of each
	type of ECN combination and CE-marked packet ratio to the ingress.		type of ECN combination and the CE-marked packet ratio to the
	This section specifies how the messages should be conveyed.		ingress. This section specifies how the messages should be conveyed.
	IPFIX recommends, but does not require, use of SCTP [RFC4960] in		IPFIX recommends, but does not require, use of SCTP [RFC4960] in
	partial reliability mode [RFC3758] for the transport of its messages.		partial reliability mode [RFC3758] for the transport of its messages.
	This mode allows loss of some packets, which is tolerable because		This mode allows loss of some packets, which is tolerable because
	IPFIX communicates cumulative statistics. IPFIX over SCTP over IP		IPFIX communicates cumulative statistics. IPFIX over SCTP over IP
	SHOULD be used directly where there is IP connectivity between the		SHOULD be used directly where there is IP connectivity between the
	ingress and egress; however, there might be different transport		ingress and egress; however, there might be different transport
	protocols or address spaces used in different regions of an SFC		protocols or address spaces used in different regions of an SFC
	domain that make such direct IP connectivity problematic. The NSH		domain that make such direct IP connectivity problematic. The NSH
	provides the general method of routing traffic within an SFC domain		provides the general method of routing traffic within an SFC domain
	so the encapsulation of the required IPFIX traffic in NSH MUST be		so the encapsulation of the required IPFIX traffic in NSH MUST be
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	implemented and, when IP connectivity is not available, IPFIX over		implemented and, when IP connectivity is not available, IPFIX over
	NSH SHOULD be used along with configuration of appropriate SFC paths		NSH SHOULD be used along with configuration of appropriate SFC paths
	for the IPFIX over NSH traffic.		for the IPFIX over NSH traffic.
	Typically IPFIX messages could travel along the same path as network		IPFIX messages could travel along the same path as network data
	data traffic. In any case, an IPFIX message packet may get lost in		traffic. In any case, an IPFIX message packet may get lost in case of
	case of network congestion. Even though the missing information could		network congestion. Even though the missing information could be
	be recovered because of the use of cumulative counts, the message		recovered because of the use of cumulative counts, the message SHOULD
	SHOULD be transmitted at a higher priority than users' traffic flows.		be transmitted at a higher priority than users' traffic flows.
	The ingress node can do congestion management at different		The ingress node can do congestion management at different
	granularity which means both the overall aggregated inner tunnel		granularity which means both the overall aggregated inner tunnel
	congestion level and congestion level contributed by certain traffic		congestion level and congestion level contributed by certain traffic
	flows could be measured for different congestion management purpose.		flows could be measured for different congestion management purpose.
	For example, if the ingress only wants to limit congestion volume		For example, if the ingress only wants to limit congestion volume
	caused by certain traffic flows, such as UDP-based traffic, then		caused by certain traffic flows, such as UDP-based traffic, then
	congestion volume for that traffic will be fed back; or if the		congestion volume for that traffic will be fed back; or if the
	ingress is doing overall congestion management, the aggregated		ingress is doing overall congestion management, the aggregated
	congestion volume will be fed back.		congestion volume will be fed back.
	skipping to change at page 20, line 37 ¶		skipping to change at page 20, line 37 ¶
	When sending IPFIX messages from ingress to egress, the ingress acts		When sending IPFIX messages from ingress to egress, the ingress acts
	as IPFIX exporter and egress acts as IPFIX collector; When feedback		as IPFIX exporter and egress acts as IPFIX collector; When feedback
	congestion level information from egress to ingress, then the egress		congestion level information from egress to ingress, then the egress
	acts as IPFIX exporter and ingress acts as IPFIX collector.		acts as IPFIX exporter and ingress acts as IPFIX collector.
	The combination of congestion level measurement and congestion		The combination of congestion level measurement and congestion
	information delivery procedures should be as following:		information delivery procedures should be as following:
	o The ingress node determines the IPFIX template record to be used.		o The ingress node determines the IPFIX template record to be used.
	The template record can be pre-configured or determined at		The template record can be pre-configured or determined at
	runtime, the content of template record will be determined		runtime, the content of the template record will be determined
	according to the granularity of congestion management; if the		according to the granularity of congestion management; if the
	ingress wants to limit congestion volume contributed by specific		ingress wants to limit congestion volume contributed by specific
	traffic flow then the elements such as source IP address,		traffic flows then the elements such as source IP address,
	destination IP address, flow id and CE-marked packet volume of the		destination IP address, flow ID and CE-marked packet volume of the
	flow, etc., will be included in the template record.		flows, etc., will be included in the template record.
	o Metering on the ingress measures traffic volume according to		o Metering at the ingress measures traffic volume according to the
	template record chosen and then the measurement records are sent		template record chosen and then the measurement records are sent
	to the egress.		to the egress.
	o Metering on the egress measures congestion level information		o Metering on the egress measures congestion level information
	according to template record which should be the same as the		according to template record which SHOULD be the same as the
	template record sent by the ingress.		template record sent by the ingress.
	o The egress sends measurement records together with the measurement		o The egress sends its measurement records together with the
	records of ingress back to the ingress.		measurement records of the ingress back to the ingress.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	4.3 IPFIX Extensions		4.3 IPFIX Extensions
	This section specifies new IPFIX Information Elements according to		This section specifies the new IPFIX Information Elements needed. It
	[RFC7013].		conforms to [RFC7013].
	4.3.1 nshServicePathID		4.3.1 nshServicePathID
	In order to identify SFC flows, so that congestion can be measured		In order to identify SFC flows, so that congestion can be measured
	and reported at that granularity, it is necessary for IPFIX to be		and reported at that granularity, it is necessary for IPFIX to be
	able to classify traffic based on the Service Path Identifier field		able to classify traffic based on the Service Path Identifier field
	of the NSH [RFC8300]. Thus an NSH Service Path Identifier		of the NSH [RFC8300]. Thus an NSH Service Path Identifier
	(nshServicePathID) IPFIX Information Element [RFC7012] is specified.		(nshServicePathID) IPFIX Information Element [RFC7012] is specified.
	Name: nshServicePathID		Name: nshServicePathID
	skipping to change at page 21, line 37 ¶		skipping to change at page 21, line 35 ¶
	Abstract Data Type: unsigned32		Abstract Data Type: unsigned32
	Data Type Semantics: identifier		Data Type Semantics: identifier
	ElementId: tbd0		ElementId: tbd0
	Status: current		Status: current
	4.3.2 tunnelEcnCeCeByteTotalCount		4.3.2 tunnelEcnCeCeByteTotalCount
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets withthe
	CE|CE ECN marking combination at the Observation Point since		CE|CE ECN marking combination at the Observation Point since
	the Metering Process (re-)initialization for this Observation		the Metering Process (re-)initialization for this Observation
	Point.		Point.
	Abstract Data Type: unsigned64		Abstract Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	ElementId: TBD1		ElementId: TBD1
	Statues: current		Statues: current
	Units: bytes		Units: bytes
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	4.3.3 tunnelEcnEctNectBytetTotalCount		4.3.3 tunnelEcnEctNectBytetTotalCount
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	ECT|N-ECT ECN marking combination (ECT(0) and ECT(1) are		the ECT|N-ECT ECN marking combination (ECT(0) and ECT(1) are
	treated as the same) at the Observation Point since the		treated the same as each other) at the Observation Point since
	Metering Process (re-)initialization for this Observation		the Metering Process (re-)initialization for this Observation
	Point.		Point.
	Abstract Data Type: unsigned64		Abstract Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	ElementId: TBD2		ElementId: TBD2
	Statues: current		Statues: current
	Units: bytes		Units: bytes
	4.3.4 tunnelEcnCeNectByteTotalCount		4.3.4 tunnelEcnCeNectByteTotalCount
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|N-ECT ECN marking combination at the Observation Point since		the CE|N-ECT ECN marking combination at the Observation Point
	the Metering Process (re-)initialization for this Observation		since the Metering Process (re-)initialization for this
	Point.		Observation Point.
	Abstract Data Type: unsigned64		Abstract Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	ElementId: TBD3		ElementId: TBD3
	Statues: current		Statues: current
	Units: bytes		Units: bytes
	4.3.5 tunnelEcnCeEctByteTotalCount		4.3.5 tunnelEcnCeEctByteTotalCount
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|ECT ECN marking combination (ECT(0) and ECT(1) are treated		the CE|ECT ECN marking combination (ECT(0) and ECT(1) are
	as the same) at the Observation Point since the Metering		treated the same as each other) at the Observation Point since
	Process (re-)initialization for this Observation Point.		the Metering Process (re-)initialization for this Observation
			Point.
	Abstract Data Type: unsigned64		Abstract Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	ElementId: TBD4		ElementId: TBD4
	Statues: current		Statues: current
	Units: bytes		Units: bytes
	4.3.6 tunnelEcnEctEctByteTotalCount		4.3.6 tunnelEcnEctEctByteTotalCount
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	ECT|ECT ECN marking combination (ECT(0) and ECT(1) are treated		the ECT|ECT ECN marking combination (ECT(0) and ECT(1) are
	as the same) at the Observation Point since the Metering		treated the same as each other) at the Observation Point since
	Process (re-)initialization for this Observation Point.		the Metering Process (re-)initialization for this Observation
			Point.
	Abstract Data Type: unsigned64		Abstract Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	ElementId: TBD5		ElementId: TBD5
	Statues: current		Statues: current
	Units: bytes		Units: bytes
	4.3.7 tunnelEcnCEMarkedRatio		4.3.7 tunnelEcnCEMarkedRatio
	Description: The ratio of CE-marked Packet at the Observation		Description: The ratio of CE-marked packets at the Observation
	Point.		Point.
	Abstract Data Type: float32		Abstract Data Type: float32
	ElementId: TBD6		ElementId: TBD6
	Statues: current		Statues: current
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	5. Example of Use		5. Example of Use
	This subsection provides an example of how the solution described in		This subsection provides an example of how the solution described in
	this document could work.		this document could work.
	First of all, IPFIX template records are exchanged between ingress		First of all, IPFIX template records are exchanged between ingress
	and egress to negotiate the format of data records. The example here		and egress to negotiate the format of data records. The example here
	is to measure the congestion level for the overall tunnel caused by		is to measure the congestion level for the overall tunnel caused by
	all the traffic. After the negotiation is finished, the ingress sends		all the traffic. After the negotiation is finished, the ingress sends
	in-band messages to egress containing the number of each kind of ECN-		in-band messages to the egress containing the number of each kind of
	marked packets (i.e.. CE|CE, ECT|N-ECT and ECT|ECT) received until it		ECN-marked packets (i.e., CE|CE, ECT|N-ECT and ECT|ECT) received
	sent the message.		before it sent the message.
	After the egress receives the message, the egress calculates the CE-		After the egress receives the message, the egress calculates the CE-
	marked packet ratio and counts the number of different kinds of ECN-		marked packet ratio and counts the number of different kinds of ECN-
	marking packets received until it received the message. Then the		marking packets received before it received the message. Then the
	egress sends a feedback message containing the counts together with		egress sends a feedback message containing the counts together with
	the information in the ingress's message back to the ingress.		the information in the ingress's message back to the ingress.
	Figures 7 to 10 below show the example procedure between ingress and		Figures 7 to 10 below show the example procedure between ingress and
	egress.		egress.
	+---------------------------------+----------------------+		+---------------------------------+----------------------+
	|Set ID=2 Length=40 |		|Set ID=2 Length=40 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|Template ID=256 Field Count=8 |		|Template ID=256 Field Count=8 |
	skipping to change at page 25, line 5 ¶		skipping to change at page 25, line 5 ¶
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|tunnelEcnCeNectByteTotalCount Field Length=8 |		|tunnelEcnCeNectByteTotalCount Field Length=8 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|tunnelEcnCeEctByteTotalCount Field Length=8 |		|tunnelEcnCeEctByteTotalCount Field Length=8 |
	+---------------------------------|----------------------+		+---------------------------------|----------------------+
	|tunnelEcnCEMarkedRatio Field Length=4 |		|tunnelEcnCEMarkedRatio Field Length=4 |
	+---------------------------------+----------------------+		+---------------------------------+----------------------+
	Figure 7. Template Record Sent From Egress to Ingress		Figure 7. Template Record Sent From Egress to Ingress
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	+---------------------------------+----------------------+		+---------------------------------+----------------------+
	|Set ID=2 Length=28 |		|Set ID=2 Length=28 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|Template ID=257 Field Count=3 |		|Template ID=257 Field Count=3 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|tunnelEcnCeCeByteTotalCount Field Length=8 |		|tunnelEcnCeCeByteTotalCount Field Length=8 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|tunnelEcnEctNectByteTotalCount Field Length=8 |		|tunnelEcnEctNectByteTotalCount Field Length=8 |
	|---------------------------------|----------------------|		|---------------------------------|----------------------|
	|tunnelEcnEctEctByteTotalCount Field Length=8 |		|tunnelEcnEctEctByteTotalCount Field Length=8 |
	skipping to change at page 26, line 4 ¶		skipping to change at page 26, line 4 ¶
	+-+		+-+
	|M| : Message Packet		|M| : Message Packet
	+-+		+-+
	+-+		+-+
	|P| : User Packet		|P| : User Packet
	+-+		+-+
	Figure 9. Traffic flow Between Ingress and Egress		Figure 9. Traffic flow Between Ingress and Egress
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Set ID=257, Length=28		Set ID=257, Length=28
	+------+ A1 +------+		+------+ A1 +------+
	| | B1 | |		| | B1 | |
	| | C1 | |		| | C1 | |
	| | <----------------------------- | |		| | <----------------------------- | |
	| | | |		| | | |
	| | | |		| | | |
	| | SetID=256, Length=72 | |		| | SetID=256, Length=72 | |
	| | A1 | |		| | A1 | |
	| | B1 | |		| | B1 | |
	skipping to change at page 26, line 31 ¶		skipping to change at page 26, line 28 ¶
	| | D | |		| | D | |
	| | E | |		| | E | |
	| | R | |		| | R | |
	| | ----------------------------> | |		| | ----------------------------> | |
	| | | |		| | | |
	+------+ +------+		+------+ +------+
	Figure 10. Messages Between Ingress and Egress		Figure 10. Messages Between Ingress and Egress
	The following provides an example of how the tunnel congestion level		The following provides an example of how the tunnel congestion level
	could be calculated:		can be calculated (see Figure 10):
	The congestion Level could be divided into two categories: (1)		The congestion Level could be divided into two categories: (1)
	slight congestion (no packets dropped); (2) serious congestion		slight congestion (no packets dropped); (2) serious congestion
	(packets are being dropped).		(packets are being dropped).
	For slight congestion, the congestion level is indicated as the		For slight congestion, the congestion level is indicated by the
	ratio of CE-marked packet:		ratio of CE-marked packets:
	ce_marked = R;		ce_marked = R;
	For serious congestion, the congestion level is indicated as the		For serious congestion, the congestion level is indicated as the
	number of volume loss:		number of volume loss:
	total_ingress = (A1 + B1 + C1)		total_ingress = (A1 + B1 + C1)
	total_egress = (A2 + B2 + C2 + D + E)		total_egress = (A2 + B2 + C2 + D + E)
	volume_loss = (total_ingress - total_egress)		volume_loss = (total_ingress - total_egress)
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	6. IANA Considerations		6. IANA Considerations
	The following subsections provide IANA assignment considerations.		The following subsections provide IANA assignment considerations.
	6.1 SFC NSH Header ECN Bits		6.1 SFC NSH Header ECN Bits
	IANA is requested to assign two contiguous bits in the NSH Base		IANA is requested to assign two contiguous bits in the NSH Base
	Header Bits registry for ECN (bits 16 and 17 suggested) and note this		Header Bits registry for ECN (bits 16 and 17 suggested) and note this
	assignment as follows:		assignment as follows:
	skipping to change at page 27, line 39 ¶		skipping to change at page 27, line 37 ¶
	Status: current		Status: current
	Description: The Network Service Header [RFC8300] Service Path		Description: The Network Service Header [RFC8300] Service Path
	Identifier.		Identifier.
	ElementID: TBD1		ElementID: TBD1
	Name: tunnelEcnCeCePacketTotalCount		Name: tunnelEcnCeCePacketTotalCount
	Data Type: unsigned64		Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	Status: current		Status: current
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|CE ECN marking combination at the Observation Point since		the CE|CE ECN marking combination at the Observation Point
	the Metering Process (re-)initialization for this Observation		since the Metering Process (re-)initialization for this
	Point.		Observation Point.
	Units: octets		Units: octets
	ElementID: TBD2		ElementID: TBD2
	Name: tunnelEcnEctNectPacketTotalCount		Name: tunnelEcnEctNectPacketTotalCount
	Data Type: unsigned64		Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	Status: current		Status: current
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	ECT|N-ECT ECN marking combination at the Observation Point		the ECT|N-ECT ECN marking combination at the Observation Point
	since the Metering Process (re-)initialization for this		since the Metering Process (re-)initialization for this
	Observation Point.		Observation Point.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Units: octets		Units: octets
	ElementID: TBD3		ElementID: TBD3
	Name: tunnelEcnCeNectPacketTotalCount		Name: tunnelEcnCeNectPacketTotalCount
	Data Type: unsigned64		Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	Status: current		Status: current
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|N-ECT ECN marking combination at the Observation Point since		the CE|N-ECT ECN marking combination at the Observation Point
	the Metering Process (re-)initialization for this Observation		since the Metering Process (re-)initialization for this
	Point.		Observation Point.
	Units: octets		Units: octets
	ElementID: TBD4		ElementID: TBD4
	Name: tunnelEcnCeEctPacketTotalCount		Name: tunnelEcnCeEctPacketTotalCount
	Data Type: unsigned64		Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	Status: current		Status: current
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|ECT ECN marking combination at the Observation Point since		the CE|ECT ECN marking combination at the Observation Point
	the Metering Process (re-)initialization for this Observation		since the Metering Process (re-)initialization for this
	Point.		Observation Point.
	Units: octets		Units: octets
	ElementID: TBD5		ElementID: TBD5
	Name: tunnelEcnEctEctPacketTotalCount		Name: tunnelEcnEctEctPacketTotalCount
	Data Type: unsigned64		Data Type: unsigned64
	Data Type Semantics: totalCounter		Data Type Semantics: totalCounter
	Status: current		Status: current
	Description: The total number of bytes of incoming packets with		Description: The total number of bytes of incoming packets with
	CE|ECT(0) ECN marking combination at the Observation Point		the CE|ECT(0) ECN marking combination at the Observation Point
	since the Metering Process (re-)initialization for this		since the Metering Process (re-)initialization for this
	Observation Point.		Observation Point.
	Units: octets		Units: octets
	ElementID: TBD6		ElementID: TBD6
	Name: tunnelEcnCEMarkedRatio		Name: tunnelEcnCEMarkedRatio
	Data Type: float32		Data Type: float32
	Status: current		Status: current
	Description: The ratio of CE-marked Packet at the Observation		Description: The ratio of CE-marked Packet at the Observation
	Point.		Point.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	7. Security Considerations		7. Security Considerations
	For general NSH security considerations, see [RFC8300].		For general NSH security considerations, see [RFC8300].
	For security considerations concerning tampering with ECN signaling,		For security considerations concerning tampering with ECN signaling,
	see [RFC3168]. For security considerations concerning ECN and		see [RFC3168]. For security considerations concerning ECN and
	encapsulation, see [RFC6040].		encapsulation, see [RFC6040].
	For general IPFIX security considerations, see [RFC7011]. If deployed		For general IPFIX security considerations, see [RFC7011]. If deployed
	in an untrusted environment, the signaling traffic between ingress		in an untrusted environment, the signaling traffic between ingress
	and egress can be protected utilizing the security mechanisms		and egress can be protected utilizing the security mechanisms
	provided by IPFIX (see Section 11 in [RFC7011]).		provided by IPFIX (see Section 11 in [RFC7011]). The tunnel
			endpoints (the ingress and egress for an SFC domain) are assumed to
	The tunnel endpoints (the ingress and egress for an SFC domain) are		be in the same administrative domain, so they will trust each other.
	assumed to be in the same administrative domain, so they will trust
	each other.
	The solution in this document does not introduce any greater		The solution in this document does not introduce any greater
	potential to invade privacy than would have been available without		potential to invade privacy than would have been available without
	the solution.		the solution.
	8. Acknowledgements		8. Acknowledgements
	Most of the material on Tunnel Congestion Feedback was originally in		Most of the material on Tunnel Congestion Feedback was originally in
	draft-ietf-twvwg-tunnel-congestion-feedback. After discussion with		draft-ietf-tsvwg-tunnel-congestion-feedback. After discussion with
	the authors of that draft, the authors of this draft, and the Chairs		the authors of that draft, the authors of this draft, and the Chairs
	of the TSVWG and SFC Working Groups, the Tunnel Congestion Feedback		of the TSVWG and SFC Working Groups, the Tunnel Congestion Feedback
	draft was merged into this draft.		draft was merged into this draft.
	The authors wish to thank the following for their comments,		The authors wish to thank the following for their comments,
	suggestions, and reviews:		suggestions, and reviews:
	David Black, Sami Boutros, Anthony Chan, Lingli Deng, Liang Geng,		David Black, Sami Boutros, Anthony Chan, Lingli Deng, Liang Geng,
	Joel Halpern, Jake Holland, John Kaippallimalil, Tal Mizrahi,		Joel Halpern, Jake Holland, John Kaippallimalil, Tal Mizrahi,
	Vincent Roca, Lei Zhu		Vincent Roca, Lei Zhu
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Normative References		Normative References
	[RFC2119] - Bradner, S., "Key words for use in RFCs to Indicate		[RFC2119] - Bradner, S., "Key words for use in RFCs to Indicate
	Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119,		Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119,
	March 1997, <http://www.rfc-editor.org/info/rfc2119>.		March 1997, <http://www.rfc-editor.org/info/rfc2119>.
	[RFC3168] - Ramakrishnan, K., Floyd, S., and D. Black, "The Addition		[RFC3168] - Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
	of Explicit Congestion Notification (ECN) to IP", RFC 3168, DOI		of Explicit Congestion Notification (ECN) to IP", RFC 3168, DOI
	10.17487/RFC3168, September 2001, <http://www.rfc-		10.17487/RFC3168, September 2001, <http://www.rfc-
	editor.org/info/rfc3168>.		editor.org/info/rfc3168>.
	skipping to change at page 31, line 5 ¶		skipping to change at page 31, line 5 ¶
	editor.org/info/rfc7567>.		editor.org/info/rfc7567>.
	[RFC8174] - Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC		[RFC8174] - Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
	2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May		2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May
	2017, <http://www.rfc-editor.org/info/rfc8174>		2017, <http://www.rfc-editor.org/info/rfc8174>
	[RFC8300] - Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed.,		[RFC8300] - Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed.,
	"Network Service Header (NSH)", RFC 8300, DOI 10.17487/RFC8300,		"Network Service Header (NSH)", RFC 8300, DOI 10.17487/RFC8300,
	January 2018, <https://www.rfc-editor.org/info/rfc8300>.		January 2018, <https://www.rfc-editor.org/info/rfc8300>.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Informative References		Informative References
	[RFC4301] - Kent, S. and K. Seo, "Security Architecture for the		[RFC4301] - Kent, S. and K. Seo, "Security Architecture for the
	Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, December		Internet Protocol", RFC 4301, DOI 10.17487/RFC4301, December
	2005, <https://www.rfc-editor.org/info/rfc4301>.		2005, <https://www.rfc-editor.org/info/rfc4301>.
	[RFC4960] - Stewart, R., Ed., "Stream Control Transmission Protocol",		[RFC4960] - Stewart, R., Ed., "Stream Control Transmission Protocol",
	RFC 4960, DOI 10.17487/RFC4960, September 2007,		RFC 4960, DOI 10.17487/RFC4960, September 2007,
	<https://www.rfc-editor.org/info/rfc4960>.		<https://www.rfc-editor.org/info/rfc4960>.
	skipping to change at page 32, line 5 ¶		skipping to change at page 32, line 5 ¶
	[RFC8311] - Black, D., "Relaxing Restrictions on Explicit Congestion		[RFC8311] - Black, D., "Relaxing Restrictions on Explicit Congestion
	Notification (ECN) Experimentation", RFC 8311, DOI		Notification (ECN) Experimentation", RFC 8311, DOI
	10.17487/RFC8311, January 2018, <https://www.rfc-		10.17487/RFC8311, January 2018, <https://www.rfc-
	editor.org/info/rfc8311>.		editor.org/info/rfc8311>.
	[ecnL4S] - De Schepper, K., and B. Briscoe, "Identifying Modified		[ecnL4S] - De Schepper, K., and B. Briscoe, "Identifying Modified
	Explicit Congestion Notification (ECN) Semantics for Ultra-Low		Explicit Congestion Notification (ECN) Semantics for Ultra-Low
	Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s-id, work in		Queuing Delay (L4S)", draft-ietf-tsvwg-ecn-l4s-id, work in
	progress.		progress.
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Authors' Addresses		Authors' Addresses
	Donald E. Eastlake, 3rd		Donald E. Eastlake, 3rd
	Futurewei Technologies		Futurewei Technologies
	2386 Panoramic Circle		2386 Panoramic Circle
	Apopka, FL 32703 USA		Apopka, FL 32703 USA
	Tel: +1-508-333-2270		Tel: +1-508-333-2270
	Email: d3e3e3@gmail.com		Email: d3e3e3@gmail.com
	skipping to change at page 33, line 5 ¶		skipping to change at page 33, line 5 ¶
	Email: agmalis@gmail.com		Email: agmalis@gmail.com
	Xinpeng Wei		Xinpeng Wei
	Huawei Technologies		Huawei Technologies
	Beiqing Rd. Z-park No.156, Haidian District,		Beiqing Rd. Z-park No.156, Haidian District,
	Beijing, 100095, P. R. China		Beijing, 100095, P. R. China
	EMail: weixinpeng@huawei.com		EMail: weixinpeng@huawei.com
	INTERNET-DRAFT NSH ECN & Congestion Feedback
	Copyright and IPR Provisions		Copyright and IPR Provisions
	Copyright (c) 2021 IETF Trust and the persons identified as the		Copyright (c) 2021 IETF Trust and the persons identified as the
	document authors. All rights reserved.		document authors. All rights reserved.
	This document is subject to BCP 78 and the IETF Trust's Legal		This document is subject to BCP 78 and the IETF Trust's Legal
	Provisions Relating to IETF Documents		Provisions Relating to IETF Documents
	(http://trustee.ietf.org/license-info) in effect on the date of		(http://trustee.ietf.org/license-info) in effect on the date of
	publication of this document. Please review these documents		publication of this document. Please review these documents
	carefully, as they describe your rights and restrictions with respect		carefully, as they describe your rights and restrictions with respect
End of changes. 65 change blocks.
	159 lines changed or deleted		90 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/