< draft-krishnan-ipfix-flow-aware-packet-sampling-05.txt   draft-krishnan-ipfix-flow-aware-packet-sampling-06.txt >
IPFIX IPFIX
Internet Draft R. Krishnan Internet Draft R. Krishnan
Intended status: Informational Brocade Communications Intended status: Informational Brocade Communications
Expires: December 2013 Ning So Expires: April 2014 Ning So
June 16, 2013 Tata Communications October 2013 Tata Communications
S. D'Antonio
University of Napoli "Parthenope"
Flow-state dependent packet selection techniques Flow-state Dependent Packet Selection Techniques
draft-krishnan-ipfix-flow-aware-packet-sampling-05.txt draft-krishnan-ipfix-flow-aware-packet-sampling-06.txt
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. This document may not be modified, provisions of BCP 78 and BCP 79. This document may not be modified,
and derivative works of it may not be created, except to publish it and derivative works of it may not be created, except to publish it
as an RFC and to translate it into languages other than English. as an RFC and to translate it into languages other than English.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 35 skipping to change at page 1, line 37
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on December 18, 2013. This Internet-Draft will expire on April 18, 2009.
Copyright Notice Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 20 skipping to change at page 2, line 22
communications etc. Using sampling techniques, for a given sampling communications etc. Using sampling techniques, for a given sampling
rate, the amount of samples that need to be processed is increasing rate, the amount of samples that need to be processed is increasing
exponentially especially for applications like security threat exponentially especially for applications like security threat
detection. This draft elaborates on flow-state dependent packet detection. This draft elaborates on flow-state dependent packet
selection techniques and the relevant information models. It selection techniques and the relevant information models. It
describes how these techniques can be effectively used to reduce the describes how these techniques can be effectively used to reduce the
number of samples for applications like security threat detection. number of samples for applications like security threat detection.
Table of Contents Table of Contents
1. Introduction...................................................2 1. Introduction...................................................3
1.1. Acronyms..................................................3 1.1. Acronyms..................................................3
1.2. Terminology...............................................3 1.2. Terminology...............................................3
2. Flow-state dependent packet selection techniques...............3 2. Flow-state dependent packet selection techniques...............3
2.1. Information Model for flow-state dependent packet selection 2.1. Information Model for flow-state dependent packet selection
technique configuration........................................4 technique configuration........................................4
2.2. Handling Inactive/Misidentified Large Flows...............5 2.2. Handling Inactive/Misidentified Large Flows...............5
2.3. Flow-state dependent packet selection - sample and hold...6 2.3. Flow-state dependent packet selection - sample and hold...5
2.4. IANA Considerations.......................................6 2.4. IANA Considerations.......................................5
2.4.1. Registration of Information Elements.................6 2.4.1. Registration of Information Elements.................5
2.4.1.1. largeFlowObservationInterval....................6 2.4.1.1. largeFlowObservationInterval....................5
2.4.1.2. largeFlowBandwidthThreshold.....................6 2.4.1.2. largeFlowBandwidthThreshold.....................6
3. Current sampling techniques for security threat detection......7 3. Current sampling techniques for security threat detection......6
4. Application of flow-state dependent packet selection techniques 4. Application of flow-state dependent packet selection techniques
for security threat detection.....................................7 for security threat detection.....................................7
4.1. Applicability of flow-state dependent packet selection 4.1. Analysis of various flow-state dependent packet selection
technique suggested in [ESVA].......Error! Bookmark not defined. techniques.....................................................8
4.2. Applicability of flow-state dependent packet selection 4.2. Simulation................................................8
technique suggested in [VRM]........Error! Bookmark not defined. 5. Security Considerations........................................8
4.3. Simulation................................................9 6. Operational Considerations.....................................8
5. Security Considerations........................................9 7. Acknowledgements...............................................8
6. Operational Considerations.....................................9 8. References.....................................................9
7. Acknowledgements...............................................9 8.1. Normative References......................................9
8. References....................................................10 8.2. Informative References....................................9
8.1. Normative References.....................................10
8.2. Informative References...................................10
1. Introduction 1. Introduction
This draft expands on the flow-state dependent packet selection This draft expands on the flow-state dependent packet selection
techniques described in [FLSEC] for identifying long-lived large techniques described in [RFC 7014] for identifying long-lived large
flows and the relevant information models. This draft also describes flows and the relevant information models. This draft also describes
a practical use case for efficient behavioral security detection, a practical use case for efficient behavioral security detection,
like Denial of Service (DOS) attacks etc., using flow-state dependent like Denial of Service (DOS) attacks etc., using flow-state dependent
packet selection techniques. packet selection techniques.
1.1. Acronyms 1.1. Acronyms
DOS: Denial of Service DOS: Denial of Service
GRE: Generic Routing Encapsulation GRE: Generic Routing Encapsulation
skipping to change at page 3, line 25 skipping to change at page 3, line 30
MPLS: Multi Protocol Label Switching MPLS: Multi Protocol Label Switching
NVGRE: Network Virtualization using Generic Routing Encapsulation NVGRE: Network Virtualization using Generic Routing Encapsulation
TCAM: Ternary Content Addressable Memory TCAM: Ternary Content Addressable Memory
STT: Stateless Transport Tunneling STT: Stateless Transport Tunneling
VXLAN: Virtual Extensible LAN VXLAN: Virtual Extensible LAN
1.2. Terminology 1.2. Terminology
Large flow(s): long-lived large flow(s) Large flow(s): long-lived large flow(s)
Small flow(s): long-lived small flow(s) and short-lived small/large Small flow(s): long-lived small flow(s) and short-lived small/large
flow(s) flow(s)
2. Flow-state dependent packet selection techniques 2. Flow-state dependent packet selection techniques
Expanding on the work in [FLSEC] and [RFC 5475], this draft suggests Expanding on the work in [RFC 7014] and [RFC 5475], this draft
additional techniques for flow-state dependent packet selection for suggests additional techniques for flow-state dependent packet
identifying large flows. One of these techniques is called Multistage selection for identifying large flows. One of these techniques is
Filters which is described in [ESVA]. This technique helps in called Multistage Filters which is described in [ESVA]. This
automatically identifying large flows with a low false positive rate. technique helps in automatically identifying large flows with a low
This technique can be implemented as an inline solution in false positive rate. This technique can be implemented as an inline
switches/routers and would be expected to operate at line rate. solution in switches/routers and would be expected to operate at line
rate.
Besides the Multistage filters technique described in [ESVA], Besides the Multistage filters technique described in [ESVA],
1) The technique suggested in [VRM] is also applicable. [VRM] 1) The technique suggested in [VRM] is also applicable. [VRM]
suggests techniques for automatically identifying large flows suggests techniques for automatically identifying large flows
using rotating conservative counting Bloom filters with periodic using rotating conservative counting Bloom filters with periodic
decay. This technique has a low false positive rate in large flow decay. This technique has a low false positive rate in large flow
misidentification. misidentification.
2) The sample and hold technique suggested in [ESVA] is also 2) The sample and hold technique suggested in [ESVA] is also
skipping to change at page 4, line 19 skipping to change at page 4, line 22
The large flows which are automatically identified using the above The large flows which are automatically identified using the above
techniques are populated in the IPFIX flow cache [RFC 6728]. If a techniques are populated in the IPFIX flow cache [RFC 6728]. If a
large flow already exists in the IPFIX flow cache, the above large flow already exists in the IPFIX flow cache, the above
techniques are not applied - this is the reason these are called techniques are not applied - this is the reason these are called
flow-state dependent packet selection techniques. flow-state dependent packet selection techniques.
Please note that there is a finite probability of small flows being Please note that there is a finite probability of small flows being
misidentified as large flows. These are handled as described in the misidentified as large flows. These are handled as described in the
section 2.2 "Handling Inactive/Misidentified Large Flows". section 2.2 "Handling Inactive/Misidentified Large Flows".
2.1. Information Model for flow-state dependent packet selection technique configuration 2.1. Information Model for flow-state dependent packet selection
technique configuration
From a bandwidth and time duration perspective, in order to identify From a bandwidth and time duration perspective, in order to identify
large flows we define an observation interval and observe the large flows we define an observation interval and observe the
bandwidth of the flow over that interval. A flow that exceeds a bandwidth of the flow over that interval. A flow that exceeds a
certain minimum bandwidth threshold over that observation interval certain minimum bandwidth threshold over that observation interval
would be considered a large flow. would be considered a large flow.
The two configuration parameters -- the observation interval, and the The two configuration parameters -- the observation interval, and the
minimum bandwidth threshold over that observation interval -- should minimum bandwidth threshold over that observation interval -- should
be programmable in a switch or a router to facilitate handling of be programmable in a switch or a router to facilitate handling of
skipping to change at page 5, line 14 skipping to change at page 5, line 4
largeFlowBandwidthThreshold: The minimum bandwidth of the flow during largeFlowBandwidthThreshold: The minimum bandwidth of the flow during
the observation interval for declaring the flow a large flow. Unit is the observation interval for declaring the flow a large flow. Unit is
in Mbps. in Mbps.
For example, a flow which is at or above 10 Mbps for a time period of For example, a flow which is at or above 10 Mbps for a time period of
at least 30 seconds could be declared a large flow. at least 30 seconds could be declared a large flow.
Below is the list of flow-state dependent packet selection technique Below is the list of flow-state dependent packet selection technique
Information Elements: Information Elements:
+-----+---------------------------------+-------+------------------------------+
| ID | Name | ID | Name |
+-----+----------------------------------+------+------------------------------+
| TBD | largeFlowObservationInterval | TBD | largeFlowBandwidthThreshold |
| 1 | | 2 | |
+-----+----------------------------------+------+------------------------------+
2.2. Handling Inactive/Misid entified Large Flows +-----+-------------------------------+
| ID | Name |
+-----+-------------------------------+
| TBD | largeFlowObservationInterval |
| 1 | |
+-----+-------------------------------+
| TBD | largeFlowBandwidthThreshold |
| 2 | |
+-----+-------------------------------+
2.2. Handling Inactive/Misidentified Large Flows
Once a flow has been recognized as a large flow, it should continue Once a flow has been recognized as a large flow, it should continue
to be recognized as a large flow as long as the traffic received to be recognized as a large flow as long as the traffic received
during an observation interval exceeds some fraction of the bandwidth during an observation interval exceeds some fraction of the bandwidth
threshold, for example 80% of the bandwidth threshold. If the traffic threshold, for example 80% of the bandwidth threshold. If the traffic
received during an observation interval falls below a fraction of the received during an observation interval falls below a fraction of the
bandwidth threshold, the large flow should be removed from the IPFIX bandwidth threshold, the large flow should be removed from the IPFIX
flow cache. flow cache.
2.3. Flow-state dependent packet selection - sample and hold 2.3. Flow-state dependent packet selection - sample and hold
[FLSEC] suggests some information model parameters for the sample and [RFC 7014] suggests some information model parameters for the sample
hold technique suggested in [ESVA]. The large flow information model and hold technique suggested in [ESVA]. The large flow information
parameters suggested in section 2.1 are complementary to these. model parameters suggested in section 2.1 are complementary to these.
2.4. IANA Considerations 2.4. IANA Considerations
2.4.1. Registration of Information Elements 2.4.1. Registration of Information Elements
IANA will register the following IEs in the IPFIX Information IANA will register the following IEs in the IPFIX Information
Elements registry at http://www.iana.org/assignments/ipfix/ipfix.xml Elements registry at http://www.iana.org/assignments/ipfix/ipfix.xml
IANA Note: please replace TBD1, TBD2, with the assigned values, IANA Note: please replace TBD1, TBD2, with the assigned values,
throughout the document. throughout the document.
2.4.1.1. largeFlowObservationInterval 2.4.1.1. largeFlowObservationInterval
skipping to change at page 7, line 38 skipping to change at page 7, line 5
(aka large) flows and a small percentage of the packet samples (aka large) flows and a small percentage of the packet samples
comprise of other (aka small) flows. The large flows aka top-talkers comprise of other (aka small) flows. The large flows aka top-talkers
consume a large percentage of the bandwidth and small percentage of consume a large percentage of the bandwidth and small percentage of
the flow space. the flow space.
The small flows, which are the typical cause of security threats like The small flows, which are the typical cause of security threats like
Denial of Service (DOS) attacks, scanning attacks etc., consume a Denial of Service (DOS) attacks, scanning attacks etc., consume a
small percentage of the bandwidth and a large percentage of the flow small percentage of the bandwidth and a large percentage of the flow
space. space.
4. Application of flow-state dependent packet selection techniques for security threat detection 4. Application of flow-state dependent packet selection techniques for
security threat detection
Using the flow-state dependent packet selection techniques described Using the flow-state dependent packet selection techniques described
in Section 2, the large flows or top-talkers can be detected in real- in Section 2, the large flows or top-talkers can be detected in real-
time with a high degree of accuracy. Only the small flows need to be time with a high degree of accuracy. Only the small flows need to be
sampled -- this makes security threat detection more effective with sampled -- this makes security threat detection more effective with
minimal sampling overhead. minimal sampling overhead.
The steps in security threat detection are described below The steps in security threat detection are described below
1) Large Flow Identification: 1) Large Flow Identification:
skipping to change at page 8, line 23 skipping to change at page 7, line 36
categories as detailed below. categories as detailed below.
a. Well behaved (steady rate) large flows, e.g. video streams a. Well behaved (steady rate) large flows, e.g. video streams
b. Bursty (fluctuating rate) large flows e.g. Peer-to-Peer b. Bursty (fluctuating rate) large flows e.g. Peer-to-Peer
traffic traffic
The large flows can be sampled at a low rate for further analysis The large flows can be sampled at a low rate for further analysis
or need not be sampled. If desired, the large flows could be or need not be sampled. If desired, the large flows could be
exported to a central entity, e.g. Netflow Collector, using IPFIX exported to a central entity, e.g. Netflow Collector, using IPFIX
protocol [RFC 5101] for further analysis. protocol [RFC 7011] for further analysis.
3) Small Flow Processing: 3) Small Flow Processing:
The small flows (excluding the large flows) can be sampled at a The small flows (excluding the large flows) can be sampled at a
normal rate. The small flows can be examined for determining normal rate. The small flows can be examined for determining
security threats like DOS attacks (for e.g. SYN floods), Scanning security threats like DOS attacks (for e.g. SYN floods), Scanning
attacks etc. [FDDOS, PDSN, and ALDS] attacks etc. [FDDOS, PDSN, and ALDS]
Thus, we can see that, security threat detection is possible with Thus, we can see that, security threat detection is possible with
minimal sampling overhead. minimal sampling overhead.
4.1. Analysis of various flow-state dependent packet selection techniques 4.1. Analysis of various flow-state dependent packet selection
techniques
The multistage filter technique suggested in [ESVA] for automatic The multistage filter technique suggested in [ESVA] for automatic
identification works well for standard applications generating large identification works well for standard applications generating large
flows, for e.g. video content like movies and catch-up episodes, flows, for e.g. video content like movies and catch-up episodes,
backup transactions etc. with a detection time of approximately 30-60 backup transactions etc. with a detection time of approximately 30-60
seconds. These detection times ensure that short-lived large flows, seconds. These detection times ensure that short-lived large flows,
for e.g. HD video clips, are not unnecessarily recognized. for e.g. HD video clips, are not unnecessarily recognized.
If faster large flow identification times are desired (much shorter If faster large flow identification times are desired (much shorter
than 30s), the multistage filter technique suggested in [ESVA] may than 30s), the multistage filter technique suggested in [ESVA] may
pose the following problem that the effective filtered flow size is pose the following problem that the effective filtered flow size is
phase-dependent: that is, relatively smaller constant-rate flows, for phase-dependent: that is, relatively smaller constant-rate flows, for
e.g. HD video clips, beginning early within a counting Bloom filter e.g. HD video clips, beginning early within a counting Bloom filter
reset interval would be unnecessarily detected with the same reset interval would be unnecessarily detected with the same
probability as relatively larger flows beginning toward the interval. probability as relatively larger flows beginning toward the interval.
[VRM] suggests techniques for addressing the above problem using [VRM] suggests techniques for addressing the above problem using
rotating conservative counting Bloom filters with periodic decay. rotating conservative counting Bloom filters with periodic decay.
4.2. Simulation 4.2. Simulation
Simulation results for these flow-state dependent packet selection Simulation results for these flow-state dependent packet selection
techniques are presented in Appendix A. The goal of the simulation is techniques are presented in Appendix A. The goal of the simulation is
to demonstrate the effectiveness of these techniques for security to demonstrate the effectiveness of these techniques for security
threat detection in a multi-tenant video streaming data center. threat detection in a multi-tenant video streaming data center.
5. Security Considerations 5. Security Considerations
This document does not directly impact the security of the Internet This document does not directly impact the security of the Internet
infrastructure or its applications. In fact, it proposes techniques infrastructure or its applications. In fact, it proposes techniques
skipping to change at page 10, line 5 skipping to change at page 9, line 5
techniques, the operator should adjust the programmable parameters techniques, the operator should adjust the programmable parameters
largeFlowObservationInterval and largeFlowBandwidthThreshold in largeFlowObservationInterval and largeFlowBandwidthThreshold in
switches/routers based on the applications which are being deployed. switches/routers based on the applications which are being deployed.
7. Acknowledgements 7. Acknowledgements
The authors would like to thank Juergen Quittek, Brian Carpenter, The authors would like to thank Juergen Quittek, Brian Carpenter,
Michael Fargano, Michael Bugenhagen, Jianrong Wong, Brian Trammell Michael Fargano, Michael Bugenhagen, Jianrong Wong, Brian Trammell
and Paul Aitken for all the support and valuable input. and Paul Aitken for all the support and valuable input.
8. References 8. References
8.1. Normative References 8.1. Normative References
8.2. Informative References 8.2. Informative References
[RFC 5474] N. Duffield et al., "A Framework for Packet Selection and [RFC 5474] N. Duffield et al., "A Framework for Packet Selection and
Reporting", March 2009. Reporting", March 2009.
[RFC 5475] T. Zseby et al., "Sampling and Filtering Techniques for IP [RFC 5475] T. Zseby et al., "Sampling and Filtering Techniques for IP
Packet Selection", March 2009. Packet Selection", March 2009.
[RFC 5476] B. Claise, Ed. et al., "Packet Sampling (PSAMP) Protocol [RFC 5476] B. Claise, Ed. et al., "Packet Sampling (PSAMP) Protocol
Specifications", March 2009. Specifications", March 2009.
[RFC 5477] T. Dietz et al., "Information Model for Packet Sampling [RFC 5477] T. Dietz et al., "Information Model for Packet Sampling
Exports", March 2009. Exports", March 2009.
[RFC 5101] B. Claise, "Specification of the IP Flow Information [RFC 7011] B. Claise, "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Export (IPFIX) Protocol for the Exchange of Flow
Information", January 2008 Information", September 2013
[RFC 6728] G. Muenz et al., "Configuration Data Model for the IP Flow [RFC 6728] G. Muenz et al., "Configuration Data Model for the IP Flow
Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols" Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols"
[VRM] G. Bianchi et al., "Measurement Data Reduction through [VRM] G. Bianchi et al., "Measurement Data Reduction through
Variation Rate Metering", INFOCOM 2010 Variation Rate Metering", INFOCOM 2010
[PDSN] Ignasi Paredes-Oliva et al., "Portscan Detection with Sampled [PDSN] Ignasi Paredes-Oliva et al., "Portscan Detection with Sampled
NetFlow", TMA 2009 NetFlow", TMA 2009
[ALDS] Z. Morley Mao et al., "Analyzing Large DDoS Attacks Using [ALDS] Z. Morley Mao et al., "Analyzing Large DDoS Attacks Using
Multiple Data Sources", SIGCOMM 2006 Multiple Data Sources", SIGCOMM 2006
[FDDOS] David Holmes, "The DDoS Threat Spectrum", F5 White paper 2012 [FDDOS] David Holmes, "The DDoS Threat Spectrum", F5 White paper 2012
[ESVA] C. Estan and G. Varghese, "New Directions in Traffic [ESVA] C. Estan and G. Varghese, "New Directions in Traffic
Measurement and Accounting", ACM SIGCOMM Internet Measurement Measurement and Accounting", ACM SIGCOMM Internet Measurement
Workshop 2001, San Francisco (CA) Nov. 2001. Workshop 2001, San Francisco (CA) Nov. 2001.
[RFC 7014] S. D'Antonio et al., "Flow Selection Techniques",
September 2013
Appendix A: Simulation of Flow aware packet sampling Appendix A: Simulation of Flow aware packet sampling
Goal: Goal:
Demonstrate the effectiveness of flow aware packet sampling in a Demonstrate the effectiveness of flow aware packet sampling in a
practical use case, for e.g. multi-tenant video streaming in a data practical use case, for e.g. multi-tenant video streaming in a data
center. center.
Test Topology: Test Topology:
Multiple virtual servers (server hosted on a virtual machine) Multiple virtual servers (server hosted on a virtual machine)
connected to a virtual switch (vSwitch) which in turn connects to the connected to a virtual switch (vSwitch) which in turn connects to the
skipping to change at page 12, line 4 skipping to change at page 10, line 50
o Other traffic - 500Mbps (Video clips, DOS attacks (for e.g. o Other traffic - 500Mbps (Video clips, DOS attacks (for e.g.
SYN floods), Scanning attacks etc.) SYN floods), Scanning attacks etc.)
. Aggregate traffic - 3.1Gbps . Aggregate traffic - 3.1Gbps
Total traffic on 2 servers - 5.8Gbps Total traffic on 2 servers - 5.8Gbps
Existing techniques: Existing techniques:
Normal sampling rate - 1:1000 Normal sampling rate - 1:1000
Total sampled traffic = 5.8Gbps/1000 = 5.8Mbps
Total sampled traffic = 5.8Gbps/1000 = 5.8Mbps
Flow aware sampling technique: Flow aware sampling technique:
Large flow recognition parameters Large flow recognition parameters
. Observation interval for large flow - 60 seconds . Observation interval for large flow - 60 seconds
. Minimum bandwidth threshold over the observation interval - . Minimum bandwidth threshold over the observation interval -
2Mbps 2Mbps
Aggregate bit rate of large flows = 4.8Gbps Aggregate bit rate of large flows = 4.8Gbps
skipping to change at line 484 skipping to change at page 11, line 46
Phone: +001-408-406-7890 Phone: +001-408-406-7890
Email: ramk@brocade.com Email: ramk@brocade.com
Ning So Ning So
Tata Communications Tata Communications
Plano, TX 75082, USA Plano, TX 75082, USA
Phone: +001-972-955-0914 Phone: +001-972-955-0914
Email: ning.so@tatacommunications.com Email: ning.so@tatacommunications.com
Salvatore D'Antonio
University of Napoli "Parthenope"
Centro Direzionale di Napoli Is. C4
Naples 80143
Italy
Phone: +39 081 5476766
EMail: salvatore.dantonio@uniparthenope.it
 End of changes. 29 change blocks. 
54 lines changed or deleted 66 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/