< draft-malas-performance-metrics-07.txt		draft-malas-performance-metrics-08.txt >
	Internet-Draft D. Malas		PMOL Working Group D. Malas
	Expires: November 2007 Level 3 Communications		Internet-Draft Level 3 Communications
	May 24, 2007		Expires: June 2008 December 6, 2007
	SIP End-to-End Performance Metrics		SIP End-to-End Performance Metrics
	draft-malas-performance-metrics-07.txt		draft-malas-performance-metrics-08.txt
	Status of this Memo		Status of this Memo
	By submitting this Internet-Draft, each author represents that		By submitting this Internet-Draft, each author represents that
	any applicable patent or other IPR claims of which he or she is		any applicable patent or other IPR claims of which he or she is
	aware have been or will be disclosed, and any of which he or she		aware have been or will be disclosed, and any of which he or she
	becomes aware will be disclosed, in accordance with Section 6 of		becomes aware will be disclosed, in accordance with Section 6 of
	BCP 79.		BCP 79.
	Internet-Drafts are working documents of the Internet Engineering		Internet-Drafts are working documents of the Internet Engineering
	skipping to change at page 1, line 33 ¶		skipping to change at page 1, line 33 ¶
	and may be updated, replaced, or obsoleted by other documents at any		and may be updated, replaced, or obsoleted by other documents at any
	time. It is inappropriate to use Internet-Drafts as reference		time. It is inappropriate to use Internet-Drafts as reference
	material or to cite them other than as "work in progress."		material or to cite them other than as "work in progress."
	The list of current Internet-Drafts can be accessed at		The list of current Internet-Drafts can be accessed at
	http://www.ietf.org/ietf/1id-abstracts.txt		http://www.ietf.org/ietf/1id-abstracts.txt
	The list of Internet-Draft Shadow Directories can be accessed at		The list of Internet-Draft Shadow Directories can be accessed at
	http://www.ietf.org/shadow.html		http://www.ietf.org/shadow.html
	This Internet-Draft will expire on November 24, 2007.		This Internet-Draft will expire on May 6, 2008.
	Abstract		Abstract
	This document defines a set of metrics and their usage to evaluate		This document defines a set of metrics and their usage to evaluate
	the performance of end-to-end Session Initiation Protocol (SIP) based		the performance of end-to-end Session Initiation Protocol (SIP) based
	services in both production and testing environments. The purpose of		services in both production and testing environments. The purpose of
	this document is to combine a set of common metrics, allowing		this document is to combine a set of common metrics, allowing
	interoperable performance measurements, easing the comparison of		interoperable performance measurements, easing the comparison of
	industry implementations.		industry implementations.
	skipping to change at page 2, line 5 ¶		skipping to change at page 2, line 4 ¶
	industry implementations.		industry implementations.
	Table of Contents		Table of Contents
	1. Introduction...................................................2		1. Introduction...................................................2
	2. Terminology....................................................3		2. Terminology....................................................3
	3. SIP Performance Metrics........................................4		3. SIP Performance Metrics........................................4
	3.1. Registration Request Delay (RRD)..........................5		3.1. Registration Request Delay (RRD)..........................5
	3.1.1. Successful REGISTER Completion RRD...................5		3.1.1. Successful REGISTER Completion RRD...................5
	3.1.2. Failed REGISTER Attempt RRD..........................6		3.1.2. Failed REGISTER Attempt RRD..........................6
	3.2. Session Request Delay (SRD)...............................7		3.2. Session Request Delay (SRD)...............................7
	3.2.1. Successful Session Setup SRD.........................7		3.2.1. Successful Session Setup SRD.........................7
	3.2.2. Failed Session Setup SRD.............................8		3.2.2. Failed Session Setup SRD.............................8
	3.2.3. Instant Messaging....................................9		3.2.3. Instant Messaging....................................9
	3.3. Session Disconnect Delay (SDD)............................9		3.3. Session Disconnect Delay (SDD)............................9
	3.3.1. Successful session completion SDD...................10		3.3.1. Successful session completion SDD...................10
	3.3.2. Failed session completion SDD.......................11		3.3.2. Failed session completion SDD.......................11
	3.4. Session Duration Time (SDT)..............................12		3.4. Session Duration Time (SDT)..............................12
	3.4.1. Successful session completion SDT...................13		3.4.1. Successful session completion SDT...................13
	3.4.2. Failed session completion SDT.......................14		3.4.2. Failed session completion SDT.......................14
	3.5. Average Hops per Request (AHR)...........................15		3.5. Average Hops per Request (AHR)...........................15
	3.6. Session Establishment Rate (SER).........................17		3.6. Session Establishment Rate (SER).........................17
	3.6.1. Instant Messaging...................................18		3.6.1. Instant Messaging...................................18
	3.7. Session Establishment Efficiency Rate (SEER).............18		3.7. Session Establishment Efficiency Rate (SEER).............18
	3.8. Session Defects (SD).....................................19		3.8. Session Defects (SD).....................................19
	3.9. Ineffective Session Attempts (ISA).......................19		3.9. Ineffective Session Attempts (ISA).......................19
	3.10. Session Disconnect Failures (SDF).......................20		3.10. Session Disconnect Failures (SDF).......................20
	3.11. Session Completion Rate (SCR)...........................20		3.11. Session Completion Rate (SCR)...........................20
	3.11.1. Successful Session Completion......................21		3.11.1. Successful Session Completion......................21
	3.11.2. Failed Session Completion..........................22		3.11.2. Failed Session Completion..........................22
	3.12. Session Success Rate (SSR)..............................23		3.12. Session Success Rate (SSR)..............................22
	4. Metric Correlations...........................................23		4. Metric Correlations...........................................23
	5. Additional Considerations.....................................23		5. Additional Considerations.....................................23
	5.1. Back-to-back User Agent (B2BUA)..........................23		5.1. Back-to-back User Agent (B2BUA)..........................23
	5.2. Authorization and Authentication.........................23		5.2. Authorization and Authentication.........................23
	5.3. Forking..................................................24		5.3. Forking..................................................24
	5.4. Data Collection..........................................24		5.4. Data Collection..........................................24
	5.5. Testing Documentation....................................25		5.5. Testing Documentation....................................24
	5.6. Metric Template..........................................25		5.6. Metric Template..........................................25
	6. Security Considerations.......................................25		6. Security Considerations.......................................25
	7. IANA Considerations...........................................25		7. IANA Considerations...........................................25
	8. Conclusions...................................................26		8. Conclusions...................................................25
	9. Contributors..................................................26		9. Contributors..................................................26
	10. Acknowledgments..............................................26		10. Acknowledgments..............................................26
	11. References...................................................26		11. References...................................................26
	11.1. Normative References....................................26		11.1. Normative References....................................26
	11.2. Informative References..................................27		11.2. Informative References..................................27
	Author's Addresses...............................................27		Author's Addresses...............................................27
	Intellectual Property Statement..................................27		Intellectual Property Statement..................................27
	Disclaimer of Validity...........................................28		Disclaimer of Validity...........................................28
	Copyright Statement..............................................28		Copyright Statement..............................................28
	Acknowledgment...................................................28		Acknowledgment...................................................28
	skipping to change at page 18, line 4 ¶		skipping to change at page 18, line 4 ¶
	3.6. Session Establishment Rate (SER)		3.6. Session Establishment Rate (SER)
	This metric is used to detect the ability of a terminating UA or		This metric is used to detect the ability of a terminating UA or
	downstream proxy to successfully establish sessions per INVITE		downstream proxy to successfully establish sessions per INVITE
	request. SER is defined as the number of INVITE requests resulting		request. SER is defined as the number of INVITE requests resulting
	in a 200 OK response, to the total number of attempted INVITE		in a 200 OK response, to the total number of attempted INVITE
	requests less INVITE requests resulting in a 3XX response. This		requests less INVITE requests resulting in a 3XX response. This
	metric is similar to Answer Seizure Rate (ASR) [8] in telephony		metric is similar to Answer Seizure Rate (ASR) [8] in telephony
	applications of SIP. It is measured at the UAC only. The output		applications of SIP. It is measured at the UAC only. The output
	value of this metric is numerical and should be adjusted to indicate		value of this metric is numerical and should be adjusted to indicate
	a percentage (likely a fractional percentage) of successfully		a percentage of successfully established sessions. The following
	established sessions. The following represents the calculation for		represents the calculation for this metric:
	this metric:
	# of INVITE Requests w/ associated 200OK		# of INVITE Requests w/ associated 200OK
	SER = ---------------------------------------------------------------		SER = ---------------------------------------------------------------
	(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)		(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
	The following flow provides an example of identifiable events		The following flow provides an example of identifiable events
	necessary for inputs in determining session establishment as		necessary for inputs in determining session establishment as
	described above:		described above:
	UA1 UA2		UA1 UA2
	skipping to change at page 19, line 8 ¶		skipping to change at page 19, line 7 ¶
	3.7. Session Establishment Efficiency Rate (SEER)		3.7. Session Establishment Efficiency Rate (SEER)
	This metric is complimentary to SER, but is intended to exclude the		This metric is complimentary to SER, but is intended to exclude the
	potential effects of the terminating UAS from the metric. SEER is		potential effects of the terminating UAS from the metric. SEER is
	defined as the number of INVITE requests resulting in a 200 OK		defined as the number of INVITE requests resulting in a 200 OK
	response and INVITE requests resulting in a 480, 486, or 600; to the		response and INVITE requests resulting in a 480, 486, or 600; to the
	total number of attempted INVITE requests less INVITE requests		total number of attempted INVITE requests less INVITE requests
	resulting in a 3XX response. This metric is similar to Network		resulting in a 3XX response. This metric is similar to Network
	Efficiency Rate (NER) [8] in telephony applications of SIP. It is		Efficiency Rate (NER) [8] in telephony applications of SIP. It is
	measured at the UAC only. The output value of this metric is		measured at the UAC only. The output value of this metric is
	numerical and should be adjusted to indicate a percentage (likely a		numerical and should be adjusted to indicate a percentage of
	fractional percentage) of successfully established sessions less		successfully established sessions less common UAS failures. The
	common UAS failures. The following represents the calculation for		following represents the calculation for this metric:
	this metric:
	# of INVITE Requests w/ associated 200OK, 480, 486, or 600		# of INVITE Requests w/ associated 200OK, 480, 486, or 600
	SER = ---------------------------------------------------------------		SER = ---------------------------------------------------------------
	(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)		(Total # of INVITE Requests)-(# of INVITE Requests w/ 3XX Response)
	Reference the example flow is Section 3.6.		Reference the example flow is Section 3.6.
	3.8. Session Defects (SD)		3.8. Session Defects (SD)
	Session defects provide a subset of SIP failure responses, which		Session defects provide a subset of SIP failure responses, which
	consistently indicate a failure in dialog processing. Defects are		consistently indicate a failure in dialog processing. Defects are
	necessary to provide input to calculations such as Defects per		necessary to provide input to calculations such as Defects per
	Million (DPM) or other similar metrics. These failure responses are		Million (DPM) or other similar metrics. These failure responses are
	in response to initial session setup requests, such as a new INVITE.		in response to initial session setup requests, such as a new INVITE.
	The output value of this metric is numerical and should be adjusted		The output value of this metric is numerical and should be adjusted
	to indicate a percentage (likely a fractional percentage) of		to indicate a percentage of defective sessions. The following
	defective sessions. The following failure responses provide a		failure responses provide a guideline for defective criterion:
	guideline for defective criterion:
	. 500 Server Internal Error		. 500 Server Internal Error
	. 503 Service Unavailable		. 503 Service Unavailable
	. 504 Server Timeout		. 504 Server Timeout
	This set of failure responses was derived through correlating more		This set of failure responses was derived through correlating more
	granular ISUP failure responses as described in RFC 3398.		granular ISUP failure responses as described in RFC 3398.
	3.9. Ineffective Session Attempts (ISA)		3.9. Ineffective Session Attempts (ISA)
	Ineffective session attempts occur when a proxy or agent internally		Ineffective session attempts occur when a proxy or agent internally
	releases a setup request with a failed or congested condition. This		releases a setup request with a failed or congested condition. This
	metric is similar to Ineffective Machine Attempts (IMA) in telephony		metric is similar to Ineffective Machine Attempts (IMA) in telephony
	applications of SIP, and was adopted from Telcordia GR-512-CORE [7].		applications of SIP, and was adopted from Telcordia GR-512-CORE [7].
	The output value of this metric is numerical and should be adjusted		The output value of this metric is numerical and should be adjusted
	to indicate a percentage (likely a fractional percentage) of		to indicate a percentage of ineffective session attempts. The
	ineffective session attempts. The following failure responses		following failure responses provide a guideline for this criterion:
	provide a guideline for this criterion:
	. 408 Request Timeout		. 408 Request Timeout
	. 500 Server Internal Error		. 500 Server Internal Error
	. 503 Service Unavailable		. 503 Service Unavailable
	. 504 Server Timeout		. 504 Server Timeout
	This set was derived in a similar manner as described in Section 3.6,		This set was derived in a similar manner as described in Section 3.6,
	in addition 408 failure responses is indicative a congested state		in addition 408 failure responses is indicative a congested state
	with a downstream element.		with a downstream element.
	This metric is calculated as a percentage of total session setup		This metric is calculated as a percentage of total session setup
	requests. The following represents the calculation for this metric:		requests. The following represents the calculation for this metric:
	# of ISA		# of ISA
	skipping to change at page 20, line 37 ¶		skipping to change at page 20, line 33 ¶
	Multiplexing) media gateway, and the media gateway notifies the UA of		Multiplexing) media gateway, and the media gateway notifies the UA of
	a failure condition causing the loss of media related to an		a failure condition causing the loss of media related to an
	established session. The UA will release the session with a BYE, but		established session. The UA will release the session with a BYE, but
	should include a REASON header indicating the session was		should include a REASON header indicating the session was
	disconnected abnormally. The REASON value is utilized to determine		disconnected abnormally. The REASON value is utilized to determine
	the disconnect was a failure. This metric is similar to Cutoff Calls		the disconnect was a failure. This metric is similar to Cutoff Calls
	(CC) in telephony applications of SIP, and was adopted from Telcordia		(CC) in telephony applications of SIP, and was adopted from Telcordia
	GR-512-CORE [7]. The input variables for this metric are captured		GR-512-CORE [7]. The input variables for this metric are captured
	from the originating UAC or proxy perspective as relative to the end-		from the originating UAC or proxy perspective as relative to the end-
	to-end network under measurement. The output value of this metric is		to-end network under measurement. The output value of this metric is
	numerical and should be adjusted to indicate a percentage (likely a		numerical and should be adjusted to indicate a percentage of session
	fractional percentage) of session disconnect failures.		disconnect failures.
	This metric is calculated as a percentage of total session completed		This metric is calculated as a percentage of total session completed
	successfully as defined in Section 3.5. The following represents the		successfully as defined in Section 3.5. The following represents the
	calculation for this metric:		calculation for this metric:
	# of SDF's		# of SDF's
	SDF % = -------------------------------		SDF % = -------------------------------
	Total # of Session Requests		Total # of Session Requests
	3.11. Session Completion Rate (SCR)		3.11. Session Completion Rate (SCR)
	A session completion is defined as a SIP dialog, which completes		A session completion is defined as a SIP dialog, which completes
	without failing due to a lack of response from an intended proxy or		without failing due to a lack of response from an intended proxy or
	UA. This metric is only used when at least one proxy is involved in		UA. This metric is only used when at least one proxy is involved in
	the dialog. This metric is similar to Call Completion Rate (CCR) in		the dialog. This metric is similar to Call Completion Rate (CCR) in
	telephony applications of SIP. The output value of this metric is		telephony applications of SIP. The output value of this metric is
	numerical and should be adjusted to indicate a percentage (likely a		numerical and should be adjusted to indicate a percentage of
	fractional percentage) of successfully completed sessions.		successfully completed sessions.
	This metric is calculated as a percentage of total sessions completed		This metric is calculated as a percentage of total sessions completed
	successfully. The following represents the calculation for this		successfully. The following represents the calculation for this
	metric:		metric:
	# of Successfully Completed Sessions		# of Successfully Completed Sessions
	SCR % = ---------------------------------------		SCR % = ---------------------------------------
	Total # of Session Requests		Total # of Session Requests
	3.11.1. Successful Session Completion		3.11.1. Successful Session Completion
End of changes. 15 change blocks.
	26 lines changed or deleted		23 lines changed or added
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/