SIMPLE WG A. Houri Internet-Draft IBM Intended status: Standards Track T. Rang Expires: April 26, 2007 Microsoft Corporation E. Aoki AOL LLC October 23, 2006 Problem Statement for SIP/SIMPLE draft-rang-simple-problem-statement-01.txt Status of this Memo By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. This Internet-Draft will expire on April 26, 2007. Copyright Notice Copyright (C) The Internet Society (2006). Houri, et al. Expires April 26, 2007 [Page 1] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 Abstract As more experience of deploying SIMPLE based presence systems is accumulated, it seems that deploying a large SIMPLE presence system is actually a very hard task. The document analyzes several aspects of SIMPLE based presence system deployment and shows that difficulties around the amount of messages (network), state management (memory) and processing load (cpu). Although this document is a problem statement document and not a BCP document, several possible optimizations and directions are listed at the end of the document in addition to an initial set of requirements for what should be the characteristic of the solution to the problem stated in the document This document is intended to be used by the SIMPLE WG in order to work on possible solutions that will make the deployment of a presence server more reasonable task. Note that the document does not try to compare the SIP based presence server to other types of presence servers but only analyzes the SIP based presence server. Houri, et al. Expires April 26, 2007 [Page 2] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 Table of Contents 1. Requirements notation . . . . . . . . . . . . . . . . . . . . 4 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 3. Message Load . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Known Optimizations . . . . . . . . . . . . . . . . . . . 7 3.2. Analysis . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.3. SIMPLE with no optimizations . . . . . . . . . . . . . . . 9 3.4. SIMPLE with suggested optimizations . . . . . . . . . . . 10 3.5. Presence Federations . . . . . . . . . . . . . . . . . . . 11 3.5.1. Widely distributed inter-domain presence . . . . . . . 11 3.5.2. Associated inter-domain presence . . . . . . . . . . . 13 3.5.3. Very large network peering . . . . . . . . . . . . . . 14 3.5.4. Intra-domain peering . . . . . . . . . . . . . . . . . 16 4. State Management . . . . . . . . . . . . . . . . . . . . . . . 19 4.1. State Size Calculations . . . . . . . . . . . . . . . . . 20 4.1.1. Tiny System . . . . . . . . . . . . . . . . . . . . . 20 4.1.2. Medium System . . . . . . . . . . . . . . . . . . . . 20 4.1.3. Large System . . . . . . . . . . . . . . . . . . . . . 20 4.1.4. Very Large System . . . . . . . . . . . . . . . . . . 21 5. Processing complexities . . . . . . . . . . . . . . . . . . . 22 5.1. Aggregation . . . . . . . . . . . . . . . . . . . . . . . 22 5.2. Partial Publish and Notify . . . . . . . . . . . . . . . . 22 5.3. Filtering . . . . . . . . . . . . . . . . . . . . . . . . 23 5.4. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 23 6. Resource List Service . . . . . . . . . . . . . . . . . . . . 24 7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 7.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 25 7.2. Optimizations . . . . . . . . . . . . . . . . . . . . . . 26 8. Extremely Optimized Model . . . . . . . . . . . . . . . . . . 29 9. Security Considerations . . . . . . . . . . . . . . . . . . . 31 10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 32 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33 11.1. Normative References . . . . . . . . . . . . . . . . . . . 33 11.2. Informational References . . . . . . . . . . . . . . . . . 33 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 Intellectual Property and Copyright Statements . . . . . . . . . . 36 Houri, et al. Expires April 26, 2007 [Page 3] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 1. Requirements notation The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [1]. Houri, et al. Expires April 26, 2007 [Page 4] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 2. Introduction As more experience of deploying SIMPLE based presence systems is accumulated, it seems that deploying a large SIMPLE presence system is actually a very hard task. The document analyzes several aspects of SIMPLE based presence system deployment and shows that difficulties around the amount of messages (network), state management (memory) and processing load (cpu). Although this document is a problem statement document and not a BCP document, several possible optimizations and directions are listed at the end of the document in addition to an initial set of requirements for what should be the characteristic of the solution to the problem stated in the document This document is intended to be used by the SIMPLE WG in order to work on possible solutions that will make the deployment of a presence server more reasonable task. Note that the document does not try to compare the SIP based presence server to other types of presence servers but only analyzes the SIP based presence server. The document discusses the following areas. In each area we try to show the complexity and the load that the presence server has to handle in order to provide its service. o Messages load - By computing the number of messages that are required for connecting presence systems the document shows that the number of messages is enormous and it is quite obvious that some optimizations are needed. o State management - Due to the nature of the service that the presence server provides, the presence server has to manage a relatively big and complex state and some computations are provided in the document. o Processing complexities - The presence server maintains many small objects and has to do frequent operations on these objects. We show that these operations and especially the optimizations that are intended to save on the amount of data that is being sent between watchers and presence servers, are not so simple and may create a very heavy processing load on the presence server. o Groups - Resource List Servers [10] optimize that number of sessions that are created between the watchers and the presence server. On the other hand, this optimization may create an exponential size of subscription due to the unbearable ease of subscribing to large groups. Houri, et al. Expires April 26, 2007 [Page 5] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 The term presence domain or presence system appears in this document several time. By this term we refer to a presence system that provides presence subscription and notification services to its users. The system can be a system that is deployed in a small enterprise or in a very large consumer network. Houri, et al. Expires April 26, 2007 [Page 6] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 3. Message Load Even though some optimizations are approved or are being defined, we show in this section that a very large number of messages are needed in order to establish federation between presence systems of large communities. Further thinking is needed in order to make large deployment of presence systems less resource demanding. Note that very similar situations occur between RLSs [10] and presence servers due to the amount of subscriptions that RLSs may need to create to presence servers in order to handle the resource list subscriptions 3.1. Known Optimizations The current optimizations that are approved or considered in the SIMPLE group can be divided into two categories: o Dialogs saving optimization - Here we refer to optimizations as the resource list RFC [10] or to the Uri list subscriptions draft [14]. These documents define ways to reduce the number of dialogs that are required between the subscriber and the presence system. o Notification optimizations - Here we refer to the optimizations that are suggested in the subnot-etags draft [16]. This draft suggests ways to suppress the sending of unnecessary notifies when for example a subscription is refreshed. There are other drafts that reduce the size of messages as partial notifies or filtering but in this document we mostly care about the amount of messages. 3.2. Analysis The basic SIMPLE subscription dialog involves the following message- transfer: o SUBSCRIBE/200 o Initial NOTIFY/200 o (j) NOTIFY/200 where 'j' is the number of presence changes seen by the watcher o (k) SUBSCRIBE/200 where 'k' is the number of subscription dialog refresh periods o SUBSCRIBE/200 with Expires = 0 to terminate the dialog o NOTIFY/200 ending the dialog An individual watcher will generate X number of SIMPLE subscription Houri, et al. Expires April 26, 2007 [Page 7] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 dialogs corresponding to the number of presentities it chooses to watch. The amount of traffic generated is significantly affected by several factors: o Number of watchers connected to the system o Number of presentities connected to the system o Complexity of presence & frequency of presence change This document contains several calculations that show the expected message rate between presence domains. The following explains the assumptions and methods behind the calculations: o (A01) Subscription lifetime (hours)- The assumed lifetime of a subscription in hours. Here we assume 8 hours for all calculations. o (A02) Presence state changes / hour - The average time that a presentity changes his/hers status in one hour. We assumed 3 times a hour for most calculations. Note that for some users in consumer messaging systems, the actual numbers are likely to be much higher. o (A03) Subscription refresh interval / hour - The duration of the SUBSCRIBE session after which it needs to be refreshed. We assumed that the duration is one hour. o (A04) Total federated presentities per watcher - The number of presentities that the watcher is watching. The number here changes in this document according to the type of the specific deployment o (A05) Number of dialogs to maintain per watcher - The number of the SUBSCRIBE dialogs that are maintained per watcher. if a dialog optimization is not assumed this number is equal to A4, otherwise it is 1 o (A06) Number of watchers in a federated presence domain - The number of watchers in one presence domain that watch users in the other domain. The number here varies according to the assumptions for a specific deployment o (A07) Initial SUBSCRIBE/200 per watcher = A5*2 (message and an OK) o (A08) Initial NOTIFY/200 per watcher = A5*2 (message and an OK) Houri, et al. Expires April 26, 2007 [Page 8] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 o (A09) Total initial messages = (A7+A8)*A6 o (A10) NOTIFY/200 per watched presentity = (A2*A1*A4*2) (message and an OK) o (A11) SUBSCRIBE/200 refreshes = (A1/A3)*A5*2 (message and an OK) o (A12) NOTIFY/200 due to subscribe refresh - In a deployment where the notification optimization is not deployed this number will be ((A1/A3)*A5), otherwise it is 0 o (A13) Number of steady state messages = (A10+A11+A12)*A6 o (A14) SUBSCRIBE termination = A5*2 (message and an OK) o (A15) NOTIFY terminated = A5*2 (message and an OK) o (A16) Number of sign-out messages = (A7+A8)*A6 o (A17) Total messages between domains (both directions where users from domain A subscribe to users from domain B and vice versa)= (A9+A13+A16)*2 o (A18) Total number of messages / second = A17/A1/3600 (seconds in hour) 3.3. SIMPLE with no optimizations The following table uses some common presence characteristics to demonstrate the effect these factors have on state and message rate within a presence domain using base SIMPLE protocols without any proposed optimizations. In this example, there are two presence domains, each with 20,000 federating users with an average of 4 contacts in the peer domain Houri, et al. Expires April 26, 2007 [Page 9] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)...........................8 (A02) Presence state changes / hour...........................3 (A03) Subscription refresh interval / hour....................1 (A04) Total federated presentities per watcher................4 (A05) Number of dialogs to maintain per watcher...............4 (A06) Number of watchers in a federated presence domain..20,000 (A07) Initial SUBSCRIBE/200 per watcher.......................8 (A08) Initial NOTIFY/200 per watcher..........................8 (A09) Total initial messages............................320,000 (A10) NOTIFY/200 per watched presentity.....................192 (A11) SUBSCRIBE/200 refreshes................................64 (A12) NOTIFY/200 due to subscribe refresh....................64 (A13) Number of steady state messages.................6,400,000 (A14) SUBSCRIBE termination...................................8 (A15) NOTIFY terminated.......................................8 (A16) Number of sign-out messages.......................320,000 (A17) Total messages between domains.................14,080,000 (A18) Total number of messages / second.....................489 SIMPLE with no optimizations 3.4. SIMPLE with suggested optimizations The same analysis provided above is repeated here with the assumption that both the dialog and the notification optimizations are applied. Note that while the sign-in (ramp up) and sign-out messages flows are positively affected, the steady state rates are not. The optimizations enable the creation of a single dialog to the other domain from each subscriber for the set of presentities it is subscribing to. The optimizations also enable that there will be no need for a NOTIFY upon refreshing a SUBSCRIBE since the NOTIFY should not be sent in the refresh since it should be the same one as was sent when there was a state change for the presentity. Houri, et al. Expires April 26, 2007 [Page 10] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)...........................8 (A02) Presence state changes /hour............................3 (A03) Subscription refresh interval / hour....................1 (A04) Total federated presentities per watcher................4 (A05) Number of dialogs to maintain per watcher...............1 (A06) Number of watchers in a federated presence domain..20,000 (A07) Initial SUBSCRIBE/200 per watcher.......................2 (A08) Initial NOTIFY/200 per watcher..........................2 (A09) Total initial messages.............................80,000 (A10) NOTIFY/200 per watched presentity.....................192 (A11) SUBSCRIBE/200 refreshes................................16 (A12) NOTIFY/200 due to subscribe refresh.....................0 (A13) Number of steady state messages.................4,160,000 (A14) SUBSCRIBE termination...................................2 (A15) NOTIFY terminated.......................................2 (A16) Number of sign-out messages........................80,000 (A17) Total messages between domains..................8,640,000 (A18) Total number of messages / second.....................300 SIMPLE with optimizations 3.5. Presence Federations While these scalability issues exist in any large deployment, certain characteristics make the deployment conducive to the existing resource- list optimizations, and others have characteristics that cannot be exploited with the existing SIMPLE model. Following is a list of federation relationships that have varying usage characteristics. For each, a message rate table is provided reflecting typical changes message rates. Those characteristics can alter the overall effectiveness of existing optimizations. 3.5.1. Widely distributed inter-domain presence In some environments presence federation may be very common, perhaps even more common than intra-domain presence. An example of this type of environment is a small ISV or public server. Users in that small ISV are not likely to subscribe to the presence of other users in the their server since they do not necessarily have any relationship with each other aside from receiving service from the same provider. They are much more likely to be subscribed to the presence of users in one of the federated domains (whether in consumer domains, academic, other ISVs, etc). Common characteristics of this deployment are: Houri, et al. Expires April 26, 2007 [Page 11] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 o Federated subscriptions are the majority of subscription traffic o Individual users are likely to subscribe to multiple users in any one domain o The intersection of users in the deployment watching the same presentities is quite small (i.e., probability that watchers in the domain subscribe to the same presentity) To account for the extraordinarily high percentage of federation traffic, the number of federated presentities is increased to 20. The number of watchers in the domain could also be adjusted to account for an expected larger community of users being peered with, it is omitted here for simplification The first table below provides the calculations without optimizations the second table provides the calculations with optimization. Note that the number of messages per second decreases by a quarter with the optimizations but it is still quite big. (A01) Subscription lifetime (hours)...........................8 (A02) Presence state changes / hour...........................3 (A03) Subscription refresh interval / hour....................1 (A04) Total federated presentities per watcher...............20 (A05) Number of dialogs to maintain per watcher..............20 (A06) Number of watchers in a federated presence domain..20,000 (A07) Initial SUBSCRIBE/200 per watcher......................40 (A08) Initial NOTIFY/200 per watcher.........................40 (A09) Total initial messages..........................1,600,000 (A10) NOTIFY/200 per watched presentity.....................960 (A11) SUBSCRIBE/200 refreshes...............................320 (A12) NOTIFY/200 due to subscribe refresh...................320 (A13) Number of steady state messages................32,000,000 (A14) SUBSCRIBE termination..................................40 (A15) NOTIFY terminated......................................40 (A16) Number of sign-out messages.....................1,600,000 (A17) Total messages between domains.................70,400,000 (A18) Total number of messages / second...................2,444 Widely distributed inter-domain with no optimizations Houri, et al. Expires April 26, 2007 [Page 12] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)...........................8 (A02) Presence state changes / hour...........................3 (A03) Subscription refresh interval / hour....................1 (A04) Total federated presentities per watcher...............20 (A05) Number of dialogs to maintain per watcher...............1 (A06) Number of watchers in a federated presence domain..20,000 (A07) Initial SUBSCRIBE/200 per watcher.......................2 (A08) Initial NOTIFY/200 per watcher..........................2 (A09) Total initial messages.............................80,000 (A10) NOTIFY/200 per watched presentity.....................960 (A11) SUBSCRIBE/200 refreshes................................16 (A12) NOTIFY/200 due to subscribe refresh.....................0 (A13) Number of steady state messages................19,520,000 (A14) SUBSCRIBE termination...................................2 (A15) NOTIFY terminated.......................................2 (A16) Number of sign-out messages........................80,000 (A17) Total messages between domains.................39,360,000 (A18) Total number of messages / second...................1,367 Widely distributed inter-domain with optimizations 3.5.2. Associated inter-domain presence In this type of environment, the domain is a collection of associated users such as an enterprise. Here, federation is once again very common. However, there is also a strong association between some users in the deployment. These associations make it somewhat more likely that users in that domain will be watchers of the same presentity. This can occur because of business relationships (e.g. two co-workers on a project federating with a partner company). Common characteristics of this deployment are: o Federated subscriptions are large minority or small majority of subscription traffic o Individual users are likely to subscribe to multiple users in any one domain, especially their own o The intersection of users in the deployment watching the same presentities increases This federation type has traffic rates similar to the previous examples but with different levels of association of the users. Houri, et al. Expires April 26, 2007 [Page 13] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 3.5.3. Very large network peering In this environment, two or more very large networks create a peering relationship allowing their users to subscribe to presence in the other domains. Where as the number of users in other deployment types ranges from hundreds to several hundred thousand, these large networks host up to hundreds of millions of users. Examples of these networks are large wireless carriers at the low end and consumer IM networks at the high end. Common characteristics of this deployment are: o As users become accustomed to network boundaries disappearing, federated subscriptions become as common as subscriptions within the same domain o Individual users are highly likely to want to see presence of multiple presentities in the peer network o The intersection of users in the deployment watching the same presentities is very high (i.e., two or more users in network A are extremely likely to be watching a same user in network B) o Status changes increase greatly due to typical observed consumer behavior The first table below provides the calculations without optimizations the second table provides the calculations with optimization. Even though the optimization help a lot (almost cut the number of messages by half), the numbers are still very concerning. Houri, et al. Expires April 26, 2007 [Page 14] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................6 (A03) Subscription refresh interval / hour.......................1 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher.................10 (A06) Number of watchers in a federated presence domain.10,000,000 (A07) Initial SUBSCRIBE/200 per watcher.........................20 (A08) Initial NOTIFY/200 per watcher............................20 (A09) Total initial messages...........................400,000,000 (A10) NOTIFY/200 per watched presentity........................960 (A11) SUBSCRIBE/200 refreshes..................................160 (A12) NOTIFY/200 due to subscribe refresh......................160 (A13) Number of steady state messages...............12,800,000,000 (A14) SUBSCRIBE termination.....................................20 (A15) NOTIFY terminated.........................................20 (A16) Number of sign-out messages....................4,000,000,000 (A17) Total messages between domains................27,200,000,000 (A18) Total number of messages / second....................944,444 Very large network peering with no optimizations Houri, et al. Expires April 26, 2007 [Page 15] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................6 (A03) Subscription refresh interval / hour.......................1 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher..................1 (A06) Number of watchers in a federated presence domain.10,000,000 (A07) Initial SUBSCRIBE/200 per watcher..........................2 (A08) Initial NOTIFY/200 per watcher.............................2 (A09) Total initial messages............................40,000,000 (A10) NOTIFY/200 per watched presentity........................960 (A11) SUBSCRIBE/200 refreshes...................................16 (A12) NOTIFY/200 due to subscribe refresh........................0 (A13) Number of steady state messages................9,760,000,000 (A14) SUBSCRIBE termination......................................2 (A15) NOTIFY terminated..........................................2 (A16) Number of sign-out messages.......................40,000,000 (A17) Total messages between domains................19,680,000,000 (A18) Total number of messages / second....................683,333 Very large network peering with optimizations 3.5.4. Intra-domain peering Within a particular domain, multiple presence infrastructures are deployed with users split between the two. This scenario is unique in that federated messages do not pass outside the administrative domain's network. The two infrastructures peer directly inside the domain. A common example of this is an enterprise IT system with multiple independent vendor presence solutions deployed(e.g., a presence solution for desktop messaging deployed alongside a presence solution for IP telephony). Common characteristics of this deployment are o The difference between subscriptions to presentities in one system vs. the other are completely arbitrary. Any one presentity is as likely to be homed on one infrastructure as the other o Active users are almost guaranteed of subscribing to many users in the peer infrastructure o The level of intersection of presentities is extremely high The first table below provides the calculations without optimizations Houri, et al. Expires April 26, 2007 [Page 16] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 the second table provides the calculations with optimization. Even though the relatively conservative numbers are used, the amount of messages is still very high even though optimization may cut the traffic by more then half (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................3 (A03) Subscription refresh interval / hour.......................1 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher.................10 (A06) Number of watchers in a federated presence domain.....60,000 (A07) Initial SUBSCRIBE/200 per watcher.........................20 (A08) Initial NOTIFY/200 per watcher............................20 (A09) Total initial messages.............................2,400,000 (A10) NOTIFY/200 per watched presentity........................480 (A11) SUBSCRIBE/200 refreshes..................................160 (A12) NOTIFY/200 due to subscribe refresh......................160 (A13) Number of steady state messages...................48,400,000 (A14) SUBSCRIBE termination.....................................20 (A15) NOTIFY terminated.........................................20 (A16) Number of sign-out messages........................2,400,000 (A17) Total messages between domains...................105,600,000 (A18) Total number of messages / second......................3,667 Inter-domain peering with no optimizations Houri, et al. Expires April 26, 2007 [Page 17] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................3 (A03) Subscription refresh interval / hour.......................1 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher..................1 (A06) Number of watchers in a federated presence domain.....60,000 (A07) Initial SUBSCRIBE/200 per watcher..........................2 (A08) Initial NOTIFY/200 per watcher.............................2 (A09) Total initial messages...............................240,000 (A10) NOTIFY/200 per watched presentity........................480 (A11) SUBSCRIBE refreshes.......................................16 (A12) NOTIFY/200 due to subscribe refresh........................0 (A13) Number of steady state messages...................29,760,000 (A14) SUBSCRIBE termination......................................2 (A15) NOTIFY terminated..........................................2 (A16) Number of sign-out messages..........................240,000 (A17) Total messages between domains....................60,480,000 (A18) Total number of messages / second......................2,100 Inter-domain peering with optimizations Houri, et al. Expires April 26, 2007 [Page 18] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 4. State Management In previous section we have discussed the big amount of messages that need to be sent to/from a presence server In this section the state that needs to be maintained by a presence server will be analyzed and shown to be far from trivial. The presence server has two parallel tasks. 1. Maintain the state of the resources to which subscribers subscribe. 2. Maintain the state of the subscriptions of subscribers and provide timely updates to the subscription. For a single subscription from a single watcher on a resource, the presence server has to maintain the following state: o Subscription state including all the parameter that are needed in order to maintain the subscription as timers. o Optional filtering information that was requested by the subscriber. This includes enough information that is needed for doing the filtering. In addition additional information has to be maintained if partial notification is being supported for the subscription o Optional rate management information as throttling o Watcher information [5], [6] that is resulted from the subscription itself so users that are subscribed to can see who is watching them For each resource that has been subscribed to in the presence server, the presence server has to maintain the following state: o A list of the subscriptions for the resource. Note that this is already taken care of from the size calculation point of view by the subscription state above. o Privacy information for the resource. For each resource for which there was any publication and the resource has a state other then a default value, the presence server has to maintain the current value of the resource. Houri, et al. Expires April 26, 2007 [Page 19] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 4.1. State Size Calculations Lets assume the following sizes: o Subscription size - 2K bytes. This includes watcher information that need to be created by the presence server for each subscription. o Subscribed to resource - 1K bytes (for privacy information and other management info. The subscriptions themselves are already calculated in the previous bullet. o Resource with a state - 6K bytes. This is a moderate assumption if we take into account the amount of data that is being put in a presence document as multiple devices, calendar and geographical information. 4.1.1. Tiny System o 10K subscriptions = 19M bytes. o 5K subscribed to resources = 5M bytes. o 10K resources with state = 58M bytes. Total is 82M bytes. 4.1.2. Medium System o 100K subscriptions = 195M bytes. o 50K subscribed to resources = 49M bytes. o 100K resources with state = 586M bytes. Total is 830M bytes. 4.1.3. Large System o 6M subscriptions = 11,718M bytes. o 3M subscribed to resources = 2,929M bytes. o 4M resources with state = 23437M bytes. Total is 38G bytes. Houri, et al. Expires April 26, 2007 [Page 20] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 4.1.4. Very Large System o 150M subscriptions = 292,969M bytes. o 75M subscribed to resources = 73,242M bytes. o 100M resources with state = 585,937M bytes. Total is 952G bytes which is a very big number for a very dynamic storage as needed by the presence server. Although the numbers above may seem moderate enough for the sizes that the presence server is handling we should consider the following: o Dynamic state - Although the state may seem not so big for databases even for the very large system, we need to remember that this state is a very dynamic state. Subscriptions come and go all the time, the status of resources is being updated and so forth. This means that the presence server has to manage its state in a medium that is very dynamic and for such large sizes this task is not trivial. o Intelinked state - The subscriptions and the subscribed to resources are dependent on each other. There need to be a link from the resource to the subscriptions and vice versa. This means that it is not trivial to e.g. separate the storage of subscriptions from the the resources. The intelinking is becoming a much more complex task when the presence server is deployed as a cluster of servers where each of the servers need to manage part of the overall state. o Moderate assumptions - The size assumptions that were made above are quite moderate. As presence is becoming more a core middleware functionality that holds a lot of data on the user. In real-life the numbers above may be even higher and the presence server can have additional overhead as managing the SIP sessions, networking and more. Although the calculations above do not show that there is a real issue with state management of presence in medium systems, the state size in large and very large systems is very big and may need special technology to enable a presence server to handle the sate efficiently. Houri, et al. Expires April 26, 2007 [Page 21] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 5. Processing complexities The basic presence paradigm consists from a subscriber and a resource to which the subscriber subscribes to. It sounds simple enough but there are many additions and extensions that the presence server has to manage that make the processing of the presence server very complex one. In this section we show that in addition to the large amount of messages and the big state that the presence server has to handle, it has also to handle quite intensive processing for aggregation, partial notify and publish, filtering and privacy. This makes the task for the presence server challenging in all possible fronts: network, memory and cpu. 5.1. Aggregation A presence document does not represent a single resource only but a user may have many resources that update the presence documents. These resources can be devices of the user, external providers of presence information for the user as geographical and calendar information and more. The presence server need to be able to get the updates from all the resource and aggregate them correctly into a single presence document. Although this is just "XML processing" task, the amount of updates that the presence server may get, the need to keep the presence document aligned with its schema and the need to notify the users as soon as possible create a significant processing burden on the presence server 5.2. Partial Publish and Notify Drafts [11], [12] define a way for the subscriber to request getting only what was changed in the presence document and for the publisher of presence information to publish only what was changed in the presence document since the last publish. Although these optimizations help in reducing the amount of actual data that is sent from/to the presence server, these optimizations create additional processing burden on the presence server. When a partial publish is arriving to the presence server, the presence server has to be able to process the partial publish, change only what is indicated in the partial publish while keeping the presence document in a well formed shape according to the schema. In partial notify the processing is even more complex. In partial notify each watcher need to get the partial update based on the last Houri, et al. Expires April 26, 2007 [Page 22] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 update that was received by that watcher. Therefore [11] specifies a versioning mechanism that enables the watcher to get the updates based on the previous state that it has seen. However this versioning mechanism has to be maintained by the presence server for each watcher to the subscribed to resource that requires partial publish. 5.3. Filtering Filtering as defined in RFCs [8], [9] enables a watcher to request to be notified only when the presence document fulfills certain conditions. Although this is a very convenient feature for watchers, the burden that is put on the presence server is quite big. For each change in the presence document, the presence server needs to compute the filtering expressions which can be very complex, decide whether and what to send to the watcher that have requested filtering. 5.4. Privacy Draft [13] defines presence authorization rules that can be used by presentities to define who can see what from their presence documents. The processing that the presence server has to do here is very similar to filtering. When there is a change to any presence document that has filtering defined for it, the presence server needs to create different notification for different watchers according to what is defined in the authorization rules. Houri, et al. Expires April 26, 2007 [Page 23] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 6. Resource List Service RFC [10] defines a way to subscribe on a single URI while that URI is actually a list of resources that are being subscribed to by a single subscription. Although this is quite useful mechanism and it significantly saves on the number of sessions between the watcher and the presence server (as we show in the message calculations), this feature has the potential to make the number of subscriptions in a presence system to grow exponentially. There are two types of groups that may be used with this feature, private groups that are defined for each watcher and public groups that are defined in the system and can be used by any user. Public groups can be a source of making the number of subscriptions in the system grow exponentially. They are convenient to use, usually system administrator do not limit the size of these groups and in many cases we can find public groups that cover almost all the enterprise... Another aspect of the exponentially of resource lists is that as it is defined today by [10] and with the current way that authorization between servers can be done, the RLS needs to subscribe on each resource that appear in the resource list even if that resource has been subscribed to by the RLS dozen times for other resource lists. This way of behavior creates an exponential and complex mesh of subscriptions between RLSs and presence servers. Houri, et al. Expires April 26, 2007 [Page 24] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 7. Summary In the previous sections we have shown several areas where the deployment of a presence system is far from being trivial, these include network load, memory load and CPU load. In this section we are listing an initial set of requirements to a possible optimizations in this area and we also list the existing and some suggested suggestions for opimizations. 7.1. Requirements The following is an initial list requirements for optimizations that seem to be necessary in presence systems: Backward compatibility requirements o The solution should not hinder the ability of existing SIMPLE clients and/or servers from peering with a domain or client implementing the solution. No changes may be required of existing servers to interoperate o It does NOT constrain any existing RFC functional or security requirements for presence o Systems that are not using the new additions to the protocol should operate at the same level as they do today Policy, privacy, permissions requirements o The solution does not limit the ability for presentities to present different views of presence to different watchers o The solution does not restrict the ability of a presentity to obtain its list of watchers o The solution MUST NOT create any new or make worse any existing privacy holes Scalability requirements o It is highly desirable for any presence system (intra or inter- domain) to scale linearly as number of watchers and presentities increase linearly o The solution SHOULD NOT require significantly more state in order to implement the solution Houri, et al. Expires April 26, 2007 [Page 25] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 o It MUST be able to scale to tens of millions of concurrent users in each domain and in each peer domain o It MUST support a very high level of watcher/presentity intersections in various intersection models o Protocol changes MUST NOT prohibit optimizations in different deployment models esp. where there is a high level of cross subscriptions between the domains o New functionalities and extensions to the presence protocol SHOULD take into account scalability with respect to the number of messages, state size and management and processing load. Topology requirement o The solution SHOULD allow for arbitrary federation topologies including direct peering and intermediary routing 7.2. Optimizations This section lists the current optimizations that have been defined, are being worked on or are suggested. o Subnot-etags - Draft [16]. This draft suggests ways to suppress the sending of unnecessary notifies when for example a subscription is refreshed. This suggestion seems to be an efficient optimization since it saves both the number of messages sent and on the processing time of the presence server. o Resource List Service - [10] enable creating a single subscription session between the watcher and the presence server for subscribing on a list of users. This saves the amount of sessions that are created between watchers and presence servers. On the other hand, this mechanism enables creating very large amount of subscriptions in the presence server/RLS system thus enabling the creation of a very large number of subscriptions between presence servers and RLSs with relatively few clients especially if large public groups are used. It seems that in order to really optimize in this area, the usage of large public groups should not be considered as BCP and there should be a way for an RLS to create a single subscription for multiple occurrences of the same resource in resource lists. See consolidates subscriptions below. o Partial notify/publish - Drafts [11], [12] define a way for the subscriber to request getting only what was changed in the presence document and for the publisher of presence information to publish only what was changed in the presence document since the Houri, et al. Expires April 26, 2007 [Page 26] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 last publish. Although these optimizations help in reducing the amount of actual data that is sent from/to the presence server, these optimizations create additional processing burden on the presence server as was discussed above. o Filtering as defined in RFCs [8], [9] enables a watcher to request to be notified only when the presence document fulfills certain conditions. Although this optimization enables saving on the amount of messages that are sent from the presence server to the watcher, this optimization puts more burden on the processing time of the presence server as was discussed above. o Throttling [draft-niemi-sipping-event-throttle-04.txt - expired at the time of the writing of this document] defines a mechanism in which a watcher requires to be updated only in certain intervals. Although this mechanism may give some extra load on the processing time of the presence server, that load is negligible and the reduction on the amount of messages sent from the presence server to the watchers is significant. This optimization is even more important with resource lists where there can be many resources in the resource lists and if the traffic of updates on resource list is not regulated, the watcher may get very large amount of notifications. o Presence specific sigcomp dictionary [15] defines a SIGCOMP [3] dictionary for presence. This optimization will enable to reduce the number of bytes that are transferred in presence systems by compressing the textual SIP messages and using the specialized presence dictionary the compression may be more significant then just using SIGCOMP as is. Note that number of actual messages will remain the same and a calculation of the amount of bytes that will be saved may be useful here. o Content Indirection [7] enables sending only the URI of the presence document to the watcher thus offloading the presence server from sending the presence document to the watcher. This optimization may be useful in some cases but in reality it may have several drawbacks: 1. Due to partial/privacy/filtering and other functionalities, it will be relatively a rare case where many watchers will get exactly the same presence document. 2. There should be a mechanism that will enable removing the content from the content server at the appropriate time. Defining the appropriate time is far from trivial since the removal should be synchronized with all the watcher that need to get the content. Houri, et al. Expires April 26, 2007 [Page 27] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 o Resubscription to resource list [10] requires that a full state will be sent for subside refreshes. In large resource lists the amount of data that needs to be sent for each subscribe refresh may be very big. Having an optimization that will enable sending only partial information at subscribe refreshes may let RLS subscriptions be more optimized. o No Resubscriptions - Due to the nature of SIP that is network agnostic and always assumes the worst for the network layer, resubscriptions are part of the SIP sub/notify model [2]. In many cases it should be possible to negotiate a special connection between watchers and presence servers, this type of connection will use a different mechanism of e.g. keep alives and will not necessitate resubscribes. This will be mostly important between presence domains and between RLSs and presence servers and may save many messages. o Consolidated Subscriptions - In many of cases described in this document there are many subscriptions between peers that may be consolidated into a single subscription. One example of such subscriptions are subscriptions between domains, another example of such subscriptions are subscriptions that are created by the resource list server (RLS) to the presence servers that hold the presence document. The reason for not being able to do consolidation of subscriptions is privacy. A presence server needs to know who is the actual watcher in order to know if and what to send on the subscription. Enabling a single subscription from one presence server or an RLS to another presence server on behalf of many watchers contradicts the privacy considerations but on the other hand this single optimization can have a very big impact of the amount of the subscriptions that are required in the presence system. Some suggestions have been suggested but there is not concrete proposal on how to enable consolidated subscriptions while adhering to the privacy requirments. Houri, et al. Expires April 26, 2007 [Page 28] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 8. Extremely Optimized Model The following calculations are made assuming that the following optimizations are deployed: o No resubscriptions are necessary. o Consolidates Subscriptions are possible. The following table shows the amount of messages that are required in this model using the very large network model numbers. We assume that even though there are 10M watchers from one domain to the other, the number of actually watched resources is only 3M. (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................6 (A03) Subscription refresh interval / hour.......................0 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher..................1 (A06) Number of watchers in a federated presence domain.10,000,000 (A06-1) Number of resources watched......................3,000,000 (A07) Initial SUBSCRIBE/200 per watcher..........................2 (A08) Initial NOTIFY/200 per watcher.............................2 (A09) Total initial messages............................12,000,000 (A10) NOTIFY/200 per watched presentity........................960 (A11) SUBSCRIBE/200 refreshes....................................0 (A12) NOTIFY/200 due to subscribe refresh........................0 (A13) Number of steady state messages................2,880,000,000 (A14) SUBSCRIBE termination......................................2 (A15) NOTIFY terminated..........................................2 (A16) Number of sign-out messages.......................12,000,000 (A17) Total messages between domains.................5,808,000,000 (A18) Total number of messages / second....................201,333 Very large network peering with extreme optimizations Note that we get almost a 3 fold less messages by only assuming that 10M absorbers subscribe to 3M resources while consolidated subscriptions are possible. Note that due to the usage of the subnot-etags [16] optimization the total removal of resubscribes does not save many messages as the following table shows: Houri, et al. Expires April 26, 2007 [Page 29] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 (A01) Subscription lifetime (hours)..............................8 (A02) Presence state changes / hour..............................6 (A03) Subscription refresh interval / hour.......................1 (A04) Total federated presentities per watcher..................10 (A05) Number of dialogs to maintain per watcher..................1 (A06) Number of watchers in a federated presence domain.10,000,000 (A06-1) Number of resources watched......................3,000,000 (A07) Initial SUBSCRIBE/200 per watcher..........................2 (A08) Initial NOTIFY/200 per watcher.............................2 (A09) Total initial messages............................12,000,000 (A10) NOTIFY/200 per watched presentity........................960 (A11) SUBSCRIBE/200 refreshes...................................16 (A12) NOTIFY/200 due to subscribe refresh........................0 (A13) Number of steady state messages................2,928,000,000 (A14) SUBSCRIBE termination......................................2 (A15) NOTIFY terminated..........................................2 (A16) Number of sign-out messages.......................12,000,000 (A17) Total messages between domains.................5,904,000,000 (A18) Total number of messages / second....................205,000 Very large network extreme optimizations+resubscribe "Only" additional 3.5K messages per second are needed if we re- introduce re-subscriptions, since the subnot-etags [16] optimization is used. Houri, et al. Expires April 26, 2007 [Page 30] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 9. Security Considerations This document discusses scalability issues with the existing SIP/ SIMPLE presence protocol and model. Therefore, there are no security considerations to be considered for this document. Houri, et al. Expires April 26, 2007 [Page 31] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 10. Acknowledgments We would like to thank Jonathan Rosenberg, Markus Isomaki and David Viamonte for their ideas and input. Houri, et al. Expires April 26, 2007 [Page 32] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 11. References 11.1. Normative References [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 11.2. Informational References [2] Roach, A., "Session Initiation Protocol (SIP)-Specific Event Notification", RFC 3265, June 2002. [3] Price, R., Bormann, C., Christoffersson, J., Hannu, H., Liu, Z., and J. Rosenberg, "Signaling Compression (SigComp)", RFC 3320, January 2003. [4] Rosenberg, J., "A Presence Event Package for the Session Initiation Protocol (SIP)", RFC 3856, August 2004. [5] Rosenberg, J., "A Watcher Information Event Template-Package for the Session Initiation Protocol (SIP)", RFC 3857, August 2004. [6] Rosenberg, J., "An Extensible Markup Language (XML) Based Format for Watcher Information", RFC 3858, August 2004. [7] Burger, E., "A Mechanism for Content Indirection in Session Initiation Protocol (SIP) Messages", RFC 4483, May 2006. [8] Khartabil, H., Leppanen, E., Lonnfors, M., and J. Costa- Requena, "Functional Description of Event Notification Filtering", RFC 4660, September 2006. [9] Khartabil, H., Leppanen, E., Lonnfors, M., and J. Costa- Requena, "An Extensible Markup Language (XML)-Based Format for Event Notification Filtering", RFC 4661, September 2006. [10] Roach, A., Campbell, B., and J. Rosenberg, "A Session Initiation Protocol (SIP) Event Notification Extension for Resource Lists", RFC 4662, August 2006. [11] Lonnfors, M., "Session Initiation Protocol (SIP) extension for Partial Notification of Presence Information", draft-ietf-simple-partial-notify-08 (work in progress), July 2006. [12] Lonnfors, M., "Publication of Partial Presence Information", draft-ietf-simple-partial-publish-05 (work in progress), Houri, et al. Expires April 26, 2007 [Page 33] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 July 2006. [13] Rosenberg, J., "Presence Authorization Rules", draft-ietf-simple-presence-rules-07 (work in progress), June 2006. [14] Camarillo, G., "Subscriptions to Request-Contained Resource Lists in the Session Initiation Protocol (SIP)", draft-ietf-sipping-uri-list-subscribe-05 (work in progress), May 2006. [15] Garcia-Martin, M., "The Presence-specific Dictionary for the Signaling Compression (Sigcomp) Framework", draft-garcia-simple-presence-dictionary-00 (work in progress), June 2006. [16] Niemi, A., "An Extension to Session Initiation Protocol (SIP) Events for Issuing Conditional Subscriptions", draft-niemi-sip-subnot-etags-01 (work in progress), June 2006. Houri, et al. Expires April 26, 2007 [Page 34] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 Authors' Addresses Avshalom Houri IBM Science Park Building 18/D Rehovot, Israel Email: avshalom@il.ibm.com Tim Rang Microsoft Corporation One Microsoft Way Redmond, WA 98052 USA Email: timrang@microsoft.com Edwin Aoki AOL LLC 360 W. Caribbean Drive Sunnyvale, CA 94089 USA Email: aoki@aol.net Houri, et al. Expires April 26, 2007 [Page 35] Internet-Draft Problem Statement for SIP/SIMPLE October 2006 Full Copyright Statement Copyright (C) The Internet Society (2006). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Intellectual Property The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. Acknowledgment Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). Houri, et al. Expires April 26, 2007 [Page 36]