idnits 2.17.1 draft-kurtis-anycast-bcp-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1.a on line 17. -- Found old boilerplate from RFC 3978, Section 5.5 on line 708. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 685. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 692. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 698. ** The document seems to lack an RFC 3978 Section 5.1 IPR Disclosure Acknowledgement. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. ** The document uses RFC 3667 boilerplate or RFC 3978-like boilerplate instead of verbatim RFC 3978 boilerplate. After 6 May 2005, submission of drafts without verbatim RFC 3978 boilerplate is not accepted. The following non-3978 patterns matched text found in the document. That text should be removed or replaced: This document is an Internet-Draft and is subject to all provisions of Section 3 of RFC 3667. By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 15 longer pages, the longest (page 13) being 71 lines == It seems as if not all pages are separated by form feeds - found 0 form feeds but 16 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack separate sections for Informative/Normative References. All references will be assumed normative when checking for downward references. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year == Line 575 has weird spacing: '...ibed in havin...' == The document doesn't use any RFC 2119 keywords, yet seems to have RFC 2119 boilerplate text. -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (October 22, 2004) is 7120 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Unused Reference: '1' is defined on line 610, but no explicit reference was found in the text == Unused Reference: '3' is defined on line 616, but no explicit reference was found in the text ** Obsolete normative reference: RFC 1142 (ref. '1') (Obsoleted by RFC 7142) ** Downref: Normative reference to an Informational RFC: RFC 1321 (ref. '2') ** Obsolete normative reference: RFC 1247 (ref. '3') (Obsoleted by RFC 1583) ** Downref: Normative reference to an Informational RFC: RFC 1546 (ref. '4') ** Obsolete normative reference: RFC 1771 (ref. '5') (Obsoleted by RFC 4271) ** Obsolete normative reference: RFC 2401 (ref. '7') (Obsoleted by RFC 4301) ** Obsolete normative reference: RFC 2267 (ref. '8') (Obsoleted by RFC 2827) ** Downref: Normative reference to an Informational RFC: RFC 3765 (ref. '11') -- Possible downref: Non-RFC (?) normative reference: ref. '12' -- Possible downref: Non-RFC (?) normative reference: ref. '13' -- Possible downref: Non-RFC (?) normative reference: ref. '14' -- Possible downref: Non-RFC (?) normative reference: ref. '15' Summary: 14 errors (**), 0 flaws (~~), 8 warnings (==), 11 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 GROW K. Lindqvist 2 Internet-Draft Netnod Internet Exchange 3 Expires: April 22, 2005 J. Abley 4 ISC 5 October 22, 2004 7 Operation of Anycast Services 8 draft-kurtis-anycast-bcp-00.txt 10 Status of this Memo 12 This document is an Internet-Draft and is subject to all provisions 13 of section 3 of RFC 3667. By submitting this Internet-Draft, each 14 author represents that any applicable patent or other IPR claims of 15 which he or she is aware have been or will be disclosed, and any of 16 which he or she become aware will be disclosed, in accordance with 17 RFC 3668. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as 22 Internet-Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet-Draft will expire on April 22, 2005. 37 Copyright Notice 39 Copyright (C) The Internet Society (2004). 41 Abstract 43 As the Internet has grown, many services with high availability 44 requirements have emerged. The requirements of these services have 45 increased the demands on the reliability of the infrastructure on 46 which those services rely. 48 Many techniques have been employed to increase the availability of 49 services deployed on the Internet. This document presents 50 operational experience of wide-scale service distribution using 51 anycast, and proposes a series of recommendations for others using 52 this approach. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 3. Anycast Service Distribution . . . . . . . . . . . . . . . . . 4 61 3.1 General Description . . . . . . . . . . . . . . . . . . . 4 62 3.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . 5 64 4. Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 65 4.1 Protocol Suitability . . . . . . . . . . . . . . . . . . . 5 66 4.2 Node Placement . . . . . . . . . . . . . . . . . . . . . . 6 67 4.3 Routing Systems . . . . . . . . . . . . . . . . . . . . . 6 68 4.3.1 Anycast within an IGP . . . . . . . . . . . . . . . . 6 69 4.3.2 Anycast within the Global Internet . . . . . . . . . . 7 70 4.4 Routing Considerations . . . . . . . . . . . . . . . . . . 7 71 4.4.1 Signalling Service Availability . . . . . . . . . . . 7 72 4.4.2 Covering Prefix . . . . . . . . . . . . . . . . . . . 8 73 4.4.3 Equal-Cost Paths . . . . . . . . . . . . . . . . . . . 8 74 4.4.4 Route Dampening . . . . . . . . . . . . . . . . . . . 9 75 4.4.5 Reverse Path Forwarding Checks . . . . . . . . . . . . 10 76 4.4.6 Propagation Scope . . . . . . . . . . . . . . . . . . 10 77 4.4.7 Other Peoples' Networks . . . . . . . . . . . . . . . 11 78 4.5 Data Synchronisation . . . . . . . . . . . . . . . . . . . 11 79 4.6 Node Autonomy . . . . . . . . . . . . . . . . . . . . . . 11 81 5. Service Management . . . . . . . . . . . . . . . . . . . . . . 12 82 5.1 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . 12 83 5.2 Self-Healing Nodes . . . . . . . . . . . . . . . . . . . . 12 85 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 87 7. Protocol Considerations . . . . . . . . . . . . . . . . . . . 13 89 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 91 9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 13 93 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 14 95 Intellectual Property and Copyright Statements . . . . . . . . 16 97 1. Introduction 99 To distribute a service using anycast, the service is first 100 associated with a stable set of IP addresses, and reachability to 101 those addresses is advertised in a routing system from multiple, 102 independent service nodes. Various techniques for anycast deployment 103 of services are discussed in RFC 1546 [4], ISC-TN-2003-1 [12] and 104 ISC-TN-2004-1 [13]. 106 Anycast has in recent years become increasingly popular for adding 107 redundancy to DNS servers. Several root server operators have 108 distributed their servers widely around the Internet, and both 109 resolver and authority servers are commonly distributed within the 110 networks of service providers. Anycast distribution has been used by 111 commercial DNS authority server operators for several years. The use 112 of anycast is not limited to the DNS, although the use of anycast 113 imposes some additional requirements on the nature of the service 114 being distributed, including transaction longevity, transaction state 115 held on servers and data synchronization capabilities. 117 Although anycast is conceptually simple, its implementation 118 introduces some pitfalls for operation of the service. For example, 119 monitoring the availability of the service becomes more difficult; 120 the observed availability changes according to the source of the 121 query, and the client catchment of individual anycast nodes is not 122 static, nor especially deterministic. 124 This document will describe the use of anycast for both local scope 125 distribution of services using an Interior Gateway Protocol (IGP) and 126 global distribution using BGP [5]. Many of the issues for monitoring 127 and data synchronization are common to both, but deployment issues 128 differ substantially. 130 2. Terminology 132 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 133 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 134 document are to be interpreted as described in RFC 2119. 136 Service Address: an IP address associated with a particular service 137 (e.g. the address of a nameserver). 138 Anycast: the practice of making a particular Service Address 139 available in multiple, discrete, autonomous locations, such that 140 datagrams sent are routed to one of several available locations. 141 Anycast Node: an internally-connected collection of hosts and routers 142 which together provide service for an anycast service address. 144 Local-Scope Anycast: reachability information for the anycast service 145 address is propagated through a routing system in such a way that 146 a particular anycast node is only visible to a subset of the whole 147 routing system. 148 Local Node: an Anycast Node providing service using a Local-Scope 149 Anycast address. 150 Global-Scope Anycast: reachability information for the anycast 151 service address is propagated through a routing system in such a 152 way that a particular anycast node is potentially visible to the 153 whole routing system. 154 Global Node: an Anycast Node providing service using a Global-Scope 155 Anycast address. 157 3. Anycast Service Distribution 159 3.1 General Description 161 Anycast is the name given to the practice of making one or more 162 Service Addresses available to a routing system at Anycast Nodes in 163 two or more discrete locations. The service provided by each node is 164 necessarily consistent regardless of the particular node chosen by 165 the routing system to handle a particular request. 167 For services distributed using anycast, there is no inherent 168 requirement for referrals to other servers or name-based service 169 distribution ("round-robin DNS"), although those techniques could be 170 combined with anycast service distribution if an application required 171 it. The routing system makes the decision of the node to be used for 172 each request, based on the topological design of the routing system 173 and the point in the network at which the request originates. 175 The Anycast Node chosen to service a particular query can be 176 influenced by the traffic engineering capabilities of the routing 177 protocols which make up the routing system. The degree of influence 178 available to the operator of the node depends on the scale of the 179 routing system within which the Service Address is anycast. 181 Load-balancing between Anycast Nodes is typically difficult to 182 achieve (load distribution between nodes is generally unbalanced in 183 terms of request and traffic load). Distribution of load between 184 nodes for the purposes of reliability, and coarse-grained 185 distribution of load for the purposes of making popular services 186 scalable can often be accommodated, however. 188 The scale of the routing system through which a service is anycast 189 can vary from a small Interior Gateway Protocol (IGP) connecting a 190 small handful of components, to the Border Gateway Protocol (BGP) [5] 191 connecting the global Internet, depending on the nature of the 192 service distribution that is required. 194 3.2 Goals 196 A service may be anycast for a variety of reasons. A number of 197 common objectives are: 199 1. Coarse ("unbalanced") distribution of load across nodes, to allow 200 infrastructure to scale to increased numbers of queries and to 201 accommodate transient query peaks; 202 2. Mitigation of non-distributed denial of service attacks by 203 localizing damage to single anycast nodes; 204 3. Constraint of distributed denial of service attacks or flash 205 crowds to local regions around anycast nodes (perhaps restricting 206 query traffic to local peering links, rather than paid transit 207 circuits); 208 4. Triangulation of traffic sources, in the case of attack (or 209 query) traffic which incorporates spoofed source addresses; 210 5. Improvement of query response time, by reducing the network RTT 211 between client and server with the provision of a local Anycast 212 Node. 213 6. Reduction of a list of servers to a single, distributed address. 214 For example, a large number of authoritative nameservers for a 215 zone may be deployed using a small set of anycast service 216 addresses; this approach can increase the accessibility of zone 217 data in the DNS without increasing the size of a referral 218 response from a parent nameserver. 220 4. Design 222 4.1 Protocol Suitability 224 When a service is anycast between two or more nodes, the routing 225 system effectively makes the node selection decision on behalf of a 226 client. Since it is usually a requirement that a single 227 client-server interaction is carried out between a client the same 228 server node for the duration of the transaction, it follows that the 229 routing system's node selection decision ought to be stable for an 230 order of magnitude longer than the expected transaction time, if the 231 service is to be provided reliably. 233 Some services have very short transaction times, and may even be 234 carried out using a single packet request and a single packet reply 235 in some cases (the DNS is an example of this). Other services 236 involve far longer-lived transactions (e.g. bulk file downloads and 237 audio-visual media streaming). 239 Some anycast deployments have very predictable routing systems, which 240 can remain stable for long periods of time (e.g. anycast within an 241 IGP, where node selection changes only occur as a response to node 242 failures). Other deployments have far less predictable 243 characteristics (e.g. a densely-deployed array of nodes across the 244 global Internet). 246 The stability of the routing system together with the transaction 247 time of the service should be carefully compared when deciding 248 whether a service is suitable for distribution using anycast. 250 4.2 Node Placement 252 Decisions as to where Anycast Nodes should be placed will depend to a 253 large extent on the goals of the service distribution. For example: 255 o A recursive resolver service might be distributed within an ISP's 256 network, one Anycast Node per PoP. 257 o A root server service might be distributed throughout the Internet 258 with nodes located in regions with poor external connectivity, to 259 ensure that the DNS functions adequately within the region during 260 times of external network failure. 261 o An FTP mirror service might include local nodes located at 262 exchange points, so that ISPs connected to that exchange point 263 could download bulk data more cheaply than if they had to use 264 expensive transit circuits. 266 In general node placement decisions should be made with consideration 267 of likely traffic requirements, the potential for flash crowds or 268 denial-of-service traffic, the stability of the local routing system 269 and the failure modes with respect to node failure, or local routing 270 system failure. 272 4.3 Routing Systems 274 4.3.1 Anycast within an IGP 276 There are several common motivations for the distribution of a 277 Service Address within the scope of an IGP: 279 1. to improve service response times, by hosting a service close to 280 other users of the network; 281 2. to improve service reliability by providing automatic fail-over 282 to backup nodes; and 283 3. to keep service traffic local, to avoid congesting wide-area 284 links. 286 In each case the decisions as to where and how services are 287 provisioned can be made by network engineers without requiring such 288 operational complexities as regional variances in the configuration 289 of client computers, or DNS tricks which respond differently to 290 requests from clients in different locations. 292 When a service is anycast within an IGP the routing system is 293 typically under the control of the same organization who is providing 294 the service, and hence the relationship between service transaction 295 characteristics and network stability are likely to be relatively 296 well-understood. This technique is consequently applicable to a 297 larger number of applications than Internet-wide anycast service 298 distribution (see Section 4.1). 300 By reducing the scope of the IGP to just the hosts providing service 301 (together with one or more gateway routers) this technique can be 302 applied to the construction of server clusters. This application is 303 discussed in some detail in [13]. 305 4.3.2 Anycast within the Global Internet 307 Service Addresses may be anycast within the global Internet routing 308 system in order to distribute services across the entire network. 309 The principal differences between this application and the IGP-scope 310 distribution discussed in Section 4.3.1 are that: 312 1. the routing system is, in general, controlled by other people; 313 and 314 2. the routing protocol concerned (BGP), and commonly-accepted 315 practices in its deployment, impose some additional constraints 316 (see Section 4.4). 318 4.4 Routing Considerations 320 4.4.1 Signalling Service Availability 322 When a routing system is provided with reachability information for a 323 Service Address from an individual node, packets addressed to that 324 Service Address will start to arrive at the node. Since it is 325 desirable for the node to be ready to accept requests before they 326 start to arrive, a coupling between the routing information and the 327 availability of the service at a particular node is desirable. 329 Where a routing advertisement from a node corresponds to a single 330 Service Address, this coupling might be such that availability of the 331 service triggers the route advertisement, and non-availability of the 332 service triggers a route withdrawal. This can be achieved using 333 routing protocol implementations on the same servers which provide 334 the service being distributed. 336 Where a routing advertisement from a node corresponds to two or more 337 Service Addresses, it may not be appropriate to trigger a route 338 withdrawal due to the non-availability of a single service. Another 339 approach is to tunnel requests from nodes that cannot handle 340 individual services to other nodes that can, perhaps using an IGP 341 which extends over tunnels between nodes, in which servers 342 participate. Circumstances which might lead to multiple Service 343 Addresses being covered by a single route are discussed in Section 344 4.4.2. 346 4.4.2 Covering Prefix 348 In some routing systems (e.g. the BGP-based routing system of the 349 global Internet) it is not possible, in general, to propagate a host 350 route with confidence that availability of the route will be signaled 351 throughout the network. This is a consequence of operational policy, 352 and not a protocol restriction. 354 In such cases it is necessary to propagate a route which covers the 355 Service Address, and which has a sufficiently short prefix that it 356 will not be caught by commonly-deployed import policies. In many 357 cases this will be a 24-bit prefix, but there are other 358 well-documented examples of import polices which filter on RIR 359 allocation boundaries, and hence some experimentation may be prudent. 361 Where multiple Service Addresses are covered by the same covering 362 route, there is no longer a tight coupling between the advertisement 363 of that route and the individual services associated with the covered 364 host routes. The resulting impact on signaling availability of 365 individual services is discussed in Section 4.4.1. 367 4.4.3 Equal-Cost Paths 369 Some routing systems support equal-cost paths to the same 370 destination. Where multiple, equal-cost paths exist and lead to 371 different anycast nodes, there is a risk that request packets 372 associated with a single transaction might be delivered to more than 373 one node. Services provided over TCP necessarily involve 374 transactions with multiple request packets, due to the TCP setup 375 handshake. 377 Equal cost paths are commonly supported in IGPs. Multi-node 378 selection for a single transaction can be avoided in most cases by 379 careful consideration of IGP link metrics, or by applying equal-cost 380 multi-path (ECMP) selection algorithms which cause a single node to 381 be selected for a single multi-packet transaction. For a description 382 of hash-based ECMP selection, see [13]. 384 For services which are distributed across the global Internet using 385 BGP, equal-cost paths are normally not a consideration: BGP's exit 386 selection algorithm usually selects a single, consistent exit for a 387 single destination regardless of whether multiple candidate paths 388 exist. Implementations of BGP exist that support multi-path exit 389 selection, however, and corner cases where dual selected exits route 390 to different nodes are possible. Analysis of the likely incidence of 391 such corner cases for particular distributions of Anycast Nodes are 392 recommended for services which involve multi-packet transactions. 394 4.4.4 Route Dampening 396 Frequent advertisements and withdrawals of individual prefixes in BGP 397 are known as flaps. Rapid flapping can lead to CPU exhaustion on 398 routers quite remote from the source of the instability, and for this 399 reason rapid route oscillations are frequently "damped", as described 400 in [9]. 402 A dampened path will be suppressed by routers for an interval which 403 increases according to the frequency of the observed oscillation; a 404 suppressed path will not propagate. Hence a single router can 405 prevent the propagation of a flapping prefix to the rest of an 406 autonomous system, affording other routers in the network protection 407 from the instability. 409 Common implementations of flap dampening penalizes oscillating 410 advertisements based on the observed AS_PATH, and not on the NLRI. 411 For this reason, network instability which leads to route flapping 412 from a single anycast node ought not to cause advertisements from 413 other nodes (which have different AS_PATH attributes) to be dampened. 415 As dampening works on advertisements with the same AS_PATH attribute, 416 care should be taken so that the AS_PATH is as diverse as possible 417 for the anycasted nodes. The Anycasted nodes should have the same 418 origin AS for their advertisements, but they should have different 419 upstream AS:es for each node. If the upstream AS is the same at all 420 locations, there is a risk that the upstream AS will peer with the 421 AS:es at multiple locations and could therefor propagate the same 422 AS_PATH, but for different Anycast nodes. This could render the 423 service for multiple Anycast nodes unavailable due to dampening 424 caused by only one of them. 426 It is possible that other implementations of flap dampening may 427 become prevalent in the future, causing individual nodes' instability 428 to result in stable nodes becoming unavailable. Judicious deployment 429 of Local Nodes in combination with especially stable Global Nodes 430 may help mitigate such problems, should they ever arise. 432 4.4.5 Reverse Path Forwarding Checks 434 Reverse Path Forwarding (RPF) checks, first described in [8], are 435 commonly deployed as part of ingress interface packet filters on 436 routers in the global Internet in order to deny packets whose source 437 addresses are spoofed (see also [10]). Deployed implementations of 438 RPF make available two modes of operation: a loose mode, and a strict 439 mode. 441 Strict-mode RPF checks can cause non-spoofed packets to be denied 442 when they originate from multi-homed site, since selected paths might 443 legitimately not correspond with the ingress interface of non-spoofed 444 packets from the multi-homed site. A collection of anycast nodes 445 deployed across the Internet is largely indistinguishable from a 446 distributed, multi-homed site to the routing system, and hence this 447 risk also exists for anycast nodes, even if individual nodes are not 448 multi-homed. 450 Care should be taken to ensure that strict-mode RPF is not enabled in 451 peer networks connecting to anycast nodes. 453 4.4.6 Propagation Scope 455 In the context of Anycast service distribution across the global 456 Internet, Global Nodes are those which are capable of providing 457 service to clients anywhere in the network; reachability information 458 for the service is propagated globally, without restriction, by 459 advertising the routes covering the Service Addresses for global 460 transit to one or more providers. 462 More than one Global Node can exist for a single service (and indeed 463 this is often the case, for reasons of redundancy and load-sharing). 465 In contrast, it is sometimes desirable to deploy an Anycast Node 466 which only provides services to a local catchment of autonomous 467 systems, and which is deliberately not available to the entire 468 Internet; such nodes are referred to in this document as Local Nodes. 469 An example of circumstances in which a Local Node may be appropriate 470 are nodes designed to serve a region with rich internal connectivity 471 but unreliable, congested or expensive access to the rest of the 472 Internet. 474 Local Nodes advertise covering routes for Service Addresses in such a 475 way that their propagation is restricted. This might be done using 476 well-known community string attributes such as NO_EXPORT [6] or 477 NOPEER [11], or by arranging with peers to apply a conventional 478 "peering" import policy instead of a "transit" import policy, or some 479 suitable combination of measures. 481 4.4.7 Other Peoples' Networks 483 When Anycast services are deployed across networks operated by 484 others, their reachability is dependent on routing polices and 485 topology changes (planned and unplanned) which are unpredictable and 486 sometimes difficult to identify. Consequently, routing policies used 487 by Anycast Nodes should be conservative, individual nodes' internal 488 and external/connecting infrastructure should be scaled to support 489 loads far in excess of the average, and the service should be 490 monitored proactively (Section 5.1) from many points in order to 491 avoid unpleasant surprises. 493 4.5 Data Synchronisation 495 As a client contacting a anycasted service will expect all possible 496 servers to serve the same data, the Anycast service needs to assure 497 data consistency across all Anycast Nodes. This includes periodic 498 updating of all data, and verification of a successful transfer of 499 data. 501 How data is synchronized depends on the service being Anycasted. The 502 methods used could for example be a zone transfer for an 503 authoritative set of DNS-servers, rsync for a FTP archive or no 504 synchronization needed for a DNS resolver service. In the DNS 505 examples, synchronization comes with the service and the associated 506 protocol. For other services, this will be an external mechanism to 507 the protocol. In both cases, the synchronization needs to be run 508 from a local IP address that is not the service address. The data 509 transfer should be authenticated in order to prevent spoofing of the 510 data on the Anycasted nodes and the data should be verified. 512 Verification can be done with for example TSIG for DNS, or for 513 example a MD5 hash[2] for verification of other data. The method 514 might vary but should verify that all data was transfered, and that 515 the data is correct and not manipulated. 517 Authentication of the data source can be based either on the protocol 518 in use, as is the case with TSIG for DNS, or some other external 519 mechanism. For example a IP tunnel protected by authentication and 520 encryption as described in [7]. 522 4.6 Node Autonomy 524 For an Anycast deployment whose goals include improved reliability 525 through redundancy, it is important to minimize the opportunity for a 526 single defect to compromise many (or all) nodes, or for the failure 527 of one node to provide a cascading failure bringing down additional 528 successive nodes until the service as a whole is defeated. 530 Codependencies are avoided by making each node as autonomous and 531 self-sufficient as possible. The degree to which nodes can survive 532 failure elsewhere depends on the nature of the service being 533 delivered, but for services which accommodate disconnected operation 534 (e.g. the timed propagation of changes between master and slave 535 servers in the DNS) a high degree of autonomy can be achieved. 537 The possibility of cascading failure due to load can also be reduced 538 by the deployment of both Global and Local Nodes for a single 539 service, since the effective fail-over path of traffic is, in 540 general, from Local Node to Global Node; traffic that might sink one 541 Local Node is unlikely to sink all Local Nodes, except in the most 542 degenerate cases. 544 The chance of cascading failure due to a software defect in an 545 operating system or server can be reduced in many cases by deploying 546 nodes running different software implementations. 548 5. Service Management 550 5.1 Monitoring 552 Monitoring a service which is distributed is more complex than 553 monitoring a non-distributed service, since the observed accuracy and 554 availability of the service is, in general, different when viewed 555 from clients attached to different parts of the network. When a 556 problem is identified, it is also not always obvious which node 557 served the request, and hence which node is malfunctioning. 559 It is recommended that distributed services are monitored from probes 560 distributed representatively across the routing system, and, where 561 possible, the identity of the node answering individual requests is 562 recorded along with performance and availability statistics. 564 Monitoring the routing system (from a variety of places, in the case 565 of routing systems where perspective counts) can also provide useful 566 diagnostics for troubleshooting service availability. This can be 567 achieved using dedicated probes, or public route measurement 568 facilities on the Internet such as RIPE's Routing 569 Information Service [14] and the University of 570 Oregon Route 571 Views Project [15]. 573 5.2 Self-Healing Nodes 575 As is described in having the Anycast Node avoid black-holing 576 traffic in the event of a failure on the software or subsystem 577 providing the service should be avoided. As described, this can be 578 done with withdrawing the announcement of the prefix corresponding to 579 the service address, or the covering route. However, the nodes could 580 also try and handle the failure in a number of ways. This can be 581 with as also previously described tunneling to other instances of the 582 Anycasted service, and using a IGP over the tunnels, route incoming 583 client queries to the other destination. The Anycasted node could 584 also contain separate systems for trying to restart the service in 585 question, and if successful again re-announce the service prefix. 587 6. Security Considerations 589 This document describes mechanisms for deploying services on the 590 Internet which can be used to mitigate vulnerability to attack. 592 The distribution of a service across several (or many) autonomous 593 nodes imposes an increased monitoring load on the operator of the 594 service, and which also imposes an additional systems administration 595 load on the service operator which might reduce the effectiveness of 596 host and router security. It is recommended that these factors be 597 taken into account when assessing the risks and benefits of 598 distributing services using anycast. 600 7. Protocol Considerations 602 This document does not impose any protocol considerations. 604 8. IANA Considerations 606 This document requests no action from IANA. 608 9 References 610 [1] Oran, D., "OSI IS-IS Intra-domain Routing Protocol", RFC 1142, 611 February 1990. 613 [2] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, April 614 1992. 616 [3] Moy, J., "OSPF Version 2", RFC 1247, July 1991. 618 [4] Partridge, C., Mendez, T. and W. Milliken, "Host Anycasting 619 Service", RFC 1546, November 1993. 621 [5] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", 622 RFC 1771, March 1995. 624 [6] Chandrasekeran, R., Traina, P. and T. Li, "BGP Communities 625 Attribute", RFC 1997, August 1996. 627 [7] Kent, S. and R. Atkinson, "Security Architecture for the 628 Internet Protocol", RFC 2401, November 1998. 630 [8] Ferguson, P. and D. Senie, "Network Ingress Filtering: 631 Defeating Denial of Service Attacks which employ IP Source 632 Address Spoofing", RFC 2267, January 1998. 634 [9] Villamizar, C., Chandra, R. and R. Govindan, "BGP Route Flap 635 Damping", RFC 2439, November 1998. 637 [10] Ferguson, P. and D. Senie, "Network Ingress Filtering: 638 Defeating Denial of Service Attacks which employ IP Source 639 Address Spoofing", BCP 38, RFC 2827, May 2000. 641 [11] Huston, G., "NOPEER Community for Border Gateway Protocol (BGP) 642 Route Scope Control", RFC 3765, April 2004. 644 [12] Abley, J., "Hierarchical Anycast for Global Service 645 Distribution", March 2003, 646 . 648 [13] Abley, J., "A Software Approach to Distributing Requests for 649 DNS Service using GNU Zebra, ISC BIND 9 and FreeBSD", March 650 2004, . 652 [14] 654 [15] 656 Authors' Addresses 658 Kurt Erik Lindqvist 659 Netnod Internet Exchange 660 Bellmansgatan 30 661 118 47 Stockholm 662 Sweden 664 EMail: kurtis@kurtis.pp.se 665 URI: http://www.netnod.se/ 666 Joe Abley 667 Internet Systems Consortium, Inc. 668 950 Charter Street 669 Redwood City, CA 94063 670 USA 672 Phone: +1 650 423 1317 673 EMail: jabley@isc.org 674 URI: http://www.isc.org/ 676 Intellectual Property Statement 678 The IETF takes no position regarding the validity or scope of any 679 Intellectual Property Rights or other rights that might be claimed to 680 pertain to the implementation or use of the technology described in 681 this document or the extent to which any license under such rights 682 might or might not be available; nor does it represent that it has 683 made any independent effort to identify any such rights. Information 684 on the procedures with respect to rights in RFC documents can be 685 found in BCP 78 and BCP 79. 687 Copies of IPR disclosures made to the IETF Secretariat and any 688 assurances of licenses to be made available, or the result of an 689 attempt made to obtain a general license or permission for the use of 690 such proprietary rights by implementers or users of this 691 specification can be obtained from the IETF on-line IPR repository at 692 http://www.ietf.org/ipr. 694 The IETF invites any interested party to bring to its attention any 695 copyrights, patents or patent applications, or other proprietary 696 rights that may cover technology that may be required to implement 697 this standard. Please address the information to the IETF at 698 ietf-ipr@ietf.org. 700 Disclaimer of Validity 702 This document and the information contained herein are provided on an 703 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 704 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 705 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 706 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 707 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 708 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 710 Copyright Statement 712 Copyright (C) The Internet Society (2004). This document is subject 713 to the rights, licenses and restrictions contained in BCP 78, and 714 except as set forth therein, the authors retain all their rights. 716 Acknowledgment 718 Funding for the RFC Editor function is currently provided by the 719 Internet Society.