INTERNET-DRAFT M. Shand IPv6 Work in Progress Digital Equipment Co. Ltd. M. Thomas Digital Equipment Corp. Expiration Date: Aug. 1996 19 Feb. 1996 Multi-homing Support in IPv6 Status of this Memo This document is a submission to the IPng Working Group of the Internet Engineering Task Force (IETF). Comments should be submitted to the ipng@sunroof.eng.sun.com mailing list. This document is not at this time a product of the IPng Working Group, but a proposal to become a product of the IPng Working Group. This document is an Internet-Draft. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as ``work in progress.'' To learn the current status of any Internet-Draft, please check the ``1id-abstracts.txt'' listing contained in the Internet- Drafts Shadow Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast). Distribution of this document is unlimited. Abstract This document defines various forms of multihoming, identifies the requirements associated with each and suggests means by which they could be supported in the IPv6 environment. The intention is to provoke discussion, leading to a general consensus as to which scenarios and requirements are important, and which potential solutions are appropriate for further study. Expires August 1996 [Page 1] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 1. Introduction The development of the new version of the Internet Protocol, IPv6, while adhering to many of the architectural principles of IPv4, nevertheless requires the definition of new pieces of architecture and this offers the opportunity to provide support for features which have not been available in IPv4. One such feature is multihoming. Multihoming is currently a vaguely defined term. One of the objectives of this paper is to provide some well understood and agreed taxonomy with which to discuss the subject. At its simplest, multihoming means a node having multiple addresses and or multiple interfaces. There are many existing reasons for requiring multiple addresses on a host, and the desire to employ multiple interfaces for reasons of resilience and bandwidth sharing is increasingly common. In this document we identify the possible topologies and addressing configurations associated with multihoming, and analyze which of these are important. Suggested mechanisms by which these configurations may be supported are then developed. 2. Types of Multihoming In order to facilitate discussion, a number of different types of multihoming are identified. Some features are common to multiple types. 2.1 Type 1 (Single Interface) This is the simplest form of multihoming. The host has a single interface, which has multiple IP addresses. Each address may be used independently. Transmission of a packet from a type 1 multihomed host requires a decision about which source address to use and possibly which destination address (since there may be some correlation between which address is used to reach a destination and which address the destination should use for the return traffic). There are two subtypes associated with this type, depending on the surrounding network environment. 2.1.1 Type 1a (Equivalent addresses) In this sub type the multiple addresses are exactly equivalent. i.e. packets addressed to or from the interface are routed identically Expires August 1996 [Page 2] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 irrespective of which address is used. In this case the choice of source address for an initial packet is arbitrary. However for subsequent packets associated with the same "conversation" (e.g. TCP connection or UDP exchange) there will most likely be a requirement to select a source address for a responding packet which is the same as the destination address of the received packet. ============+================= | |A::x,B::y H 2.1.2 Type 1b (Non-equivalent addresses) In this sub type the choice of source address may be important even for an initial packet. For example, if the multiple addresses correspond to multiple service provider prefixes, then the choice of source address may reflect the desired service provider. In particular, it may influence the return destination address, and hence which service provider is used for the return traffic. It may be necessary to select different source addresses when establishing conversations with different destinations. In some cases this may be to ensure optimal routing. Communication would still be possible if a different address were chosen. In other case certain sources may be completely unreachable from certain destinations, and the correct pairing must be selected before communication is possible. Provider A Provider B | | R1 R2 | | ====+=======+=======+========= | |A::x,B::y H 2.2 Type 2 (Multiple Interfaces - Multiple Addresses) Here the host has multiple interfaces and each interface has one or more independent addresses. In its simplest case each interface has exactly one address, and that simplification will be assumed here for Expires August 1996 [Page 3] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 clarity. Where there are multiple addresses per interface, the type 1 classification can be applied to each interface in addition to the type 2 classification described below. Transmission of a packet from a type 2 multihomed host requires a decision about which interface to use and also about which address pair should be used over that interface (although in the simple single address per interface case the former may imply the latter). Rather like type 1 multihoming there are two sub-types 2.2.1 Type 2a (Equivalent addresses) In this sub type each interface has a distinct address, but the addresses (and hence the interfaces) are exactly equivalent. For example a host may have multiple interfaces onto the same physical subnetwork. ============+=======+========= \ / A::x \ / B::y \ / H 2.2.1.1 Type 2a' (Single Address) A variant of the above configuration theoretically exists in which both interfaces have the same IP address (or set of addresses), but retain distinct link-layer addresses. Note that to deal effectively with this case would require that neighbor discovery caches allowed multiple link-layer addresses for a single IP address. However it may be possible to deal with this sub-type as a special case of type 3 configurations (see below). The case where the two interfaces have both the same IP address and the same link-layer address is not interesting. It is essentially a single dual ported interface. 2.2.2 Type 2b (Non-equivalent addresses) Here the addresses are not equivalent. The most common example of this configuration is the case where the multiple interfaces are connected to different physical subnetworks, and hence the destination address used when sending a packet to the host determines which physical interface is used to receive the packet. Expires August 1996 [Page 4] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 Similarly the source address used when transmitting packets may determine which interface is used to transmit the packet. (It may be possible to transmit a packet over interface (a) using the source IP address associated with interface (b). The result would probably be that any responding packets would be addressed to, and received over, interface (b) ). ============+============= | |A::x H |B::y | ============+============= The set of addresses reachable in the rest of the network from the two interfaces may be: 1. Identical i.e. any address reachable over interface A is also reachable over interface B 2. Completely distinct i.e. any address reachable over interface A is unreachable over interface B and vice versa. 3. Partially overlapping Clearly in cases 1 and 3 the reachability may be equivalent, or one path may be more optimal. Which path is optimal may depend on the destination address. Case 2 might arise typically in firewall applications where the two interface have deliberately been assigned to distinct partitions of the network. 2.3 Type 3 (High Resilience Multiple Interfaces) Here the host has multiple interfaces (typically two, although more are possible) which are functionally equivalent, but have been dupli- cated for reasons of resilience to failure. A typical configuration Expires August 1996 [Page 5] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 might be as illustrated below. ========+========+==============+======== LAN1 | | | | | | H1 H2 R----------....----D | | | | | | ========+========+==============+======== LAN2 Two hosts H1, and H2 are both connected to LAN1 and LAN2. They can communicate with each other using either LAN1 or LAN2. In the case of the failure of one LAN (or the interface connecting either host to that LAN) communication is still possible using the other LAN. Typically these LANs (often called dual rail LANs) would both be con- nected to one or more routers, which provide the connection to the rest of the network and between the LANs. Even if one interface on each host should fail, such that there is no common LAN, communica- tion should still be possible via the router(s). Some other host (D) in some other part of the network may require to communicate with the multihomed hosts via one or more routers. The following are common requirements for configurations of this type (although not all requirements may exist in all situations, and some requirements may be very difficult, if not impossible to achieve in practice). It is not the intention that any solution adopted should necessarily be required to meet all these requirements. The word "conversation" is used to represent both TCP connections and multi packet UDP pseudo-connections. 1. Failure of either LAN must not prevent establishment of new conversations. i.e. they can be established over the alternate LAN. 2. Failure of either LAN during a conversation must not require the re-establishment of that conversation. i.e. it must be possible to move traffic from one LAN to the other without breaking the connection. 3. A remote host (i.e. D) must not require special knowledge that Expires August 1996 [Page 6] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 the hosts H1 and H2 are operating in dual rail environment. i.e. the algorithms for the multihomed and non-multihomed cases must be the same. 4. When both LAN1 and LAN2 are operational the traffic must be divided between them. Note that this does not necessarily imply that traffic for a single conversation must be split (e.g. by sending alternate packets over each LAN). The load splitting could be achieved by assigning each conversation to a single LAN, but distributing the assignments between the two LANs. Exact load balancing is not required, but it is not acceptable for one LAN to be approaching saturation, while the other remains idle. 5. As 4 but path splitting of a single conversation is explicitly forbidden (perhaps to avoid the possibility of out of order packet delivery). 6. As 4, but path splitting of a single conversation is required. 7. As 4 and 6, but dynamic load balancing is required. i.e. the distribution of the packets between the LANs should be dynami- cally adjusted to maintain approximately equal loading on each LAN. 8. The traffic is required to be partitioned between the LANs according to some policy. A typical example might be for a stock trading system. Under no fault conditions LAN1, is used exclusively by the live data, and LAN2 is used exclusively for management and other non-revenue generating traffic. Under failure conditions, LAN2 is permitted to be used as a backup for live data, but under no circumstance is LAN1 to be used for non-revenue earning traffic. Note that requirement of this type are mutually exclusive with requirements of types 4 to 7. 9. Failover from one LAN to the other must be rapid (of the order of one second). i.e. the service interruption time must be short and bounded. 10. Where path splitting or load balancing is a requirement, and traffic has been moved off a LAN because of failure (or under Expires August 1996 [Page 7] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 operator request), the traffic must be returned reasonably quickly (say within a minute) once the LAN is again operational. 3. Some possible solutions Since the type 3 case is the most interesting and potentially impor- tant, we will focus on that case. The essential difficulty lies in the fact that in order to treat the multiple interfaces as completely equivalent it is necessary to allow data to and from the host to be received and transmitted over either interface. However, the higher level protocols use the addressing information associated with each interface to identify conversation streams. For example TCP includes the source and destination IP addresses as part of the connection identifiers. Similar consideration apply to UDP based protocols. There are essentially three possible approaches to solving this prob- lem, each of which may exist in multiple variants. a) Use a single IP address without modification of the TCP/UDP pro- tocols b) Use multiple IP addresses, but pass some other identification in the IP packet to uniquely identify the end point. c) Use multiple IP addresses, but modify TCP/UDP to deal with them. We examine these in more detail below. 3.1 Using a single IP address 3.1.1 Multi-lan subnets The IP (and IPv6) architecture effectively precludes this option. However it would theoretically be possible to allow a single subnet to span multiple physical subnetworks. It would be necessary for the routers attached to those subnetworks to maintain host routes (i.e. full addresses rather than subnet prefixes) for the multihomed addresses. To achieve this they would have to perform neighbor discovery on all the possible physical subnetworks and choose the interface for forwarding on the basis of which addresses were visible on which subnetwork. Where necessary they would also have to exchange host routes with neighboring routers. However the scope of this exchange could be limited to the immediate location of the multi-lan subnet. Outside of this domain, the host routes can be summarized under the subnet prefix. To allow non-multihomed hosts to operate in this environment without Expires August 1996 [Page 8] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 modification it would be necessary for the routers to provide proxy neighbor solicitation responses for those hosts visible only on the other subnetwork. A multihomed host could have a conventional single subnet address on each interface to allow specific per interface addressing, (for exam- ple to ensure that traffic is confined to a specific subnetwork, or for diagnostic purposes) in addition to the single address which is present on both subnetworks. 3.1.2 Routing Hosts Another alternative is for the multihomed hosts to appear as routers with the single address present on a private subnet e.g. ============+=================== | |a::x (H)- c::z |b::y | ============+=================== In this scenario, the private subnet (C::) only exists local to the host (H). A separate subnet is required for each multihomed host, since the routing protocols will route all traffic for the subnet to the associated host. There are a number of problems with this approach. a) The host must participate (at least to some extent) in the rout- ing protocols. At its simplest this would mean sending RIP advertisements. However the use of RIP as a routing protocol in this environment doesn't meet the rapid failover requirements. A more rapidly converging routing protocol, such as OSPF, would be a major burden on the hosts, and would also increase the com- plexity of the entire LSA database for the genuine routers. b) The use of a separate subnet for each multihomed host seems a profligate waste of addressing space. Although this is much less of an issue with IPv6 than with IPv4, it still seems undesir- able. c) Each multihomed host would still necessitate an entry (for its Expires August 1996 [Page 9] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 private subnet this time, rather than for its full address, but there is a one to one correspondence between the two) in the routing tables. As before these could all be summarized to a single entry for distribution outside of the dual rail LAN environment provided the private subnets were suitably struc- tured. d) Like the previous proposal, this goes against the conventional wisdom of the IP architecture (See for example the architectural assumptions in RFC 1122 Section 1.1.2 (c) quoted below. "Routing complexity should be in the gateways. Routing is a complex and difficult problem, and ought be performed by the gateways, not the hosts. An important objective is to insulate host software from changes by the inevitable evolution of the Internet architecture." 3.1.3 Using an anycast address In some ways the semantics associated with the single address of the multihomed host resemble those of an anycast address. i.e. the packet should be delivered at least once to the nearest instance of the address. It might therefore be possible to use an anycast address as the destination address of the multihomed node. The scope of an anycast address can be controlled by its prefix, and is permit- ted to be wider than a single LAN. However there are the following problems. 1. The use of an anycast address for host addressing is currently deprecated. This restriction is only in place until the use of anycast addresses is more fully understood. It could potentially be lifted for this use. 2. More seriously the use of anycast addresses as source addresses is forbidden. This is with good reason, since the anycast address does not uniquely identify a source of the transmission. In the case of the normal use of anycast addresses this has the potential to cause untold confusion, since multiple hosts might attempt to participate in a single TCP connection. However, in this specialized context it could be guaranteed that the sources were indeed the same entity and confusion would not arise. It might therefore be possible to allow the use of an anycast address as a source address under these constraints. Expires August 1996 [Page 10] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 Alternatively it might be possible to define another address range similar to the anycast range, but with the modified con- straints. 3. Delivery semantics do not preclude the delivery to multiple own- ers of the same anycast address. It is not clear how anycast routing will actually work, but it would clearly be unsatisfac- tory if all packets were delivered to all interfaces. 4. Anycast addressing would appear to require routing information to be held for the individual anycast addresses, at least within the scope of the anycast address. i.e. the routing issues are the same as with the previous two suggestions. The subnet prefix chosen for the anycast address could follow one of two possible models. 1. The anycast subnet prefix could be the same as the natural sub- net prefix of one of the LANs. In this case traffic on that LAN would be dealt with normally. On the other LANs, the routers would have form specific host routes. 2. The anycast subnet prefix could be distinct from any of the natural subnet prefixes. In this case all the LANs have to employ host routes. Although requiring slightly more context in the routers, this model might be preferable since it avoids the asymmetry inherent in the first model. 3.2 Using multiple IP addresses with a single identifier Again there are a number of possible variants of this scheme. a) EIDs (Endpoint Identifiers) b) use of destination headers 3.2.1 EIDs This path has been thoroughly explored in the past and rejected, at least as far as including EIDs in the IPv6 header is concerned. How- ever it may still be possible to include some equivalent functional- ity in an extension header. This leads to the next possibility Expires August 1996 [Page 11] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 3.2.2 Use of destination headers Note: These ideas are further developed in [BOUND]. 3.2.2.1 EID like It would be possible to include something like an EID in the destina- tion header. Packets transmitted by the multihomed node would use the IP address of the interface over which they were transmitted as their source address, but would include the source 'EID' in the des- tination header and compute any higher layer checksums as if that were the source address (rather like the use of the routing header for destination addresses). The recipient would, on processing the destination option replace the original source address with the EID and use that for checksum verification and TCP connection identifica- tion. It could also build a mapping between EID and source IP addresses. When attempting to return a packet addressed by the upper layer to the EID, it would consult its mapping for the EID, choose one of the real IP addresses stored for that EID, place that address in the des- tination address, and place the destination EID in the destination option. For communication between two multihomed hosts it would be necessary to store both the source EID and destination EID in the destination header. The option would therefore occupy at least 34 bytes, which is a considerable overhead. 3.2.2.2 Anycast Addresses in destination headers A variant of the above would be to use an anycast address as the des- tination address when transmitting to the multihomed node. In pack- ets transmitted by the multihomed node, the source address would be the real IP address of the interface used, and the corresponding any- cast address would be placed in the destination header. Substitution for checksum and TCP identification would take place as before, but the anycast address could be used directly as the destination address of returned packets, without the need to maintain an EID mapping. This effectively uses the anycast address as an EID which can be routed. There is considerable advantage in this approach, since it places the burden of determining the best destination interface over which to communicate with the multihomed node on the routing infrastructure. In approaches which use multiple destination addresses, the burden of selecting the correct destination address rests with the host. Expires August 1996 [Page 12] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 3.2.2.3 Security considerations Both the above schemes have serious security implications, since they effectively provide a means to change a packet's apparent source address by inserting a destination header. However, since the address included in the destination header is the one used for pseu- doheader construction, it doesn't provide any less security than the albeit minimal security already existing for conventionally addressed packets. 3.3 Modifications to TCP An alternative approach which provides similar functionality to those outlined above, but within the TCP layer rather than within the IP layer, is that described in [HUITEMA], to which the reader is referred for details. In essence, rather than an EID with global uniqueness scope, a con- text identifier with only local uniqueness is carried within an extended TCP packet and used to identify connections. The initiator proposes extended operation by including its locally unique context ID. Any packets returned, from whatever address, quoting this con- text ID will be accepted as belonging to that connection. This allows the underlying IP addresses to be changed at will without affecting the connection. For return traffic, the node maintains a cache of recently seen source addresses associated with the specific context ID and chooses one of those (using an appropriate algorithm; least recently used, most recently seen, round robin etc.) for use as the destination address of the packet. While this provides an elegant solution to some of the issues sur- rounding multihoming, as well as renumbering, by its very nature it is applicable only to TCP. The author makes a good case that TCP represents the most important case, but there are nevertheless many UDP based applications which would wish to benefit from the use of multihoming. NFS is a prime example. Again, the burden of destination address (and hence interface) selec- tion rests with the transmitting host. 3.4 Use of Mobile IP techniques An alternative way of making use of destination option information is to extend the mobile IP binding update option to be able to contain multiple addresses. A multihomed node could then send packets using the source address appropriate to the interface used (another possi- bility would be to use some other address, but discussion below indi- cates that using per interface source addresses has some advantages), Expires August 1996 [Page 13] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 but add a binding update referencing all of its addresses. The receiving node could then cache this information and use the addresses in rotation as the destination address of returned packets. Should the set of operational interfaces (or the addresses on them) change, then a new binding update could be issues with the updated information. As with the other approaches which rely on the host to select appropriate destination addresses, there is no guarantee that the multihomed host is reachable use all (or indeed any) of the interface addresses which it advertises in the binding update. This is dis- cussed below. 3.5 Selection of destination address by the transmitting host Just because the multihomed node thinks its interface is operational doesn't mean that the destination node can actually reach that inter- face. Indeed it is possible that the multihomed node could success- fully transmit over a particular interface, but return traffic to that interface's address would be dropped. This means that all schemes which rely on the transmitting host to select the appropriate destination address may attempt to send traffic to the wrong address. One way around this would be for all nodes (multihomed or not) to operate a similar scheme to that suggested below for determining the output interface, but use it for determining the destination address to use. i.e. if the node failed to make progress with the upper layer protocol using one destination address it could leave that address out of the set. Clearly all schemes which use separate destination addresses for each interface and rely on the sender to sort out the multiplexing will suffer from this difficulty. With single destination address schemes, it is the responsibility of routing to do the path split- ting, and routing is in a position to determine which interface is reachable. Especially if the decision is taken locally to the desti- nation. {But what about the simple dual railed LAN case. Suppose we have two MH hosts and a router spanning the LANs. } 4. Selecting the output interface In any of the above mechanisms it is still necessary for the multi- homed host to be able to determine the correct output interface over which to transmit packets. To be able to determine this accurately Expires August 1996 [Page 14] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 would require the host to participate in the routing algorithms. As has already been discussed, this is undesirable for the following reasons a) it would add significantly to the complexity of the host imple- mentation, especially since simple routing protocols such as RIP do not exhibit the required rapid failover characteristics. b) it would pollute the routing databases of other routers. However, a good approximation to the desired behavior may be possible using neighbor discovery, neighbor unreachability detection, and some heuristics. The reachability of the destination over the two (or more) interfaces can be classified as follows:- a) The destination is reachable over both interfaces, and the two paths are equivalent. i.e. in routing terms they both exhibit the same metric value. b) The destination is reachable over both interfaces, but one exhi- bits a "better" metric than the other. c) The destination is only reachable over one interface. d) The destination is reachable over neither interface. In case (a) the desired behavior is to split the traffic over both interfaces. In cases (b) and (c), the traffic should be sent over the "better" or "only" interface. It is possible for the reachability to change dynamically and hence one case may be transformed to another. It is necessary to adapt the behavior to the new case as soon as possible. The information which is available from neighbor discovery includes the state (REACHABLE, INCOMPLETE, STALE, DELAY, PROBE) of the next hop, and whether or not the next hop is the destination itself (the isRouter flag). For the purposes of determining the best path, these indications can Expires August 1996 [Page 15] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 be ranked as follows, starting with the best 1) REACHABLE, isRouter false (i.e. the destination is directly reachable on this interface) 2) REACHABLE, isRouter true (i.e. the destination may be reachable over this interface via a router, which is itself known to be reachable) 3) STALE, DELAY or PROBE (i.e. the destination was reachable over this interface, but this needs to be re-verified) 4) INCOMPLETE (i.e. the destination doesn't (yet) appear to be reachable over this interface). Transmitted data should be round robinned between all the interfaces with the highest equal ranking of reachability. No data (except pos- sibly for probes, see later) should be sent over lower ranking inter- faces. Note that this is different from the normal case, where data for stale entries is still sent while the reachability is being re- verified. In the normal case, the choice is only between sending or not sending at all, and "to travel hopefully is a better thing than to arrive" [ R L Stevenson]. However, in the multihomed case, where we know we have a good path, we would rather concentrate the data on that path than risk it to a path of dubious viability. By setting the ReachableTime quite short, it is possible to allow failing paths to be rapidly detected. Provided the traffic provides "hints from the upper layers", this need not impose too much of an overhead. Note that where the next hop is NOT the destination, neighbor discovery only provides confirmation (via NUD) that the next hop is reachable. It does not confirm that the destination is reachable via that next hop. However the "hints from the upper layer protocols" that the connection is making forward progress, do confirm that the destination is reachable, but not necessarily over that interface. Suppose the data for a connection is being split over two interfaces, both having a router as the next hop. Alternate packets are sent over each interface, but the path over interface A always drops the packet at some point in the path, while the path over interface B always delivers it. Using a window size of one, packet 1 is sent over interface A. No acknowledgement is received, so the TCP layer Expires August 1996 [Page 16] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 retransmits the packet (1), this time over interface B. An ack- nowledgement IS received for this packet, so packet 2 is now transmitted over interface A. Again it is dropped, and re- transmitted successfully over interface B. The connection is indeed making forward progress (albeit, painfully slowly), but the path over interface A is invalid. Indeed the same scenario applies if the next hop router over interface A is broken. In order to use forward progress as a path validator it is necessary to tie the forward progress to the packets being sent over that par- ticular interface. One way of doing this would be to keep a record of the interface last used for the transmission of each sequence number, and on receipt of an acknowledgement, only update the vali- dity of the interfaces which had last transmitted the sequence numbers covered by the acknowledgement. This requires the saving of a large amount of state. Hopefully, rather more efficient algorithms can be found. It might be possible to only initiate the validation following a packet loss. However, even assuming such an algorithm existed, this technique is only suitable for use on TCP connections. An alternative might be to actively probe the destination (perhaps with ICMP echo packets) over each interface. This would impose an intolerable overhead on the net- work, especially since to meet the required failover characteristics, the probe frequency would have to be of the order of once a second. Frequently it may be the case that although the next hop router is reachable, the destination is unreachable from that router. Thus neighbor reachability information alone is not sufficient to deter- mine the best interface. If the router knows of a better path, it will issue a redirect to the better router, however if it has no path it will issue an ICMP unreachable error message back to the source. Receipt of an ICMP unreachable message can therefore be used as an indication that a path is failing and the interface concerned should be downgraded. Note that the path may not be completely broken. The unreachable message may just indicate a temporary condition, but in the absence of any other information it is still probably better to avoid that interface for a while. A similar problem to that with forward progress monitoring arises in this situation, since it is necessary to identify which interface was used to transmit the failed packet. We cannot use the interface on which the ICMP error message arrived, since it may well have been delivered to the "good" interface. Fortunately the ICMP error mes- sage holds the original packet header, and in particular the IP source address used for that packet. Therefore (provided a multihom- ing scheme which uses separate source addresses for the interfaces has been chosen), the ICMP error message can be accurately associated Expires August 1996 [Page 17] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 with the appropriate interface. Although negative acknowledgement via ICMP messages allows some meas- ure of detection of failed paths, nevertheless is cannot be relied upon to give perfect notification, since the ICMP packets themselves may be being lost for some reason. Only the positive acknowledgement provided by the upper layer protocols can be relied upon to prove that the packets are being correctly delivered. So where positive acknowledgement is available it should be used. Although the above mechanisms distinguish between the case of a directly reachable destination and one which must be reached via a router, they do not distinguish between paths which both reach a des- tination via routers, but which have different metrics. This is prob- ably unimportant, since it can be assumed that dual rail configura- tions are generally constructed with a reasonable degree of symmetry. It might be possible to have a mechanism whereby hosts could solicit per destination routing metric information from their nearby router, but this seems fraught with difficulty, not least that of keeping the information up to date without involving large amounts of stored state, or frequent message passing. 4.1 Probing for better paths. When an interface is not being used because it has a lower ranking than other interfaces, it is still necessary to occasionally poll the state of that interface for the destination in question, in case the situation has improved. There are 3 possible levels of polling: 1) Neighbor discovery probing. 2) Sending a duplicate packet over the suspect interface to deter- mine if an ICMP error report is still generated. 3) Sending a data packet ONLY over that interface to determine if the connection continues to make progress using that interface. These should be used in order. Clearly there is no point in sending any data until the next hop is known to be reachable. Sending a duplicate packet then allows the new path to be assessed without com- mitting the integrity of the data stream to that path. Finally the positive acknowledgement can be obtained by sending a data packet only over that interface. An alternative to 2 and 3 would be to use an ICMP echo packet, and check for the correct reception of the reply. Although this technique Expires August 1996 [Page 18] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 would be impractical for verifying the continued viability of the path (as discussed earlier), it may be acceptable in this case since the frequency of polling need only be of the order of once per minute. 5. Link Local Addresses Currently Link Local Addresses have not been considered 6. Security Considerations These will be addressed as they become apparent. 7. Author's Addresses Mike Shand REO2 F-D9 Digital Equipment Co. Ltd. Digital Park Imperial Way Reading Berkshire RG2 0TE UK Tel: +44 1734 204424 Fax: +44 1734 203133 Email: shand@shand.reo.dec.com Matt Thomas Digital Equipment Corporation 550 King Street Littleton MA 01460 - 1289 Phone: (508) 486 -5074 Email: matthew.thomas@lkg.dec.com 8. References [BOUND} J. Bound, P Roque, "Dynamic Reassignment of IP Addresses for TCP and UDP" [HUITEMA] C. Huitema, "Multi-homed TCP" Expires August 1996 [Page 19] Internet Draft Multihoming Support in IPv6 19 Feb. 1996 9. Acknowledgements Many of the ideas contained in this document originated in discus- sions which took place during the Dallas IETF meeting in December 1995 including (but not limited to) Mike Shand, Matt Thomas, Dan McDonald, Jack McCann, Jim Bound, Keith Sklower, Thomas Narten. (End of Draft) Expires August 1996 [Page 20]