Mbone Deployment Working Group                  Hugh LaMaster
 INTERNET-DRAFT                                  Steve Shultz
 Category: Informational                         NASA ARC/NREN
 draft-ietf-mboned-mix-00.txt                    John Meylor
 Operations and Management Area                  David Meyer
 Internet Engineering Task Force                 Cisco Systems
 12 November 1998
 Expires May 1999
 
 
 
 
               Multicast-Friendly Internet Exchange (MIX)
                      <draft-ietf-mboned-mix-00.txt
 
 
 
 
 Status of this Memo
 
 
 
    This document is an Internet-Draft. Internet-Drafts are working
    documents of the Internet Engineering Task Force (IETF), its areas,
    and its working groups. Note that other groups may also distribute
    working documents as Internet-Drafts.
 
    Internet-Drafts are draft documents valid for a maximum of six months
    and may be updated, replaced, or obsoleted by other documents at any
    time. It is inappropriate to use Internet-Drafts as reference
    material or to cite them other than as ``work in progress.''
 
    To learn the current status of any Internet-Draft, please check the
    ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow
    Directories on ftp.is.co.za (Africa), nic.nordu.net (Europe),
    munnari.oz.au (Pacific Rim), ds.internic.net (US East Coast), or
    ftp.isi.edu (US West Coast).
 
 
 
 
    Abstract
 
    This document describes an architecture for a Multicast-friendly
    Internet eXchange (MIX), and the actual implementation at the NASA
    Ames Research Center Federal Internet eXchange (FIX-West, or FIX).
    The MIX has three objectives: native IP multicast routing, scalable
    interdomain policy-based route exchange, and to allow a variety of
 
 
 
 LaMaster, et al.                                                [Page 1]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    IGP protocols and topologies for intra-domain use. In support of
    these objectives, the MIX architecture defines the following
    components: a peer-peer routing protocol, a method for multicast
    forwarding, a method for exchanging information about active sources,
    and a medium which provides native multicast.  This document
    describes the protocols and configurations necessary to provide a
    current, working multicast-friendly internet exchange, or MIX.
 
    This memo is a product of the MBONE Deployment Working Group (MBONED)
    in the Operations and Management Area of the Internet Engineering
    Task Force. Submit comments to <mboned@ns.uoregon.edu or the
    authors.
 
 
 
 
 
    Copyright Notice
 
    Copyright (C) The Internet Society (1998).  All Rights Reserved.
 
 
 
 
 
    Acknowledgments
 
    Thanks to the NASA HPCC program for supporting the NREN staff portion
    of this project; thanks to William P. Jones of the NASA ARC Gateway
    Facility for making the gateway facility available for housing this
    project.
 
 
 
 
    1.  Introduction
 
 
 
 
    The MIX objective was to use current technology to implement a
    scalable, high-performance, efficient, native IP multicast
    architecture.
 
    Past experience at ARC, NASA WANs, and at FIX-West, had shown that
    mrouted/DVMRP "Mbone" tunnels were an inefficient of routing
    multicast through an exchange point.  Specifically, at FIX-West, the
    large number of tunnels often resulted in unicast traffic loads on
 
 
 
 LaMaster, et al.                                                [Page 2]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    the FIX FDDI that were 10 times the underlying multicast load.  In
    addition, some WANs had multiple tunnels criss-crossing the same
    physical links, resulting in wasted WAN bandwidth.  And, the separate
    workstation and router infrastructure for the "Mbone" tunnels created
    numerous problems.  Maintenance of Unix system and tunnel
    configurations was often ad hoc, because some of the network
    operators lacked the necessary expertise.  And the hardware and
    software configuration and performance of the tunnel infrastructure
    was often out of step with the underlying router-based unicast
    structure. In addition, use of a single, shared, distance-vector IGP
    in the inter-domain space led to instability.
 
    Therefore, it was desired to implement a new multicast internet
    exchange from the ground up, using current technology, and
    significantly improving performance, efficiency, and reliability.
 
    Four elements were identified as being necessary for the MIX
    architecture in order to meet the objectives.  These were to define a
    peer-peer routing protocol, a method for multicast forwarding, a
    method for exchanging information about distant sources and groups,
    and a non-switched broadcast medium.
 
    NASA Ames Research Center hosts the Federal Internet eXchange (FIX-
    West, or, "the FIX") as well as hosting the Ames Internet eXchange
    (AIX), which is connected at high speed to the MAE-West, and, which
    also shares the same address space as the MAE-West. These facilities
    are co-located at the Ames Telecommunications Gateway Facility.  It
    was felt that this would be an excellent location to test the
    viability of the native multicast technologies. The Multicast-
    friendly Internet eXchange (MIX) is co-located adjacent to the FIX
    for easy access from the existing FIX routers.
 
    Choices were made for each element, and the MIX was implemented
    adjacent to the existing NASA ARC FIX gateway facility.  At the time
    of writing, there are eight direct participants in the MIX, peering
    and exchanging routes and multicast traffic natively, and the
    performance and reliability have already far exceeded the tunneled
    infrastructure the MIX replaced.
 
 
 
 
    2. Requirements and Technology
 
 
 
 
    In order to meet the objectives for this multicast exchange, all
 
 
 
 LaMaster, et al.                                                [Page 3]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    peering partners had to agree mutually to standardize on the
    following four elements.  These are:
 
     - the protocol to be used for multicast route exchange
     - the method for performing multicast forwarding
     - the method for identifying active sources
     - the physical medium for the multicast exchange
 
    The elements chosen to implement the MIX were BGP4+ (also known as
    "MBGP") for routing and route exchange [BGP4+], PIM-DM and PIM-SM for
    multicast forwarding on the exchange, dense-mode flooding, and, the
    MSDP protocol for information on sources and groups, and, FDDI for
    the multicast medium.
 
 
 
 
 
    2.1 Routing
 
    Two of the objectives of the MIX were to provide an EGP for scalable
    interdomain policy-based route exchange, and to allow a variety of
    IGP protocols and topologies for intra-domain use.  As with unicast
    interdomain routing, BGP could be used as the EGP to exchange routes
    for multicast.  However, the unicast and multicast routing paths and
    policies would have to be completely congruent.  In practice, this is
    sometimes not the case.  It is possible, however, to take advantage
    of the extensions in BGP4+ to deal with these policy and path
    incongruencies.
 
    BGP4+ [BGP4+] describes extensions to (unicast) BGP that allow use of
    the existing BGP machinery to provide the necessary scalability,
    policy control, and route stability features and mechanisms to be
    applied to both unicast and multicast routes consistently.
 
    BGP4+ allows routes to be marked "unicast forwarding", "multicast
    forwarding", or "both unicast and multicast forwarding".  In this
    way, BGP4+ supports different multicast and unicast forwarding paths
    and policies.  This removes the dependency on unicast-only routing.
 
    The ability of BGP4+ to support separate paths and policies for
    multicast is important for meeting the objectives of the exchange in
    various ways.  It allows for a participant's multicast routing policy
    to be independent of its established unicast routing policy.  This is
    important in order that the exchange can support providers migrating
    to BGP4+ as an IDMR.  This is because it allows for the exchange of
    routes previously exchanged via DVMRP, even though those routes would
    not meet the existing unicast routing policy.  It allows for
 
 
 
 LaMaster, et al.                                                [Page 4]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    different policy in the interim.  For example, routes may be
    exchanged for BGP4+ multicast forwarding even though they would not
    be permitted under existing unicast routing policy.  BGP4+ also
    provides for the possibility that even after full migration is
    complete, a separate multicast routing policy can be applied.
 
    The exchange architecture imposes no requirements on the IGP or the
    multicast forwarding protocol or topology used internal to an AS.
 
 
 
 
 
    2.2 Multicast Forwarding
 
    The first requirement for the multicast forwarding protocol is that
    it be able to use routes exchanged via BGP4+.  For this reason, PIM
    was selected.  For the MIX, PIM-Dense-Mode (PIM-DM) was selected
    initially for the mutually agreed upon multicast forwarding process.
    By flooding data using PIM-DM, it was possible to provide information
    about active sources to PIM-SM RP's co-located on the MIX.  Migration
    to PIM-Sparse-Mode (PIM-SM) with MSDP is underway.
 
    The use of PIM on a shared LAN has certain consequences. It is
    necessary for all MIX participants to agree on certain configuration
    conventions affecting PIM forwarding on multi-access LANs.  In
    particular, it is necessary to establish a standard protocol "metric
    preference" (also known as "distance" or process "precedence") to be
    used by all peers for the PIM Assert process, because the PIM Assert
    process [PIM-SM] uses the "metric preference" [PIM-SM] as a mechanism
    by which the multicast forwarder is chosen.  If all parties are not
    following the convention, there may be black holes, in which a route
    appears to be valid, but traffic does not flow, or, there may be
    multicast loops, which can have deleterious consequences.
 
    For the MIX, a standard set of metric preferences are applied to the
    BGP4+ routes as the convention for the PIM forwarding mechanism.
 
 
 
 
    2.3 Active Sources
 
    There are two current methods for distributing information about
    active sources to participating AS's.  The AS's may be dense-mode
    regions, or, they may contain PIM-SM RP's.  One method is to use
    dense-mode to flood data packets to dense-mode regions and to
    sparse-mode RPs co-located on the exchange.  The second method is to
 
 
 
 LaMaster, et al.                                                [Page 5]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    use a protocol that allows each AS to share information about the
    sources contained within it.
 
    For the MIX, it was decided use dense-mode, and, all participating
    sparse-mode peers would co-locate their RP's on the router directly-
    connected to the MIX.
 
    Dense-mode, including PIM-DM, and (mrouted-based) DVMRP, uses data
    flooding to propagate information about active source-group or <S,G
    pairs throughout the global multicast routing world. Unwanted sources
    are pruned back, and are periodically re-flooded in order to fully
    refresh forwarding state in mrouters.  This is a simple and very
    reliable method of propagating information on source-group pairs, but
    the effectiveness of dense-mode depends upon reliable pruning, and
    flooding traffic to propagate <S,G information over WANs does not
    scale well.
 
    Recently, a new protocol, MSDP [MSDP] has been proposed that, when
    combined with PIM-SM, will allow independent AS's to share
    information about distant sources and groups without flooding.
    Instead of flooding all data, only <S,G information is flooded, and
    then, only to systems, such as PIM-SM RP's, which require the
    information.  MSDP allows each AS to choose its own mode, sparse or
    dense, and also to run its own sparse-mode region independent of all
    other sparse-mode regions.
 
    MSDP has now been deployed on many of the MIX routers, and some MIX-
    connected AS's are now running sparse-mode internally.  This
    deployment is ongoing, and is not yet complete.
 
 
    2.4 Medium
 
 
    The objective for the MIX medium was to provide support for native
    multicast among multiple peering partners.
 
    There exist a number of unresolved issues regarding use of layer-2
    switched media at interexchange points, and, until these issues are
    resolved, running native multicast on such media is problematic.
 
    Fortunately, BGP4+ permits unicast and multicast to be carried on
    different media, permitting a multicast medium to be used
    independently of the unicast medium.
 
    A FDDI concentrator was selected to provide the native multicast
    exchange medium. It was router-efficient, because it permitted the
    medium to do the multicast packet replication, with a single copy
 
 
 
 LaMaster, et al.                                                [Page 6]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    from a router being replicated to all neighbors.  Using a simple
    broadcast medium eliminates the complexity of using a switch for
    multicast.  And FDDI was considered operationally convenient by most
    of the participants.  Unicast traffic continues to be routed over the
    existing unicast exchange media.
 
 
    3.  The NASA Ames Research Center Multicast-Friendly Internet
    Exchange
 
    The Ames Multicast-friendly Internet eXchange, or MIX, began with the
    first beta-test trials in March 1998, and became operational,
    exchanging BGP4+ routes externally and using BGP4+ between multiple
    AS's, in May 1998.  NREN implemented BGP4+ and internal BGP4+ and
    began trial external peerings in the same time frame, evolving from
    the first trials, to full deployment by October. As of October 1998,
    there were 8 AS's peering using BGP4+ and actively exchanging
    multicast on the MIX FDDI.  One of the AS's, AS10888, represents a
    multi-router virtual BGP4+ backbone, and a router within AS10888 has
    been located on the MIX by NREN, as a gateway router.  The physical
    and logical topologies are as follows:
 
 
 
 
                           AS10888---R----"MBone"
                                     |
                                MIX  |  multicast_exchange
                              ----------------
                                 /         \
                                /           \
                   bgp4+_peer---R             R---bgp4+_peer
                                \           /
                                 \         /
                               ---------------
                            FIX unicast_exchanges
 
 
 
 
 
    AS10888 acts as a transit AS to connect other multicast-friendly
    exchanges to the NASA ARC MIX.  It also acts as a gateway between
    the DVMRP-based "Mbone" and the BGP4+ area.
 
 
    4.  Topology, Architecture, and Special Considerations
 
 
 
 
 LaMaster, et al.                                                [Page 7]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    BGP4+
 
     -PIM Asserts and Metric preference
 
    The PIM Assert mechanism requires that all routing protocols
    "compete" to see which router is allowed for forward onto the
    shared medium.  To first order, the protocol metric preference
    is used to determine the forwarder.  All MIX peers must coordinate
    routing protocol parameters so that one router does not inadvertantly
    win PIM asserts over a neighbor which has a functional path.
    This requires that BGP4+ routes have preference over other
    routes, such as BGP, OSPF, and DVMRP.  In particular,
    it was necessary to standardize protocol metric preferences,
    and give BGP4+ routes the lowest, preferred, dynamic routing
    protocol metric preferences.  For this reason, the standard
    set of BGP4+ metric preferences was chosen to be less than any
    other dynamic unicast routing protocol metric preferences.
    Any MIX routers which are using DVMRP must use a DVMRP metric
    preference higher than the BGP4+ metric preferences, rather than
    what many people have used previously as the DVMRP metric preference,
    of 0.
 
     -Default
 
    One transitional requirement is the necessity to have routes
    to "Mbone" sources, that is, sources within the global DVMRP
    routing region.  Currently, the mechanism used is to have
    a single router in AS10888 on the MIX originate MBGP default
    to all external peers.
 
 
    DVMRP routing
 
     -DVMRP route redistribution
 
    At present, all BGP4+ routes tagged with a particular community
    are redistributed at the MIX into DVMRP within AS10888.  This is
    to provide DVMRP region users access to sources originating
    within AS's that are being routed via BGP4+ exclusively.
    Unless a particular community string is set, it is
    assumed that redistribution is not desired.  In the reverse
    direction, instead of sending DVMRP routes into BGP4+,
    BGP4+ default is originated from the intermediary router.
 
    In addition, local, stub-region DVMRP routes are redistributed
    into BGP4+ internally by several of the peers.  As long as the
    regions remain stub regions, there is no danger, but, the
    possibility of a backdoor into the Mbone presents an ever-present
 
 
 
 LaMaster, et al.                                                [Page 8]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    threat of loops unless care is taken to redistribute
    only the routes which are known to be owned within the AS.
 
 
 
    5.  Conclusions and Recommendations
 
     -Provide support for native multicast
     -Use BGP4+ as a method of exchanging routes for
      inter-domain multicast
     -Use PIM-DM, or PIM-SM with MSDP
     -Concurrent use of BGP4+ and DVMRP for inter-domain
      routing is not recommended.  It is strongly
      recommended to use BGP4+ for inter-domain route exchange.
 
 
    6.  Security Considerations
 
    There are no security considerations unique to the multicast exchange.
 
 
    7.  References
 
 
    [DVMRP]   T. Pusateri, "Distance Vector Multicast Routing
                 Protocol", <draft-ietf-idmr-dvmrp-v3-07.txt,
                 August 1998.
 
    [BGP4+]   T. Bates, R. Chandra, D. Katz, Y. Rekhter,
                 "Multiprotocol Extensions for BGP-4", RFC 2283,
                 February 1998.
 
    [BGP4+2]  T. Bates, R. Chandra, D. Katz, Y. Rekhter,
                 "Multiprotocol Extensions for BGP-4", Internet Draft,
                 <draft-ietf-idr-bgp4-multiprotocol-v2-01.txt,
                 August 1998.
 
    [PIM-SM]  D. Estrin, D. Farinacci, A. Helmy, D. Thaler, S. Deering,
                 M. Handley, V. Jacobson, C. Liu, P. Sharma, L. Wei,
                 "Protocol Independent Multicast-Sparse Mode (PIM-SM):
                 Protocol Specification", RFC 2362, June 1998.
 
    [PIM-DM]  S. Deering, D. Estrin, D. Farinacci, V. Jacobson, A. Helmy,
                 D. Meyer, L. Wei, "Protocol Independent Multicast
                 Version 2 Dense Mode Specification", Internet Draft,
                 <draft-ietf-pim-v2-dm-01.txt, November 1998.
 
    [MSDP]    D. Farinacci, Y. Rekhter, P. Lothberg, H. Kilmer, J. Hall,
 
 
 
 LaMaster, et al.                                                [Page 9]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
                 "Multicast Source Discovery Protocol (MSDP)",
                 <draft-farinacci-msdp-00.txt, June 1998.
 
 
    Author's Address
 
    Hugh LaMaster
    Steve Shultz
    NASA Ames Research Center
    Mail Stop 233-21
    Moffett Field, CA 94035-1000
    email: hlamaster@arc.nasa.gov
           shultz@arc.nasa.gov
 
    David Meyer
    John Meylor
    Cisco Systems
    San Jose, CA
    email: dmm@cisco.com
           jmeylor@cisco.com
 
 
 
    8.  Full Copyright Statement
 
    Copyright (C) The Internet Society (1998).  All Rights Reserved.
 
    This document and translations of it may be copied and furnished to
    others, and derivative works that comment on or otherwise explain it or
    assist in its implmentation may be prepared, copied, published and
    distributed, in whole or in part, without restriction of any kind,
    provided that the above copyright notice and this paragraph are included
    on all such copies and derivative works.  However, this document itself
    may not be modified in any way, such as by removing the copyright notice
    or references to the Internet Society or other Internet organizations,
    except as needed for the purpose of developing Internet standards in
    which case the procedures for copyrights defined in the Internet
    languages other than English.
 
    The limited permissions granted above are perpetual and will not be
    revoked by the Internet Society or its successors or assigns.
 
    This document and the information contained herein is provided on an "AS
    IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK
    FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
    LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT
    INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR
    FITNESS FOR A PARTICULAR PURPOSE."
 
 
 
 LaMaster, et al.                                               [Page 10]
 
 
 
 
 
 <draft-ietf-mboned-mix-01.txt                                November 1998
 
 
    Table of Contents
 
 
    1 Introduction ....................................................    2
    2 Requirements and Technology .....................................    3
    3 The NASA Ames MIX ...............................................    7
    4 Topology, Architecture, and Special Considerations ..............    7
    5 Conclusions and Recommendations .................................    9
    6 Security Considerations .........................................    9
    7 References ......................................................    9
    8 Full Copyright Statement ........................................   10
 
  LaMaster, et al.                                               [Page 11]