<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE rfc SYSTEM 'rfc2629.dtd' []>

<rfc ipr="trust200902" category="info" docName="draft-defoy-coms-subnet-interconnection-04">
  <?rfc toc="yes"?>
  <?rfc symrefs="yes"?>
  <?rfc sortrefs="yes"?>
  <?rfc compact="yes"?>
  <?rfc subcompact="no"?>
  <?rfc private=""?>
  <?rfc topblock="yes"?>
  <?rfc comments="no"?>
  <front>
    <title abbrev="Network slicing">Interconnecting (or Stitching) Network Slice
      Subnets
    </title>

    <author initials="X." surname="de Foy" fullname="Xavier de Foy">
      <organization>InterDigital Inc.</organization>
      <address>
        <postal>
          <street>1000 Sherbrooke West</street>
          <city>Montreal</city>
          <code></code>
          <country>Canada</country>
          <region></region>
        </postal>
        <phone></phone>
        <email>Xavier.Defoy@InterDigital.com</email>
        <uri></uri>
      </address>
    </author>
    <author initials="A." surname="Rahman" fullname="Akbar Rahman">
      <organization>InterDigital Inc.</organization>
      <address>
        <postal>
          <street>1000 Sherbrooke West</street>
          <city>Montreal</city>
          <code/>
          <country>Canada</country>
          <region/>
        </postal>
        <phone></phone>
        <email>Akbar.Rahman@InterDigital.com</email>
        <uri></uri>
      </address>
    </author>
    <author fullname="Alex Galis" initials="A." surname="Galis">
      <organization>University College London</organization>
      <address>
        <postal>
          <street>Torrington Place</street>
          <city>London</city>
          <code>WC1E 7JE</code>
          <country>United Kingdom</country>
        </postal>
        <email>a.galis@ucl.ac.uk</email>
      </address>
    </author>
    <author initials="K." surname="Makhijani" fullname="Kiran Makhijani">
      <organization>Huawei Technologies</organization>
      <address>
        <postal>
          <street>2890 Central Expressway</street>
          <city>Santa Clara</city>
          <code>CA 95050</code>
          <country>USA</country>
          <region/>
        </postal>
        <phone></phone>
        <email>kiran.makhijani@huawei.com</email>
        <uri></uri>
      </address>
    </author>
    <author fullname="Li Qiang" initials="L." surname="Qiang">
      <organization>Huawei Technologies</organization>
      <address>
        <postal>
          <street>Huawei Campus, No. 156 Beiqing Rd.</street>
          <city>Beijing</city>
          <code>100095</code>
          <country>China</country>
          <region/>
        </postal>
        <phone/>
        <email>qiangli3@huawei.com</email>
        <uri/>
      </address>
    </author>
    <author fullname="Shunsuke Homma" initials="S." surname="Homma">
      <organization abbrev="NTT">NTT, Corp.</organization>
      <address>
        <postal>
          <street>3-9-11, Midori-cho</street>
          <city>Musashino-shi</city>
          <region>Tokyo</region>
          <code>180-8585</code>
          <country>Japan</country>
        </postal>
        <email>homma.shunsuke@lab.ntt.co.jp</email>
      </address>
    </author>
    <author fullname="Pedro Martinez-Julia" initials="P." surname="Martinez-Julia">
      <organization abbrev="NICT">National Institute of Information and Communications Technology</organization>
      <address>
        <postal>
          <street/>
          <city/>
          <region/>
          <code/>
          <country>Japan</country>
        </postal>
        <email>pedro@nict.go.jp</email>
      </address>
    </author>

    <date year="2020" month="March" day="8"/>
    <area>Internet</area>
    <workgroup>none</workgroup>
    <keyword>Network Slicing</keyword>


    <abstract>
      <t>
        This document defines the network slice (NS) subnet as a general management plane
        concept that augments a baseline YANG network slice model with management
        attributes and operations enabling interconnections (or stitching) between
        network slices.

        The description of NS subnet interconnections is technology agnostic, and
        is not tied to a particular implementation of the interconnection in data plane.
      </t>
    </abstract>
  </front>

  <middle>

    <section anchor="introduction" title="Introduction">


      <t>
        Network Slicing enables deployment and management of services with
        diverse requirements on end-to-end partitioned virtual
        networks over the same infrastructure, including networking,
        compute and storage resources.

        There were recent efforts in the IETF to define a transport
        slice (<xref target="I-D.nsdt-teas-transport-slice-definition"/>)
        and to define a north-bound interface for such a transport slice
        (<xref target="I-D.contreras-teas-slice-nbi"/>).

        The mapping of transport slices in 5G mobile systems is also
        studied in
        <xref target="I-D.clt-dmm-tn-aware-mobility"/> and
        <xref target="I-D.geng-teas-network-slice-mapping"/>.
      </t>

      <t>
        Network slices may be managed through usage of YANG data models.

        For example, <xref target="I-D.liu-teas-transport-network-slice-yang"/>
        describes how existing YANG models can be augmented with network slice attributes.

        Nevertheless, defining and managing a network slice (NS) end-to-end
        does not always have to be done directly. It may be convenient to define and
        manage separately subsets of an end-to-end slice.

        The concept of network slice subnet is defined originally in
        <xref target="NGMN_Network_Slicing"/>, though we only need to retain its
        definition in the most universal form: network slice subnets
        are similar to network slices in most ways
        but cannot be operated in isolation as a complete network slice
        (e.g., a NS subnet can be seen as a network slice with unconnected links).

        NS subnets are interconnected with
        other NS subnets to form a complete, end-to-end network slice
        (i.e. interconnection and/or stitching of NS subnets).

        In the present draft, we describe a data model for describing
        interconnections between NS subnets, that enables assembling them in
        a hierarchical fashion.
      </t>

    <section anchor="intro11" title="Motivation and Roles of NS Subnet">

      <t>
        NS subnet is a management plane concept that facilitates interconnections (also
        known as stitching) of network slices.

        It augments the base slice information model, that can
        be used to represent an end-to-end network slice.

        The extensions described in this document can be used to represent a slice subnet instead,
        and can also be used to represent an interconnection inside an end-to-end slice, i.e.
        they aim to represent interconnection points both "before" and "after"
        the interconnection takes place.

        Operations such as stitching subnets are also described.
      </t>
      <t>
        The description of NS subnet interconnections is technology agnostic
        following the approach of the slice information model.

        Some interconnections may be implemented using the interplay between management
        plane and gateways in the data plane.

        <xref target="I-D.homma-rtgwg-slice-gateway"/> describes the requirements
        on such data plane network elements, and will provide input for the
        management plane mechanisms described in the present document.

      </t>
    </section>


      <section anchor="rationale" title="Usage of NS Subnets">

        <t>
          Using NS subnets can help:
          <list style="symbols">
            <t>Isolate management and maintenance of different portions of a network slice,
              over multiple infrastructure domains, or even within a single domain.
              For example, in Figure 1, NS orchestrator (NSO) 2 manages subnet A, in isolation from
              subnets B and C managed by NSO 3. NSO 1 can still manage the end-to-end slice
              as a whole, but it does not need to deal in detail with each subnet.
            </t>
            <t>Isolate mapping towards different infrastructure technologies, even within the same domain.
              This can simplify NS orchestrator implementation, since each NSO can
              specialize in managing a smaller set of technologies.</t>
            <t>Enable advanced functions such as sharing a slice subnet between several slices, or substituting one slice
              subnet for another, e.g. for coping with load.
            </t>
          </list>
        </t>

        <figure anchor="fig-interconnection0" align="center"
                title="Overview of Network Slice Subnets Interconnection">
          <artwork align="center"><![CDATA[
                   +-----------+
             ******| NS Orch. 1|********
             *     +-----------+       *
             *                         *
             *                         *
        +-----------+              +-----------+
        | NS Orch. 2|              | NS Orch. 3|*****
        +-----------+              +-----------+    *
             *                         *            *
             *                         *            *
             *   A-B Inter-            * B-C Inter- *
             *   connection            * connection *
+-----------------+   .  +-----------------+  .  +-----------------+
|      +--+       |   .  |      +--+       |  .  |      +--+       |
|      |  +---------------------+  +--------------------+  |       |
|      ++-+       |   .  |      ++-+       |  .  |      ++-+       |
|       |         |   .  |       |         |  .  |       |         |
| +---+ |  +---+  |   .  | +---+ |  +---+  |  .  | +---+ |  +---+  |
| |   +-+--+   +-----------+   +-+--+   +----------+   +-+--+   |  |
| +---+    +---+  |   .  | +---+    +---+  |  .  | +---+    +---+  |
+-----------------+   .  +-----------------+  .  +-----------------+

<.. NS subnet A ..>      <.. NS subnet B ..>     <.. NS subnet C ..>

<....................... end-to-end slice .........................>
]]></artwork>
        </figure>


        <t>
          Figure 1 illustrates how an end-to-end network slice may be composed of
          multiple slice subnets, each managed independently by a same or different NSO.

          In multi-administrative domain scenarios, using NS subnets can help limiting the information that needs to be
          shared between domains.

          At the infrastructure layer (i.e. in the data plane), the interconnection between NS subnets may involve:
          <list style="symbols">
            <t>a gateway, that performs protocol and/or identifier/label translation as needed,</t>
            <t>two gateways, especially in cases where interconnected NS subnets are in different administrative domains,</t>
            <t>nothing at all, in cases where the interconnection point can be abstracted away, e.g.
              when the NS subnets share a common infrastructure. In this case nodes from both NS subnets
              end up being directly interconnected between each other.</t>
          </list>

        </t>

        <t>
          More detailed usage scenarios are described in <xref target="stitching-scenarios"/>.
        </t>

      </section>

      <section anchor="requirements-language" title="Terminology">

        <t>Network slicing terminology, especially focusing on transport slices, is defined in
          <xref target="I-D.nsdt-teas-transport-slice-definition"/>.
        </t>

        <t>Network Slice Subnet (NS subnet): a network slice designed to be
          interconnected with other network slices.
        </t>

        <t>
          NS Stitching: a management operation consisting in
          creating an end-to-end NS or a larger NS subnet, by interconnecting a set
          of NS subnets together.
        </t>

        <t>
          Interconnection Anchor: a management plane entity, part of a NS subnet model, representing
          an end point for use in future stitching operation.
        </t>

        <t>
          Interconnection Instance (or Interconnect): a management plane entity,
          part of a NS subnet model, representing an interconnection realized by a stitching operation.
          It is distinct from a (data plane) gateway: an interconnect may be realized with or without using a gateway
          in the data plane.
        </t>

      </section>

    </section>

    <section anchor="InfoModel" title="Information Model">

      <section anchor="BaseInfoModel" title="Base Information Model">

        <t>
          The information model we use as base for network slicing is
          the network topology model ietf-network defined in
          <xref target="RFC8345"/>, in which networks
          are composed of nodes and links, and in which termination points (TP), defined in nodes,
          are used to define source and destination of links.
        </t>

        <t>
          A network slice data model instance, i.e. a YANG data model
          augmented using <xref target="I-D.liu-teas-transport-network-slice-yang"/>), represents
          a network slice.

          When such a data model instance includes at least an "interconnection anchor", as defined below,
          it represents a network slice subnet instance.
        </t>
        <t>
          At high level, the extensions defined in this document will augment nodes and
          termination points:
        </t>
        <t>
          <figure>
            <artwork align="left" type="ascii-art"><![CDATA[module: ietf-network
+--rw networks
   +--rw network* [network-id]
      +--rw network-id
      +--rw network-types
      +--rw supporting-network* [network-ref]
      |  +--rw network-ref
      +--rw node* [node-id]
      |  +--... (augmented with attributes for
      |  |       anchor/interconnection nodes)
      |  +--rw nt:termination-point* [tp-id]
      |  |  ... (augmented with attributes for
      |  |       anchor/interconnection TP)
]]></artwork>
          </figure>
        </t>


      </section>

      <section anchor="Anchors" title="Interconnection Anchors">

        <t>
          To represent an anchor point for future interconnections (i.e. an unconnected end of a link),
          a simple solution is to use an "interconnection anchor" termination point (or anchor TP).

          Within the data model describing a subnet, any link not entirely contained within the NS subnet
          must be terminated with such an anchor TP as source or destination.

          An anchor TP belongs to a "node" attribute, which we refer to as
          interconnection anchor node (or anchor node).

          Several anchor TPs can be grouped together in an anchor node, and such grouping may be used
          as a hint during a stitching operation (e.g. to place all interconnection points at a same
          location).
        </t>

        <t>
          Figure 2 represents 2 interconnected network slice subnets.
        </t>

        <figure anchor="fig-interconnection" align="center"
                title="Network Slice Subnets Interconnection">
          <artwork align="center"><![CDATA[
                            Slice Provider
                                  |
+---------------------------------v---------------------------------+
|  Network Slice Orchestrator                                       |
|                                                                   |
| +---------------------------------------------------------------+ |
| |   Data model: network slice composed of NS subnet 1 and 2     | |
| |                                                               | |
| |      Network Slice Subnet 1            Network Slice Subnet 2 | |
| | +---------------------------+  +----------------------------+ | |
| | |     cross-subnet link     |  |   cross-subnet             | | |
| | |    +----------------+     |  |       link    +------+     | | |
| | |    |                |     |  |      +--------o node |     | | |
| | |    |                |Interconnection|        +---o--+     | | |
| | |+---o--+     +-------|-----+--+------|------+     |        | | |
| | || node |     |       |     |  |      |      |     |        | | |
| | |+---o--+     | +-----|---+ |  | +----|----+ |     |        | | |
| | |    |        | |     |   | |  | |    |    | |     |        | | |
| | |    |        | |     O - - - - - - - O    | |     |        | | |
| | |    |        | |         | |  | |         | |     |        | | |
| | |    |        | | anchor  | |  | | anchor  | |     |        | | |
| | |    |        | |  node   | |  | |  node   | |     |        | | |
| | |    |        | |         | |  | |         | |     +---+    | | |
| | |    |        | |     O - - - - - - - O    | |         |    | | |
| | |    |        | |     |   | |  | |    |    | |         |    | | |
| | |    |        | +-----|---+ |  | +----|----+ |     +---o--+ | | |
| | |    |        |       |     |  |      |      |     | node | | | |
| | |    |        +-------|-----+--+------|------+     +---o--+ | | |
| | |    | +------+       |     |  |      |                |    | | |
| | |    +-o node o-------+     |  |      +----------------+    | | |
| | |      +------+ cross-subnet|  |         cross-subnet       | | |
| | |                link       |  |           link             | | |
| | +---------------------------+  +----------------------------+ | |
| +---------------------------------------------------------------+ |
+--------------------------------+----------------------------------+
                                 |
                                 v
                         Network Infrastructure


     Legend: o = termination point, O = anchor termination point
]]></artwork>
        </figure>

        <t>Attributes of interconnection anchor nodes and termination points include:
          <list style="symbols">
            <t>Information enabling NS orchestrators to match anchor nodes and TPs
              from both NS during a stitching operation. A label may be a simple
              way to enable this.</t>
            <t>Information to help locate the interconnection. For example, it could
              be a (sub-)domain name or geo-location information,
              that indicates where the interconnection point should be located.
              This can help for example in cases where the subnet is instantiated before stitching.</t>
            <t>Information to help select the type of interconnection establishment:
              for example, this can indicate a preference for using interconnection over a gateway,
              or for abstracting away the interconnection point in the infrastructure plane.</t>
          </list>
        </t>

        <t>
          <figure>
            <artwork align="left" type="ascii-art"><![CDATA[      +--rw node* [node-id]
         +-- (...)
         +-- anchor_node_config
         |   +-- label (and/or other auto stitching help)
         |   +-- hint for location (domain, geolocation, etc.)
         |   +-- hint for type (1 gateway, 2 gateways, ...)
         +--rw nt:termination-point* [tp-id]
             +-- (...)
             +-- anchor_tp_config
                 +-- label (and/or other auto stitching help)
                 +-- location (domain, geolocation, etc.)
                 +-- type (1 gateway, 2 gateways, ...)
]]></artwork>
          </figure>
        </t>

      </section>

      <section anchor="Instances" title="Interconnection Instances">

        <t>
          There are two options for representing post-stitching network slices (or subnets).
          They are not mutually exclusive:
          <list style="symbols">
            <t>
              Option 1: subnet data models are updated with information describing the
              interconnection (e.g. anchor TPs and nodes are updated with new attributes
              representing the existing connection, if necessary).
            </t>
            <t>
              Option 2: a new data model is generated to represent the resulting network slice (or subnet).
              In this composite data model, the interconnection may or may not be represented, this
              can be a choice made by the operator.
            </t>
          </list>
          Option 1 and 2 can be used concurrently in a network. For example, a parent NS orchestrator
          may manage stitched NS subnets through underlying NS orchestrators, and at the same
          time expose to the NS operator a composite data model representing the resulting end-to-end slice.
        </t>

        <t>
          To represent an existing interconnection in option 1, a simple solution is to
          add attributes to existing anchor nodes and anchor TPs. Those attributes will be described
          below. They aim to describe state and configuration associated with an active
          interconnection.
        </t>

        <t>
          To represent an existing interconnection in option 2, a simple solution is to
          create new interconnection instance nodes and termination point.

          The same attributes as in option 1 may be associated with
          these nodes and TPs.
        </t>

        <t>Attributes of interconnection instance nodes and termination points include:
          <list style="symbols">
            <t>State information (interconnection type, status, location...).</t>
            <t>Service assurance related information: besides measurements (on throughput, loss rate, etc.),
              triggers depending on throughput, latency, etc. can be linked with a management
              action or event. A NS operator can use such events to take the decision to disable
              a NS subnet, replace a NS subnet with another, etc. to maintain overall service
              performance.</t>
          </list>
        </t>

        <t>
          <figure>
            <artwork align="left" type="ascii-art"><![CDATA[      +--rw node* [node-id]
         +-- (...)
         +-- interconnection_instance_node_state
         |   +-- status
         |   +-- location (domain, geolocation, etc.)
         |   +-- type (1 gateway, 2 gateways, ...)
         +-- interconnection_instance_node_service_assurance
         |   +-- events (including triggers and event IDs)
         |   +-- measurements
         +--rw nt:termination-point* [tp-id]
             +-- (...)
             +-- interconnection_instance_tp_state
             |   +-- status
             |   +-- location (domain, geolocation, etc.)
             |   +-- type (1 gateway, 2 gateways, ...)
             +-- interconnection_instance_node_service_assurance
                 +-- events (including triggers and event IDs)
                 +-- measurements
]]></artwork>
          </figure>
        </t>

      </section>

      <section anchor="StitchingOperations" title="Stitching Operation">

        <section anchor="stitching-operation" title="Operation Overview">
          <t>
            Stitching is an operation that takes two or more NS subnets as input,
            and produces a single composite NS subnet or end-to-end slice.

            It may occur when the slice subnets are being instantiated, or later.
          </t>

          <t>
            The first step in this operation is to identify the anchors
            that will be used in the interconnection.

            This may be done
            by an automated algorithm that matches the possible interconnection points
            and decides which one will be used, according to the policies established by
            the NS operator.

            The operation in this case will require the
            presence of semantically-rich attributes in the candidate anchors to enable
            automatic matching without human intervention.
          </t>

          <t>
            Other attributes of slices and anchors will also influence the operation and
            the resulting stitched (composite) object.

            For instance, network links that
            are interconnected must have compatible QoS attributes.

            Moreover, available
            networking protocols must also match among the underlying network elements that are being
            stitched.

            Otherwise, the operation will fail unless the NS operator (based on policy and/or NS subnet
            attributes) enables it to search for, and use, some "bridge" element in the underlying
            infrastructure.
          </t>
        </section>

        <section anchor="stitching-scenarios" title="Stitching Scenarios">
          <t>This section briefly describes examples of usage for subnet stitching.
          </t>
          <t>Traversal through a transport network.
            <list style="hanging">
              <t>
                Let's consider a network slice composed of (NS) subnet-A,
                and subnet-C (<xref target="Fig_scenario"/>).

                Subnet-A and subnet-C are deployed in independent domains and are mapped into
                a slice information model; in order to stitch these two together a transport segment is needed.

                N1 and N2 are anchor nodes within NS subnets A and C.

                Segment-B could be a simple link between the two NS subnets but it may also be a
                TE-link made available by a transport network provider.

                Segment-B may be involved in the stitching operation in one of several ways:

                <list>
                  <t>
                    Segment-B may be set up as part of the stitching operation between NS subnets A and C, as
                    a form of "bridge" mentioned in <xref target="StitchingOperations"/>.

                    Segment-B will need to comply with service specific traffic constraints that
                    are determined during the stitching operation, possibly using attributes
                    from NS subnets A and C.

                    In this case, the data plane implementation of N1 and N2 in the composite slice
                    may be, for example, 2 distinct gateway functions terminating segment-B.
                  </t>
                  <t>
                    Segment-B may alternatively be represented as a distinct NS subnet, e.g. in cases where
                    segment-B is complex and/or involves multiple network functions.

                    In this case, the stitching operation may therefore involve 3 NS subnets A-B-C.
                  </t>
                </list>
              </t>
            </list>
          </t>

          <figure anchor="Fig_scenario" align="center" title="Example of NS subnets interconnection through transport network">
            <artwork align="center" type="ascii-art"><![CDATA[
          +-----------+                     +----------+
          |   +--+    |      ______         |   +--+   |
          |   |N1+==========(______)============|N2|   |
          |   +--+    |   --transport--     |   +--+   |
          +-----------+                     +----------+
          --subnet-A---  --segment-B------  --subnet-C--
          <---------------end to end slice ------------>
]]></artwork></figure>
          <t>Subnets in a single domain.
            <list style="hanging">
              <t>
                In this scenario multiple network slice subnets
                are defined as basic building blocks with specific service functions (or chains),
                topologies and traffic handling characteristics.

                These building blocks can be assembled through stitching to build end-to-end
                customized slices, but also to dynamically extend slices to adapt to traffic load.

                Additionally, stitching can also be used to share building blocks between multiple
                slices, e.g. to interconnect multiple slices with a shared function.

                In all these cases, interconnection instances may be entirely abstracted away,
                although they may also be implemented through one or multiple gateways,
                e.g. when stitched subnets belong to different sub-domains.
              </t>
            </list>
          </t>
        </section>

      </section>

    </section>

    <section anchor="security-considerations" title="Security Considerations">
      <t>
        Security aspects relative to network slices (e.g., for transport slices, in
        <xref target="I-D.liu-teas-transport-network-slice-yang"/>)
        are applicable to slice subnets, including transport security aspects,
        access control and protection of write operation on newly introduced nodes
        (e.g., termination-point).
      </t>
    </section>

    <!--
    <section anchor="dataModel" title="YANG Data Model">
      <t>
        TODO
      </t>
    </section>
    -->

    <!--
    <section anchor="acknowledgements" title="Acknowledgements">
      <t>TBD
      </t>
    </section>
    -->

    <section anchor="iana" title="IANA Considerations">
      <t>This document has no actions for IANA.
      </t>
    </section>

  </middle>
  <back>
    <references title="Informative References">
      <?rfc
            include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.nsdt-teas-transport-slice-definition.xml"?>
      <?rfc
            include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.contreras-teas-slice-nbi.xml"?>
      <?rfc
            include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.liu-teas-transport-network-slice-yang.xml"?>

      <?rfc
          include="https://xml2rfc.ietf.org/public/rfc/bibxml/reference.RFC.8345.xml"?>
      <?rfc
          include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.homma-rtgwg-slice-gateway.xml"?>

      <?rfc
            include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.clt-dmm-tn-aware-mobility.xml"?>
      <?rfc
            include="https://xml2rfc.tools.ietf.org/public/rfc/bibxml3/reference.I-D.geng-teas-network-slice-mapping.xml"?>

      <reference anchor="NGMN_Network_Slicing"
                 target="https://www.ngmn.org/uploads/media/161010_NGMN_Network_Slicing_framework_v1.0.8.pdf">
        <front>
          <title>
            Description of Network Slicing Concept
          </title>
          <author>
            <organization>NGMN</organization>
          </author>
          <date day="16" month="10" year="2016"/>
        </front>
        <format type="PDF"
                target="https://www.ngmn.org/uploads/media/161010_NGMN_Network_Slicing_framework_v1.0.8.pdf"/>
      </reference>

    </references>

  </back>
</rfc>
