< draft-liu-detnet-large-scale-requirements-01.txt   draft-liu-detnet-large-scale-requirements-02.txt >
Deterministic Networking Working Group P. Liu Deterministic Networking Working Group P. Liu
Internet-Draft China Mobile Internet-Draft China Mobile
Intended status: Informational Y. Li Intended status: Informational Y. Li
Expires: 15 August 2022 Huawei Expires: 13 October 2022 Huawei
T. Eckert T. Eckert
Futurewei Technologies USA Futurewei Technologies USA
Q. Xiong Q. Xiong
ZTE Corporation ZTE Corporation
J. Ryoo J. Ryoo
ETRI ETRI
11 February 2022 11 April 2022
Requirements for Large-Scale Deterministic Networks Requirements for Large-Scale Deterministic Networks
draft-liu-detnet-large-scale-requirements-01 draft-liu-detnet-large-scale-requirements-02
Abstract Abstract
Aiming at the large-scale deterministic network, this document Aiming at the large-scale deterministic network, this document
specifies the technical and operational requirements when the describes the technical and operational requirements when the
different deterministic levels of applications co-exist and are different deterministic levels of applications co-exist and are
transported over a wide area. transported over a wide area. This document also describes the
corresponding Deterministic Networking (DetNet) data plane
enhancement requirements.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 15 August 2022. This Internet-Draft will expire on 13 October 2022.
Copyright Notice Copyright Notice
Copyright (c) 2022 IETF Trust and the persons identified as the Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License. provided without warranty as described in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions Used in This Document . . . . . . . . . . . . . . 4 2. Conventions Used in This Document . . . . . . . . . . . . . . 4
3. The Overall Characteristics of Large-Scale Deterministic 3. The Overall Characteristics of Large-Scale Deterministic
Networks . . . . . . . . . . . . . . . . . . . . . . . . 4 Networks . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Technical Requirements in Large-Scale Deterministic 4. Technical Requirements in Large-Scale Deterministic
Networks . . . . . . . . . . . . . . . . . . . . . . . . 5 Networks . . . . . . . . . . . . . . . . . . . . . . . . 6
4.1. Tolerate Time Asynchrony . . . . . . . . . . . . . . . . 5 4.1. Tolerate Time Asynchrony . . . . . . . . . . . . . . . . 6
4.1.1. Support Asynchronous Clocks Across Domains . . . . . 6 4.1.1. Support Asynchronous Clocks Across Domains . . . . . 6
4.1.2. Tolerate Clock Jitter & Wander within a Clock 4.1.2. Tolerate Clock Jitter & Wander within a Clock
Synchronous Domain . . . . . . . . . . . . . . . . . 6 Synchronous Domain . . . . . . . . . . . . . . . . . 7
4.1.3. Provide Mechanisms not Requiring Full Time 4.1.3. Provide Mechanisms not Requiring Full Time
Synchronization . . . . . . . . . . . . . . . . . . . 7 Synchronization . . . . . . . . . . . . . . . . . . . 7
4.1.4. Support Asynchronization based Methods . . . . . . . 7 4.1.4. Support Asynchronization based Methods . . . . . . . 7
4.2. Support Large Single-hop Propagation Latency . . . . . . 7 4.2. Support Large Single-hop Propagation Latency . . . . . . 7
4.3. Accommodate the Higher Link Speed . . . . . . . . . . . . 8 4.3. Accommodate the Higher Link Speed . . . . . . . . . . . . 8
4.4. Be Scalable . . . . . . . . . . . . . . . . . . . . . . . 9 4.4. Be Scalable to Numerous Network Devices and Massive Traffic
4.4.1. Be Scalable to Numerous Network Devices . . . . . . . 9 Flows . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.4.2. Be Scalable to Massive Traffic Flows . . . . . . . . 9
4.5. Tolerate Failures of Links or Nodes and Topology 4.5. Tolerate Failures of Links or Nodes and Topology
Changes . . . . . . . . . . . . . . . . . . . . . . . . . 10 Changes . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.6. Support Configuration of Multiple Queueing Mechanisms . . 10 4.6. Support Configuration of Multiple Queueing Mechanisms . . 10
4.7. Support Queueing Mechanisms Switchover Crossing 4.7. Support Queueing Mechanisms Switchover Crossing
Multi-domains . . . . . . . . . . . . . . . . . . . . . . 11 Multi-domains . . . . . . . . . . . . . . . . . . . . . . 11
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 11 5. Data Plane Enhancement Requirements . . . . . . . . . . . . . 11
6. Security Considerations . . . . . . . . . . . . . . . . . . . 11 5.1. Support Aggregated Flow Identification . . . . . . . . . 11
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 5.2. Support Queuing Related Information . . . . . . . . . . . 12
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 11 5.3. Support Redundancy Related Fields . . . . . . . . . . . . 12
9. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 11 5.4. Support Explicit Path Selection . . . . . . . . . . . . . 12
10. Normative References . . . . . . . . . . . . . . . . . . . . 12 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 12
7. Security Considerations . . . . . . . . . . . . . . . . . . . 12
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13
10. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 13
11. Normative References . . . . . . . . . . . . . . . . . . . . 13
Appendix A. Examples of Large-Scale Deterministic Network Appendix A. Examples of Large-Scale Deterministic Network
Trials . . . . . . . . . . . . . . . . . . . . . . . . . 14 Trials . . . . . . . . . . . . . . . . . . . . . . . . . 15
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16
1. Introduction 1. Introduction
Packet networks are evolving from bandwidth-guaranteed Quality of Packet networks are evolving from bandwidth-guaranteed Quality of
Service (QoS) to latency-guaranteed QoS that guarantees bounded Service (QoS) to latency-guaranteed QoS that guarantees bounded
latency and definite latency. Bounded latency and definite latency latency and definite latency. Bounded latency and definite latency
can be further understood as in-time delivery, in which a packet can be further understood as in-time delivery, in which a packet
arrives without exceeding a predetermined time, and on-time delivery, arrives without exceeding a predetermined time, and on-time delivery,
in which a packet arrives at a predetermined time, respectively. In in which a packet arrives at a predetermined time, respectively. In
addition, network survivability, which typically guarantees traffic addition, network survivability, which typically guarantees traffic
skipping to change at page 3, line 28 skipping to change at page 3, line 28
Networking (DetNet) technology are considered to be essential. Networking (DetNet) technology are considered to be essential.
TSN is a set of standards developed by the IEEE 802.1 TSN Task Group TSN is a set of standards developed by the IEEE 802.1 TSN Task Group
(TG) [IEEE802.1TSN] and specifies mechanisms and protocols necessary (TG) [IEEE802.1TSN] and specifies mechanisms and protocols necessary
to realize highly available IEEE 802.1 networks with bounded latency to realize highly available IEEE 802.1 networks with bounded latency
to carry time-sensitive, real-time application traffic. to carry time-sensitive, real-time application traffic.
DetNet, of which architecture is defined in RFC 8655 [RFC8655], DetNet, of which architecture is defined in RFC 8655 [RFC8655],
provides a capability to carry specified unicast or multicast data provides a capability to carry specified unicast or multicast data
flows for real-time applications with extremely low data loss rates flows for real-time applications with extremely low data loss rates
and bounded latency within a network domain. Various documents on and bounded latency within a network domain. The overall framework
data planes and their interworking technologies to extend the service for DetNet data plane is provided in [RFC8938], and various documents
range of data that TSN intends to deliver to the IP (Internet on different data plane technologies and their interworking
Protocol) and MPLS (Multi-Protocol Label Switching) networks have technologies to extend the service range of data that TSN intends to
been standardized. deliver to the IP (Internet Protocol) and MPLS (Multi-Protocol Label
Switching) networks have been standardized.
Since TSN and DetNet were proposed, application use cases have always Since TSN and DetNet were proposed, application use cases have always
been one of the hottest topics. As documented in RFC 8578 [RFC8578], been one of the hottest topics. As documented in RFC 8578 [RFC8578],
the scope of networks addressed by the current DetNet is limited to the scope of networks addressed by the current DetNet is limited to
networks that can be centrally controlled, i.e., an "enterprise" (aka networks that can be centrally controlled, i.e., an "enterprise" (aka
"corporate") network, excluding "the open Internet," explicitly. "corporate") network, excluding "the open Internet," explicitly.
After years of development, TSN has been used in several industries, After years of development, TSN has been used in several industries,
and has enough public awareness of the industry for its scope. and has enough public awareness of the industry for its scope.
DetNet also has done a lot of work and the standards are mature, and DetNet also has done a lot of work and the standards are mature, and
people become concerned about how to meet deterministic service people become concerned about how to meet deterministic service
demand in large-scale networks. The current DetNet is limited to a demand in large-scale networks. The current DetNet is limited to a
single administrative domain network, and there are technical single administrative domain network, and there are technical
elements necessary for application to a large-scale network spanning elements necessary for application to a large-scale network spanning
multiple domains. multiple domains.
This document describes requirements for large-scale deterministic This document describes requirements for large-scale deterministic
networks where different deterministic levels of applications co- networks where different deterministic levels of applications co-
exist and large-scale deterministic networking across multiple exist and large-scale deterministic networking across multiple
administrative domains is possible. administrative domains is possible. This document also describes the
requirements for enhancing the DetNet data plane defined prior to
this document.
2. Conventions Used in This Document 2. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119][RFC8174] when, and only when, they appear in all 14 [RFC2119][RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
While [RFC2119] and [RFC8174] describe interpretations of these key While [RFC2119] and [RFC8174] describe interpretations of these key
skipping to change at page 4, line 49 skipping to change at page 5, line 12
just as a network provider provides different network services for just as a network provider provides different network services for
personal business or enterprise business. personal business or enterprise business.
One kind has critical SLA requirement, such as remote control or One kind has critical SLA requirement, such as remote control or
cloud Programmable Logic Controller (PLC) of manufacturing and cloud Programmable Logic Controller (PLC) of manufacturing and
differential protection of electricity. If these services exceed the differential protection of electricity. If these services exceed the
boundaries of latency and jitter, it will bring property losses and boundaries of latency and jitter, it will bring property losses and
security risks, so they cannot tolerate with any non-deterministic security risks, so they cannot tolerate with any non-deterministic
situation and can pay more on the network service. situation and can pay more on the network service.
Another kind has relatively lower levels of SLA requirement, such as Another kind has relatively losse levels of SLA requirement, such as
cloud gaming, cloud VR and online meeting for "consumer" networks. cloud gaming, cloud VR and online meeting for "consumer" networks.
The users of these applications hope to have a better network The users of these applications hope to have a better network
experience, but they can tolerate it to a certain extent. If the experience, but they can tolerate it to a certain extent. If the
network quality is not good sometime, they might be willing to spend network quality is not good sometime, they might be willing to spend
more money for high-quality network services. In some aspects, more money for high-quality network services. In some aspects,
because such services have no industry barriers and can tolerate because such services have no industry barriers and can tolerate
exceeding the upper boundary of latency within a small probability, exceeding the upper boundary of latency within a small probability,
they have relatively lower requirements for the network and may be they have relatively lower requirements for the network and may be
easier to deploy. easier to deploy.
skipping to change at page 6, line 4 skipping to change at page 6, line 12
Figure 1: Figure 1: Different levels of application requirements Figure 1: Figure 1: Different levels of application requirements
4. Technical Requirements in Large-Scale Deterministic Networks 4. Technical Requirements in Large-Scale Deterministic Networks
Due to the different kinds of application requirements in large-scale Due to the different kinds of application requirements in large-scale
networks, the corresponding technical requirements should be networks, the corresponding technical requirements should be
considered. considered.
4.1. Tolerate Time Asynchrony 4.1. Tolerate Time Asynchrony
4.1.1. Support Asynchronous Clocks Across Domains 4.1.1. Support Asynchronous Clocks Across Domains
A large-scale network may span over multiple networks with one or A large-scale network may span over multiple networks with one or
more administrators. One of DetNet's objectives is to stitch TSN more administrative domains. One of DetNet's objectives is to stitch
islands together. All devices inside a TSN domain are time- TSN islands together. All devices inside a TSN domain are time-
synchronized, and most of TSN technologies rely on precise time synchronized, and most of TSN technologies rely on precise time
synchronization[IEEE802.1Qbv][IEEE802.1Qch][IEEE802.1Qav]However, synchronization[IEEE802.1Qbv][IEEE802.1Qch][IEEE802.1Qav]However,
different TSN islands may have different clocks which are not different TSN islands may have different clocks which are not
synchronized as shown in Figure 2, where the time difference of two synchronized as shown in Figure 2, where the time difference of two
TSN domains is D. DetNet needs to connect these two TSN domains TSN domains is D. DetNet needs to connect these two TSN domains
together and provide end-to-end deterministic latency service. The together and provide end-to-end deterministic latency service. The
mechanism adopted by a large-scale deterministic network MUST support mechanism adopted by a large-scale deterministic network MUST support
the interaction across time domains, so that time domains are the interaction across time domains, so that time domains are
synchronized. This can be done, for example, by putting extra buffer synchronized. This can be done, for example, by putting extra buffer
space at the ingress of a new domain, increasing the dead time as a space at the ingress of a new domain, increasing the dead time as a
skipping to change at page 9, line 5 skipping to change at page 9, line 15
Another aspect to consider is the aggregation of the flows. In the Another aspect to consider is the aggregation of the flows. In the
large-scale network, the number of flows can be hundreds or tens of large-scale network, the number of flows can be hundreds or tens of
thousands. They can be aggregated into a small number of thousands. They can be aggregated into a small number of
deterministic path or tunnels. It is practical to have a few flow- deterministic path or tunnels. It is practical to have a few flow-
based or aggregated-flow based status in the local network. But in based or aggregated-flow based status in the local network. But in
higher speed and larger scale networks, it is hardly feasible. higher speed and larger scale networks, it is hardly feasible.
If[IEEE802.1Qcr]is in use, it requires more buffers comparing to the If[IEEE802.1Qcr]is in use, it requires more buffers comparing to the
other full/partial time synchronized mechanisms. Therefore, it other full/partial time synchronized mechanisms. Therefore, it
requires optimizations to support higher link speeds. requires optimizations to support higher link speeds.
4.4. Be Scalable 4.4. Be Scalable to Numerous Network Devices and Massive Traffic Flows
Comparing to a LAN, a large-scale network may have more network Comparing to a LAN, a large-scale network may have more network
devices and traffic flows, and there is a greater possibility of devices and traffic flows, and there is a greater possibility of
adding or removing network devices and traffic flows. The adding or removing network devices and traffic flows. The
deterministic latency forwarding mechanisms MUST scale to networks of deterministic latency forwarding mechanisms MUST scale to networks of
significant size with numerous network devices and a massive traffic significant size with numerous network devices and a massive traffic
flows. flows.
4.4.1. Be Scalable to Numerous Network Devices
The increase or decrease of network devices in large-scale networks The increase or decrease of network devices in large-scale networks
is more frequent than that in LANs. The change of the number of is more frequent than that in LANs. The change of the number of
devices may affect the implementation and adjustment of deterministic devices may affect the implementation and adjustment of deterministic
network mechanism, such as the topology discovery, queuing mechanism network mechanism, such as the topology discovery, queuing mechanism
and packet replication and elimination. A simple use case to and packet replication and elimination. A simple use case to
understand is ultra-low-latency (public) 5G transport networks, which understand is ultra-low-latency (public) 5G transport networks, which
would require DetNet extend to every 5G base station. For some would require DetNet extend to every 5G base station. For some
network operators, their networks may need to connect to ~100 K base network operators, their networks may need to connect to ~100 K base
stations (serving multiple mobile networks operators), and this stations (serving multiple mobile networks operators), and this
number will only increase with 5G. number will only increase with 5G.
4.4.2. Be Scalable to Massive Traffic Flows
It is almost impossible to identify individual IP flows at the DetNet It is almost impossible to identify individual IP flows at the DetNet
data plane because of the large overhead and resource reservation for data plane because of the large overhead and resource reservation for
a massive number of flows. DetNet allows the leverage of the flow a massive number of flows. DetNet allows the leverage of the flow
aggregation. With the large scaling of the network, proper provision aggregation. With the large scaling of the network, proper provision
at the control plane to accommodate such higher aggregation is at the control plane to accommodate such higher aggregation is
required. Individual flows may join and exit the aggregated flow required. Individual flows may join and exit the aggregated flow
rapidly which causes the dynamic in identification of the aggregated rapidly which causes the dynamic in identification of the aggregated
DetNet flow. The wildcards and value ranges used in the DetNet flow. The wildcards and value ranges used in the
identification may have to change in order to ensure the aggregated identification may have to change in order to ensure the aggregated
flows have compatible deterministic characteristics. flows have compatible deterministic characteristics.
skipping to change at page 11, line 26 skipping to change at page 11, line 26
taken into consideration. For example, when a flow is forwarded taken into consideration. For example, when a flow is forwarded
across multiple network domains based on different queueing across multiple network domains based on different queueing
mechanisms, such as a time synchronous Qbv mechanism[IEEE802.1Qbv] mechanisms, such as a time synchronous Qbv mechanism[IEEE802.1Qbv]
and an asynchronous Qcr mechanism [IEEE802.1Qcr], a collaboration and an asynchronous Qcr mechanism [IEEE802.1Qcr], a collaboration
mechanism crossing multi-domains MUST be considered, such as mechanism crossing multi-domains MUST be considered, such as
increasing the buffer of inter-domain devices to provide enough increasing the buffer of inter-domain devices to provide enough
adjustment space for the flow to cross different queueing mechanisms, adjustment space for the flow to cross different queueing mechanisms,
so as to provide end-to-end deterministic services across multiple so as to provide end-to-end deterministic services across multiple
network domains. network domains.
5. Conclusion 5. Data Plane Enhancement Requirements
According to[RFC8938], the DetNet data plane can provide or carry two
metadata in MPLS and IP data planes: Flow-ID and sequence number.
The Flow-ID could be used for identification of the DetNet flow or
aggregate flow, and the sequence number could be used for PREOF for
each DetNet flow. The Flow-ID is used by both the service and
forwarding sub-layers, but the sequence number is only used by the
service layer. Metadata can also be used for OAM indications and
instrumentation of DetNet data plane operation.
Generally speaking, more data plane metadata and related processing
SHOULD be supported in the large-scale networks. Native IPv6 data
plane should be supported. This section lists the data plane
enhancement requirements based on but not limited to the technical
requirements in Section 4.
5.1. Support Aggregated Flow Identification
Current IPv6 aggregated flow identification is generally based on 5
or 6 tuples, IP prefixes, or wildcards as indicated in [RFC8938].
However, in large-scale deterministic networks the number of
individual flows is huge, and they may randomly join and leave the
aggregated flow at each hop. Such behaviours lead to the difficulty
in identifying aggregated flows by relying on the prefixes or
wildcards.
In addition, flow identification is also used to quickly push a
packet to a suitable queue. In a large-scale network, there are mix
of flows requiring deterministic latency service and normal
forwarding service. Explicit flow identification makes it easier to
quickly distinguish the DetNet flows without requiring the longest
match rule on multiple tuples in IP data plane. Therefore, explicit
aggregated flow identification SHOULD be supported.
5.2. Support Queuing Related Information
According to Section 4.1, a large-scale network should support
synchronized or asynchronized queuing mechanisms. Different queueing
mechanisms require different metadata to be defined to help
regulation and queue management. For instance, the data plane MUST
support the identification of cycle for cyclic queuing or the timing
related information for time based queuing.
5.3. Support Redundancy Related Fields
Sequence number is the only metadata currently defined for redundancy
feature of Detnet. MPLS data plane uses Detnet-over-MPLS label stack
to carry it. At the same time, native IPv6 data plane should be able
to carry this information too. If specific IP encapsulation or
tunnel is in use, this meta data should be defined explicitly for
that data plane.
5.4. Support Explicit Path Selection
Explicit route at the control plane and/or management is required so
that the "best" path can be selected to meet the latency requirement
for DetNet flows. At the data planes, MPLS label stack can be used
for this purpose. IP data plane enhancement is required to support
the explicit path selection based on IP source routing or SRv6.
6. Conclusion
This document specifies the technical requirements when ensuring the This document specifies the technical requirements when ensuring the
deterministic features in the large-scale networks. Some of the deterministic features in the large-scale networks, and the
proposed queueing mechanisms are analyzed and the authors of the corresponding data plane enhancement requirements to support the
document think those proposals give reasonably sound insights to them. Some of the proposed queueing mechanisms and trials are cited
enhancement the current queueing mechanisms to meet the deterministic and the authors of the document think those proposals give reasonably
requirements of the large-scale networks. sound insights to enhancement the current queueing mechanisms to meet
the deterministic requirements of the large-scale networks.
6. Security Considerations 7. Security Considerations
There are no IANA actions required by this document. There are no IANA actions required by this document.
7. IANA Considerations 8. IANA Considerations
This section will be described later. This section will be described later.
8. Acknowledgements 9. Acknowledgements
The authors would like to thank Yaakov Stein for helpful suggestions. The authors would like to thank Yaakov Stein for helpful suggestions.
The authors also would like to thank Liang Geng, Peter Willis, The authors also would like to thank Liang Geng, Peter Willis,
Shunsuke Homma and Li Qiang for their previous works. Shunsuke Homma and Li Qiang for their previous works.
9. Contributors 10. Contributors
The following people have substantially contributed to this document: The following people have substantially contributed to this document:
Zongpeng Du Zongpeng Du
China Mobile China Mobile
EMail: duzongpeng@chinamobile.com EMail: duzongpeng@chinamobile.com
10. Normative References 11. Normative References
[Fast-Ethernet-MII-clock] [Fast-Ethernet-MII-clock]
"Fast Ethernet MII clock". "Fast Ethernet MII clock".
[G.8262] International Telecommunication Union, "Timing [G.8262] International Telecommunication Union, "Timing
characteristics of a synchronous equipment slave clock", characteristics of a synchronous equipment slave clock",
ITU-T Recommendation G.8262, November 2018. ITU-T Recommendation G.8262, November 2018.
[G.8273] International Telecommunication Union, "Framework of phase [G.8273] International Telecommunication Union, "Framework of phase
and time clocks", ITU-T Recommendation G.8273, March 2018. and time clocks", ITU-T Recommendation G.8273, March 2018.
skipping to change at page 14, line 18 skipping to change at page 15, line 28
[RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases", [RFC8578] Grossman, E., Ed., "Deterministic Networking Use Cases",
RFC 8578, DOI 10.17487/RFC8578, May 2019, RFC 8578, DOI 10.17487/RFC8578, May 2019,
<https://www.rfc-editor.org/info/rfc8578>. <https://www.rfc-editor.org/info/rfc8578>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655, "Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019, DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>. <https://www.rfc-editor.org/info/rfc8655>.
[RFC8938] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
Bryant, "Deterministic Networking (DetNet) Data Plane
Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020,
<https://www.rfc-editor.org/info/rfc8938>.
Appendix A. Examples of Large-Scale Deterministic Network Trials Appendix A. Examples of Large-Scale Deterministic Network Trials
Some trials have been carried out to verify the concept of large- Some trials have been carried out to verify the concept of large-
scale deterministic networks. scale deterministic networks.
In order to verify the deterministic technology of large-scale In order to verify the deterministic technology of large-scale
networks, a trial of Deterministic IP on China Environment for networks, a trial of Deterministic IP on China Environment for
Network Innovations (CENI), which is a network built for new network Network Innovations (CENI), which is a network built for new network
technology trial, was deployed. A network with a distance of 3,000 technology trial, was deployed. A network with a distance of 3,000
km over 13 hops was tested, and the jitter was controlled within km over 13 hops was tested, and the jitter was controlled within
 End of changes. 28 change blocks. 
46 lines changed or deleted 118 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/