< draft-lai-bmwg-istn-methodology-00.txt   draft-lai-bmwg-istn-methodology-01.txt >
Benchmarking Methodology Working Group Z. Lai Benchmarking Methodology Working Group Z. Lai
Internet-Draft H. Li Internet-Draft H. Li
Intended status: Informational Y. Deng Intended status: Informational Y. Deng
Expires: 28 April 2022 Q. Wu Expires: 29 October 2022 Q. Wu
J. Liu J. Liu
Tsinghua University Tsinghua University
25 October 2021 27 April 2022
Problems and Requirements of Evaluation Methodology for Integrated Space Problems and Requirements of Evaluation Methodology for Integrated Space
and Terrestrial Networks and Terrestrial Networks
draft-lai-bmwg-istn-methodology-00 draft-lai-bmwg-istn-methodology-01
Abstract Abstract
With the rapid evolution of the aerospace industry, many "NewSpace" With the rapid evolution of the aerospace industry, many "NewSpace"
upstarts are actively deploying their mega-constellations in low upstarts are actively deploying their mega-constellations in low
Earth orbits (LEO) and building integrated space and terrestrial earth orbits (LEO) and building integrated space and terrestrial
networks (ISTN), promising to provide pervasive, low-latency, and networks (ISTN), promising to provide pervasive, low-latency, and
high-throughput Internet service globally. Due to the high high-throughput Internet service globally. Due to the high
manufacturing, launching, and updating cost of LEO mega- manufacturing, launching, and updating cost of LEO mega-
constellations, it is expected that ISTNs can be well designed and constellations, it is expected that ISTNs can be well designed and
evaluated before the launch of satellites. However, the progress of evaluated before the launch of satellites. However, the progress of
designing, assessing, and understanding new network functionalities designing, assessing, and understanding new network functionalities
and protocols for futuristic ISTNs faces a substantial obstacle: lack and protocols for futuristic ISTNs faces a substantial obstacle: lack
of standardized evaluation methodology with acceptable realism, of standardized evaluation methodology with acceptable realism (e.g.
flexibility, and cost that can involve the unique dynamic behaviors can involve the unique dynamic behaviors of ISTNs), flexibility, and
of ISTNs. This memo first reviews the unique characteristics of LEO cost. This memo first reviews the unique characteristics of LEO
mega-constellations. Further, it analyzes the limitation of existing mega-constellations. Further, it analyzes the limitation of existing
network evaluation and analysis methodologies under ISTN evaluation and analysis methodologies under ISTN environments.
environments. Finally, it outlines the key requirements of future Finally, it outlines the key requirements of future evaluation
evaluation methodology tailored for ISTNs. methodology tailored for ISTNs.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 28 April 2022. This Internet-Draft will expire on 29 October 2022.
Copyright Notice Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the Copyright (c) 2022 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components and restrictions with respect to this document. Code Components
extracted from this document must include Simplified BSD License text extracted from this document must include Revised BSD License text as
as described in Section 4.e of the Trust Legal Provisions and are described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Simplified BSD License. provided without warranty as described in the Revised BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Notation and Terminology . . . . . . . . . . . . . . . . . . 4 2. Notation and Terminology . . . . . . . . . . . . . . . . . . 4
3. Quick Primer for Integrated Space and Terrestrial Networks . 5 3. Quick Primer for Integrated Space and Terrestrial Networks . 5
3.1. Mega-constellation . . . . . . . . . . . . . . . . . . . 5 3.1. Mega-constellation . . . . . . . . . . . . . . . . . . . 5
3.2. Topological Dynamics . . . . . . . . . . . . . . . . . . 6 3.2. Topological Dynamics . . . . . . . . . . . . . . . . . . 6
3.3. Long Manufacturing and Deployment Duration . . . . . . . 7 3.3. Limited Resources . . . . . . . . . . . . . . . . . . . . 7
3.4. Long Manufacturing and Deployment Duration . . . . . . . 8
4. Problem Statement: We Need the Right Evaluation 4. Problem Statement: We Need the Right Evaluation
Methodology . . . . . . . . . . . . . . . . . . . . . . . 8 Methodology . . . . . . . . . . . . . . . . . . . . . . . 8
4.1. Live networks and platforms . . . . . . . . . . . . . . . 8 4.1. Live networks and platforms . . . . . . . . . . . . . . . 9
4.2. Network Simulators . . . . . . . . . . . . . . . . . . . 9 4.2. Network Simulators . . . . . . . . . . . . . . . . . . . 10
4.3. Network Emulators . . . . . . . . . . . . . . . . . . . . 10 4.3. Network Emulators . . . . . . . . . . . . . . . . . . . . 11
4.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.4. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 12
5. Requirements: New Evaluation Methodology Tailored for 5. Requirements: New Evaluation Methodology Tailored for
ISTNs . . . . . . . . . . . . . . . . . . . . . . . . . . 12 ISTNs . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.1. Realism . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.1. Realism . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.2. Flexibility . . . . . . . . . . . . . . . . . . . . . . . 13 5.2. Flexibility . . . . . . . . . . . . . . . . . . . . . . . 13
5.3. Low-cost and Easy-to-use . . . . . . . . . . . . . . . . 13 5.3. Low-cost and Easy-to-use . . . . . . . . . . . . . . . . 13
5.4. Cross-domain Dataset Support . . . . . . . . . . . . . . 13 5.4. Cross-domain Dataset Support . . . . . . . . . . . . . . 13
6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 14 6. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 14
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
9. Security Considerations . . . . . . . . . . . . . . . . . . . 14 9. Security Considerations . . . . . . . . . . . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 14
10.1. Normative References . . . . . . . . . . . . . . . . . . 14 10.1. Normative References . . . . . . . . . . . . . . . . . . 14
10.2. Informative References . . . . . . . . . . . . . . . . . 15 10.2. Informative References . . . . . . . . . . . . . . . . . 15
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 16 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction 1. Introduction
Integrated Space and Terrestrial Networks (ISTN), combining diverse Integrated Space and Terrestrial Networks (ISTN), combining diverse
spacecraft and ground infrastructures, are extending the frontier of spacecrafts and ground infrastructures, are extending the frontier of
today's terrestrial network, promising to provide low-latency, high- today's terrestrial network, promising to provide low-latency, high-
bandwidth Internet access with broader coverage globally. bandwidth Internet access with broader coverage globally.
Low Earth Orbit (LEO) satellites are the key building block for Low earth orbit (LEO) satellites are the key building block for
constructing ISTN. Recently, we have witnessed a renaissance in the constructing ISTNs. Recently, we have witnessed a renaissance in the
space industry, stimulating an exponential increase in constructing space industry, stimulating an exponential increase in constructing
mega-constellations. As compared to their predecessor, cutting-edge mega-constellations. As compared with their predecessor, cutting-
satellites can be equipped with high-resolution sensors, space-grade edge satellites can be equipped with high-resolution sensors, space-
multi-core processors, high-data-rate communication links, and grade multi-core processors, high-data-rate communication links, and
multifunctional space software. multifunctional space software.
While ISTNs hold great promise, to completely unleash the network While ISTNs hold great promise, to completely unleash the network
potential of emerging ISTN, it still needs to address a series of new potential of emerging ISTN, it still needs to address a series of new
research issues. The unique characteristics of LEO satellites (e.g., technical issues. The unique characteristics of LEO satellites
high-dynamics), not only impose new challenges at various layers of (e.g., high-dynamics), not only impose new challenges at various
the ISTN networking stack but also open the door to many new research layers of the ISTN networking stack but also open the door to many
problems. With many unexplored problems facing the "NewSpace" new technical problems. With many unexplored problems facing the
industry, it is thus foreseen that in the near future, there will be "NewSpace" industry, it is thus foreseen that in the near future,
a surge of new research (e.g. topology, addressing, routing, there will be a surge of new efforts (e.g. topology, addressing,
transport, etc.) to rethink and reshape network functionalities and routing, transport, etc.) to rethink and reshape the networking stack
protocols in ISTNs. In addition, the cost/ timeline of in ISTNs. In addition, the cost/timeline of manufacturing,
manufacturing, launching, operating, and updating satellite launching, operating, and updating satellite constellations is
constellations is typically much higher/longer than that in typically much higher/longer than that in traditional terrestrial
traditional terrestrial networks. Therefore, it is expected that new networks. Therefore, it is expected that new network functionalities
network functionalities and protocols can be well evaluated before and protocols can be well evaluated before they are launched and
they are launched and deployed in realistic satellite constellations. deployed in realistic satellite constellations.
However, the network community lacks the proper analysis tools and However, the network community lacks the proper analysis tools and
evaluation methodologies that can mimic the unique dynamic behavior evaluation methodologies that can mimic the unique dynamic behavior
to analyze many of the ISTN challenges that have been highlighted by to analyze many of the ISTN challenges that have been highlighted by
prior studies. At high level, existing evaluation methodologies in prior works. At high level, existing evaluation methodologies in the
the network community can typically be grouped into three major network community can typically be grouped into three major
categories: live networks or platforms, simulation, and emulation. categories: live networks or platforms, simulation, and emulation.
However, the feasibility and flexibility of live satellite networks However, the feasibility and flexibility of live satellite networks
are technically and economically limited. The abstraction level of are technically and economically limited. The abstraction level of
network simulation might be too high to capture low-level system network simulation could be too high to capture low-level system
effects. Existing network emulators fail to characterize the high effects. Existing network emulators fail to characterize the high
dynamicity of LEO satellites and thus cannot accomplish an dynamicity of LEO satellites and thus cannot accomplish an
environment with acceptable fidelity. The community hence needs a environment with acceptable fidelity. The community hence needs a
reasonable and standardized evaluation methodology to build proper reasonable and standardized evaluation methodology to build proper
experimental environments which can mimic the behavior of ISTNs, experimental environments which can mimic the behavior of ISTNs,
supporting the community to deeply understand the problems, and to supporting the community to deeply understand the problems, and to
evaluate new functionalities and protocols (e.g. for topology, evaluate new functionalities and protocols (e.g. for topology,
addressing, routing, transport, etc.) for ISTNs, before the mega- addressing, routing, transport, etc.) for ISTNs, before the mega-
constellation is completely deployed. In this memo, we first review constellation is completely deployed. In this memo, we first review
the unique characteristics of emerging LEO mega-constellations and the unique characteristics of emerging LEO mega-constellations and
skipping to change at page 5, line 15 skipping to change at page 5, line 15
3. Quick Primer for Integrated Space and Terrestrial Networks 3. Quick Primer for Integrated Space and Terrestrial Networks
Emerging mega-constellations with inter-satellite links (ISLs) can Emerging mega-constellations with inter-satellite links (ISLs) can
build a satellite network in outer space, and further be integrated build a satellite network in outer space, and further be integrated
with terrestrial ground infrastructures to construct an integrated with terrestrial ground infrastructures to construct an integrated
space and terrestrial network (ISTN). space and terrestrial network (ISTN).
3.1. Mega-constellation 3.1. Mega-constellation
A constellation is a group of satellites working as a system to give A constellation is a group of satellites working as a system to give
a coverage of the Earth surface, among which satellites are a coverage of the earth surface, among which satellites are
positioned in fixed orbital planes with regular trajectories. LEO positioned in fixed orbital planes with regular trajectories. LEO
and MEO satellites often belong to a constellation, because a single and MEO satellites often belong to a constellation, because a single
satellite only covers a small area with high angular velocity. Thus, satellite only covers a small area with high angular velocity. Thus,
continuous coverage over an area could be maintained by the relay continuous coverage over an area could be maintained by the relay
within a constellation, as compared with GEO satellites that only within a constellation, as compared with GEO satellites that only
provides a permanent coverage over a target area. Walker Delta provides a permanent coverage over a target area. Walker Delta
constellation is the most common formation for constellations. It is constellation is the most common formation for constellations. It is
defined as a bunch of circular orbits with a fixed inclination, defined as a bunch of circular orbits with a fixed inclination,
satellite number, number of equally spaced planes and the relative satellite number, number of equally spaced planes and the relative
spacing between satellites in adjacent planes. The famous Ballard spacing between satellites in adjacent planes. The famous Ballard
skipping to change at page 7, line 39 skipping to change at page 7, line 39
| Kuiper | 5.0562 | | Kuiper | 5.0562 |
+----------+--------------+ +----------+--------------+
| OneWeb | 10.6824 | | OneWeb | 10.6824 |
+----------+--------------+ +----------+--------------+
| Telesat | 45.5696 | | Telesat | 45.5696 |
+----------+--------------+ +----------+--------------+
Table 2: Space-ground Table 2: Space-ground
link churn interval. link churn interval.
3.3. Long Manufacturing and Deployment Duration 3.3. Limited Resources
Space resources (e.g. CPU, energy) on satellites are limited, as
compared with terrestrial network. Since resource-constrained
satellites such as nanosatellites are only able to carry certain
sennsing or transferring missions, energy-consuming or complex tasks
may not be achievable in these satellites. Such complicated tasks
include on-board target identification and instant and continuous
disaster monitoring.
For example, the CPU frequency of current spaceborne processors (e.g.
RAD5545 [RAD5545], RAD750 [RAD750]) is only up to 466MHz per core.
More recently, some low energy-consuming commodity processors are
used in space to complete certain remote sensing missions under a
limited CPU capacity. [raspberry-pi] With a constrained computation
ability and limited storage and energy, satellite functions and
lifetime are greatly repressed.
3.4. Long Manufacturing and Deployment Duration
Different from terrestrial network infrastructures, the timeline of Different from terrestrial network infrastructures, the timeline of
manufacturing and deploying satellite networks could be much longer manufacturing and deploying satellite networks could be much longer
due to the high cost and complex process during the development and due to the high cost and complex process during the development and
launch period. Satellites, as well as the orbit and spectrum they launch period. Satellites, as well as the orbit and spectrum they
used, have to be regulated, and launches have to be carefully used, have to be regulated, and launches have to be carefully
scheduled (e.g. to avoid the impact of poor weather conditions). In scheduled (e.g. to avoid the impact of poor weather conditions). In
addition, the maintenance and update cost of a satellite network is addition, the maintenance and update cost of a satellite network is
also typically much higher than that of a terrestrial network. also typically much higher than that in a terrestrial network.
For example, a review of 24 Air Force and Navy space vehicle (SV) For example, a review of 24 Air Force and Navy space vehicle (SV)
development programs found that on average it took about 7.5 years development programs found that on average it took about 7.5 years
from contract start to launch a government satellite. from contract start to launch a government satellite.
[Development-Timeline] Commercial satellite programs typically take 2 [Development-Timeline] Commercial satellite programs typically take 2
to 3 years from contract start to launch. [Production-Cycles] to 3 years from contract start to launch. [Production-Cycles]
SpaceX's Starlink constellation plan to launch about 42,000 SpaceX's Starlink constellation plan to launch about 42,000
satellites to construct a mega-constellation in outer space. On 15 satellites to construct a mega-constellation in outer space. On 15
October 2019, the United States Federal Communications Commission October 2019, the United States Federal Communications Commission
(FCC) submitted filings to the International Telecommunication Union (FCC) submitted filings to the International Telecommunication Union
(ITU) on SpaceX's behalf to arrange spectrum for 30,000 additional (ITU) on SpaceX's behalf to arrange spectrum for 30,000 additional
Starlink satellites to supplement the 12,000 Starlink satellites Starlink satellites to supplement the 12,000 Starlink satellites
already approved by the FCC. As of the date of September 2021, two already approved by the FCC. As of the date of April 2022, SpaceX
years after the first launch in May 2019, SpaceX has launched about has launched about 2,100 Starlink satellites, which is about 5% of
1791 Starlink satellites, which is about 4% of the ultimate the ultimate constellation plan consisting of 42,000 satellites.
constellation plan consisting of 42,000 satellites. Foreseeably, it Foreseeably, it may take many years to complete the entire
may take many years to complete the entire constellation deployment. constellation deployment. Even the first phase of Starlink which
Even the first phase of Starlink which consists of about 4400 consists of about 4400 satellites is not expected to be completed
satellites is not expected to be completed until 2024. until 2024.
4. Problem Statement: We Need the Right Evaluation Methodology 4. Problem Statement: We Need the Right Evaluation Methodology
The unique characteristics of LEO mega-constellations involve new The unique characteristics of LEO mega-constellations involve new
challenges on various layers of the networking stack of ISTNs. On challenges on various layers of the networking stack of ISTNs. On
one hand, it is foreseen that in the near future, there will be a one hand, it is foreseen that in the near future, there will be a
surge of new network functionalities and protocols designed or surge of new network functionalities and protocols designed or
optimized for ISTNs. On the other hand, because the cost/timeline of optimized for ISTNs. On the other hand, because the cost/timeline of
manufacturing, launching, operating, and updating satellite manufacturing, launching, operating, and updating satellite
constellations is typically much higher/longer than that in constellations is typically much higher/longer than that in
skipping to change at page 9, line 13 skipping to change at page 9, line 29
companies. And for the resource competition, each independent companies. And for the resource competition, each independent
experiment needs to completely monopolize a part of the test bed, so experiment needs to completely monopolize a part of the test bed, so
the researcher cannot deploy the experiment until being allocated the researcher cannot deploy the experiment until being allocated
with enough nodes. PlanetLab [PlanetLab] is truly global ground with enough nodes. PlanetLab [PlanetLab] is truly global ground
testbed prototype. Started from 2003, it consists of 1353 nodes at testbed prototype. Started from 2003, it consists of 1353 nodes at
717 sites spanning 48 countries. Together the nodes form a global 717 sites spanning 48 countries. Together the nodes form a global
network system to support new design of network services. network system to support new design of network services.
The live platforms described above were initially proposed for The live platforms described above were initially proposed for
terrestrial networks and they are developed and repaired at the same terrestrial networks and they are developed and repaired at the same
time. The key limitation of them in ISTN environment is that they time. The key limitation of them in an ISTN environment is that they
are designed for terrestrial network experiments, and do not are designed for terrestrial network experiments, and do not
incorporate the realistic characteristic of LEO mega-constellations incorporate the realistic characteristic of LEO mega-constellations
to support experiments and evaluations in ISTNs. to support experiments and evaluations in ISTNs.
We may search for help from live satellites, but still there is only We may search for help from live satellites, but still there is only
limited help. It seems that with the help of live ISTN, researchers limited help. It seems that with the help of live ISTN, researchers
are capable to assess, verify and evaluate their ideas and thoughts. are capable to assess, verify and evaluate their ideas and thoughts.
Live ISTN can give a real constellation-consistency and stack- Live ISTN can give a real constellation-consistency and stack-
consistency testing environment. However, current satellites only consistency testing environment. However, current satellites only
provide users a bent-pipe service, which is purely relaying the provide users a bent-pipe service, which is purely relaying the
skipping to change at page 9, line 38 skipping to change at page 10, line 7
satellites. Besides, the access to satellites is also limited. satellites. Besides, the access to satellites is also limited.
Therefore, live networks or platforms for terrestrial networks can Therefore, live networks or platforms for terrestrial networks can
give us a large-scale experimental environment but they lack the give us a large-scale experimental environment but they lack the
support for ISTN characteristics. On the other hand, live ISTN is support for ISTN characteristics. On the other hand, live ISTN is
able to guarantee a real space environment, but it is not that able to guarantee a real space environment, but it is not that
affordable and flexible. affordable and flexible.
4.2. Network Simulators 4.2. Network Simulators
Simulators are testing methodology tools that enable researchers to Simulators are tools that enable researchers to reproduce their
reproduce their testing experiments by simulating a real-world testing experiments by simulating a real-world process or system over
process or system over time. Simulators work by using discrete event time. Simulators work by using discrete event simulation to
simulation to calculate the interactive states among all the network calculate the interactive states among all the network entities,
entities, ranging from switches, routers, nodes, access points, links ranging from switches, routers, nodes, access points, links and so
and so on. While working fast and efficiently, the fidelity is only on. While working fast and efficiently, the fidelity is only brought
brought by the state variable changes at discrete points. by the state variable changes at discrete points.
Such tools like Systems Tool Kit (STK) [Systems-Tool-Kit] and General Such tools like Systems Tool Kit (STK) [Systems-Tool-Kit] and General
Mission Analysis Tool (GMAT) [General-Mission-Analysis-Tool] are good Mission Analysis Tool (GMAT) [General-Mission-Analysis-Tool] are good
for orbit analysis. STK is a powerful tool to help researchers to for orbit analysis. STK is a powerful tool to help researchers to
model the behavior of mission entities in aerospace, model the behavior of mission entities in aerospace,
telecommunications and so forth. It also provides visualization and telecommunications and so forth. It also provides visualization and
analysis functions. GMAT is a similar tool for space trajectory analysis functions. GMAT is a similar tool for space trajectory
optimization and mission modeling. Nevertheless, both tools do not optimization and mission modeling. Nevertheless, these tools do not
support networking simulations such as topology and protocol support networking simulations such as topology and protocol
simulations. ns-3 [ns-3] goes a step further with support for simulations. ns-3 [ns-3] goes a step further with support for
Internet simulation, but on the contrary, it was not designed for Internet simulation, but on the contrary, it was not designed for
ISTN and lacks the support for high-dynamics of ISTN. StarPerf ISTN and lacks the support for high-dynamics of ISTN. StarPerf
[StarPerf] is a simulator that helps researchers to study network [StarPerf] is a simulator that helps researchers to study network
performance under a range of constellation conditions. But still, it performance under a range of constellation conditions. But still, it
lacks the ability to support interactive network traffic simulation lacks the ability to support interactive network traffic simulation
and system codes in the systems. and system codes in the systems.
Overall, while flexible and low-cost, the realism of simulators is Overall, while flexible and low-cost, the realism of simulators is
not content enough, because they are too abstract to realize the low- not content enough, because they are difficult to describe the low-
level characteristics. In other words, simulators are being too level characteristics. In other words, simulators are being too
object-oriented to involve additional overhead in the actual object-oriented to involve additional overhead in the actual
execution of programs. Besides, when accessing the network execution of programs. Besides, when accessing the network
performance, a number of recent emerging algorithms for congestion performance, a number of recent emerging algorithms for congestion
control, reliable transmission or even protocols are not supported, control, reliable transmission or even protocols are not supported,
for example ns-3 [ns-3] only supports basic congestion control like for example ns-3 [ns-3] only supports basic congestion control like
Reno [RFC6582] and so forth, so the need to work with some new Reno [RFC6582] and so forth, so the need to work with some new
algorithms cannot be satisfied and the research to discover new algorithms cannot be satisfied and the research to discover new
mechanisms, such as new routing algorithms and re-transmission mechanisms, such as new routing algorithms and re-transmission
schemes, is extensively prohibited. Another problem of simulators, schemes, is extensively prohibited. Another problem of simulators,
such as ns-3 [ns-3], is that it difficult to trace or understand the such as ns-3 [ns-3], is that it difficult to trace or understand the
previous codes, without appropriate documentations. Simulators previous codes, without appropriate documentations. Simulators
usually face the additional compatibility problem, which means they usually face the additional compatibility problem, which means they
is not portable with other systems, or they do not support kernel are not portable with other systems, or they do not support kernel
codes. Since there are multiple simulators developed by different codes. Since there are multiple simulators developed by different
group of users, sometimes users are required to be familiar with the group of users, sometimes users are required to be familiar with the
writing language, scripting style and modelling technique. So, the writing language, scripting style and modelling technique.
Tool Command Language might be difficult to understand and write.
4.3. Network Emulators 4.3. Network Emulators
Emulators are another kind of paradigm for testing methodology tool Emulators are another kind of paradigm for network evaluation over a
over a virtual network. The difference between a simulator and an virtual network. The difference between a simulator and an emulator
emulator is that emulators leverage VM or containers to keep the is that emulators leverage VM or containers to keep the realism which
realism which is close to actual performances. Therefore, in is close to actual performances. Therefore, in emulators, virtual
emulators, virtual nodes. virtual network links, virtual models of nodes. virtual network links, virtual models of traffic, and
traffic, and protocols are all applied. Emulators are capable to run protocols are all applied. Emulators are capable to run real kernel
real kernel and application code. Thus, emulators not only support and application code. Thus, emulators not only support diverse
diverse topology design, but also protocol emulation in a synthetic topology design, but also protocol emulation in a synthetic network
network environment. They emulate the network behavior in a more environment. They emulate the network behavior in a more real way.
real way. Mininet [Mininet] is commonly regarded as the most Mininet [Mininet] is commonly regarded as the most illustrious
illustrious emulator for networking with its strong ability to emulator for networking with its strong ability to support
support experiments with Software-Defined Networking (SDN) experiments with Software-Defined Networking (SDN)
[Software-defined-networking] systems. EstiNet [EstiNet] is another [Software-defined-networking] systems. EstiNet [EstiNet] is another
emulator that supports evaluating and testing the performances of emulator that supports evaluating and testing the performances of
software-defined networks. Based on containers, they can emulate software-defined networks. Based on containers, they can emulate
real TCP/IP protocol stack in the Linux kernel. However, existing real TCP/IP protocol stack in the Linux kernel. However, existing
emulation tools lack the ability to construct the dynamic links and emulation tools lack the ability to construct the dynamic links and
orbits in ISTN like simulators. Thus, more problems could happen in orbits in ISTN like simulators. Thus, more problems could happen in
higher-level protocols such as routing protocols (e.g. OSPF and higher-level protocols such as routing protocols (e.g. OSPF and
BGP). Besides, since emulators run containers or virtual machines BGP). Besides, since emulators run containers or virtual machines
which occupy more software overhead, compared with simulators, it which occupy more software overhead, as compared with simulators, it
will be hard to emulate the large-scale mega-constellations. will be hard to emulate the large-scale mega-constellations.
Existing work has shown the capability of 25 physical machines
working together as a system to emulate 250 network nodes, but still,
it is far less for ISTN scalability.
To conclude, emulators are relatively good methodologies for network To conclude, emulators are relatively good methodologies for network
experiments, but emulators still have limitations when using them for experiments, but emulators still have limitations when using them for
ISTN research. While keeping a moderate realism by using VM or ISTN research. While keeping a moderate realism by using VM or
containers for entity emulation and flexibility, emulators still lack containers for entity emulation and flexibility, emulators still lack
the supports for ISTN characteristics, such as frequent link changes, the supports for ISTN characteristics, such as frequent link changes,
satellite network topology uncertainty, and so on. More satellite network topology uncertainty, and so on. More
specifically, current emulators only support fixed network topology specifically, current emulators only support fixed network topology
emulation. It is not flexible to emulate the time-varying link emulation. It is not flexible to emulate the time-varying link
packet loss, bandwidth, and other traits. A possible way is to packet loss, bandwidth, and other traits. A possible way is to
skipping to change at page 11, line 39 skipping to change at page 12, line 8
In addition, it is still difficult to emulate thousands or ten In addition, it is still difficult to emulate thousands or ten
thousand of satellites for ISTN even with VM or containers, subject thousand of satellites for ISTN even with VM or containers, subject
to hardware limitations. For flexibility, some emulators do not to hardware limitations. For flexibility, some emulators do not
support a good network animator tool. Especially in ISTN emulation, support a good network animator tool. Especially in ISTN emulation,
GUI is important for users to observe and analyze orbit trajectories GUI is important for users to observe and analyze orbit trajectories
and real time satellite positions. and real time satellite positions.
4.4. Summary 4.4. Summary
In this section, we explain the necessity of an evaluation In this section, we explain the necessity of an evaluation
methodology specifically for ISTN. Then we demonstrate the problems methodology specifically for ISTNs. Then we demonstrate the problems
with existing methodologies related to ISTN. The performance with existing methodologies related to ISTNs. The performance
comparison result is shown in Table 3. Above all, ISTN should be comparison result is shown in Table 3. Above all, ISTNs should be
designed first and then launched. Live satellites enable good designed first and then launched. Live satellites enable good
realism but they lack flexibility and require very high cost as well realism but they lack flexibility and require very high cost as well
as a very long deployment period. Other testing tools such as as a very long deployment period. Other testing tools such as
simulators and emulators are either functional for merely aerospace simulators and emulators are either functional for merely aerospace
analysis or simply terrestrial networks. None of the existing analysis or simply terrestrial networks. None of the existing
methodologies guarantees a practical and user-friendly methodology methodologies guarantees a practical and user-friendly methodology
while keeping the evaluation environment realism with low costs. while keeping the evaluation environment realism with low costs.
+================+=========+=============+======+=================+ +================+=========+=============+======+=================+
| Platform/Tool | Realism | Flexibility | Cost | Cross-domain | | Platform/Tool | Realism | Flexibility | Cost | Cross-domain |
skipping to change at page 12, line 39 skipping to change at page 13, line 8
A proper evaluation methodology tailored for ISTNs is expected to A proper evaluation methodology tailored for ISTNs is expected to
help developers, researchers, engineers to explore various design- help developers, researchers, engineers to explore various design-
space of the networking stack of ISTNs in a technically and space of the networking stack of ISTNs in a technically and
economically feasible manner. Based on the comparative analysis economically feasible manner. Based on the comparative analysis
results in the prior section, we sum up the following requirements results in the prior section, we sum up the following requirements
for the new evaluation methodology in ISTNs. for the new evaluation methodology in ISTNs.
5.1. Realism 5.1. Realism
The first requirement is realism. Realism represents the testing The first requirement is realism. Realism represents the testing
authenticity and fidelity, compared with real ISTN. It could be authenticity and fidelity. The evaluation methodology is expected to
further divided into constellation-consistency and networking stack keep the actual characteristics of mega-constellations. In other
realism. Constellation-consistency requires the testing to keep the words, the orbit-level information including the latitude, longitude,
actual characteristics of mega-constellations both spatially and and height of each satellite in any given time and the same
temporally. In other words, the orbit-level information including information for GS and elevation angles of antennas of each GS. Note
the latitude, longitude, and height of each satellite in any given that the constellation information also determines the visibility,
time and the same information for GS and elevation angles of antennas links and even topology of ISTN.s Since the mega-constellations are
of each GS spatially. Note that the spatial information also unstable, how the temporal satellite locations, visibility, link
determines the visibility, links and even topology of ISTN. Since propagation delays and so on should also be considered carefully. In
the mega-constellations are unstable, how the temporal satellite addition, it requires the network nodes to communicate and negotiate
locations, visibility, link propagation delays and so on should also their messages following the actual protocol process. For example,
be considered carefully. Similarly, the networking stack realism when doing a test for OSPF in an ISTN, we would like the nodes to
requires the network nodes to communicate and negotiate their send Hello packets, Link-State-Request (LSR) packets, Link-State-
messages following the actual protocol process. For example, when Update (LSU) packets and so on. A real network stack is preferred to
doing a test for OSPF in an ISTN, we would like the nodes to send
Hello packets, Link-State-Request (LSR) packets, Link-State-Update
(LSU) packets and so on. A real network stack is preferred to
provide researchers an opportunity to see the performance of provide researchers an opportunity to see the performance of
different protocols in ISTN. different protocols in ISTNs.
5.2. Flexibility 5.2. Flexibility
Another requirement is flexibility and feasibility. The testing Another requirement is flexibility and feasibility. The testing
methodology should be technically easy to use and easy to learn. methodology should be technically easy to use and easy to learn.
Without extra modifications or process, the methodology should help Without extra modifications or process, the methodology should help
researchers learn and use it without much effort and can evaluate researchers learn and use it without much effort and can evaluate
their ideas as they wish, which means it should support multiple their ideas as they wish, which means it should support flexible,
environments for researchers. controllable environments for researchers.
5.3. Low-cost and Easy-to-use 5.3. Low-cost and Easy-to-use
Meanwhile, the evaluation methodology is expected to be low-cost. A Meanwhile, the evaluation methodology is expected to be low-cost. A
well-acceptable methodology should be economically feasible for users well-acceptable methodology should be economically feasible for users
to create an experimental network environment. Researchers do not to create an evaluation environment. Researchers do not want to
want to conduct their tests all in live ISTN, which is over- conduct their tests all in live ISTN, which is over-cumbersome and
cumbersome and unaffordable, let alone launching their own unaffordable, let alone launching their own spacecraft. Even if
spacecraft. Even if there are a number of orbiting satellites, there are a number of orbiting satellites, whether users can easily
whether users can easily gain access to satellites is also a problem. gain access to satellites is also a problem.
5.4. Cross-domain Dataset Support 5.4. Cross-domain Dataset Support
The evaluation methodology is expected to be driven by realistic The evaluation methodology is expected to be driven by realistic
datasets from multi-dimensions to support its realism. Multi- datasets from multi-dimensions to support its realism. Multi-
dimension refers to multi-disciplinary research on ISTN. Since a dimension refers to multi-disciplinary research on ISTNs. Since a
standard ISTN evaluation methodology not only contains high-level standard ISTN evaluation methodology not only contains high-level
benchmarks from topology, routing to transmission, but also considers benchmarks from topology, routing to transmission, but also considers
the low-level traits such as wireless link conditions, weather the low-level traits such as wireless link conditions, weather
conditions and Earth rotations. To be more concrete, the former one conditions and Earth rotations. To be more concrete, the former one
requires knowledge in networks while the latter one relies more on requires knowledge in networks while the latter one relies more on
aerospace. Hence, to build a high-fidelity methodology, we need aerospace. Hence, to build a high-fidelity methodology, we need
community efforts both from networks and aerospace. On the other community efforts both from networks and aerospace. On the other
hand, an authentic dataset is an indispensable element for data hand, an authentic dataset is an indispensable element for data
driven testing methodology. Actual data is the first step to obtain driven testing methodology. Actual data is the first step to obtain
a realistic emulation. with characteristics of a real ISTN. Thus, a realistic emulation. with characteristics of a real ISTN. Thus,
skipping to change at page 16, line 11 skipping to change at page 16, line 19
[PlanetLab] [PlanetLab]
"PlanetLab", <https://citeseerx.ist.psu.edu/viewdoc/ "PlanetLab", <https://citeseerx.ist.psu.edu/viewdoc/
download?doi=10.1.1.99.7006&rep=rep1&type=pdf>. download?doi=10.1.1.99.7006&rep=rep1&type=pdf>.
[Production-Cycles] [Production-Cycles]
"Production-Cycles", "Production-Cycles",
<http://www.futron.com/upload/wysiwyg/Resources/ <http://www.futron.com/upload/wysiwyg/Resources/
Whitepapers/Satellite_Manufacturing_Productio Whitepapers/Satellite_Manufacturing_Productio
n_Cycles_0504.pdf>. n_Cycles_0504.pdf>.
[RAD5545] "RAD5545", <https://www.baesystems.com/en-media/
uploadFile/20210407074148/1434594567983.pdf>.
[RAD750] "RAD750", <https://www.baesystems.com/en-media/
uploadFile/20210407041505/1434555689265.pdf>.
[raspberry-pi]
"raspberry-pi", <https:// www.raspberrypi.com/news/
raspberry-pi-in-space/>.
[Software-defined-networking] [Software-defined-networking]
"Software-defined-networking", "Software-defined-networking",
<https://en.wikipedia.org/wiki/Software- <https://en.wikipedia.org/wiki/Software-
defined_networking>. defined_networking>.
[Sparta] "Sparta", <https://s3-us-west- [Sparta] "Sparta", <https://s3-us-west-
2.amazonaws.com/ieeeshutpages/xplore/xplore-shut- 2.amazonaws.com/ieeeshutpages/xplore/xplore-shut-
page.html>. page.html>.
[Starlink] "Starlink", <https://en.wikipedia.org/wiki/Starlink>. [Starlink] "Starlink", <https://en.wikipedia.org/wiki/Starlink>.
skipping to change at page 16, line 45 skipping to change at page 17, line 17
20200526-00053/2378318.pdf>. 20200526-00053/2378318.pdf>.
Authors' Addresses Authors' Addresses
Zeqi Lai Zeqi Lai
Tsinghua University Tsinghua University
30 ShuangQing Ave 30 ShuangQing Ave
Beijing Beijing
100089 100089
China China
Email: zeqilai@tsinghua.edu.cn Email: zeqilai@tsinghua.edu.cn
Hewu Li Hewu Li
Tsinghua University Tsinghua University
30 ShuangQing Ave 30 ShuangQing Ave
Beijing Beijing
100089 100084
China China
Email: lihewu@cernet.edu.cn Email: lihewu@cernet.edu.cn
Yangtao Deng Yangtao Deng
Tsinghua University Tsinghua University
30 ShuangQing Ave 30 ShuangQing Ave
Beijing Beijing
100089 100084
China China
Email: dengyt21@mails.tsinghua.edu.cn Email: dengyt21@mails.tsinghua.edu.cn
Qian Wu Qian Wu
Tsinghua University Tsinghua University
30 ShuangQing Ave 30 ShuangQing Ave
Beijing Beijing
100089 100084
China China
Email: wuqian@cernet.edu.cn Email: wuqian@cernet.edu.cn
Jun Liu Jun Liu
Tsinghua University Tsinghua University
30 ShuangQing Ave 30 ShuangQing Ave
Beijing Beijing
100089 100084
China China
Email: juneliu@mail.tsinghua.edu.cn Email: juneliu@mail.tsinghua.edu.cn
 End of changes. 48 change blocks. 
121 lines changed or deleted 138 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/