idnits 2.17.1 draft-irtf-ipnrg-arch-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- == No 'Intended status' indicated for this document; assuming Proposed Standard == The page length should not exceed 58 lines per page, but there was 1 longer page, the longest (page 43) being 60 lines Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 2 instances of lines with control characters in the document. == There are 2 instances of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- Couldn't find a document date in the document -- date freshness check skipped. Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) No issues found here. Summary: 3 errors (**), 0 flaws (~~), 4 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 IPN Research Group V. Cerf 2 Internet Draft Worldcom/Jet Propulsion Laboratory 3 May 2001 S. Burleigh 4 Expires November 2001 A. Hooke 5 L. Torgerson 6 NASA/Jet Propulsion Laboratory 7 R. Durst 8 K. Scott 9 The MITRE Corporation 10 E. Travis 11 Global Science and Technology 12 H. Weiss 13 SPARTA, Inc. 15 Interplanetary Internet (IPN): Architectural Definition 17 draft-irtf-ipnrg-arch-00.txt 19 Status of this Memo 21 This document is an Internet-Draft and is in full conformance with 22 all provisions of Section 10 of RFC2026. 24 Internet-Drafts are working documents of the Internet Engineering 25 Task Force (IETF), its areas, and its working groups. Note that other 26 groups may also distribute working documents as Internet-Drafts. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 The list of current Internet-Drafts can be accessed at 33 http://www.ietf.org/ietf/1id-abstracts.txt. 35 The list of Internet-Draft Shadow Directories can be accessed at 36 http://www.ietf.org/shadow.html. 38 Abstract 40 This document describes the Interplanetary Internet: a communication 41 system to provide Internet-like services across interplanetary 42 distances in support of deep space exploration. Our approach, which 43 we refer to as bundling, builds a store-and-forward overlay network 44 above the transport layers of underlying networks. Bundling uses 45 many of the techniques of electronic mail, but is directed toward 46 interprocess communication, and is designed to operate in 47 environments that have very long speed-of-light delays. We partition 48 the Interplanetary Internet into IPN Regions, and discuss the 49 implications that this has on naming and routing. We discuss the way 50 that bundling establishes dialogs across intermittently connected 51 internets, and go on to discuss the types of bundle nodes that exist 52 in the interplanetary internet, followed by a discussion of security 53 in the IPN, a discussion of the IPN backbone network, and a 54 discussion of remote deployed internets. 56 Table of Contents 58 Status of this Memo................................................1 59 Abstract...........................................................1 60 Table of Contents..................................................2 61 Copyright Notice...................................................2 62 Desiderata of Interplanetary Internetworking.......................3 63 Acknowledgments....................................................3 64 Foreward...........................................................4 65 Executive Summary..................................................5 66 1. Introduction....................................................8 67 1.1. Preliminary Considerations .............................9 68 1.2. The IPN Operating Environment .........................11 69 1.3. A "Postal" Communications Model .......................14 70 2. IPN Architectural Overview.....................................14 71 3. Inter-Internet Dialogs.........................................15 72 3.1. Principles of Design ..................................15 73 3.2. Information Carried by the Bundle Layer ...............19 74 3.3. Reliability at the Bundle Layer .......................21 75 3.4. Bandwidth Allocation via Market Mechanisms: 76 "Starbucks" ...........................................21 77 4. IPN Nodes......................................................23 78 4.1. Types of IPN Nodes ....................................24 79 4.2. Example end-to-end transfer ...........................24 80 4.3. Error Conditions at the Bundle Layer ..................32 81 4.4. Support of existing Internet applications .............35 82 5. Security in the IPN............................................36 83 5.1. Assumptions Regarding Required IPN Security 84 Mechanisms ............................................36 85 5.2. Secure Email Technology ...............................38 86 5.3. Application of Secure Email Technology to the IPN .....40 87 5.4. Protecting IPN Data and the IPN Backbone 88 Infrastructure ........................................41 89 6. Building a Stable Backbone for the IPN.........................42 90 6.1. Backbone Design Considerations ........................43 91 7. Deployed Internets in the IPN..................................46 92 7.1. Applications of deployed internets in the IPN .........47 93 7.2. Characteristics of remote deployed internets 94 in the IPN ............................................48 95 7.3. Effects of environmental characteristics on 96 protocols for the IPN RDIs ............................49 97 7.4. Summary ...............................................53 98 8. Working Conclusions............................................53 99 9. Security Considerations........................................56 100 10. Authors' Addresses............................................57 102 Copyright Notice 104 Copyright (C) The Internet Society (2001). All Rights Reserved. 106 Desiderata of Interplanetary Internetworking 108 Go thoughtfully in the knowledge that all interplanetary 109 communication derives from the modulation of radiated energy, and 110 sometimes a planet will be between the source and the destination. 111 Therefore rely not on end-to-end connectivity at any time, for the 112 universe does not work that way. 114 Neither rely on ample bandwidth, for power is scarce out there and 115 the bit error rates are high. Know too that signal strength drops 116 off by the square of the distance, and there is a lot of distance. 118 Consider the preciousness of interplanetary communication links, and 119 restrict access to them with all your heart. Protect also the 120 confidentiality of application data or risk losing your customers. 122 Remember always that launch mass costs money. Think not, then, that 123 you may require all the universe to adopt at once the newest 124 technologies. Be backward compatible. 126 Never confuse patience with inaction. By waiting for acknowledgement 127 to one message before sending the next, you squander tracking pass 128 time that will never come to you again in this life. Send as much as 129 you can, as early as you can, and meanwhile confidently await 130 responses for as long as they may take to find their way to you. 132 Therefore be at peace with physics, and expect not to manage the 133 network in closed control loops -- neither in the limiting of 134 congestion nor in the negotiation of connection parameters nor even 135 in on-demand access to transmission bands. Each node must make its 136 own operating choices in its own understanding, for all the others 137 are too far away to ask. Truly the solar system is a large place and 138 each one of us is on his or her own. Deal with it. 140 S. Burleigh 142 Acknowledgments 144 Robert Braden, of USC ISI, Deborah Estrin, of UCLA, and Craig 145 Partridge, of BBN all contributed useful thoughts and criticisms to 146 this document. 148 This work was performed under DOD Contract DAA-B07-00-CC201, DARPA AO 149 H912; JPL Task Plan No. 80-5045, DARPA AO H870; and NASA Contract 150 NAS7-1407. 152 Foreward 154 This document presents the current state of our team's efforts to 155 define an end-to-end architecture for the Interplanetary Internet. 156 This is a "living" document, and is frequently updated. In this 157 version of the document, we introduce a new construct, a tuple 158 consisting of a topologically significant routing handle and the 159 administrative name. In this model, the routing handle identifies an 160 "IPN Region," an area of the Interplanetary Internet in which A) the 161 administrative name is resolvable, and in which B) a route can be 162 formed from anywhere within the region to the address returned when 163 the administrative name is resolved. 165 On the presumption that the exploration of space will eventually lead 166 to the need for communication among planets, satellites, asteroids, 167 robotic spacecraft and crewed vehicles, our project has a heavy focus 168 on advocating the development of a stable interplanetary backbone 169 network. We have concluded that simply extending the Internet suite 170 to operate end-to-end over interplanetary distances is not feasible. 171 Rather, we envision a "network of internets" _ ordinary internets 172 that are interconnected by a system of gateways that cooperate to 173 form the stable backbone across interplanetary space. Each 174 internet's protocols are terminated at its local gateway, which then 175 uses a specialized long-haul transport protocol to communicate with 176 peer gateways. An end-to-end "bundle" protocol will operate above 177 the transport layer to carry necessary information from one internet 178 to another. Bundling will, to the extent possible, remove any 179 "chattiness" from the local protocols, forming atomic units that will 180 be shipped across the backbone. Bundles may be protected from 181 unauthorized access and unauthorized modification. The IPN will have 182 a global namespace that is broken into a number of name-to-address 183 binding regions, referred to as IPN regions. Names carried in the 184 bundles consist of a tuple, one element identifying the destination 185 IPN region and used for routing and a second element that carries the 186 administrative name of the destination in the namespace that is 187 relevant within the destination IPN region. The administrative name 188 will be bound to an address that is routable within the destination 189 IPN region. Finally, strong authentication and strict access 190 controls at several levels will protect the IPN from tampering. 192 In summary, the best way to envision the fundamental architecture of 193 the Interplanetary Internet is to picture a network of internets. 194 Ordinary internets (many being wireless in nature) are placed on the 195 surface of moons and planets as well as in free-flying spacecraft. 196 These remotely deployed internets run ordinary Internet protocols. A 197 system of Interplanetary Gateways connected by deep-space 198 transmission links form a backbone communication infrastructure that 199 provides connectivity for each of the deployed internets. New long- 200 haul protocols, some confined to the backbone network and some 201 operating end-to-end, allow the deployed internets to communicate 202 with each other. 204 Executive Summary 206 This document describes the Interplanetary Internet: a communication 207 system to provide Internet-like services across interplanetary 208 distances in support of deep space exploration. The communications 209 environment is characterized by high bandwidth-delay products 210 resulting from very long signal propagation delays, intermittent 211 connectivity that results in long periods of network partitioning, 212 and discontinuities in the capabilities of adjacent networks. Many 213 of these characteristics are similar to those facing emerging 214 communication services in the terrestrial Internet. For example, 215 terabit networks exhibit very high bandwidth-delay products, mobility 216 can result in partitioning of both nodes and subnetworks, and 217 interconnecting different physical layer technologies can result in 218 "impedance mismatches." It is possible to build an end-to-end 219 solution to address any of these, but difficult to build one that is 220 adequate for all of them simultaneously. 222 The long bandwidth-delay products shared by the IPN and very high- 223 speed terrestrial networks argue in favor of non-chatty communication 224 protocols. In the IPN, the long delays mean that protocols that use 225 many round-trips to accomplish some task pay a significant time 226 penalty. In terrestrial terabit networks where switches can forward 227 data very fast but take a (relatively) long time to reconfigure, the 228 penalty is one of lost efficiency in the use of the resources. Both 229 environments benefit from protocols that pack as much as possible 230 into each transmission and minimize the number of round-trips needed. 231 Thus a file transfer protocol that can place the entire file and 232 associated control information together in a single atomic 233 transaction completes faster within the IPN and makes more efficient 234 use of terrestrial high-speed networks. 236 Most of the problems cited above have existed and been considered, 237 albeit separately, during the evolution of what is now the Internet. 238 Many of the solutions, however, represent branches of the Internet's 239 evolutionary tree that have withered and died as a result of the 240 availability of infrastructure to mitigate environmental differences. 241 This effective homogeneity is diminishing, however, with the rapid 242 deployment of new technologies with fundamentally different 243 characteristics, such as wireless communication and Dense Wavelength 244 Division Multiplexing (DWDM) networks. One solution, electronic 245 mail, provides a means to span very different networks that are not 246 necessarily always connected. The electronic mail approach has a 247 number of attractions. First, there is no expectation of continuous 248 or instantaneous connectivity. Second, electronic mail embodies the 249 concept of indirection as a means of providing store-and-forward 250 traversal of different, sometimes-disconnected networks. Finally, 251 the electronic mail concept is generally considered to be a non- 252 interactive communications mechanism, potentially well suited to long 253 delay environments. The electronic mail approach has limitations, 254 though. Without and end-to-end retransmission mechanism, electronic 255 mail does not provide true end-to-end reliability. Additionally, 256 electronic mail is oriented toward human use rather than interprocess 257 communication. Further, the protocol that typically provides 258 electronic mail services in the Internet, SMTP, is highly interactive 259 in its control traffic, even though the electronic mail concept is 260 only minimally interactive. 262 Our approach, which we refer to as bundling, builds a store-and- 263 forward overlay network above the transport layers of underlying 264 networks. Thus two nodes that are adjacent in bundle space may be 265 many hops apart in the context of the underlying network topology. 266 We see the bundle layer as a second "thin waist of the hourglass" 267 that allows applications built on top of it to communicate across 268 discontinuities in connectivity, and to communicate efficiently over 269 a multitude of underlying transport technologies. This discontinuity 270 in connectivity may result from fluctuations in link availability or 271 from artificial discontinuities, such as are imposed by firewalls. 272 For efficient communication, the bundle layer attempts to minimize 273 interactivity of its control traffic, and expects applications to do 274 likewise. The bundle layer also provides a level of indirection 275 between applications and the specific services of the underlying 276 network protocols. Bundle applications can specify requested 277 "handling instructions," such as reliability and quality of service 278 requests that are mapped into the most appropriate mechanisms 279 available in the underlying networks. 281 Bundling uses many of the techniques of electronic mail, but is 282 directed toward interprocess communication. Bundle nodes use the 283 capabilities of the underlying networks, including transport layer 284 retransmission protocols, to effect the transfer of bundles between 285 bundle nodes. Optional end-to-end reliability at the bundle layer 286 facilitates end-to-end reliability at the application layer. In 287 addition, the bundle layer allows any bundle node in the path to take 288 custody of a bundle. When custody is transferred, the receiving 289 bundle node assumes responsibility for delivering the bundle 290 according to its handling instructions, and the previous bundle 291 custodian is allowed to recover its storage resources. The bundle 292 protocol is designed to function over simplex and half-duplex links, 293 and custody transfers may occur between non-adjacent bundle nodes. 294 Finally, because the bundle layer operates over networks that are 295 often disconnected, the bundle-layer reliability mechanisms adapt the 296 operation of timers to accommodate this episodic connectivity. 298 To illustrate how bundling allows connectivity across intermittently 299 connected networks, one could envision a "strobe light" that 300 illuminates the portions of the network's topology that are connected 301 at a given time. Lit portions of the network are available for 302 bundle forwarding. If one imagines a high-persistence CRT capturing 303 an ordered sequence of illuminations that proceed from source to 304 destination, one can envision the available routes for bundle 305 forwarding. This environment, therefore, requires a mechanism to 306 route based on both current and expected connectivity. This routing 307 mechanism exploits predictability, such as that provided by orbital 308 mechanics or by scheduled network events (possibly including rate 309 changes, such as "5c Sundays"). It is important to note that this 310 routing mechanism may choose to defer communication even though a 311 current path toward the destination exists, if a substantially more 312 attractive path is expected to become available. 314 We believe that the bundle layer functionality has utility within the 315 context of the terrestrial Internet as well. The Internet is 316 evolving to encompass very different networking technologies, 317 architectures, and applications. Some of these, such as firewalls, 318 result in logical partitioning of the Internet, while others, such as 319 extreme rate mismatches, may result in inefficient use of network 320 resources. The bundle layer extends the Internet architecture to 321 provide consistent end-to-end communication in the current and 322 emerging partitioned environments, facilitating the deployment of new 323 applications that can operate reliably while allowing networks to 324 operate efficiently. 326 Readers may find more about the project at http://www.ipnsig.org/, 327 the Interplanetary Internet Special Interest Group of the Internet 328 Society (http://www.isoc.org). 330 1. Introduction 332 The exploration of space began with naked-eye observations of the 333 stars and moon. In more recent centuries, this exploration was aided 334 by new Earth-based technologies such as optical and radio telescopes. 335 Even more recently, we have placed observing resources into near 336 Earth orbit, such as the Hubble Space Telescope. We have also sent 337 robotic missions to other parts of the Solar system for a closer 338 look. 340 It is the latter activity -- the current exploration of the Solar 341 System by robotic means and possibly later by missions crewed by 342 people -- that motivates our interest in an Interplanetary Internet. 343 The new technology of the terrestrial Internet needs to be extended 344 into space. We believe that the creation and adoption of Internet- 345 friendly standards for space communication will enhance our ability 346 to build a common interplanetary communication infrastructure. We 347 think this infrastructure will be needed to support he expansion of 348 human intelligence throughout the Solar System. The current 349 terrestrial Internet and its technology provide a robust basis to 350 support these missions in an efficient and scalable manner. 352 Although many missions during the last 40 years of space exploration 353 have by necessity provided a substantial portion of their own 354 communication resources, a significant amount of shared 355 infrastructure is already in place. For example, the multi-mission 356 Deep Space Network (DSN) provides a complex of large-diameter (up to 357 70-meter) tracking and data acquisition dishes at three points on the 358 Earth's surface, each about 120 degrees from each other. Similarly, 359 the Tracking and Data Relay Satellite System (TDRSS) and a global 360 network of small ground tracking stations are used to relay data to 361 and from many near-Earth missions. 363 In terms of the communications protocols that support space 364 exploration, the Consultative Committee for Space Data Systems 365 (CCSDS) has for almost twenty years been developing internationally 366 agreed standards for the physical and link layers that interconnect 367 remote spacecraft with their ground control systems. NASA, through 368 CCSDS, has also been working since 1993 on general application of 369 terrestrial Internet or Internet-like protocols for space data use. 370 Such standardization opens up the possibility of re-purposing and re- 371 using existing and planned communication facilities for multiple 372 subsequent missions. 374 Early in 1998, it became apparent to our team that the space 375 communications research community and the Internet research and 376 development community were pursuing technology paths that can 377 potentially lead to a kind of convergence. A few members of the 378 Internet community began thinking about adaptation of the terrestrial 379 Internet to deep space communications. The space communications 380 research community was already trying out variants of the Internet's 381 TCP/IP protocol suite to support space-based applications. Mutual 382 recognition led to the formation of a program of work that was aimed 383 at extending the notion of Internet to interplanetary scale. The 384 heretofore-independent "rivers" of evolving space technology and 385 Internet technology are converging in this program. 387 To realize this convergence, the effort to define the IPN 388 architecture is being undertaken at the Jet Propulsion Laboratory 389 (JPL) that is operated by the California Institute of Technology 390 (Caltech) for the US National Aeronautics and Space Administration 391 (NASA). Partial funding for the effort comes from the US Defense 392 Advanced Research Projects Agency (DARPA) that sponsored the original 393 Internet design work starting in 1973 and its predecessors, such as 394 the ARPANET, in 1969. NASA supplies in-kind support and staffing 395 through its standardization program with CCSDS as well as program 396 involvement from the Mars exploration enterprise. 398 An Interplanetary Internet Research Group (IPNRG) has been formed 399 under the auspices of the Internet Research Task Force of the 400 Internet Society and an Interplanetary Internet Special Interest 401 Group has also been created as a means of keeping the public informed 402 as to progress. 404 Early protocol design phases are underway now and prototype testing 405 of candidate designs is anticipated within a year. Demonstration of 406 these protocols in terrestrial environments will likely occur during 407 2001 and plans for their use in interplanetary contexts are under 408 consideration as part of the NASA Mars mission in the years beyond 409 2003. The Mars exploration program is particularly interesting as a 410 "reference model" for our work, since it includes a "Mars Network" 411 project that hopes to deploy a series of remote communications 412 satellites into orbit around the planet. These satellites will be 413 dedicated to servicing the local communication and positioning 414 requirements of the other in-orbit and surface observation and 415 exploration missions that are in the vicinity of the planet. By 2010, 416 as many as seven communications and navigation satellites could be in 417 orbit around Mars, most in low Mars orbit (LMO) but with perhaps at 418 least one in an "areo" synchronous orbit that is analogous to a 419 geosynchronous orbit around the Earth. 421 1.1. Preliminary Considerations 423 The remarkable success and growth of the Earth-bound TCP/IP protocols 424 of the Internet illustrate the power of communication 425 standardization. The simplicity of the Internet architecture, with 426 its layered structure, contributes to its ability to adapt to almost 427 any underlying communication capability. As with the terrestrial 428 Internet, the ultimate test of the IPN technology is whether it 429 successfully supports commercial applications that have a space 430 component. We expect that space will eventually be commercialized - 431 not only for communication services, but also for mining the 432 asteroids, for the operation of space-based hotels, for manufacturing 433 and medical treatments, and for general tourism. While such 434 developments may still lie decades in the future, the potential 435 investment and benefits can be appreciated as we contemplate the 436 explosion of new markets associated with the commercialization of the 437 Internet that began only ten years ago, in 1990. We will therefore 438 architect the Interplanetary Internet in anticipation of possibly 439 rapid commercialization. 441 In terms of technologies, the current Internet capabilities work well 442 on Earth where the propagation delay of light-speed signals is short. 443 The packets exchanged according to the TCP protocol reach their 444 destination in fractions of a second, for the most part. The TCP/IP 445 protocols (a system of over 150 related communication standards), are 446 therefore expected to work just as well on the surface of other 447 planets or moons, on space craft and orbiting space stations, all of 448 which involve data exchange over fairly short distances, subject to 449 the availability of sufficient power to maintain good signal-to-noise 450 ratios. However tempting it is to employ similar concepts in 451 extending the Internet into deep space, there are problems - deep 452 space communications really still are "rocket science." The distances 453 between the planets are, well, astronomical. For example, the round- 454 trip propagation delays - at the speed of light - between Earth and 455 Mars range from about 8 minutes to over 40 minutes. This makes 456 "chatty" protocols like TCP relatively unattractive because of their 457 heavy dependence on near real-time exchanges between the 458 communicating parties. 460 These large distances also impair the data rates that can be 461 sustained because of radio signal degradation and attenuation. 462 Moreover, the celestial mechanics of the solar system mean that the 463 distances between the planets change with time. While these changes 464 are essentially calculable, they still cause variations in delay, in 465 transmission capacity and occasionally in connectivity due to 466 occultation of satellites as they orbit a planet, or of ground-based 467 facilities as planets rotate. 469 Size, weight and - most of all - power are supreme challenges for 470 space-based communication systems, as they are for ground-based 471 mobile systems. Launching mass into interplanetary trajectories, 472 injecting mass into orbit, and landing mass into the gravity well of 473 another planet is currently very expensive. Mass translates directly 474 into the local availability of power. Efficient use of the 475 communications channel allows more information to be carried per unit 476 of transmitted power. But power limitations introduce asymmetries in 477 the communication capacities available between Earth, for example, 478 and the remote spacecraft and planets. There can be factors of ten or 479 more differences between the data rate that can be received on Earth 480 and the rates of reception of off-Earth resources. It is quite common 481 to be able to receive transmissions from Mars at 100 kilobits/second 482 while the Mars-based systems may only receive from Earth at 1 483 kilobits/second. 485 All of these effects combine to make the design of an interplanetary 486 backbone communication system a considerable challenge. The Deep 487 Space Network - the current interplanetary backbone - uses its three 488 terrestrial communication complexes to communicate with spacecraft, 489 orbiting satellites and ground-based resources on moons or other 490 planets of the Solar System. Because these resources must be shared 491 among many missions, it is necessary to schedule them to be aimed in 492 particular directions at particular times. Time synchronization is 493 needed among the various parts of such a system. For example, a 494 signal from Mars may take 20 minutes to reach Earth, at which time, 495 the appropriate antenna of the Deep Space network must be aimed 496 properly to receive the transmission, 20 minutes after it was sent. 497 This same antenna might then have to be repositioned to send data to 498 another spacecraft elsewhere in the Solar System, and the receiving 499 system must be ready to receive that transmission at the right time. 500 In some ways, this problem is somewhat like the problem of scheduling 501 trains on railroad tracks; since multiple trains use the tracks, they 502 must be scheduled to avoid collisions. 504 1.2. The IPN Operating Environment 506 There are a number of fundamental differences between the 507 environments for terrestrial communications and those we envision for 508 the IPN. These differences include delay, low and asymmetric 509 bandwidth, intermittent connectivity, and a relatively high bit error 510 rate. Taking these into account affects the entire model for 511 communicating; shifting us from the "telephony" model implicit in 512 current Internet communications to the "Postal", or "Pony Express," 513 model. We will first therefore describe the environmental differences 514 between terrestrial communications and the IPN and gives a brief 515 accounting of why the standard Internet protocol for reliable 516 transport, TCP, is unsuitable for end-to-end communications in the 517 IPN. 519 The most obvious difference between communicating between points on 520 Earth and communicating between planets is the delay. While round- 521 trip times in the terrestrial Internet range from milliseconds to a 522 few seconds, round-trip times to Mars range from 8 to 40 minutes, 523 depending on the planet's position, and round-trip times between 524 Earth and Europa run between 66 and 100 minutes. In addition to the 525 propagation delay, communicating over interplanetary distances 526 currently requires special equipment (large antennas, high- 527 performance receivers, etc.). For most deep-space missions, even 528 non-NASA ones, these are currently provided by NASA's Deep Space 529 Network (DSN). The communication resources of the DSN are currently 530 oversubscribed and will probably continue to be so in the future. 531 While studies have been done as to the feasibility of upgrading or 532 replacing the current DSN, the number of deep space missions will 533 probably continue to grow faster than the terrestrial infrastructure 534 needed to support them, making over-subscription a persistent 535 problem. 537 This over-subscription means that the round-trip times experienced by 538 packets will be affected not only by the propagation delay, but also 539 by the scheduling and queuing delays imposed by the Earth-based 540 resources. Thus packets to a given destination may have to be queued 541 until the next scheduled contact period, which may be hours, days, or 542 even weeks away. While queuing and scheduling delays are generally 543 known well in advance except when missions need emergency service 544 (such as during landings and maneuvers), the long and highly variable 545 delays make the design of timers, and retransmission timers in 546 particular, quite difficult. This again forms a point of departure 547 from the current Internet model, as IPN-aware applications will 548 probably need ways to track the status of a communication and to 549 apprise users of the expected delay before a response can be 550 expected. This will be complicated once the IPN moves from its 551 initial Earth-centric approach to a peer-to-peer network, since 552 notifying users of the progress of their communications will itself 553 consume precious bandwidth within the network. 555 The combined effects of large distances, the expense and difficulty 556 of deploying large antennas to distant planets, and the difficulty in 557 generating power in space all mean that the available bandwidth for 558 communications in the IPN will likely be modest compared to 559 terrestrial systems. Data rates on the order of hundreds of kilobits 560 per second to a few megabits per second will probably be the norm for 561 the next few decades. Another characteristic prevalent in today's 562 deep-space missions is bandwidth asymmetry, where data is transmitted 563 at different rates in different directions. Current missions are 564 usually designed with a much higher data return rate (from space to 565 Earth) than command rate. The reason for the asymmetry is simple: 566 nobody ever wanted a high-rate command channel, and, all else being 567 equal, it was deemed better to have a more reliable command channel 568 than a faster one. This design choice has led to data rate 569 asymmetries in excess of 100:1, sometimes approaching 1000:1. A 570 strong desire for a very robust command channel will probably remain, 571 so that any transport protocol designed for use in the IPN will need 572 to function with a relatively low bandwidth outbound channel to 573 spacecraft / landers. 575 The difficulties of generating power on and around other planets will 576 also result in relatively high bit error rates. Current deep-space 577 missions operate with very high bit error rates (on the order of 10e- 578 1, or one error in ten bits) that are then improved using heavy 579 coding. The tradeoffs between coding, bit error rate, and 580 reliability requirements will need to be reexamined in the context of 581 the IPN. 583 Finally, interplanetary communications will, at least in the near 584 future, be characterized by intermittent connectivity between nodes. 585 As satellites or moons pass behind planets, and as planets pass 586 behind the sun as seen from Earth, we lose the ability to communicate 587 with them. This effect adds to the delays experienced by packets, 588 and could push queuing delays to several weeks or a month if the 589 source and destination are in opposition (on opposite sides of the 590 sun). Inter-layer signaling, especially from the link layer to 591 provide notifications of such breaks in connectivity, will probably 592 be required. 594 We see the IPN growing outwards from Earth as we explore more and 595 more planets, moons, asteroids, and possibly other stars. Thus there 596 will always be a fringe to the fabric of the IPN, an area without a 597 rich communications infrastructure. As a result, the data rate, 598 connectivity, and error characteristics mentioned above will probably 599 always be an issue somewhere in the IPN. For the more highly 600 developed core areas of the IPN, it is interesting to note that delay 601 is the only truly immutable characteristic that differentiates the 602 IPN from terrestrial communications. Data rates, intermittent 603 connectivity, and bit error rate can all be mitigated or eliminated 604 by adding additional infrastructure, in theory if not in practice. 605 Additional infrastructure can also mitigate the scheduling and 606 queuing delays mentioned above, but the propagation delays will 607 remain unless and until we find a way to transmit information faster 608 than the speed of light. 610 These environmental characteristics: long delays, low and asymmetric 611 bandwidth, intermittent connectivity, and relatively high error rate 612 make using unmodified TCP/IP for end to end communications in the IPN 613 infeasible. Using the equations from Mathis, et al [ref: 614 http://www.psc.edu/networking/papers/model_ccr97.ps], we can 615 calculate an upper bound on the sustainable throughput of a TCP 616 connection, taking into account TCP's congestion avoidance 617 mechanisms. Even if only 1 in 100 million packets are lost, a TCP 618 connection to Mars is limited to just under 250kbps. If we assume 619 that 1 in 5000 packets is lost (this figure was reported by Paxson as 620 the packet corruption rate in the Internet ref: 621 ftp://ftp.ee.lbl.gov/papers/vp-thesis/dis.ps.gz caution: very large 622 file) then that number falls to around 1,600bps. These values are 623 upper bounds on steady-state throughput; since the number of packets 624 in a connection will generally be under 10,000, TCP performance would 625 be dominated by its behavior during slow-start. Even when Mars is at 626 its closest approach to Earth, this means that it would take a TCP 627 nearly 100 minutes to ramp up to a transmission rate of 20kbps. Lab 628 experiments using a channel emulator and standard applications show 629 that even if TCP could be pushed to work efficiently at such 630 distances, many applications either rely on several rounds of 631 handshaking or have built-in timers that render them non-functional 632 when the round-trip-time is pushed over a couple of minutes. It 633 typically takes eight round trips for FTP to get to a state where 634 data can begin flowing, for example, and an FTP server may time out 635 and reset the connection after 5 minutes of inactivity. This means 636 that a conformant standard FTP server could be unusable for 637 communicating even with the closest planets. 639 1.3. A "Postal" Communications Model 641 We have concluded that the standard Internet protocols should be 642 essentially "terminated" at the Interplanetary Gateways and the 643 information payloads conveyed through a new set of protocols better 644 suited to the long distances, variable delays and asymmetric data 645 rates of the interplanetary backbone network. In essence, the design 646 is analogous to a kind of postal relay system in which messages are 647 delivered to the intermediate Interplanetary Gateways, extracted from 648 their standard Internet protocols, and encapsulated in new link and 649 transport protocols to be forwarded to the next IPN gateway and 650 ultimately into the target internet. 652 Internet electronic mail already works in this fashion but the 653 transfer of files, the operation of the World Wide Web, and remote 654 interactive applications do not fit into this model directly. There 655 are circumstances under which researchers on planet Earth do need an 656 ability to interact with remote devices, for example to steer wheeled 657 robotic vehicles around on the surface. But because of the enormous 658 round-trip delays, such systems must work very indirectly. For 659 example, to steer the Mars Pathfinder rover, one sends instructions 660 about intermediate points that the robot must steer past. This is 661 analogous to automatic airplane pilots that are given a series of 662 coordinates through which to pass. In effect, a planned itinerary is 663 sent and the robot vehicle executes the plan, dealing with local 664 conditions as required. 666 The "store-and-forward" nature of this communication method is 667 reminiscent of bucket brigades, except that the contents of the 668 buckets are actually the payloads (i.e. data) of the applications 669 that utilize the network. The concept of "custody" is important in 670 such a system. A sender does not relinquish a copy of a transmission 671 until it is sure that the next in line has successfully received it. 673 2. IPN Architectural Overview 675 We now consider five broad areas that represent areas of significant 676 research in the Interplanetary Internet architecture: 678 * The communication conducted between independent internets 679 (termed "Inter-internet dialogs") 680 * The architecture and functions of the unique nodes of the 681 Interplanetary Internet 682 * A security architecture for meeting anticipated data and 683 infrastructure protection needs 684 * The issues in developing a stable backbone network for the 685 Interplanetary Internet 686 * The issues in deploying internets on remote planets, asteroids, 687 and spacecraft 689 The following five sections of this document consider each of these 690 areas. Following these sections we present our conclusions to-date. 692 3. Inter-Internet Dialogs 694 This section first presents four principles that guide the design of 695 the Interplanetary Internet. In doing so, we introduce the concept 696 of the "bundle" layer, a protocol layer providing end-to-end service 697 in the IPN. The section continues by discussing the information 698 carried by the bundle layer, and concludes by discussing reliability 699 at the bundle layer. 701 3.1. Principles of Design 703 3.1.1. Name Tuples Consisting of Administrative and Routing Parts are 704 the Means of Reference 706 In the (terrestrial) Internet, names are administrative in nature, 707 and are hierarchically organized. The Domain Name System (DNS) uses 708 a highly distributed database to translate the name to a numeric 709 address, and addresses are the common medium used throughout the 710 network for reference. This scheme has worked well in the Internet, 711 but the emergence of network address translators and other 712 partitioning mechanisms have begun to cause some problems with this 713 scheme. 715 One of the problems involved with using DNS names across 716 interplanetary space is the distributed nature of the DNS database. 717 This means that it is entirely possible for the portion of the 718 database that can resolve a name to its address to be far (very far, 719 in the Interplanetary Internet) from the host requesting resolution. 720 This means that the times required to resolve names to addresses can 721 become impossibly high, especially when the issues of intermittent 722 connectivity come into play. The DNS has a mechanism to replicate 723 portions of its database, using a technique known as zone transfers. 724 However, this is not a good solution in the Interplanetary Internet, 725 either. One could easily spend all of the available communication 726 time transferring the ".com" zone to another planet, rather than 727 actually transferring data. Clearly, another approach is indicated. 729 In our initial designs, we considered creating a new top-level 730 domain, e.g. ".sol", and assigning to it topological significance. 731 We constructed a scheme by which we routed on the "most significant" 732 portions of the domain name, such as ".mars.sol", and essentially bet 733 that these portions would be topologically significant, rather than 734 administratively significant. This scheme, while attractive from the 735 standpoint of making use of existing infrastructure, makes use of 736 that infrastructure in a bad way. We were forced to "grandfather" 737 existing top-level domain names to be bound to Earth's Internet, so 738 that ".com" meant ".com ON EARTH." This spawned philosophical 739 debates within the group regarding sovereignty and the current top- 740 level domain structure within the internet, but also had the 741 potential of creating serious technical problems. For example, some 742 organizations encode geographic information at fairly low levels of 743 their DNS names (consider "zurich.ibm.com", and then consider 744 "mars.ibm.com"). If all ".com's" are shipped off to Earth, the 745 "mars.ibm.com" data is going to take a serious detour. We eventually 746 concluded that this was not the right approach. 748 We have concluded that names in the Interplanetary Internet should 749 consist of a tuple that contains the administrative part plus a 750 routing part. These names must be carried end-to-end throughout the 751 Interplanetary Internet. The routing part serves the purpose of the 752 new top-level domain described above, except that it is not required 753 to conform to the naming conventions of the Domain Name System (i.e., 754 it may be numeric rather than textual), it may be hierarchically 755 organized, and it must be routed. This requires us to develop the 756 means for computing and distributing routing information, but 757 relieves us from a dependence on the relationship between 758 administrative names and network topology. 760 3.1.2. The Routing Part of an IPN Tuple Identifies an Internet 762 The routing portion of the name identifies an IPN Region. We 763 envision the IPN as a "network of Internets", and the IPN Region, to 764 some extent, allows us to route to a particular Internet. The use of 765 an IPN Region is an explicit form of aggregation that is not 766 otherwise possible using administrative names. 768 It is necessary and sufficient that the administrative portion of an 769 IPN name be resolvable to a useful address within its IPN Region. We 770 are currently treating the structure of the routing part of the name 771 in a manner similar to the structure of DNS names: the name space of 772 the routing part is a tree of text labels separated by "dots," with 773 the root node of the space having a null label. We denote a name in 774 the IPN in the following manner: {administrative part, routing 775 part}. So, if "earth.sol" were an IPN Region encompassing the entire 776 Earth, the web site for the Internet Society's IPN Special Interest 777 Group would be { www.ipnsig.org, earth.sol}. 779 In order that any node in the IPN be able to send data to any other 780 node, the routing part of the name must be interpretable everywhere. 781 That is, any IPN node, when confronted with any valid IPN Region 782 name, must be able to identify a transport layer destination that 783 moves the data toward that region. 785 3.1.3. The "Bundle Layer" Terminates Local Transport Protocols and 786 Operates End-to-End 788 In the IPN, we cannot ever assume that there is direct connectivity 789 between source and destination. That is, we cannot assume that bits 790 emitted by a source can travel, delayed only by routing and 791 transmission delays, to the destination. There may be any number of 792 reasons for this, ranging from physical (the destination is on the 793 far side of a distant planet and can't communicate with anything 794 right now), to schedule-related (a required IPN gateway is currently 795 serving other customers), to administrative (the source is only on 796 during the day, the destination only during the night). For the 797 long-haul links of the backbone, information will almost certainly 798 have to be stored for some amount of time as the antennas used for 799 the long-haul links will almost surely be highly directional. 801 Thus depending on the schedules of the nodes involved and the 802 possibility of high-priority interrupt traffic, the nodes that make 803 up the IPN may have to buffer data for hours, days, or weeks before 804 it can be forwarded. Also, the highly varying communications 805 environments that will make up the IPN, ranging from optical fiber on 806 Earth to wireless communications around Mars, to the long-haul links 807 of the backbone, suggest that different transport protocols will be 808 needed for the different environments. It makes sense, therefore, for 809 the IPN nodes to terminate the transport-layer protocols used in the 810 respective IPN regions, holding data at a higher layer before 811 forwarding it on, possibly using a different transport-layer 812 protocol. 814 We call this higher layer the "bundle layer," and the protocol used 815 to send data between the various nodes of the IPN the "bundle 816 protocol." The term "bundle" is used to connote the store-and- 817 forward aspect of communications where as much interactivity as 818 possible has been distilled out of the communication. A bundle file 819 transfer request, for example, might contain the user's 820 authentication (login/password, e.g.), the location of the file to 821 get, and where that file should be delivered in the requester's IPN 822 domain. All of this information would be transmitted as one atomic 823 "bundle," and the requested file would be returned. We use the term 824 "bundle" rather than "transaction" to avoid notions of two-phase and 825 three-phase commitment that are commonly associated with transaction 826 processing. 828 In traditional networking terminology it is generally the transport- 829 layer protocol that operates end-to-end. Since IPN nodes terminate 830 transport layer protocols in order to buffer data and to enable them 831 to use a transport protocol appropriate to the IPN region through 832 which data will be sent, it is the bundle layer in the IPN that 833 operates end-to-end. 835 It should be noted that terminating the transport protocols at the 836 IPN nodes decouples the internets in different IPN regions to a 837 significant degree. This has the desirable effect of also decoupling 838 the evolutionary rates of those internets: changes in the Earth's 839 Internet do not necessarily dictate changes in other internets. This 840 is important in an environment in which resources are and will 841 continue to be severely constrained. 843 Figure 1 illustrates the progression of a bundle of data through the 844 Interplanetary Internet, from its source at host "A" to the 845 destination at host "E." Custody transfers are indicated by 846 asterisks (*), and occur at B, C, D, and E. Host "A" initiates the 847 bundle transfer, and the bundle is transferred using internet 848 protocols (i.e., TCP and IP) to bundle node "B", which serves as the 849 gateway between the left-hand internet and the interplanetary 850 backbone. The box icon indicates that custody of the bundle is 851 transferred to that gateway. When conditions permit, the bundle is 852 forwarded on to the next hop in the store-and-forward chain. In this 853 case the next hop is another host within the interplanetary backbone 854 network: host "C". 856 Also illustrated in Figure 1 is the notion of a "return receipt" sent 857 by the ultimate destination of the data back to the source. This is 858 an optional service, much like a return receipt within the postal 859 system. If the source desires notification of delivery, that is 860 accomplished by a separate return receipt, which is transmitted as 861 its own bundle, and is subject to the same custody transfers as the 862 original transmission (similar to the fact that a postal return 863 receipt is, in itself, a postcard). It is shown figuratively as 864 bypassing the forwarding path (E to D to C to B to A), but this is 865 simply for clarity. The return receipt is forwarded in exactly the 866 same manner as the original data is. 868 An Internet The IPN Backbone An Internet 869 +----------------+ +------------------------+ +----------------+ 870 | | | | | | 871 | +---+ +---+ +---+ +---+ +---+ | 872 | / /| / /| / /| / /| / /| | 873 |+---+ | +---+ | +---+ | +---+ | +---+ | | 874 || |=+=====>| |=+=====>| |=+======>| |=+=====>| | + | 875 || A |/ | B*|/ | C*|/ | D*|/ | E*|/ | 876 |+---+ +---+| +---+ +---+| +---+ | 877 | /\ | | | | || | 878 | || | | | | || | 879 | || | | | | || | 880 +--||-----------+ +-----------------------+ +---------||-----+ 881 || || 882 +=====================================================+ 883 Return Receipt 885 Figure 1. Custody Transfers 887 Many aspects of this mode of data transfer resemble the postal 888 system, or even the Pony Express. The source depends on the store- 889 and-forward nodes in the chain to operate on its behalf to deliver 890 data that may not be retained at the source. Source notification 891 that the destination has received the data is optional, and there is 892 no guarantee or even any firm expectation on the part of the source 893 about when the data will be delivered. 895 Conceptually, the bundle protocol resides above the transport 896 layer(s), and operates end-to-end between the ultimate source and the 897 ultimate destination of the data. The bundle layer provides a number 898 of services to applications using it. For example the bundle layer 899 carries the source and destination name tuples end-to-end to support 900 the late binding of the destination name's administrative part to an 901 address. The bundle layer also carries the users' specifications for 902 reliability, quality of service, and security. Because the 903 communication resources are precious, it is desirable to provide 904 error recovery mechanisms that do not necessitate discarding of 905 bundles that contain errors. We are considering adding to the 906 information carried in a bundle some information to assist in 907 transfer-related error handling. We are thinking in terms of active 908 networking, using a combination of active packets and active nodes as 909 a means of specifying appropriate actions to take in the event of 910 problems in completing the transfer. Additionally, the bundle layer 911 provides invoking applications with a transfer identifier that is 912 carried with the bundle, and is used in "return receipts." [Ref: 913 http://www.comsoc.org/pubs/surveys/1q99issue/psounis.html] 915 3.2. Information Carried by the Bundle Layer 917 To effect the end-to-end transfers necessary in the IPN, the bundle 918 layer must carry some information end-to-end. This section documents 919 our current thinking on the information that must be carried end-to- 920 end, and notes which of those data elements may be supplied by the 921 application using the bundle service. 923 * Bundle Identifier: this is a monotonically increasing number 924 that is carried in the bundle, and also returned to the 925 application to support return receipt processing. It is not 926 necessarily a sequence number, and as a result, there is no 927 requirement that the value of Bundle Identifiers increase 928 consecutively. The requirement for monotonicity derives from a 929 need to provide robustness against system crashes, and therefore 930 must be persistent across system crashes. 931 * Remote entity name: this is the IPN name of the remote bundle 932 agent, and is a tuple, as described above. It is supplied by 933 the local application using the bundle service. 934 * Source entity name: this is the IPN name of the local bundle 935 agent, and is a tuple. It is supplied by the local bundle 936 service, since a particular host may have multiple names and one 937 may be chosen based on routing decisions or other criteria 938 opaque to the application (as in multihomed hosts). The source 939 name may be returned to the application to support return 940 receipt processing. 941 * Authentication information: this is information, such as a 942 digital signature, that is passed by the application to the 943 bundle layer to unambiguously identify the source of the bundle. 944 (Just what the source of the bundle is, person, place, address, 945 etc. is still undecided.) This information is checked for 946 validity at both the source bundle agent and the destination 947 bundle agent. The authentication information may also be used 948 for access control purposes within the network. 949 * Source application instance handle: this is similar in nature to 950 a source port number in that it identifies the sending 951 application. Since bundles are inherently non-interactive, the 952 typical use for this handle is to "reanimate" the source 953 application when a return receipt arrives. This could be hours, 954 days, or weeks after the initial transmission, so the handle may 955 well be a reference to a structure that allows the application 956 to be reinstantiated with known state. The source application 957 instance handle may be used at the destination as an identifier, 958 but may also be redundant with the end-to-end authentication 959 information for this purpose. This handle is supplied by the 960 source application. 961 * Destination application instance handle: this is essentially the 962 destination port identifier. As with ports, these must be 963 known to and supplied by the source application. We have not 964 yet fully explored the implications of port advertisement across 965 the IPN. 966 * Size of data: this is a statement of the size of the bundle, in 967 bytes. It is supplied by the source application, and is used 968 initially to ensure that sufficient space is available to store 969 the bundle for its initial transmission. Nodes receiving 970 transmitted bundles use this information in the same manner: as 971 a means of making a determination about the availability of 972 storage early in the bundle handling process. In forming the 973 initial bundle, the bundle layer at the source may use the size 974 of data parameter as a consistency check on the amount of data 975 actually delivered. 976 * Handling instructions: these are parameters supplied by the user 977 with the bundle that convey the user's preferences to the 978 network. Our thoughts on just exactly what these parameters 979 look like are not yet firm, however, our thoughts are that they 980 include some or all of the following: priority, quality of 981 service (although we are still debating what this means in the 982 context of the IPN), elapsed time after which the content of 983 this bundle is meaningless (time to live as specified by the 984 user), reliability requirements, and any error handling 985 information. For the most part, these are requests, and we 986 perceive that the bundle layer may either override some or all 987 of these requests or fail the request if local policy does not 988 permit that particular user to make that particular request at 989 that particular time. 990 * Data Descriptor: This is a reservation token that is generated 991 by the bundle layer. Its detailed definition and use are still 992 to be determined. 993 * Time to Live: This is the time after which this bundle is to be 994 discarded from the network. It is present to facilitate the 995 recovery of network resources and to terminate routing loops, 996 should they occur. 997 * Loose/Strict Source Route and Record: This information is 998 provided by the source application to facilitate debugging of 999 the network. It consists of a list of names of IPN nodes 1000 through which the bundle must pass on its way to the 1001 destination, and our intent is that it should behave similarly 1002 to the corresponding option within IP. 1003 * Current bundle custodian: the bundle protocol supports store- 1004 and-forward operation in which the custody of a bundle (that is, 1005 the responsibility for ensuring reliable delivery) may transfer 1006 from one IPN node to another as the bundle progresses through 1007 the IPN. There is not a requirement for each IPN node 1008 encountered to assume custody of a bundle. As a result, it is 1009 necessary to identify the upstream node that has custody of the 1010 bundle, in order to either request retransmissions or to accept 1011 custody of the bundle. 1012 * User data: this is intended to be all of the data that the 1013 remote entity requires to perform whatever operation is 1014 requested. Since the environmental characteristics of the IPN 1015 make interactivity difficult, the notion is that all of the 1016 information that is required to perform a particular 1017 "transaction" would be provided in a single bundle. 1019 3.3. Reliability at the Bundle Layer 1021 Because no single transport-layer protocol operates end-to-end across 1022 the IPN, end-to-end reliability can only be assured at the bundle 1023 layer. At each node along an end-to-end route, the bundle-layer 1024 protocol entity passes bundle data to the Transport layer for 1025 transmission. Each bundle layer entity is highly confident that the 1026 transport layer will successfully convey the data entrusted to it to 1027 the next bundle-layer protocol entity (which may "take custody" of 1028 the data or merely relay it; a single hop). But failures are 1029 possible (e.g., a host computer does an unplanned reboot). Just in 1030 case the highly unlikely happens and a Transport-layer transmission 1031 fails, the first subsequent node that detects the failure and is 1032 capable of taking custody of the bundle will request that the prior 1033 custodian re-transmit any missing data (again using Transport-level 1034 transmission services across, potentially, one or more relay nodes). 1036 The bundle layer's confidence in the effectiveness of the underlying 1037 Transport-layer protocols is reflected in the design of the timers 1038 for bundle-layer reliability. These timers are highly optimistic _ 1039 that is, they expire as late as possible _ in order to give the 1040 Transport protocols every opportunity to complete reliable 1041 transmission. The effect of this optimism is to minimize the chance 1042 of unnecessary bundle-layer retransmission, which could seriously 1043 degrade IPN performance by consuming valuable bandwidth. 1045 3.4. Bandwidth Allocation via Market Mechanisms: "Starbucks" 1047 To promote effective and efficient use of the IPN's scarce 1048 transmission resources, some sort of sophisticated and adaptable 1049 bandwidth allocation system will probably be needed. The scheme 1050 described below is based on a free-market notion of "fare-paying 1051 packets", where at initial transmission each bundle is issued a 1052 "draft" on some small percentage of IPN resources. These resources 1053 might well map back to actual monetary funds provided by the 1054 originator of the bundle to the provider(s) of the IPN service. The 1055 bundle in effect pays its own way across the various legs of end-to- 1056 end transmission. As it traverses each hop, the bundle spends funds 1057 from its original draft until either it is received or else its funds 1058 are exhausted. If a bundle is dropped due to insufficient funds then 1059 the hope is that all available transmission resources were allocated 1060 to bundles that were allotted more funds and therefore were 1061 presumably more important (to somebody). 1063 At the initial Send-bundle service request, the source application 1064 (bundle sender) would specify total funds allocated to getting the 1065 bundle delivered to the destination. Total funds allocation needs to 1066 be a function of (a) the total number of bytes of data and metadata 1067 to be sent, and (b) the prices charged for each "transmission class," 1068 (something like First Class, Business Class, Economy Class, Steerage, 1069 Overhead Bin; etc.). The allocation should cover the anticipated 1070 costs of traversing all the bundle hops along the anticipated route 1071 to destination. If the bundle must be split up into multiple packets 1072 (bindles, segments), the bundle agent that performs the split also 1073 distributes the bundle's total funds among individual packets. This 1074 distribution will probably be on a prorated basis. 1076 Also, the source application would supply the bundle (and, by 1077 implication, each of its packets) with traveling instructions: 1079 For each time epoch that elapses while awaiting transmission from any 1080 single bundle transmission agent: 1082 * The length of the epoch in units of time (seconds?) 1083 * The queue (transmission class) to wait in over the course of the 1084 epoch 1086 For example: get in the Steerage queue, but if 30 minutes pass and 1087 you still haven't been transmitted then pay for an upgrade to 1088 Business Class; if you still haven't been transmitted after 20 1089 minutes in Business Class, then give up. 1091 At each bundle transmission agent, each packet is handled as follows: 1093 * For each of the packet's authorized time epochs until the packet 1094 is transmitted (that is, initially handed to the underlying 1095 communication layer; retention until successful custody 1096 acquisition by downstream agent is provided at no charge): 1098 - If (epoch duration * packet length * agent's price per unit of 1099 time per byte for this epoch's transmission class, i.e. queue) 1100 is greater than packet's residual funds, then discard the 1101 packet. The packet's residual funds remain unexpended. 1103 - Else, append the packet to the requested queue. Then: 1105 - If epoch expires before packet is de-queued for transmission, 1106 then charge the source application an amount equal to (epoch 1107 duration * packet length * price per unit of time per byte for 1108 queue), reduce packet's residual funds by this amount, and 1109 start the next epoch; if no more authorized epochs, give up. 1111 - Else, upon de-queuing the packet for transmission, charge the 1112 source application an amount equal to (length of time spent in 1113 queue * packet length * price per unit of time per byte for 1114 queue) and reduce packet's residual funds by this amount. 1116 * Transmission classes _ queues _are of varying maximum length 1117 (the more expensive queues are shorter) but packets in all 1118 transmission classes are at the same level of priority; packets 1119 are de-queued in round-robin fashion to ensure that no class is 1120 altogether starved for service. Because First Class is a 1121 shorter queue than Business Class, packets that pay for First 1122 Class spend less time waiting. 1124 * Separately, an Emergency queue is provided for system-critical 1125 packets. Everything in the Emergency queue has higher priority 1126 than everything else, so nothing else gets transmitted until the 1127 Emergency queue is emptied. 1129 Periodically, each bundle protocol agent reports aggregate charge 1130 amounts back to the source applications and also to some central 1131 accounting authority ["the bank", nominally based on Earth]; this is 1132 a separate application-layer protocol. When the central authority 1133 determines that a source application is out of funds, it reports the 1134 source application's bankruptcy to all bundle agents; from that time 1135 on, all service requests and packets received from that source 1136 application are rejected. 1138 Although this particular scheme may ultimately prove too complex to 1139 be workable, we think the general principle of fine-grained bandwidth 1140 allocation could contribute significantly to the viability of the 1141 IPN. 1143 4. IPN Nodes 1145 Nodes within the IPN have a number of responsibilities. As members 1146 in a store-and-forward chain, they have the responsibility for 1147 resource allocation to support bundle transfers. These resources 1148 include, among other things, buffer space and transmission capacity. 1150 Additionally, IPN nodes have the responsibility of actually executing 1151 the bundle transfer. Reliability requirements for bundle transfers 1152 are specified by the using application, and include both reliable and 1153 unreliable transfers (possibly with some intermediate, or partial, 1154 reliability services). The IPN nodes are responsible for using 1155 whatever reliability mechanisms exist in the underlying (transport- 1156 and-below) layers, and augmenting those mechanisms as necessary to 1157 effect the required reliability. 1159 Finally, IPN nodes are responsible for routing bundles between IPN 1160 domains. IPN nodes may depend upon the services of the local 1161 internets (or the IPN backbone) for intra-domain routing. 1163 In this section, we first briefly state the types of IPN nodes that 1164 we have identified, and then we provide a number of exemplary end-to- 1165 end data transfer descriptions. Finally, we list the error 1166 conditions that we have identified that may occur at the bundle layer 1167 during the course of end-to-end data transfer. 1169 4.1. Types of IPN Nodes 1171 We identify three grades of IPN functional capability. In order of 1172 increasing scope, they are: agent capability, relay capability, and 1173 gateway capability. All IPN nodes are able to act as bundle agents; 1174 some bundle agents are additionally able to act as IPN relays; some 1175 IPN relays are additionally able to act as IPN gateways. 1177 * Bundle agents build and consume bundles. These could be stand- 1178 alone proxy devices or could be an operating system service 1179 collocated with an application. The endpoints of an end-to-end 1180 bundle transmission need not be any more than bundle agents, 1181 though they may additionally have relay or even gateway 1182 capability. 1184 * IPN Relays receive bundles and forward them toward their 1185 destinations, either within or between IPN regions. 1187 * IPN Gateways reside at the interface between IPN regions. The IPN 1188 gateways perform routing between the IPN regions. 1190 Orthogonal to these grades of capability is the ability to take 1191 custody of a bundle. The endpoints of a bundle transaction are 1192 typically the initial and final custodians of the bundle. Non- 1193 gateway relays may take custody of the bundles they receive; 1194 alternatively, they may simply provide mitigation of R2 effects 1195 (signal strength losses due to the extreme distances of 1196 interplanetary communication), but provide no custody transfer 1197 capabilities. (In the latter case they are essentially "repeaters.") 1198 Gateways normally take custody but are not required to do so. 1200 4.2. Example end-to-end transfer 1202 We provide the following example of an end-to-end transfer that 1203 crosses multiple IPN Regions: A host on Earth sends a bundle to a 1204 destination on Mars. Figure 2 illustrates the network that is the 1205 subject of this example _ the Interplanetary Internet in this example 1206 has five distinct regions interconnected by four IPN Gateways. This 1207 example highlights the actions taken by the bundle layer and the 1208 interactions of the bundle layer with applications and with 1209 underlying transport protocols. 1211 4.2.1. Backbone connectivity 1213 It is important to have some understanding of the routing that is 1214 required at the IPN Gateways. Unlike terrestrial communications, the 1215 IPN's long-haul communication links are directional, mobile, and 1216 highly scheduled. This is important, because directionality combined 1217 with mobility means that a transmitter and receiver must track each 1219 +-------------------------+ 1220 | Earth's Internet | 1221 | IPN Region: earth.sol | 1222 | +---+ | 1223 +-----------/ /|--------+ 1224 +---+ | 1225 |G/W| + 1226 +----------| 1 |/---------+ 1227 | +---+ | 1228 | The "Backbone" | 1229 | IPN Region: ipn.sol | 1230 +---+ +---+ +---+ 1231 / /|------/ /|-------/ /| 1232 +---+ | +---+ | +---+ | 1233 |G/W| + |G/W| + |G/W| + 1234 +----------------| 3 |/ +---| 4 |/-----+ | 2 |/-------------------+ 1235 | +---+ | +---+ | +---+ | 1236 | Venus's Internet | | Jupiter's | | Mars's Internet | 1237 | IPN Region | | Internet | | IPN region: | 1238 | venus.sol | | IPN Region | | mars.sol | 1239 +------------------+ | jupiter.sol | +----------------------+ 1240 +--------------+ 1242 Figure 2. An Interplanetary Internet of Five IPN Regions 1244 other in order to establish and maintain a communication link. In 1245 the IPN, much of the mobility is due to orbital mechanics and is 1246 therefore relatively predictable. However, this means that nodes 1247 that we would normally consider to be fixed, such as antennas on the 1248 surface of the Earth, are actually highly mobile as a result of the 1249 Earth's rotation and its revolution around the Sun. (In this 1250 example, we confine ourselves to our local solar system, and do not 1251 consider the motion of our sun relative to celestial bodies outside 1252 our solar system.) We can describe the predictable aspects of node 1253 mobility with an ephemeris, a table of the positions of celestial 1254 bodies at specified intervals of time. Both a directional sender and 1255 receiver must know the other's position in order to establish a link 1256 between the pair. In addition, these communication resources are 1257 highly scheduled. It is not sufficient for a receiver to point at a 1258 prospective target and just wait _ for example, a terrestrial node 1259 will typically have to point at several targets sequentially, and an 1260 interplanetary node will typically not have enough power to just wait 1261 for incoming messages. Rather, a schedule of communication 1262 opportunities must be calculated and then refined with planned 1263 communication instances. A communication opportunity establishes 1264 that the endpoints could establish a link if they were pointing at 1265 each other at the proper times. We refer to a planned communication 1266 instance as an agreement (although not irrevocable) between the two 1267 parties to establish contact and communicate for a defined period of 1268 time. The protocols for establishing the schedule of communication 1269 instances between all pairs of possible communicants will evolve -- 1270 from something primarily done manually to something more automated as 1271 the Interplanetary Internet grows. 1273 The scheduled nature of connectivity in the Interplanetary Internet, 1274 particularly across the deep-space links, means that at the time of a 1275 bundle's arrival at an IPN Gateway, some or all of the possible 1276 outbound routes may be "down." The gateway must store the bundle 1277 until the appropriate link is available and then transmit the bundle 1278 over that link. One of the fundamental differences between the 1279 Interplanetary Internet and the terrestrial Internet is this inherent 1280 use of store-and-forward mechanisms in routing bundles. With that 1281 in mind, let us consider the routing decisions made at some of the 1282 IPN Gateways in Figure 2. 1284 4.2.2. IPN Gateway routing 1286 Bundle routing at an IPN gateway will typically have to deal with a 1287 mix of continuously available links and intermittently available 1288 links. Routing across a continuously available link is a relatively 1289 straightforward activity. Routing in the presence of intermittently 1290 available links can be significantly more complex, as the gateway 1291 will need to decide not only the next hop destination but also the 1292 time at which to send the bundle. Conditions elsewhere in the 1293 network may make it more desirable to send a bundle to a next-hop 1294 destination that is not yet accessible, even when an alternative 1295 route is currently available. The routing function in IPN Gateways 1296 and other IPN nodes that have intermittent links has three distinct 1297 parts: the contact scheduler, the route evaluation algorithm, and 1298 the dispatcher algorithm. 1300 Figure 3 illustrates the relationship of these functions with each 1301 other and with other elements of the communication protocol suite. 1302 The contact scheduling application establishes the schedule for 1303 communicating with next-hop neighbors. The implications of this 1304 scheduling activity are broad _ orbital mechanics, resource 1305 management onboard the spacecraft, prospective communications loads, 1306 and so forth all play a role. Because spacecraft resources must be 1307 coordinated among all functions (not just communications), this is 1308 typically going to be part of a larger resource management 1309 application (interactions with functions such as pointing and power 1310 control are not shown in Figure 3, but are present nonetheless). It 1311 is very likely that the contact scheduler will *not* be local to the 1312 bundle node, but rather a centralized function that distributes a 1313 contact schedule to IPN nodes that perform routing. This may change 1314 in the future to a distributed contact scheduling algorithm, but for 1315 the foreseeable future, this will not be the case. The product of 1316 the contact scheduler is a schedule of planned contacts, durations, 1317 expected data rates, etc. It is intrinsically link-state in nature. 1319 +-------------------------------+ 1320 | User Applications | 1321 +-------------------------------+ 1322 +-------------------------------+ +--------+ 1323 | +-----------------------+ | | | 1324 +---------+ | | Bundle Transport | | | Policy | 1325 | Contact | | +-----------------------+ | | | 1326 |Scheduler| | +-----------------------+ | +--------+ 1327 +---------+ | | | | | 1328 | | | +---------------+ | | | 1329 V | | | Dispatcher |<------------+ 1330 /-------\ | | | Algorithm | | | +----------+ 1331 ----->| | | | | /------\ |Route | 1332 \-------/ | | +---------------+ |<--< Routes ><-|Evaluation| 1333 | | Bundle Routing | | \------/ |Algorithm | 1334 | +-----------------------+ | +----------+ 1335 | Bundle Agent Functions | 1336 +-------------------------------+ 1337 +-------------------------------+ 1338 | +------+ +------+ +------+ | 1339 | | TCP | | UDP | | LTP | | 1340 | +------+ +------+ +------+ | 1341 | +----------------+ +------+ | 1342 | | IP | | - | | 1343 | +----------------+ +------+ | 1344 +-------------------------------+ 1345 +-------------------------------+ 1346 | Link and Physical Interfaces | 1347 +-------------------------------+ 1349 Figure 3. IPN routing applications and their relation to other 1350 communication functions. 1352 The route evaluation application exchanges information with all 1353 first-hop neighbors (we hope this occurs infrequently) to build a 1354 general picture of the IPN beyond the first-hop neighbors. We 1355 currently view this in terms of a distance-vector representation, but 1356 with metrics such as the expected delay to the destination IPN region 1357 and average aggregate data rate to the destination IPN region. The 1358 mechanics for building these metrics are still in development. 1360 The dispatcher algorithms accepts routing requests from the bundle 1361 transport layer and builds a "manifest" for each next-hop contact 1362 that is subsequently consumed by the bundle routing layer. The 1363 dispatcher application consumes some or all of the following in 1364 deciding how to schedule bundle transmissions: the contact schedule, 1365 the routing information, local policy information, and the per-bundle 1366 specifics provided by the bundle transport layer (such as bundle 1367 destination, length, priority, time-to-live, and possibly 1368 "Starbucks"-related information). We envision a family of different 1369 dispatcher algorithms, operating as "plug-ins," that provide 1370 different levels of sophistication in the scheduling function to suit 1371 different needs. The simplest versions of the dispatcher might just 1372 consider destination to schedule the bundle on the soonest-possible 1373 outbound contact. More sophisticated versions might consider 1374 priority, bundle length (to preserve atomicity), and time-to-live 1375 requirements. Others could support the Starbucks model or apply 1376 optimization techniques to attempt to improve the use of each 1377 contact. 1379 The following is a conceptual description of what happens: when a 1380 bundle arrives at the bundle layer and needs to be routed, the bundle 1381 routing function posts a request to the dispatcher, noting the 1382 destination, length, priority, time to live, etc. of the bundle to be 1383 routed. The dispatcher integrates this bundle into its manifest and, 1384 at the bundle's transmission time, informs the bundle routing 1385 function to send the bundle to the appropriate next-hop destination. 1387 4.2.3. Systems participating in example bundle data transfer 1389 Figure 4 is a revision of Figure 2 to highlight those portions of the 1390 Interplanetary Internet that participate directly in the bundle 1391 transfer example. Also shown in Figure 4 are the source and 1392 destination for the bundle data transfer, and the Domain Name System 1393 equivalents in the terrestrial Internet (DNS 1), in the IPN 1394 "Backbone" (DNS 2), and in the Mars Internet (DNS 3). This figure 1395 will serve as the basis for the following bundle data transfer 1396 example. 1398 Table 1 provides the full host names of each of the primary elements 1399 in Figure 4. 1401 +---+ 1402 / /| 1403 +------------------------+---+ | 1404 +---+ Earth's Internet |DNS| + 1405 / /|IPN Region: earth.sol| 1 |/ 1406 +---+ | +---+ +---+ 1407 |SRC| +---------/ /|--------+ 1408 | |/ +---+ | 1409 +---+ |G/W| + 1410 +----------| 1 |/---------+ 1411 | +---+ | 1412 | The "Backbone" | 1413 | IPN Region: ipn.sol | 1414 +---+ +---+ +---+ 1415 / /|-------------------/ /| / /| 1416 +---+ | +---+ | +---+ | 1417 |DNS| + |G/W| + |DST| + 1418 | 2 |/ | 2 |/-------------| |/+ 1419 +---+ +---+ +---+ | 1420 | Mars's Internet | 1421 +---+ IPN region: | 1422 / /| mars.sol | 1423 +---+ |---------------------+ 1424 |DNS| + 1425 | 3 |/ 1426 +---+ 1428 Figure 4. Interplanetary Internet showing principal systems. 1430 Table 1. Host name tuples. 1431 +------------+------------------+---------------------------+ 1432 | Host | IPN Regions | Host name tuples | 1433 +------------+------------------+---------------------------+ 1434 | SRC | earth.sol | {src.jpl.nasa.gov, | 1435 | | | earth.sol} | 1436 +------------+------------------+---------------------------+ 1437 | IPN G/W1 | earth.sol | {ipngw1.jpl.nasa.gov, | 1438 | | | earth.sol} | 1439 | | ipn.sol | {ipngw1.jpl.nasa.gov, | 1440 | | | ipn.sol} | 1441 +------------+------------------+---------------------------+ 1442 | IPN G/W2 | ipn.sol | {ipngw2.nasa.mars.org, | 1443 | | | ipn.sol} | 1444 | | mars.sol | {ipngw2.nasa.mars.org, | 1445 | | | mars.sol} | 1446 +------------+------------------+---------------------------+ 1447 | DST | mars.sol | {dst.jpl.nasa.gov, | 1448 | | | mars.sol} | 1449 +------------+------------------+---------------------------+ 1451 4.2.4. Step 1: Bundle creation and first-hop transmission 1453 An application on the source host in Figure 4 has data that it wishes 1454 to send to the destination on Mars. The exact content of this data 1455 is opaque to the bundle transfer, but assume that it contains all of 1456 the information necessary to accomplish some desired function. That 1457 is, assume that application-specific instructions for storage, 1458 handling, error processing, and disposal accompany whatever data 1459 object is to be operated upon. The application invokes the bundle 1460 agent, supplying it the information shown in Table 2. 1462 Table 2. Information passed from source application to bundle 1463 agent. 1465 +-------------+---------------------+------------------------------+ 1466 | Item | Value | Description | 1467 +-------------+---------------------+------------------------------+ 1468 | Destination | {dst.jpl.nasa.gov, | IPN Name tuple of the | 1469 | host name | mars.sol} | destination | 1470 +-------------+---------------------+------------------------------+ 1471 | Destination | 0x00000008 | Similar to port number. Can| 1472 | application | | be "well-known" (i.e., | 1473 | instance | | identify a daemon) or | 1474 | handle | | "ephemeral" (i.e., identify | 1475 | | | a running, suspended, or | 1476 | | | hibernating process | 1477 +-------------+---------------------+------------------------------+ 1478 | Source | 0x1763421A | Value used to identify the | 1479 | application | | appropriate instance of the | 1480 | instance | | source application for | 1481 | handle | | response processing | 1482 +-------------+---------------------+------------------------------+ 1483 | Handling | Reliable delivery, | The services requested from | 1484 | instructions| normal priority, | the bundle layer. | 1485 | | data obsolete in 36 | | 1486 | | hours. | | 1487 +-------------+---------------------+------------------------------+ 1488 | User data | N/A | | 1489 +-------------+---------------------+------------------------------+ 1491 The bundle agent creates a bundle and stores it in persistent storage 1492 (on disk or other non-volatile memory). There are some fields of the 1493 bundle header that the bundle agent must supply: the Bundle 1494 Identifier, the Source Host name tuple, the Custodian name tuple, and 1495 the time to live. (The application may state a time after which the 1496 data are obsolete, but the actual time-to-live field in the bundle 1497 header uses the application's data in combination with network 1498 restrictions on time-to-live to initialize this field properly.) The 1499 bundle agent's routing function requests a route from the dispatcher 1500 application, and receives next-hop destination information and source 1501 information. (Since a host may reside in multiple IPN Regions, the 1502 source host name tuple is a function of the outbound route selected. 1504 The bundle agent uses this information to complete the Source Host 1505 and Custodian name tuples prior to transmission.) 1507 Note: For hosts that have continuously available communication 1508 resources, it is likely that an optimization would be implemented to 1509 reduce the required interaction with the dispatcher application. In 1510 this instance, the bundle routing function might consult a local 1511 routing table before contacting the dispatcher, and only requesting 1512 routing information from the dispatcher if there is no appropriate 1513 entry in the routing table. 1515 The bundle in this example is destined for the "mars.sol" IPN Region 1516 (per Table 1). The dispatcher (or routing table) determines that the 1517 proper next-hop destination for the mars.sol region is 1518 {ipngw1.jpl.nasa.gov, earth.sol}, and that the appropriate transport 1519 protocol to use is TCP. The bundle agent consults the Terrestrial 1520 Domain Name Server to resolve ipngw1.jpl.nasa.gov to an IP address, 1521 establishes a TCP connection with ipngw1, and transfers the bundle to 1522 it. The bundle agent at the source retains a custodial copy of the 1523 bundle in persistent storage. 1525 4.2.5. Step 2: Bundle processing at first-hop destination 1527 When the IPN Gateway {ipngw1.jpl.nasa.gov, earth.sol} receives the 1528 bundle via TCP, it stores it on persistent storage (disk). The 1529 bundle agent consults the dispatcher, and is informed that the 1530 appropriate next-hop destination is in the "ipn.sol" IPN Region: 1531 {ipngw2.nasa.mars.org, ipn.sol}. The dispatcher provides the time at 1532 which the bundle should be transmitted, at which time the contact 1533 scheduler and its associated functionality will ensure that there is 1534 a contact with ipngw2 via the Long Haul Transport Protocol (LTP). In 1535 considering alternative contacts for the bundle, the dispatcher 1536 checks the time-to-live in the bundle, which was 36 hours from the 1537 time of initial submission to the bundle agent at the source, to 1538 ensure that the route selected is consistent with the time-to-live 1539 requirements of the bundle. The bundle transport functionality of 1540 the bundle agent in ipngw1 accepts custody of the bundle, updates 1541 this information in the bundle header, and informs the source that 1542 has done so. The sources bundle agent deletes its custodial copy of 1543 the bundle. When the time indicated by the dispatching function 1544 arrives, the bundle is transmitted via LTP to the IPN Gateway that 1545 connects the ipn.sol Region with the mars.sol Region: 1546 {ipngw2.nasa.mars.org, ipn.sol}. 1548 4.2.6. Step 3: Bundle processing at gateway to destination IPN Region 1550 The Mars gateway, {ipngw2.nasa.mars.org, mars.sol}, receives the 1551 bundle from the Earth gateway via LTP. It stores the bundle in 1552 persistent storage and accepts custody of the bundle, signaling back 1553 to the Earth gateway that it has done so. When the notification of 1554 acceptance reaches the Earth gateway, ipngw1 deletes its custodial 1555 copy. The Mars gateway consults its dispatcher application to find 1556 an outbound contact to forward the bundle over. The dispatcher 1557 returns an indication that the appropriate next hop is the 1558 destination itself, that the proper transport protocol is TCP, and 1559 that the destination is accessible immediately. The gateway verifies 1560 that the time-to-live has not expired, and forwards the bundle to the 1561 destination. 1563 4.2.7. Step 4: Bundle processing at destination 1565 The destination bundle agent receives the bundle via TCP, stores it 1566 on its own persistent storage, and accepts custody of the bundle from 1567 IPN G/W2. The bundle agent "awakens" the destination application 1568 process identified by the Destination Application Instance Handle. 1569 This may involve creating a new instance of a server from a daemon 1570 process, signaling an idle running process, or reinstantiating a 1571 process that has been suspended with its state stored on persistent 1572 storage. (The specifics of this are system-dependent, and may have 1573 to be robust against system restarts, upgrades, and migration of 1574 processes from one host to another.) The bundle agent deletes the 1575 copy of the bundle from persistent storage when the application has 1576 received it. The destination application may generate an 1577 application-layer acknowledgment in a new bundle and send it to the 1578 source's application instance identifier (0x1763421A from Table 2). 1580 4.3. Error Conditions at the Bundle Layer 1582 This section describes the error conditions that may arise at the 1583 bundle layer during bundle creation and transport. When these errors 1584 occur within the sender's IPN domain, it may be possible to conduct a 1585 near-real-time dialog to correct them before the bundle is forwarded. 1586 We say `may be possible' because even if two nodes are in the same 1587 IPN domain, they may not have real-time connectivity. An example of 1588 such a situation would be if a lander were on the opposite side of 1589 the planet from its IPN gateway, and used bundles to communicate with 1590 the gateway through a low altitude orbiter, with the orbiter itself 1591 serving as a bundle agent. 1593 Table 3: Error conditions at the bundle layer. 1594 +-------+---------------------------+------------------------------+ 1595 | Error | Description | Places where Error can Occur | 1596 +-------+---------------------------+------------------------------+ 1597 | 1* | Unknown destination region| Source Bundle Agent | 1598 +-------+---------------------------+------------------------------+ 1599 | 2* | Invalid Source App. | Source Bundle Agent | 1600 +-------+---------------------------+------------------------------+ 1601 | 3* | Bundle Parameter Syntax | Source Bundle Agent | 1602 | | Error | | 1603 +-------+---------------------------+------------------------------+ 1604 | 4* | Bundle Parameter Semantic | Source Bundle Agent | 1605 | | Error | | 1606 +-------+---------------------------+------------------------------+ 1607 | 5* | Invalid Node Name in LSRR | Any bundle node | 1608 | | or SSRR | | 1609 +-------+---------------------------+------------------------------+ 1610 | 6 | Insufficient buffer space | Any bundle node | 1611 +-------+---------------------------+------------------------------+ 1612 | 7 | DNS unreachable | Any bundle node | 1613 +-------+---------------------------+------------------------------+ 1614 | 8* | Time exceeded | Any bundle node other than | 1615 | | | the source agent | 1616 +-------+---------------------------+------------------------------+ 1617 | 9* | Source Entity Access | Any bundle node other than | 1618 | | Denied | the source agent | 1619 +-------+---------------------------+------------------------------+ 1620 | 10* | Invalid Administrative | IPN gateway serving | 1621 | | Destination Name | destination IPN domain | 1622 +-------+---------------------------+------------------------------+ 1623 | 11* | Invalid Destination App. | Destination | 1624 +-------+---------------------------+------------------------------+ 1625 | 12* | End-to-end Access Denied | Destination | 1626 +-------+---------------------------+------------------------------+ 1628 The errors that can occur at the bundle layer are shown in Table 3. 1629 Error numbers marked with an asterisk (*) are reported back to the 1630 sending application. 1632 * Unknown Destination Region: This error occurs when the source 1633 bundle agent is directed to create a bundle destined for an IPN 1634 Region that is not recognized (i.e. one for which there is no 1635 applicable route known to the Dispatcher Application). Note that 1636 only the IPN Region part of the destination name has to be 1637 interpretable outside the destination's IPN Region. In 1638 particular, the administrative part of the destination name need 1639 not be interpretable to the source DNS (assuming the source and 1640 destination are in different IPN Regions), so it cannot 1641 necessarily be checked when the bundle is created. 1643 * Invalid Source Application: If the source application instance 1644 handle supplied by the source application is invalid, the source 1645 bundle agent responds with an Invalid Source Application error. 1646 This might be the case, for instance, if the source application 1647 provided an instance handle referencing a system process for which 1648 the application didn't have privileges. 1650 * Bundle Parameter Syntax Error: The source bundle agent may check 1651 the syntax of some of the bundle-creation parameters (i.e. it may 1652 ensure that the end-to-end and IPN access security certificates 1653 are well-formed, etc.) If a parameter is found to be 1654 syntactically incorrect or obviously and definitely erroneous, the 1655 bundle agent will report a Bundle Parameter Syntax Error back to 1656 the source that includes, at a minimum, the parameter that caused 1657 the error. 1659 * Bundle Parameter Semantic Error: If the source bundle agent can 1660 identify a particular bundle creation parameter as being well- 1661 formed but unserviceable, it will report a Bundle Parameter 1662 Semantic Error to the source application that includes, at a 1663 minimum, the parameter that caused the error. 1665 * Invalid Node Name in LSRR/SSRR: If an invalid node name is 1666 discovered in the loose or strict source route & record, the 1667 bundle agent that detects the error will propagate it back to the 1668 source application. Note that it may be advantageous to have 1669 bundle agents check the validity not only of the next hop in the 1670 source route but as many entries as they can. The value of 1671 checking multiple entries would be in detecting errors as soon as 1672 possible, preferably before the bundle traversed any of the long- 1673 haul links. 1675 * Insufficient Buffer Space: If a bundle agent does not have 1676 sufficient buffer space to accept a bundle, it drops the bundle 1677 and generates an Insufficient Buffer Space error. Note that a 1678 bundle node may choose to drop lower priority bundles in order to 1679 make room for higher priority ones. This error is not propagated 1680 back to the source. 1682 * DNS Unreachable: If a bundle agent needs access to its DNS (or 1683 DNS-equivalent) and cannot obtain information from it, it 1684 generates a DNS Unreachable error. This information is not 1685 propagated back to the source application. 1687 * Time Exceeded: If the time-to-live field (either the source- 1688 provided TTL or the internal bundle TTL) expires, the source is 1689 notified with a Time Exceeded message. These errors are 1690 propagated to the source application. 1692 * Source Entity Access Denied: This error indicates that the source 1693 entity does not have access to a needed resource at an IPN node. 1694 The source might not be authorized to use the node at all, or it 1695 might not have authorization to use a particular interface 1696 required by the bundle. Source Entity Access Denied errors 1697 indicate that the source is not authorized to use a particular 1698 resource; other errors (e.g. Insufficient Buffer Space) indicate 1699 that a particular resource is unavailable to a bundle. For 1700 example, an entity on the surface of a planet might be authorized 1701 to communicate, using the bundle protocol, with another entity on 1702 the other side of the planet via a low-altitude orbiter that is 1703 also an IPN gateway. The sender might not, however, be authorized 1704 to send bundles across interplanetary space. In this case bundles 1705 sent to the orbiter destined for the other side of the planet 1706 would not cause errors, while any bundles with off-planet 1707 destination addresses would. Source Entity Access Denied errors 1708 are propagated back to the source application. 1710 * Invalid Administrative Destination Name: Once a bundle has reached 1711 its destination IPN Region, the administrative part of the 1712 destination name can be verified. If the administrative part of 1713 the destination name is not valid, the source is notified with an 1714 Invalid Administrative Destination Name error message. As with 1715 LSRR/SSRR, it would probably be advantageous to check the 1716 administrative part of the destination name as soon as possible to 1717 avoid propagating misnamed bundles and error messages across the 1718 backbone. 1720 * Invalid Destination Application: If the destination bundle agent 1721 cannot instantiate the destination application (based on the 1722 destination application instance handle in the bundle), it 1723 notifies the source application with an Invalid Destination 1724 Application error message. 1726 * End-to-End Access Denied: If the bundle destination does not 1727 accept the bundle due to an authentication or access-control 1728 error, the source is notified with an End-to-End Access Denied 1729 Message. 1731 4.4. Support of existing Internet applications 1733 There is no clean way to support "legacy" applications in the IPN. 1734 One might think that application-layer proxies could protect 1735 applications from the rigors of interplanetary communications, but it 1736 turns out that this is not the case, even if the appropriate timers 1737 (at both application and transport layers) could be scaled correctly. 1738 As an example, consider a legacy FTP client on Earth trying to get a 1739 file from an FTP server on Mars. If the client's connection request 1740 to {foo.bar.com, mars.sol} is somehow coerced into binding to an 1741 Earth-resident application proxy (as can be done with nonstandard 1742 modifications to the terrestrial DNS) the proxy then has to negotiate 1743 enough information out of the client to form the bundle that will 1744 travel to Mars. This bundle needs to include at least: 1745 * The client's IPN domain name 1746 * The server's name tuple on Mars (Note that if wildcarded A- 1747 records are used to cause the off-planet server name to bind to 1748 the proxy, the proxy will not have a notion of the destination 1749 name. The proxy will have to explicitly request the source name 1750 from the source.) 1751 * The file to be retrieved 1752 * Where to deliver the retrieved file to, since it probably won't be 1753 coming back in this session 1755 While all of this information could in principle be elicited from a 1756 command-line client via appropriate server messages and extensive use 1757 of the `quote' command, there is no guarantee that other clients 1758 (specifically GUI clients) will be able to accomplish this. SMTP (e- 1759 mail) is perhaps the only application that could possibly be tuned to 1760 work over interplanetary distances, but even SMTP has embedded timers 1761 that would have to be altered significantly before it would work. 1763 5. Security in the IPN 1765 We do not have a detailed list of security requirements for the 1766 Interplanetary Internet, primarily because there currently aren't any 1767 "users" of the IPN and few, if any, of the potential users have given 1768 enough thought to security to commit to a set of security 1769 "requirements". However, we know that the Interplanetary Internet's 1770 bandwidth resources will be precious. We can also safely assume that 1771 the IPN will be a prized hacker's target (at least from Earth). We 1772 can also envision that there will be precious, private data flowing 1773 across the IPN. As a result, we assume that various security 1774 mechanisms and services will be required to provide protection for 1775 the bundles traversing the IPN and for the IPN infrastructure itself. 1777 There are two aspects to IPN security: protection of the IPN 1778 infrastructure and protection of the data traversing the IPN. The 1779 protection of the IPN infrastructure is not unlike the protection 1780 required for the Earth-based Internet infrastructure. There will be 1781 a need to exchange routing information securely among the IPN nodes 1782 as well as to securely manage them. For the IPN infrastructure to 1783 protect itself, the IPN nodes need to be mutually suspicious of one 1784 another. That is, the IPN nodes will authenticate themselves to each 1785 other to ensure that they are not being spoofed by an untrustworthy 1786 entity. One might ask if this is overkill if we believe that there 1787 will always be a small, controlled number of IPN nodes (a la the 1788 original ARPANET). However, one could equally envision that there 1789 could be many IPN nodes that are sponsored and controlled by multiple 1790 organizations (a la the current Internet). Since we would like the 1791 IPN to easily scale, we want to build mutual suspicion and security 1792 mechanisms into the IPN architecture from the outset. It should be 1793 noted that the same mechanisms could be used to provide security for 1794 both the IPN infrastructure and the data flowing through it. 1796 5.1. Assumptions Regarding Required IPN Security Mechanisms 1798 The security mechanisms assumed to be required for the IPN are: 1799 * Access control 1800 * Authentication 1801 * Data integrity 1802 * Data privacy 1804 Access controls will be required because the IPN's space-based assets 1805 will have limited resources available and because it will be a likely 1806 target from a hacking perspective. By limiting access to the IPN 1807 resources, we limit the availability of the IPN to only those users 1808 who require its services and do not allow it to be overburdened by 1809 those who are not authorized to access the IPN services. 1811 Authentication of identity will be required to perform access control 1812 mediation. To allow or disallow access to the IPN, an assured 1813 identity of the source of the network traffic will be needed. The 1814 identity might be of an individual (e.g., a payload principal 1815 investigator) or a location (e.g., a science payload control center 1816 or a spacecraft control center). 1818 Data integrity will be required to ensure that data received across 1819 the IPN is the same data that was originally sent. This is true from 1820 both an IPN infrastructure perspective (e.g., management data) as 1821 well as from a user's perspective (e.g., science data, command 1822 sequences). Data integrity assures that any unauthorized modification 1823 of the data will be detectable by the receiver. 1825 Data privacy will be required to ensure that only those who are 1826 authorized to obtain data traversing the IPN or destined to/from the 1827 IPN infrastructure will be able to do so. This mechanism is also 1828 known as confidentiality. 1830 In the network security community, there are two well-known security 1831 paradigms: hop-by-hop security (also known as link security) and end- 1832 to-end security. In the hop-by-hop paradigm, the data to be 1833 transmitted over the network is protected on a hop-by-hop basis. 1834 That is, the data is protected at its source, but in order to be 1835 routed to its final destination, it must be unprotected at a trusted 1836 routing point (e.g., a ground-based gateway, an IPN gateway) in order 1837 to be examined for onward routing. The trusted routing point must 1838 then re-protect the data and forward it on to its next routing point. 1839 Each successive hop point must unprotect and re-protect the data 1840 until it reaches its ultimate destination. A negative aspect of this 1841 security paradigm is that, depending on how the protection is 1842 applied; the data might be completely exposed while in the gateway 1843 (hence the term trusted gateway). This means that the data is 1844 potentially vulnerable to unauthorized modification and unauthorized 1845 disclosure. 1847 The end-to-end paradigm does not employ trusted gateways. Rather, it 1848 assumes that the path between the source of the data and its 1849 destination is hostile and cannot be trusted. As a result, the data 1850 is protected at its source and is not unprotected until it reaches 1851 its destination. However, in order for this scheme to work, routing 1852 information must remain unprotected so that the intermediate gateways 1853 are able to determine how to forward the data without having the 1854 opportunity to read or change the data. 1856 One problem with the end-to-end paradigm is that it can only work 1857 across a network where there are end-to-end protocols (e.g., TCP). 1858 There has to be a protocol below the data that provides the ability 1859 to route. One example of this is the Internet Engineering Task 1860 Force's (IETF) Internet Protocol Security Protocol (IPSEC). The 1861 IPSEC Encapsulating Security Payload (ESP) protocol provides an end- 1862 to-end security service between the IP and TCP protocol layers. 1864 However, in the IPN, as has already been discussed earlier, IP and 1865 TCP are not necessarily carried end-to-end protocols. Rather, those 1866 protocols can/will be terminated in a local internet (e.g. a ground 1867 segment on earth or another celestial body) and not carried end-to- 1868 end through the IPN. The data will be carried end-to-end via the 1869 bundle protocol that would in turn, be transported by other IPN 1870 protocols (e.g., LTP, TCP) using the pony express model. 1872 Since the IPN must be structured on a store-and-forward basis, and 1873 since users may not trust the IPN gateways, solutions such as IPSEC 1874 cannot be employed. In order to provide an end-to-end like security 1875 solution, security mechanisms can only be applied to the data and not 1876 any other protocol layer(s) below the bundle protocol. This is the 1877 way the Secure Sockets Layer (SSL) (now being specified within the 1878 IETF as the Transport Layer Security (TLS) protocol) and secure email 1879 techniques such as S/MIME and OpenPGP work. In essence, security 1880 services are only applied to the data portion of a packet, leaving 1881 all of the packet's protocol headers in the open and available for 1882 use by intermediate systems. For example, to transmit the string 1883 "hello world" across the Internet would entail encapsulating the 1884 string into a TCP header, an IP header, and a media access (MAC) 1885 layer header. To provide end-to-end security services, only the 1886 payload ("hello world") would have security services applied to it 1887 (e.g., it might be encrypted to provide privacy/confidentiality). 1888 However the TCP, IP, and MAC headers would all remain without 1889 security services applied to them. In comparison, when using IPSEC 1890 ESP, the TCP header would have security services applied to it to 1891 protect it as well as the payload data. And if it were operating in 1892 "tunnel" mode, ESP would encapsulate the entire payload - the TCP and 1893 IP headers - inside of an IP packet. 1895 Given that data will be transported across the IPN in bundles using 1896 an email-like paradigm, borrowing technology from email security is a 1897 solution for the IPN. 1899 5.2. Secure Email Technology 1901 Since secure electronic mail operates in a non-interactive mode, and 1902 since both communicating parties have not necessarily communicated 1903 with one another previously, a technique different than used by IPSEC 1904 was developed. One way in which email could be securely wrapped 1905 (e.g., encrypted and/or digitally signed) is via through the use of 1906 public key cryptography. Using this technology, an email sender 1907 would use the public key of the intended recipient to encrypt the 1908 payload (i.e., the message) and the recipient would use its private 1909 key to decrypt the payload. Likewise, the sender could digitally 1910 sign the message (in order to authenticate the message) using its 1911 private key and the recipient would verify the message's authenticity 1912 using the sender's public key. However, there are two problems with 1913 this scheme for the IPN. 1915 The first problem is that public key cryptography is quite 1916 consumptive of processor power. Therefore, the common practice is to 1917 use symmetric key (i.e., shared secret) algorithms for large amounts 1918 of data. With changes in technology, this problem may disappear, at 1919 least for earth-based systems. However, it is not clear that space- 1920 based assets will catch up as quickly. The second problem is that 1921 public key cryptography is consumptive of bandwidth. A public key 1922 exchange technology that allows two communicating entities to derive 1923 a shared symmetric key by virtue of knowing each other's public keys 1924 and exchanging some random information is known as a Diffie-Hellman 1925 exchange. However, the exchange of information must be performed in 1926 a near-real-time environment so that the shared key can be used to 1927 encrypt transmissions. This is not practical in the IPN since there 1928 are no real-time exchanges of data on an end-to-end basis. 1930 However, the secure email community has realized that they also 1931 operate under a pony-express model. The secure email community has 1932 also realized that secure email may be sent to entities with which 1933 one has not previously communicated. Therefore, there needed to be a 1934 base-level expectation of a common, minimal set of security services 1935 that both sides could use. 1937 The general technique used by the secure email community is that the 1938 sending entity decides what security mechanism(s) to employ (e.g., 1939 encryption for confidentiality, digital signature for authentication 1940 and integrity, or both). If the data to be sent via email is to be 1941 encrypted, the sender generates a random key that is used to encrypt 1942 the data and the data is encrypted. The sender then either has in 1943 its possession the public key of the receiver, or queries a public 1944 key server (e.g., a public key infrastructure or PKI) to obtain the 1945 receiver's public key, which would be contained in a digital 1946 certificate. At the expense of additional bandwidth, the sender can 1947 transmit its digital certificate in the email message, which the 1948 receiver can verify as genuine based on the well-known certificate 1949 authority's signature on the certificate. The key used to encrypt 1950 the data is encrypted using the public key of the receiver and the 1951 message is sent. The receiver uses its private key to first decrypt 1952 the key used to encrypt the message and then uses the decrypted key 1953 to decrypt the data. 1955 When a digital signature is used, the sender calculates a hash of the 1956 message using a digest algorithm such as MD5 or the Secure Hash 1957 Algorithm (SHA-1). The resulting hash is then encrypted using the 1958 sender's private key. The encrypted hash is sent along with the 1959 message data. The receiver uses its copy of the sender's public key 1960 to authenticate the fact that the message was sent by the sender. 1962 5.3. Application of Secure Email Technology to the IPN 1964 Given that the IPN and electronic mail share the same operational 1965 paradigm, the IPN's notion of "bundle-space" is directly analogous to 1966 a secure email MIME body part. As was previously explained, in 1967 secure email the contents of the message are encrypted using a 1968 symmetric key. The actual message contents are "containerized" 1969 (which can be analogized to the contents being "bundled") into a MIME 1970 body part. Additional MIME body parts are also attached which would 1971 contain the encrypted symmetric key and additional information needed 1972 for decryption. 1974 Therefore, the same security techniques can be applied to the IPN's 1975 notion of "bundle-space." It can be seen how bundle payload data 1976 carried through the IPN is equivalent to an email message. 1977 Therefore, the bundle payload data, like the email message, can have 1978 security services applied to it before being sent as a protected 1979 entity via bundling across the IPN. 1981 In essence, the really difficult part of deploying an IPN security 1982 solution based on secure email is the problem of deploying a public 1983 key infrastructure (PKI) in the IPN. Deploying a PKI on Earth is 1984 equally difficult as can be seen by the continuing problems faced in 1985 the various PKI pilot programs currently underway. 1987 On Earth, the problem is how do competing PKIs cross certify public 1988 keys and who acts as the root certification authority? (PKI is based 1989 on a tree hierarchy where a root certificate authority's public key 1990 is well known in order to certify public key certificates). On 1991 Earth, when a user does not have the necessary public keys, the user 1992 can go off to a public key certificate server to obtain the needed 1993 certificates which contain digitally signed (i.e., certified and 1994 authenticated) public keys. In the case of email, the sender can 1995 attach its public key certificate as a MIME body part. The receiver 1996 then validates the digital certificate and uses it to obtain the 1997 information contained in the message. Eventually, a cache of digital 1998 certificates is formed containing the information needed to securely 1999 communicate among a cadre of entities without having to resort to the 2000 use of a key/certificate server. 2002 The IPN, however, will not have the luxury of being able to query on- 2003 line PKI certificate servers (at least not in the same sense as is 2004 being contemplated on Earth). The problem is very much analogous to 2005 the one with the Domain Name System (DNS). When a user sends a 2006 secured "bundle" to an IPN entity on another celestial body, we would 2007 not want the receiver to have to first query a PKI for a public key 2008 certificate on Earth to obtain the receiver's public key. A local 2009 PKI might exist. However there is the issue of the dynamic nature of 2010 such a server and the high likelihood that it would quickly drop out 2011 of synch. It would seem reasonable that the public key certificates 2012 of the various space-based entities would not change often, but this 2013 could not be said about the Earth-based entities. 2015 An earth-based sender may be able to query a certificate authority to 2016 obtain a certificate for an IPN entity not located on earth. 2017 However, an IPN entity in space sending a bundle would not be able to 2018 perform such a query. Therefore, it would appear that a set of pre- 2019 placed, cached certificates containing the public keys of those IPN 2020 entities that are expected to be communicated with would make sense. 2021 In addition, an IPN sender should always include its public key 2022 certificate as part of the bundled transmission across the IPN as is 2023 done using secure email. This would "cost" us in terms of bandwidth 2024 (a typical X.509 public key certificate is about 1K bytes in size but 2025 can be larger when certificate chains are included). However, by 2026 including the certificate, we can be assured that the receiver will 2027 not have to use any additional bandwidth, or incur any additional 2028 delays in making use of the transmitted data. 2030 5.4. Protecting IPN Data and the IPN Backbone Infrastructure 2032 The IPN will have few and precious resources. Therefore, not only 2033 the user data flowing across the IPN will require security services - 2034 the backbone infrastructure itself will require similar services. In 2035 order for the IPN infrastructure to be self-protecting, it must be 2036 built using the paradigm of mutual suspicion. Mutual suspicion 2037 requires each entity of the IPN to not assume that another IPN entity 2038 with which it is communicating is a "friendly" entity. That is, it 2039 should not assume that the IPN entity is who it claims to be without 2040 a verified means of identification. Based on a verified identity, 2041 access controls can be performed to allow or disallow communications 2042 between entities. 2044 Infrastructure information such as routing updates, node management 2045 information, distribution of digital certificates, and certificate 2046 revocation lists need to be protected from unauthorized modification 2047 and potentially from unauthorized disclosure. The IPN nodes would 2048 use the same mechanisms as are used to provide protection for user 2049 data - namely the security services available to a bundle aware 2050 application (e.g., a "bundle-agent"). 2052 A bundle aware application would encrypt and/or digitally sign the 2053 IPN infrastructure payload to be transmitted to another IPN node. 2054 The signed and/or encrypted payload would then be presented to a 2055 bundle-agent that would prepare the payload data to be carried across 2056 the IPN in a bundle. Potentially, the bundle-agent might also sign 2057 and/or encrypt for hop-by-hop security protection, which would allow 2058 each bundle-agent receiving the bundle to authenticate the identity 2059 of the transmitting bundle-agent. This mechanism provides the 2060 ability to ensure that no other entity can masquerade as a rogue 2061 bundle-agent. 2063 The bundle-agent residing at a ground-based IPN gateway should check 2064 the signature of the bundle payload to perform an access control 2065 check to limit access to the IPN only to those who are authorized to 2066 use it. The access control check can be accomplished via an access 2067 control list. 2069 Finally, the receiving application would make the final security 2070 checks before accepting the data payload from the final receiving 2071 bundle-agent. Should all the final checks succeed, the receiving 2072 application would then be free to consume the data. 2074 6. Building a Stable Backbone for the IPN 2076 Just as the performance and capability of the terrestrial Internet 2077 are largely determined by the capacity and stability of its backbone 2078 links, so will the performance and capability of the Interplanetary 2079 Internet depend in large part on the capacity and stability of the 2080 interplanetary backbone. 2082 By "backbone" we mean a set of high-capacity, high-availability links 2083 between points of access to high-activity subnets. In the 2084 terrestrial Internet, backbone links are typically between the high- 2085 activity subnets for cities such as Houston and Chicago. In the 2086 Interplanetary Internet the backbone links will be between the high- 2087 activity "subnets" for planets such as Earth and Mars. 2089 But the IPN backbone will differ from familiar terrestrial backbones 2090 in several important ways. 2092 * The media by which information is transmitted obviously differ. 2093 Terrestrial backbone links used to be copper wire and are now 2094 optical fiber. In the Interplanetary Internet, all information 2095 will be via radiation _ either RF or optical _ through empty 2096 space. 2097 * The nature of the connectivity between backbone points of presence 2098 (POPs) will be fundamentally different. Terrestrial Internet 2099 connectivity is structural and relatively static: nodes are 2100 physically attached to fiber. Interplanetary Internet 2101 connectivity will be operational, directed, and highly dynamic: 2102 radiation must be directed at the right moment toward nodes that 2103 are prepared to detect it, and the transmitting and receiving 2104 entities will be in continuous movement relative to one another. 2105 * The costs of deploying, repairing, and upgrading infrastructure 2106 will be much higher for the interplanetary backbone than for 2107 terrestrial backbones. 2108 * The costs of configuring, operating, and managing the 2109 interplanetary backbone will likely be somewhat higher than for 2110 terrestrial backbones. One factor in this is the scarcity and 2111 high cost of electrical power at IPN nodes deployed off Earth. 2112 * Perhaps most important of all, the distances between nodes of the 2113 interplanetary backbone will be vastly greater than the distances 2114 between nodes of terrestrial backbones _ so great that the speed 2115 of light becomes a very significant constraint on backbone 2116 operations. 2118 6.1. Backbone Design Considerations 2120 The differences described above imply two general constraints on the 2121 design of the interplanetary backbone. 2123 1. 2124 Bandwidth is not free, or even cheap. Reliable delivery cost per 2125 bit will be far higher across the interplanetary backbone than 2126 across any terrestrial backbone, so bandwidth must be thoughtfully 2127 allocated. Both retransmission and forward error correction 2128 coding contribute to reliable delivery, but both cost bandwidth; 2129 we must seek a balance between the two that minimizes overall 2130 overhead. 2131 2. 2132 Interactive protocols don't work, at least not well. Negotiation 2133 _ connection establishment, flow control, congestion control _ can 2134 easily take so long as to consume all available transmission time. 2135 Reliable, in-order stream delivery can be so severely delayed by 2136 retransmission latency as to be operationally useless. 2138 These design constraints, in turn, must be accommodated at four 2139 layers of the protocol stack. 2141 At the physical layer, the relevant research is in radiant RF or 2142 optical communication. The physical infrastructure of the 2143 interplanetary backbone consists mainly of antennae, many of them 2144 mounted aboard orbiting or landed spacecraft, rather than cable in 2145 buried or undersea conduit. In the near to medium term, the 2146 principal elements of this infrastructure will be Earth-based 2147 tracking stations, such as NASA's Deep Space Network, and the planned 2148 "Marsnet" of low-Mars orbiters plus a possible areostationary relay 2149 satellite. In the long term these assets might be augmented by 2150 optical communication satellites orbiting Earth and/or other 2151 planetary bodies, and perhaps by additional relay satellites 2152 positioned at the LaGrange points of planetary orbits (possibly using 2153 solar sailing technology for autonomous station-keeping). 2155 Although this infrastructure will not be subject to some of the 2156 problems of terrestrial backbones _ for example, we needn't worry 2157 about backhoes cutting through underground fiber _ there will be 2158 other challenging operational issues. Accuracy in pointing and 2159 transmission scheduling at the backbone antennae will be critical to 2160 assuring the most efficient use of these assets. This means that all 2161 elements of the interplanetary backbone infrastructure must share a 2162 common understanding of (a) one another's orbital dynamics and (b) 2163 the current time. The latter argues for the importance of reliable 2164 space-borne clocks, together with a protocol for frequent correction 2165 of clock drift based on current navigation data. 2167 Moreover, there will be times at which connectivity between a given 2168 pair of backbone antennae will be impossible due to the interposition 2169 of large, radiant planetary bodies _ or worse, the sun. These 2170 occultations will be predictable, given good models of orbital 2171 dynamics, but will nonetheless necessarily constrain the manner in 2172 which data flow through the backbone. 2174 At the link layer, the design issues are somewhat less exotic. The 2175 interplanetary backbone will require link protocols that minimize 2176 overhead and that have no inherent expectations of low noise or low 2177 point-to-point transmission latency. CCSDS protocol standards such 2178 as Proximity-1 are plausible candidates. They support robust encoding 2179 to reduce bit error rates, at some cost in bandwidth consumption. 2181 Our current thinking is that no interplanetary backbone functionality 2182 will be required at the network layer. The reason for this is that 2183 we expect the endpoints of transport-layer protocol on the backbone 2184 to be in direct line-of-sight contact as discussed below; since there 2185 will be no intervening nodes to route through, there will be no need 2186 for routing functionality at the network layer. [Selection of 2187 endpoints for point-to-point transport-layer communication does 2188 indeed imply the need for routing and network functionality, but this 2189 requirement is addressed at the Bundling layer above Transport.] 2191 Finally, the general constraints on the design of the interplanetary 2192 backbone mandate constraints on the protocol used at the transport 2193 layer. The TCP transport protocol used in both the backbones and the 2194 subnets of the terrestrial Internet _ and of other planetary subnets 2195 of the IPN _ will not be suitable: connection negotiation and in- 2196 order stream delivery are incompatible with the enormous distances 2197 between nodes of the interplanetary backbone. 2199 As noted earlier, the bundle protocol residing just above the 2200 transport layer is by nature relatively optimistic about transmission 2201 success but it must have transport-layer-like properties, i.e., it 2202 must be able to recover from transmission failure at lower layers. 2203 The capacity for timeout detection and custodian-to-custodian 2204 retransmission has to be built into it. 2206 However, the bundle protocol's structural optimism would result in 2207 poor performance if such a capacity were exercised frequently. That 2208 optimism has to be justified by the general trustworthiness of lower 2209 layers: 2211 * When the bundle protocol runs over TCP in a deployed internet, 2212 TCP's own retransmission regime automatically recovers from errors 2213 in the network and link layers, and only a failure in TCP itself 2214 will trigger retransmission at the bundle layer. 2216 * When the bundle protocol is operating over interplanetary 2217 distances, a similar level of trustworthiness must be provided by 2218 a new interplanetary transport protocol. 2220 This protocol will necessarily be very different in operation from 2221 TCP. For example, connection time within the interplanetary backbone 2222 will be too rare and valuable to squander in waiting for reciprocal 2223 traffic of any kind. Consequently the basic mode of operation in the 2224 interplanetary transport protocol will be asynchronous: data will be 2225 issued continuously throughout each episode of connectivity, with 2226 return traffic _ acknowledgements and coordination messages _ matched 2227 up to the corresponding original data as it arrives. Data will 2228 arrive at the final destination out of order (because retransmitted 2229 data will arrive late); in many cases, even out-of-order delivery to 2230 applications maybe preferable to waiting the minutes, hours, or even 2231 days required for complete, error-free acquisition of data. 2233 Fortunately, out-of-order delivery due to retransmission delay may be 2234 made relatively rare by forward error correction at the link layer. 2235 Only a failure in that error correction need trigger retransmission 2236 at the transport layer. 2238 Accurate timeout detection will be critical to the success of this 2239 retransmission regime. Premature timeouts would result in 2240 unnecessary retransmission, consuming precious bandwidth and 2241 degrading link utilization; late detection of timeouts would delay 2242 both bundle delivery and the recovery of retransmission processing 2243 and storage resources. We believe interplanetary transmission 2244 timeouts can be detected accurately, but only when round-trip times 2245 can be calculated from firm operational data rather than estimated 2246 from past experience: the extreme variability in round-trip time 2247 introduced by intermittent connectivity means that the total round- 2248 trip time for transmission N might be hundreds of times greater than 2249 that for transmission N _ 1. The required operational data could in 2250 theory be provided regardless of the topological relationship between 2251 the Transport-layer endpoints, but we believe support for any model 2252 other than point-to-point, direct line-of-sight transmission between 2253 the endpoints will not scale. [Note that accurate clock 2254 synchronization across the interplanetary backbone is as important 2255 for implementation of timeouts at the transport layer as it is for 2256 resource scheduling at the physical layer.] 2258 Any number of blocks of transport-layer data traffic might 2259 concurrently be in various stages of transmission and retransmission, 2260 so the aggregate storage space allocated to transmission state data 2261 and retransmission buffers might be very large. Moreover, loss of 2262 this information due to (for example) an unplanned power cycle could 2263 abort any number of transmissions _ unlike an unplanned reboot of a 2264 TCP host, which will crash only the messages that are currently being 2265 transmitted on all open streams. For these reasons, it may well be 2266 desirable to retain interplanetary transport protocol data in non- 2267 volatile storage rather than dynamic memory. 2269 In some cases a set of transport protocol agents _ e.g., the set of 2270 relay satellites orbiting a planet _ may be most useful if they can 2271 operate in concert as a single "aggregate" entity. By distributing 2272 the retransmission workload across multiple agents as they 2273 successively acquire connectivity to a common peer agent, this 2274 collective behavior might reduce the total elapsed time required to 2275 effect reliable completion of a large block. If so, we'll 2276 additionally need some sort of application-layer coordination 2277 protocol operating among the members of this aggregate entity. 2279 Flow control and congestion control are possibly the least tractable 2280 interplanetary backbone design issues at the transport layer. TCP's 2281 strategies are based on reciprocal message exchange that the large 2282 delays in interplanetary communication make generally impractical. 2283 Alternative approaches remain to be discovered. 2285 7. Deployed Internets in the IPN 2287 We differentiate between two types of networks in the IPN: the long 2288 haul backbone and deployed internets. The long haul backbone, 2289 described in the previous section, is characterized by long 2290 propagation times, due to the great physical separation among the 2291 communicating elements. Its protocols are designed to operate 2292 efficiently in long-delay, intermittent-connectivity environments. 2293 As discussed in [ref: Why not TCP], as delays increase, the 2294 efficiency of the Internet suite steadily decreases until 2295 applications fail altogether due to embedded time outs. 2297 We designate a network to be a "deployed internet" if it meets the 2298 following criteria: 2300 * It has connectivity to an interplanetary gateway, and through that 2301 gateway can reach the interplanetary backbone or other deployed 2302 networks. 2303 * It has a communication environment that doesn't inherently 2304 preclude the use of (possibly enhanced) internet protocols. 2305 * It uses the Domain Name space as a common means of referencing 2306 objects and systems across deployed internets. 2308 This designation covers a wide range of possible configurations. The 2309 following list provides examples of deployed internets: 2311 * A single lander that hosts an interplanetary gateway and a (real 2312 or virtual) lander-internal network; 2313 * A small number of cooperating robots on a planetary surface, such 2314 as a single lander and a single rover; 2315 * An orbiter, a lander, and a sample-return vehicle communicating 2316 among themselves and via interplanetary gateways hosted in the 2317 orbiter and/or the lander; 2318 * Multiple "cells" of robots on a planet surface communicating 2319 beyond line of sight via low-orbiting satellites that serve as 2320 relays and as interplanetary gateways; 2321 * Similar scenarios as above, but with one or more planet-stationary 2322 satellites serving as interplanetary gateways; 2324 * Spacecraft-onboard networks that communicate with remote endpoints 2325 via interplanetary gateways; 2326 * The earth's Internet is the initial instantiation of a "deployed 2327 internet." 2329 7.1. Applications of deployed internets in the IPN 2331 It is appropriate to consider the types of activities that will 2332 exercise the deployed internets. Clearly, we do not yet know all of 2333 the types of applications that will be used in deployed internets, 2334 but we do have a basic idea of some of the types of applications that 2335 will be used in the relatively near term. In developing this notion 2336 of applications within deployed internets, we want to eventually 2337 develop the ability to characterize their use of the IPN in terms of 2338 arrival rates, data volume, latency and reliability requirements, 2339 number and location of producers, number and location of consumers 2340 (within the IPN), etc. At this moment, we have only highly 2341 qualitative notions of these kinds of data. Over time, we will 2342 develop a more quantitative representation of this traffic as an 2343 adjunct for testing. The model that we develop will probably never 2344 accurately reflect the actual use of the IPN, because the use of the 2345 IPN will be affected by its availability: users will adapt their 2346 usage patterns as their understanding of the network's capabilities 2347 develops. We don't ever expect to hit this moving target, but we 2348 hope to come close enough to discover most of the usage-related 2349 issues that might exist. 2351 One of the primary uses of a deployed internet is to return science 2352 data from the point where it is collected to the point where it will 2353 be processed. The processing point could be on a remote internet 2354 (say, on earth) or at some point in the local deployed internet. 2355 Depending on the resource availability at the gathering site versus 2356 the availability at the processing point, the transfer could be 2357 largely unprocessed data, shipped as a formatted stream or as a file. 2358 Alternatively, the data could be processed to reduce its transmission 2359 size, which would likely result in a file transfer operation. Sizes, 2360 frequency, precise reliability requirements, and so forth remain to 2361 be determined. However, in general, this data is not particularly 2362 time-sensitive (although the equipment may be quite sensitive to the 2363 amount of time that it takes to complete a transfer due to power 2364 considerations, which will be discussed later). 2366 Another primary application, the reporting of health and status 2367 telemetry, is typically accomplished via repetitive, unreliable 2368 transmission in today's spacecraft. There is underway a slow 2369 evolution toward data-summarization on board spacecraft. But there 2370 is, understandably, some resistance to filtering or summarizing data 2371 at the spacecraft, since in the event of a spacecraft catastrophe, 2372 data not previously transmitted is not available for post-mortem 2373 analysis. An important aspect of current telemetry systems is its 2374 delivery characteristics, which are either "stream-oriented" or 2375 periodically delivered. Mission operators perceive that "heartbeat"- 2376 style information that is ancillary to a periodic or "continuous" 2377 stream of telemetry is an important characteristic that will not 2378 displace event-driven telemetry for the foreseeable future. 2380 A third type of application is the command and control of in-situ 2381 elements. Command and control refers to the closed-loop control of 2382 remote systems. The endpoints of the control loop could be separated 2383 by interplanetary space, as in the case of terrestrial commanding of 2384 a rover. Alternatively, the endpoints could be in reasonably close 2385 proximity, but resident on physically separate platforms connected by 2386 wireless media. This is the case when a lander controls a rover. 2387 These types of applications must be designed to accommodate the 2388 delays intrinsic in the network, but the network can permit more 2389 responsive control operations by providing qualities of service that 2390 mitigate those delays. With command and control applications, data 2391 volumes tend to be fairly low, and the commands tend to be followed 2392 by responses (although in some instances, command responses are 2393 inferred from returned telemetry, described above). 2395 The final type of application we consider is telescience and the 2396 development of a "virtual presence." This type of application is 2397 intended to synthesize great volumes of information about the local 2398 environment in order to allow earth-resident scientists, in-situ 2399 controlling robots, or eventually in-situ astronauts to interact with 2400 high-fidelity models of a portion of a remote environment. To an 2401 extent, this technology has already been used to assist in planning 2402 the Mars Pathfinder/Sojourner excursions (ref 2403 http://www.piercorp.com/Projects/mars/mars2.htm). However, the 2404 technology for fully exploiting this type of adjunct to exploration 2405 is still in development. Terrestrial research into telepresence and 2406 tele-immersion (ref: 2407 http://www.advanced.org/tele-immersion/publications.html) may provide 2408 insight into the workload that this type of application might 2409 represent for the IPN. 2411 7.2. Characteristics of remote deployed internets in the IPN 2413 Although we consider the Earth's Internet to be the first instance of 2414 a deployed internet, the environmental characteristics affecting 2415 internets deployed in remote locations may differ significantly from 2416 those affecting Earth's Internet. In examining these 2417 characteristics, we must differentiate between two different types of 2418 environments in the Earth's Internet: the traditional, "wired" 2419 environments; and the emerging mobile, ad hoc wireless environments. 2421 First and foremost among the environmental characteristics of note is 2422 that of power availability. In the "wired" Internet on Earth, power 2423 is cheap and plentiful. Power availability is more of an issue in 2424 terrestrial mobile, ad hoc networks (MANETs), because the mobile 2425 nodes operate using portable power sources, typically batteries. 2426 However, in terrestrial MANETs, the mobile nodes eventually have 2427 access to the same cheap and abundant sources of power that the 2428 "wired" nodes use. This is not the case for remote deployed 2429 internets (RDIs) in the IPN. For the foreseeable future, the primary 2430 source of power for RDIs in the IPN will be the sun. Solar power 2431 conversion is currently relatively inefficient, but even if dramatic 2432 improvements in conversion efficiency occur, the fact remains that as 2433 one moves away from the sun, the available power diminishes. In the 2434 orbit of Mars, the average solar intensity is 590 W/m2, less than 2435 half of the 1370 W/m2 in Earth orbit [ref: 2436 http://powerweb.lerc.nasa.gov/pv/marspower.html ]. On the surface of 2437 a planet, the solar intensity is by seasonal variations, by dust 2438 build-up on solar panels, and by dust erosion of the solar panels 2439 over time. As a result, power availability is and will be of 2440 overriding importance to all aspects of communication in RDIs, and 2441 will dictate a need for efficiency at all protocol layers. 2443 The signal-to-noise ratios (SNRs) experienced in the terrestrial 2444 wired Internet are currently very high, making loss due to data 2445 corruption a rare experience. In terrestrial MANETs, SNRs are lower, 2446 due to both power availability and to node density. In remote 2447 deployed internets, SNRs will be very low, due primarily to power 2448 availability. 2450 In the terrestrial wired Internet, the bulk of the routing 2451 infrastructure is fixed in terms of its location. Conversely, in 2452 terrestrial MANETs, the infrastructure is deployed on an as-needed 2453 basis, and may be mobile. In RDIs, this infrastructure is also 2454 deployable and mobile, but will have a large component of satellite- 2455 based resources. 2457 If we examine the transmission media in use, the terrestrial wired 2458 Internet uses primarily copper cable and optical fiber. Terrestrial 2459 MANETs use free-space propagation in the radio frequency (RF) range 2460 or the infrared range of the electromagnetic spectrum. In RDIs, we 2461 anticipate that free-space RF will be the primary communication 2462 medium, even for many permanent, immobile installations as they 2463 develop, due to the cost of landing, deploying, and maintaining cable 2464 or fiber. 2466 The cost characteristics of deployment, operations, repair of RDIs is 2467 not currently well understood. However, deployment cost of anything 2468 that must be landed on the surface of another planetary body is quite 2469 high. Correspondingly, repair or replacement costs of components of 2470 the RDIs are also high. We do not yet have a clear understanding of 2471 how the operations costs of the RDIs will compare to the operating 2472 costs of terrestrial MANETs or of the terrestrial wired Internet. 2474 7.3. Effects of environmental characteristics on protocols for the 2475 IPN RDIs 2477 In considering how to deal with the anticipated operating environment 2478 for RDIs, we are encouraged that a significant amount of relevant 2479 research is currently under way to support protocols for terrestrial 2480 MANETs. This section briefly reviews some of the areas in which work 2481 is required, and is organized by protocol layer. 2483 At the physical layer, spectrum management is an issue that requires 2484 coordination. There is currently no "Federal Communications 2485 Commission" or International Telecommunications Union that regulates 2486 use of radio-frequency spectrum outside of earth. However, 2487 coordination of this sort is needed sooner rather than later, due to 2488 the attractiveness of certain frequency bands as a result of their 2489 free-space propagation loss and refractive characteristics [ref: 2490 private communication]. 2492 The cost of landing equipment on remote planetary surfaces motivates 2493 designers to keep as much communication infrastructure in orbit as 2494 possible. Low-orbiting satellites will be used extensively to 2495 support surface-to-surface communication and RDI-to-RDI 2496 communication. Antenna design to support wideband communication 2497 between surface-based mobile nodes and these low-orbiting satellites 2498 is an area where research might yield great benefit. Although we 2499 have heard of some research in these areas to support military on- 2500 the-move applications, we have seen nothing in detail and feel that 2501 it is an area that is ripe for additional study. 2503 At the link layer, management of low SNRs is of significant 2504 importance, with many areas of relevant research. While many 2505 different coding schemes are available, it is not yet clear whether 2506 one of these is best for all RDI use or whether the QOS requirements 2507 of different types of traffic will dictate the use of different 2508 coding schemes on a per-packet basis. Many codes are currently being 2509 considered, including convolutional coding, concatenated codes (such 2510 as Reed Solomon codes), and the emerging Turbo codes. Each of these 2511 has different properties related to delay, residual error rate, and 2512 link acquisition characteristics. 2514 Resource reservation schemes at the media access level may be of 2515 significant importance in supporting closed-loop control operations. 2516 Some of these schemes have the benefits of providing bounded delays 2517 and of avoiding interference, which is important in power-constrained 2518 environments. 2520 Link-layer status detection and signaling (to upper layers) is 2521 important in resource-constrained environments. We believe that the 2522 link layers must detect and signal link availability, link capacity 2523 and congestion status, and current error conditions. As the 2524 terrestrial cellular telephone industry becomes internet-enabled, 2525 some of these issues are beginning to be addressed in single-hop 2526 wireless environments. There is also ongoing research of interest in 2527 MANETs and in DARPA-sponsored research [ref. 2528 ftp://ftp.rooftop.com/pub/apis/link_api.pdf ]. 2530 At the network layer, there is a need for routing protocols to 2531 support both fast- and slow-moving mobile nodes. The routing 2532 protocols must be able to constitute and maintain networks comprised 2533 of a combination of fixed and mobile nodes. Significant research in 2534 this area is currently being performed, and is being coordinated by 2535 the IETF's MANET working group [ref. 2536 http://www.ietf.org/html.charters/manet-charter.html ]. 2538 Another area of relevant network-layer research concerns "vertical 2539 handoffs," which allow nodes to adapt to changes in available links 2540 or the resources available via particular links. This work [ref. 2541 http://daedalus.cs.Berkeley.edu/publications/monet97.ps.gz ] is 2542 focused on "last-hop" use. Its potential for use within wireless 2543 routers requires further research. 2545 As link layers enhance their capabilities to monitor and report their 2546 status internally to the network layer, network layer control 2547 protocols and algorithms must be enhanced to appropriately propagate 2548 this information to affected parties within the RDI. This topic is 2549 an active research area, with some applicable work having been done 2550 within the SCPS project [ref: http://www.scps.org ]. 2552 Just as resource reservation schemes within the link layer may be 2553 important mechanisms for providing bounded link-layer delays, 2554 resource allocation schemes at the network layer may also be 2555 important for providing such bounds on end-to-end communication 2556 within the RDI. However, resource allocation and resource 2557 reservation within mobile wireless networks is still a subject of 2558 ongoing research. Should wireless networks use the integrated 2559 services model of resource allocation? Or should they use the 2560 differentiated services model? It is relatively clear that 2561 guarantees of network capacity are not feasible in a highly dynamic 2562 environment. However, the specification of operating ranges may be 2563 more tractable, and may provide useful services 2564 * To applications (by giving them feedback on available network 2565 capacity), 2566 * To the transport layer (by providing dynamic rate control 2567 information), and 2568 * To the network layer (by ensuring that resources are not over 2569 allocated in the non-bottleneck portions of the network). 2571 The applicability to MANETs of the current integrated services versus 2572 differentiated services models is the subject of ongoing research 2573 [ref: http://comet.ctr.columbia.edu/insignia/, 2574 http://www.cs.ucla.edu/~terzis/mobile.html]. 2576 The networks that will be deployed remotely are truly without any 2577 fixed infrastructure. Therefore, the elements of this network will 2578 need to establish and maintain the set of services necessary to 2579 bootstrap the network. Self-configuration has relevance here in the 2580 areas of addresses allocation and management, name-to-address 2581 binding, and dynamic hierarchical organization. There is ongoing 2582 research in all of these areas that is of potential utility, ref: 2583 http://www.ietf.org/html.charters/zeroconf-charter.html 2584 http://www.ietf.org/html.charters/dhc-charter.html 2585 http://www.ietf.org/html.charters/svrloc-charter.html 2586 http://www.ietf.org/html.charters/dnsind-charter.html 2587 http://gershwin.utdallas.edu/publications]. 2589 At the transport layer, there is a need for development of new 2590 protocols or extensions to existing protocols that are capable of 2591 participating in power efficient and mobility-aware communication 2592 schemes [ref: 2593 http://www.isi.edu/workshop/public_html/wmcw97/nsf4.html ]. 2595 These enhanced transport protocols must be able to adapt to changing 2596 network conditions. This applies in particular to adaptation of the 2597 protocol's behavior in order to meet application Quality of Service 2598 requirements. Additionally, support is required for explicit 2599 signaling of and responses to changing network conditions such as 2600 link outages, congestion, and corruption trends. 2602 For the foreseeable future, some links in the RDIs will exhibit 2603 significant data rate asymmetry (on the order of 100s: 1). The 2604 effective asymmetries may be even higher, since the low-capacity link 2605 may not necessarily be dedicated to supporting the movement of data 2606 across the high capacity link. Accordingly, accommodation of 2607 significant data rate asymmetries will be required while still 2608 maintaining a high degree of power efficiency in the presence of 2609 errors. 2611 At the application layer, service location in mobile ad hoc networks 2612 is an issue that is closely related to self-configuration at the 2613 network layer, and is of current research interest, ref: 2614 http://www.ietf.org/html.charters/zeroconf-charter.html, 2615 http://www.ietf.org/html.charters/svrloc-charter.html ). 2616 This is important both in initial network startup, and also in the 2617 (potentially frequent) case that RDIs with low connectivity become 2618 partitioned, either expectedly or unexpectedly. 2620 Efficient, autonomous network management and control in RDIs is an 2621 area of interest. There is some ongoing research in the area, ref: 2622 http://www.ietf.org/html.charters/disman-charter.html, 2623 ftp://ftp.ece.orst.edu:/pub/users/singh/papers/anmp.ps.gz 2624 that is of potential interest, but this is an area in which further 2625 research is required. 2627 Finally, monitoring the health and status of mobile nodes (not 2628 necessarily just the networking components) is of significant 2629 importance in RDIs. The cost to land new elements on a planetary 2630 surface is extremely high. Therefore, we are motivated to perform 2631 preemptive maintenance and to repair, rather than replace, failed 2632 parts. The ability to perform these activities autonomously is an 2633 area where a significant amount of research is required (and sounds 2634 really fun). 2636 7.4. Summary 2638 The effects of power limitations in RDIs are significant and will be 2639 present in at least some portions of the IPN for the foreseeable 2640 future. These effects strongly suggest the likelihood of some 2641 divergence from the standard internet suite of protocols that is in 2642 current use on Earth. Whether these changes become the standard 2643 internet suite on Earth, as a result of the significant increase in 2644 the use of mobile, wireless internet nodes, remains to be determined. 2645 It is also likely that some of the RDIs that are deployed will use a 2646 model of organization that is substantially different than that 2647 employed in the Internet. It may be that the IPN treats these 2648 networks as single, distributed nodes, but further investigation is 2649 required. 2651 8. Working Conclusions 2653 With the increasing pace of space exploration, Earth will distribute 2654 large numbers of robotic vehicles, landers, and possibly even humans, 2655 to asteroids and other planets in the coming decades. Possible 2656 future missions include lander/rover/orbiter sets, sample return 2657 missions, aircraft communicating with orbiters, and outposts of 2658 humans or computers remotely operating rovers. All of these missions 2659 involve clusters of entities in relatively close proximity 2660 communicating with each other in challenging environments. These 2661 clusters, in turn, will be in intermittent contact with one another, 2662 possibly across interplanetary space. This dual-mode communications 2663 environment: relatively cheap round-trips and more constant 2664 connectivity when communicating with "local" elements coupled with 2665 the very long-delay and intermittent connectivity with non-local 2666 elements has led us to the architecture just described, with the 2667 following working conclusions. 2669 We need to use a "Pony-Express" model and bundles for interplanetary 2670 communication 2672 For communicating between IPN domains such as Earth and Mars, the 2673 standard Internet protocols fail, both at the application and 2674 transport layers, mainly due to the large delays. Intermittent 2675 connectivity and bit error rate are also significant issues, but 2676 delay cannot be mitigated by any current means. Communicating over 2677 these distances requires a fundamentally different model that does 2678 not assume that round-trip-times are cheap. To combat this we 2679 propose a "pony-express" model, where senders submit bundles 2680 describing entire transactions to the network, which then uses 2681 custody transfers to move bundles from node to node. Under this 2682 model the sender has little or no expectation of real-time delivery 2683 of the bundle, which may be stored at intermediate nodes for 2684 arbitrary periods of time. In time, the communications links 2685 connecting the various deployed internets will grow to form a stable 2686 interplanetary backbone network. 2688 We can't support most legacy applications, even with application-layer 2689 proxies 2691 SMTP (e-mail) is perhaps the only application that could possibly be 2692 tuned to work over interplanetary distances, but even SMTP has 2693 embedded timers that would have to be altered significantly before it 2694 would work. 2696 Name tuples consisting of a routing part and an administrative part are 2697 the means of reference 2699 To assist in interplanetary routing, we have introduced the notion of 2700 an IPN name tuple. These tuples, each consisting of a topology- 2701 related routing part and an administratively controlled 2702 administrative part are the analog of IP addresses in the terrestrial 2703 Internet. Just as with IP addresses, bundles carry these name tuples 2704 (source and destination) end-to-end. The routing part of an IPN name 2705 tuple serves as an identifier of the destination internet and is 2706 consulted to find the next bundle-hop destination whenever the bundle 2707 is NOT in its destination IPN region. The administrative part of an 2708 IPN name tuple serves the same function as DNS names in the 2709 terrestrial Internet, allowing symbolic names to be used in place of 2710 network-layer addresses. For the IPN region that includes Earth, and 2711 possibly others, we envision using actual DNS names as the 2712 administrative parts of the tuples. 2714 This two-part naming approach has a number of advantages. First, it 2715 explicitly allows routing information to be encoded in the IPN name 2716 tuple (in the routing part) without interfering with the 2717 administrative part. Second, since the administrative parts of name 2718 tuples are only resolved in the destination IPN region, 2719 administrative parts of IPN names do not have to be exchanged between 2720 IPN regions for the purpose of routing. Third, while all IPN nodes 2721 must be able to interpret the routing part of a name tuple, different 2722 name-to-address binding mechanisms for the administrative part can be 2723 used in the different regions. Finally, decoupling the 2724 administrative parts of the various IPN regions from one another 2725 allows autonomy of the administrative naming in different regions. 2727 IPN Nodes will be used as "impedance-matchers" between the (relatively) 2728 rich environment of local communications and the long-delay 2729 interplanetary environment 2731 We refer to the nodes that perform the store-and-forward operations 2732 on bundles simply as interplanetary nodes, and the bundle-agents on 2733 these nodes are responsible for routing bundles through the IPN. 2735 We need flexible and bandwidth-efficient security (particularly 2736 authentication and access control) 2738 For the foreseeable future, the Interplanetary Internet will be an 2739 expensive resource, and an irresistible lure to hackers. For these 2740 reasons, access to the IPN will need to be strictly controlled and 2741 IPN gateways and resources will need to authenticate commands sent to 2742 them. In many cases, data privacy will be required by scientists to 2743 ensure that they get first access to the data from their instruments, 2744 and integrity will of course be required for command sequences and 2745 some telemetry return. At the same time, the goal of the IPN is to 2746 allow scientists, researchers, and on occasion the general public, 2747 easy access to information from outside of Earth's domain. These 2748 competing goals will require a flexible security scheme. 2750 In addition to being flexible, the security mechanisms used over the 2751 deep-space links will also need to be bit efficient. Standard 2752 Diffie-Hellman exchanges cannot be used to generate traffic keys, as 2753 they require the exchange of several rounds of random information in 2754 near-real time. Something similar to secure email may be a better 2755 approach, where the sender applies the appropriate security to the 2756 payload of a bundle and then attaches its public key certificate to 2757 the bundle. This approach, while costly in terms of bandwidth, will 2758 allow the receiver to immediately authenticate/verify/decode the 2759 contents of received bundles. 2761 The long-haul transport protocol used to carry bundles in interplanetary 2762 space will be very different from TCP 2764 The transport layer that moves these bundles across interplanetary 2765 space will necessarily be very different from TCP, the predominant 2766 transport protocol in the terrestrial Internet. Since bandwidth is a 2767 precious commodity in the IPN, the LTP must be "connectionless" in 2768 that it cannot wait a round-trip to establish a connection before 2769 beginning to send data. It is quite possible that the LTP will 2770 provide partial data delivery where data with "holes" or errors is 2771 delivered to the application out of order instead of in order, as 2772 with TCP. A major challenge of the LTP is how to perform flow and 2773 congestion control without timely feedback. 2775 Deployed internets may or may not use internet protocols. 2777 For the "local" communications, between nodes on a spacecraft LAN, 2778 elements on the surface of a planet, or between elements on the 2779 surface and in orbit, for example, round-trip times are comparable to 2780 those in the terrestrial Internet. In these cases, it is feasible 2781 and indeed very reasonable to use protocols like the standard 2782 Internet Protocols (TCP/IP). Using an internet model, and perhaps 2783 even versions of the standard internet protocols themselves, will 2784 allow deployed devices to easily communicate with one another and to 2785 share a common communications infrastructure. A common, shared 2786 infrastructure will free each mission from having to design, build, 2787 test, and operate its own custom communications system, and will pay 2788 off in terms of development time, development cost, mass, 2789 interoperability (reliability), and efficiency. Thus we envision the 2790 deployments of fragments that are similar to Earth's Internet to 2791 remote locations of interest. Current advances such as Mobile IP and 2792 SCPS that extend the internet model to include mobile nodes and 2793 stressed environments will be useful in these environments. 2795 At the same time, it may be advantageous to use protocols entirely 2796 foreign to the internet suite in an "RDI." For example, it might be 2797 preferable in some instances to deploy a self-organizing sensor 2798 network in which only data sets, not individual nodes, are 2799 addressable. We believe that the naming method under consideration, 2800 where each entity name is divided into an IPN domain part and a 2801 domain-specific part, would support these types of RDIs. The 2802 mechanics of how to interface such a network with the rest of the IPN 2803 and whether/how such a network could be integrated with an RDI using 2804 internet protocols are topics of current discussion. 2806 Terrestrial advances in Mobile Ad Hoc networking are necessary but 2807 possibly not sufficient for RDIs 2809 There is a large and growing body of ongoing work in the IETF, 2810 universities, and government and private agencies to develop 2811 protocols for mobile ad hoc networks. While much of this work will 2812 be directly applicable to the remotely deployed internets, RDIs will 2813 remain a breed apart from terrestrial wireless networks. The main 2814 differences are in power and node density. Terrestrial MANETs, while 2815 they may be power-starved, still have the prospect of recharging at a 2816 wall socket; elements of an RDI have no such expectation. Secondly, 2817 the density of elements in an RDI may be low compared to a typical 2818 terrestrial MANET (although it is too soon to know that a "typical" 2819 terrestrial MANET will look like). The node density may be so low in 2820 fact that real-time communications between members of the RDI may not 2821 be possible. This will be the case if many landed elements 2822 communicate with one another via a small number of low-altitude 2823 orbiters, for example. 2825 9. Security Considerations 2827 Security is an integral concern of the Interplanetary Internet. 2828 Section 5 of this document is devoted to examining the security 2829 requirements in the IPN, some potential approaches for securing data 2830 in the IPN, and some approaches for securing the backbone 2831 infrastructure of the IPN. 2833 10. Authors' Addresses 2835 Dr. Vinton G. Cerf 2836 MCI WorldCom 2837 22001 Loudoun County Parkway 2838 Building F2, Room 4115, ATTN: Vint Cerf 2839 Ashburn, VA 20147 2840 Telephone +1 (703) 886-1690 2841 FAX +1 (703) 886-0047 2842 Email vcerf@mci.net 2844 Scott C. Burleigh 2845 Jet Propulsion Laboratory 2846 4800 Oak Grove Drive 2847 M/S: 179-206 2848 Pasadena, CA 91109-8099 2849 Telephone +1 (818) 393-3353 2850 FAX +1 (818) 354-1075 2851 Email Scott.Burleigh@jpl.nasa.gov 2853 Adrian J. Hooke 2854 Jet Propulsion Laboratory 2855 4800 Oak Grove Drive 2856 M/S: 303-400 2857 Pasadena, CA 91109-8099 2858 Telephone +1 (818) 354-3063 2859 FAX +1 (818) 393-3575 2860 Email Adrian.Hooke@jpl.nasa.gov 2862 Leigh Torgerson 2863 Jet Propulsion Laboratory 2864 4800 Oak Grove Drive 2865 M/S: T1710- 2866 Pasadena, CA 91109-8099 2867 Telephone +1 (818) 393-0695 2868 FAX +1 (818) 354-9068 2869 Email Leigh.Torgerson@jpl.nasa.gov 2871 Robert C. Durst 2872 The MITRE Corporation 2873 1820 Dolley Madison Blvd. 2874 M/S W650 2875 McLean, VA 22102 2876 Telephone +1 (703) 883-7535 2877 FAX +1 (703) 883-7142 2878 Email durst@mitre.org 2879 Dr. Keith L. Scott 2880 The MITRE Corporation 2881 1820 Dolley Madison Blvd. 2882 M/S W650 2883 McLean, VA 22102 2884 Telephone +1 (703) 883-6547 2885 FAX +1 (703) 883-7142 2886 Email kscott@mitre.org 2888 Eric J. Travis 2889 Global Science and Technology, Inc. 2890 6411 Ivy Lane 2891 Suite 300 2892 Greenbelt, MD 20770 2893 Telephone +1 (301) 474-9696 2894 FAX +1 (301) 474-5970 2895 Email travis@gst.com 2897 Howard S. Weiss 2898 SPARTA, Inc. 2899 9861 Broken Land Parkway 2900 Columbia, MD 21046 2901 Telephone +1 (410) 381-9400 x201 2902 FAX +1 (410) 381-5559 2903 Email hsw@columbia.sparta.com