idnits 2.17.1 draft-westphal-icnrg-arvr-icn-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 14, 2018) is 2113 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 ICNRG Working Group C. Westphal 3 Internet-Draft Huawei 4 Intended status: Informational July 14, 2018 5 Expires: January 15, 2019 7 AR/VR and ICN 8 draft-westphal-icnrg-arvr-icn-00 10 Abstract 12 This document describes the challenges of AR/VR in ICN. 14 Status of This Memo 16 This Internet-Draft is submitted in full conformance with the 17 provisions of BCP 78 and BCP 79. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF). Note that other groups may also distribute 21 working documents as Internet-Drafts. The list of current Internet- 22 Drafts is at https://datatracker.ietf.org/drafts/current/. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 This Internet-Draft will expire on January 15, 2019. 31 Copyright Notice 33 Copyright (c) 2018 IETF Trust and the persons identified as the 34 document authors. All rights reserved. 36 This document is subject to BCP 78 and the IETF Trust's Legal 37 Provisions Relating to IETF Documents 38 (https://trustee.ietf.org/license-info) in effect on the date of 39 publication of this document. Please review these documents 40 carefully, as they describe your rights and restrictions with respect 41 to this document. Code Components extracted from this document must 42 include Simplified BSD License text as described in Section 4.e of 43 the Trust Legal Provisions and are provided without warranty as 44 described in the Simplified BSD License. 46 Table of Contents 48 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 49 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 3 50 2.1. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . 4 51 2.1.1. Office productivity, personal movie theater . . . . . 4 52 2.1.2. Retail, Museum, Real Estate, Education . . . . . . . 4 53 2.1.3. Sports . . . . . . . . . . . . . . . . . . . . . . . 4 54 2.1.4. Gaming . . . . . . . . . . . . . . . . . . . . . . . 5 55 2.1.5. Maintenance, Medical, Therapeutic . . . . . . . . . . 5 56 2.1.6. Augmented maps and directions, facial recognition, 57 teleportation . . . . . . . . . . . . . . . . . . . . 5 58 3. Information-Centric Network Architecture . . . . . . . . . . 6 59 3.1. Native Multicast Support . . . . . . . . . . . . . . . . 6 60 3.2. Caching . . . . . . . . . . . . . . . . . . . . . . . . . 7 61 3.3. Naming . . . . . . . . . . . . . . . . . . . . . . . . . 7 62 3.4. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 7 63 3.5. Other benefits? . . . . . . . . . . . . . . . . . . . . . 7 64 3.6. Security Considerations . . . . . . . . . . . . . . . . . 7 65 4. References . . . . . . . . . . . . . . . . . . . . . . . . . 8 66 4.1. Normative References . . . . . . . . . . . . . . . . . . 8 67 4.2. Informative References . . . . . . . . . . . . . . . . . 8 68 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 8 70 1. Introduction 72 Augmented Reality and Virtual Reality are becoming common place. 73 Facebook and YouTube have deployed support for some immersive videos, 74 including 360 videos. Many companies, including the aforementioned 75 Facebook, Google, but also Microsoft and others, are offering devices 76 to view virtual reality, ranging from simple mechanical additions to 77 a smart phone, such as Google Cardboard to full fledged dedicated 78 devices, such as the Oculus Rift. 80 Current networks however, are still struggling to deliver high 81 quality video streams. 5G Networks will have to address the 82 challenges introduced by the new applications delivering augmented 83 reality and virtual reality services. However, it is unclear that 84 without architectural support, it will be possible to deploy such 85 applications. 87 Most surveys of augmented reality systems (say, [van2010survey]) 88 ignore the potential underlying network issues. We attempt to 89 present some of these issues in this paper. We also intend to 90 explain how an Information-Centric Network architecture is beneficial 91 for AR/VR. Information-Centric Networking has been considered for 92 enhancing content delivery by adding features that are lacking in an 93 IP network, such as caching, or the requesting and routing of content 94 at the network layer by its name rather than a host's address. 96 2. Definitions 98 We provide definitions of virtual and augmented reality (see for 99 instance [van2010survey]): 101 Augmented Reality: an AR system inserts a virtual layer over the 102 user's perception of the real objects, which combines both real and 103 virtual objects in such a way that they function in relation to each 104 other, with synchronicity and the proper depth of perception in three 105 dimensions. 107 Virtual Reality: a VR system places the user in a synthetic, virtual 108 environment with a coherent set of rules and interactions with this 109 environment and the other participants in this environment. 111 Virtual reality is immersive and potentially isolating from the real 112 world, while augmented reality inserts extra information onto the 113 real world. 115 For the purpose of this article, we restrict ourselves to the audio- 116 visual perception of the environment (even though haptic systems may 117 be used) as a first step. Many of the applications of augmented and 118 virtual reality similarly start with eyesight and sounds only. 120 Most of the AR/VR we consider here focuses on head-mounted displays, 121 such as Oculus Rift or Google Cardboard. 123 There are obvious observations derived from these descriptions of 124 virtual and augmented reality. One is that virtual reality only 125 really needs a consistent set of rules for the user to be immersed 126 into it. It could theoretically work on a different time scale, say 127 where the reaction to motion is slowler than in the real world. 128 Further, VR only needs to be self-consistent, and does not require 129 synchronization with the real world. 131 As such, there are several levels of complexity along a reality- 132 virtuality continuum. For the purpose of the networking 133 infrastructure, we will roughly label them as 360/immersive video, 134 where user is streaming a video stream with a specific viewing angle 135 and direction; virtual reality environment, where the user is 136 immersed in a virtual world and has agency (say, decide of the 137 direction of the motion, in addition to deciding of the direction of 138 her viewing angle); and augmented reality where the users' view is 139 overlayed on top of the actual real view of the user. 141 The last application requires identifying the environment, generating 142 and fetching the virtual artifacts, layering these on top of the 143 reality in the vision of the user, in real time and in 144 synchronization with the space dimensionality and the perception of 145 the user, and with the motion of the user's field of vision. Such 146 processing is very computationally heavy and would require a 147 dedicated infrastructure to be placed within the network provider's 148 domain. 150 2.1. Use Cases 152 For AR/VR specifically, there is a range of scenarios with specific 153 requirements. We denote a few below, but make no claim of 154 exhaustivity: there are plenty of other applications. 156 2.1.1. Office productivity, personal movie theater 158 This is a very simple, canonical use case, where the headmounted 159 device is only a display for the workstation of the user. This has 160 little networking requirements, as all is collocated and could even 161 be wired. For this reason, it is one of the low hanging fruits in 162 this space. The main issue is of display quality, as the user spends 163 long hour looking at a screen, with a resolution, a depth of 164 perception, and a reactivity of the headmounted display that should 165 be comfortable for the user. 167 2.1.2. Retail, Museum, Real Estate, Education 169 The application recreates the experience of being in a specific area, 170 such as a home for sale, a classroom or a specific room in a museum. 171 This is an application where the files may be stored locally, as the 172 point is to replicate an existing point of reference, and this can be 173 processed ahead of time. 175 Issues become then how to move the virtual environment onto the 176 display. Can it be prefetched ahead of time; can it be distributed 177 and cached locally near the device; can it be rendered in the device? 179 2.1.3. Sports 181 This attempts to put the user in the middle of a different real 182 environment, as in the previous case, but adds to it several 183 dimensions: that of real time, as the experience must be synchronized 184 with a live event; that of scale, as many users may be attempting to 185 participate in the experience simultanuously. 187 These new dimensions add some corresponding requirements, namely how 188 to distribute live content in a timely manner that still corresponds 189 to the potentially unique viewpoint of each of the users; how to 190 scale this distribution to a large number of concurrent experiences. 191 The viewpoint in this context also may impose different requirements, 192 if it is that of a player in a basketball game, or that of a 193 spectator in the arena. For instance, in the former case, the 194 position of the viewpoint is well defined by that of the player, 195 while in the latter, it may wildly vary. 197 2.1.4. Gaming 199 Many games place the user into a virtual environment, from Minecraft 200 to multi-user shooter game. Platform such as Unity 3D allow creation 201 of virtual worlds. Unlike the previous use case, there are now 202 interactions in between the different participants in the virtual 203 environment. This require communication of these interactions in 204 between peers, and not just from a server onto the device. There are 205 issues of consistency across users and synchronization issues. 207 2.1.5. Maintenance, Medical, Therapeutic 209 There exist a few commercial products where the AR is used to overlay 210 instructions on top of some equipment so as to assist the agent in 211 performing maintenance. Surgical assistance may fall in this 212 category as well. 214 The advantage of a specific task is that it facilitates the pattern 215 recognition and the back-end processing as it is narrowed down. 216 However, the requirements to overlay the augmented layer on top of 217 the existing reality puts stringent synchronization and round-trip 218 time requirements, both on the display and on the sensors capturing 219 the motion and position. 221 2.1.6. Augmented maps and directions, facial recognition, teleportation 223 The more general scenario of augmented reality does not focus on a 224 specific, well defined application, but absorbs the environment as 225 observed by the user (or the user's car or the pilot's plane, if the 226 display is overlayed on a windshield) and annotates this environment, 227 for instance to specify directions. This includes recognizing 228 patterns and potentially people with the help of little context 229 beyond the position of the user. Another main target of AR is 230 telepresence, where a person in a remote location could be made 231 present, as if in another location, say with others in the same 232 conference room. Teleportation plus the display of the workstation 233 of a user (as in the first scenario above) may allow remote 234 collaboration on entreprise tasks. 236 3. Information-Centric Network Architecture 238 We now turn our attention to the potential benefits that Information- 239 Centric Networks can bring to the realization of AR/VR. 241 The abstractions offered by an ICN architecture are promising for 242 video delivery. RFC7933 [RFC7933] for instance highlights the 243 challenges and potential of ICN for adaptive rate streaming. As VR 244 in particular may encompass a video component, it is natural to 245 consider ICN for AR/VR. 247 There is a lot of existing work on ICN (say, caching or traffic 248 engineering [su2013benefit]) which could be applied to satisfy the 249 QoS requirements of the AR/VR applications, when possible. 251 3.1. Native Multicast Support 253 One of the key benefits from ICN is the native support for multicast. 254 For instance, [macedonia1995exploiting] quotes: "if the systems are 255 to be geographically dispersed, then highspeed, multicast 256 communication is required." Similarly, [frecon1998dive] states that: 257 "Scalability is achieved by making extensive use of multicast 258 techniques and by partitioning the virtual universe into smaller 259 regions." 261 In the sport use case, many users will be participating in the same 262 scene. They will have potentially distinct point of views, as each 263 may look into one specific direction. However, each of these views 264 may share some overlap with the others, as there is a natural focus 265 point within the event (say, the ball in a basketball game). 267 This means that many of the users will request some common data and 268 native multicast significantly reduces the bandwidth and in the case 269 of ICN, without extra signaling. 271 Further, the multicast tree should be adhoc, and dynamic to 272 efficiently support AR/VR. Back in 1995, [funkhouser1995ring] 273 attempted to identify the visual interactions in between entities 274 representing users in a VE so as to "reduce the number of messages 275 required to maintain consistent state among many workstations 276 distributed across a wide-area network. When an entity changes 277 state, update messages are sent only to workstations with entities 278 that can potentially perceive the change i.e., ones to which the 279 update is visible.}" [funkhouser1995ring] was able to reduce the 280 number of messages processed by client workstations by a factor of 281 40. 283 It is unclear that ICN can assist in identifying which workstations 284 (or nowadays, which users) may perceive the status update of another 285 user (but naming the data at the network layer may help). 286 Nonetheless, the multicast tree to reach the set of clients that 287 would require an update is dynamically modified and the support for 288 multicast in ICN definitly supports this dynamic behavior. 290 3.2. Caching 292 The caching feature of ICN allows prefetching of data near the edge 293 some of the more static use cases; further, in the case of multiple 294 users sharing a VE, the caching allows to perform the content 295 placement phase for some users at the same time as the content 296 distribution phase of others, thereby reducing bandwidth consumption. 298 Caching is a prominent feature in an AR system: the data must be 299 nearby to reduce the round-trip time to access the data. Further, AR 300 data has a strong local component and therefore caching allows to 301 keep the information within the domain where it will be accessed. 303 ICN naturally supports caching, and provides content-based security 304 to allow any edge cache to hold and deliver the data. 306 3.3. Naming 308 Since only a partial Field of View is accessed from the whole 309 spherical view at any point in time, tiling the spherical view into 310 smaller areas and requesting the tiles that are viewed would reduce 311 the bandwidth consumption of AR/VR systems. This raises the obvious 312 question of naming semantics for tiles. New naming schemes that 313 allow for tiling should be devised. 315 3.4. Privacy 317 By enabling caching at the edge, ICN enhances the privacy of the 318 users. The user may access data locally, and thereby will not reveal 319 information beyond the network edge. 321 3.5. Other benefits? 323 TBD: any other aspects to consider. 325 3.6. Security Considerations 327 TODO. 329 4. References 331 4.1. Normative References 333 [RFC7933] "Adaptive Video Streaming over Information-Centric 334 Networking (ICN)", RFC 7933, august 2016. 336 4.2. Informative References 338 [frecon1998dive] 339 and , "DIVE: A scaleable network architecture for 340 distributed virtual environments", Distributed Systems 341 Engineering vol 5, number 3 , 1998. 343 [funkhouser1995ring] 344 and , "RING: a client-server system for multi-user virtual 345 environments", ACM symposium on Interactive 3D graphics , 346 1995. 348 [macedonia1995exploiting] 349 and , "Exploiting reality with multicast groups: a network 350 architecture for large-scale virtual environments", 351 Virtual Reality Annual International Symposium , 1995. 353 [su2013benefit] 354 and , "On the Benefit of Information Centric Networks for 355 Traffic Engineering", IEEE ICC , 2014. 357 [van2010survey] 358 and , "A survey of augmented reality technologies, 359 applications and limitations", International Journal of 360 Virtual Reality , 2010. 362 Author's Address 364 Cedric Westphal 365 Huawei 367 Email: Cedric.Westphal@huawei.com