idnits 2.17.1 draft-singh-rmcat-adaptive-fec-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (March 20, 2016) is 2957 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- == Outdated reference: A later version (-18) exists of draft-ietf-avtcore-rtp-circuit-breakers-14 == Outdated reference: A later version (-20) exists of draft-ietf-payload-flexible-fec-scheme-01 == Outdated reference: A later version (-02) exists of draft-ietf-rmcat-gcc-01 == Outdated reference: A later version (-13) exists of draft-ietf-rmcat-scream-cc-03 == Outdated reference: A later version (-10) exists of draft-ietf-rmcat-eval-test-03 Summary: 0 errors (**), 0 flaws (~~), 6 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 RMCAT WG V. Singh 3 Internet-Draft callstats.io 4 Intended status: Experimental M. Nagy 5 Expires: September 21, 2016 J. Ott 6 Aalto University 7 L. Eggert 8 NetApp 9 March 20, 2016 11 Congestion Control Using FEC for Conversational Media 12 draft-singh-rmcat-adaptive-fec-03 14 Abstract 16 This document describes a new mechanism for conversational multimedia 17 flows. The proposed mechanism uses Forward Error Correction (FEC) 18 encoded RTP packets (redundant packets) along side the media packets 19 to probe for available network capacity. A straightforward 20 interpretation is, the sending endpoint increases the transmission 21 rate by keeping the media rate constant but increases the amount of 22 FEC. If no losses and discards occur, the endpoint can then increase 23 the media rate. If losses occur, the redundant FEC packets help in 24 recovering the lost packets. Consequently, the endpoint can vary the 25 FEC bit rate to conservatively (by a small amount) or aggressively 26 (by a large amount) probe for available network capacity. 28 Status of This Memo 30 This Internet-Draft is submitted in full conformance with the 31 provisions of BCP 78 and BCP 79. 33 Internet-Drafts are working documents of the Internet Engineering 34 Task Force (IETF). Note that other groups may also distribute 35 working documents as Internet-Drafts. The list of current Internet- 36 Drafts is at http://datatracker.ietf.org/drafts/current/. 38 Internet-Drafts are draft documents valid for a maximum of six months 39 and may be updated, replaced, or obsoleted by other documents at any 40 time. It is inappropriate to use Internet-Drafts as reference 41 material or to cite them other than as "work in progress." 43 This Internet-Draft will expire on September 21, 2016. 45 Copyright Notice 47 Copyright (c) 2016 IETF Trust and the persons identified as the 48 document authors. All rights reserved. 50 This document is subject to BCP 78 and the IETF Trust's Legal 51 Provisions Relating to IETF Documents 52 (http://trustee.ietf.org/license-info) in effect on the date of 53 publication of this document. Please review these documents 54 carefully, as they describe your rights and restrictions with respect 55 to this document. Code Components extracted from this document must 56 include Simplified BSD License text as described in Section 4.e of 57 the Trust Legal Provisions and are provided without warranty as 58 described in the Simplified BSD License. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 63 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 64 3. Concept: FEC for Congestion Control . . . . . . . . . . . . . 4 65 3.1. States . . . . . . . . . . . . . . . . . . . . . . . . . 6 66 3.2. Framework . . . . . . . . . . . . . . . . . . . . . . . . 7 67 3.3. FEC Scheme . . . . . . . . . . . . . . . . . . . . . . . 8 68 3.4. Applicability to other RMCAT Schemes . . . . . . . . . . 9 69 4. Security Considerations . . . . . . . . . . . . . . . . . . . 9 70 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 71 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10 72 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 73 7.1. Normative References . . . . . . . . . . . . . . . . . . 10 74 7.2. Informative References . . . . . . . . . . . . . . . . . 11 75 Appendix A. Simulations . . . . . . . . . . . . . . . . . . . . 13 76 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 78 1. Introduction 80 The Real-time Transport Protocol (RTP) [RFC3550] is widely used in 81 voice telephony and video conferencing systems. Many of these 82 systems run over best-effort UDP/IP networks, and are required to 83 implement congestion to adapt the transmission rate of the RTP 84 streams to match the available network capacity, while maintaing the 85 user-experience [I-D.ietf-rmcat-cc-requirements]. The circuit 86 breakers [I-D.ietf-avtcore-rtp-circuit-breakers] describe a minimal 87 set of conditions when an RTP stream is causing severe congestion and 88 should cease transmission. Consequently, the congestion control 89 algorithm are expected to avoid triggering these conditions. 91 Conversational multimedia systems use Negative Acknowlegment (NACK), 92 Forward Error Correction (FEC), and Reference Picture Selection (RPS) 93 to protect against packet loss. These are used in addition to the 94 codec-dependent resilience methods (for e.g., full intra-refresh and 95 error-concealment). In this way, the multimedia system is anyway 96 trading off part of the transmission rate for redundancy or 97 retransmissions to reduce the effects of packet loss. An endpoint 98 often prefers using FEC in high latency networks where 99 retransmissions may arrive later than the playout time of the packet 100 (due to the size of the dejitter buffer) [Holmer13]. Therefore, the 101 endpoint needs to adapt the transmission rate to best fit the 102 changing network capacity and the amount of redundancy based on the 103 observed/expected loss rate and network latency. Figure 1 shows the 104 applicatbility of different error-resilience schemes based on the 105 end-to-end latency and the observed packet loss [Devadoss08]. 107 ^ 108 | .__________. 109 | | | 110 | | UEP/FEC | 111 l |____________|____. | 112 a | | | | 113 t | RPS | | | 114 e |_______. | | | 115 n | | | | | 116 c | | |____|_____| 117 y | NACK | | 118 | | | 119 +-------------------------------> 120 Packet loss 122 Figure 1: Applicability of Error Resilience Schemes based on the 123 network delay and observed packet loss 125 In this document, we describe the use of FEC packets not only for 126 error-resilience but also as a probing mechanism for congestion 127 control (ramping up the transmission rate). 129 2. Terminology 131 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 132 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 133 document are to be interpreted as described in BCP 14, [RFC2119] and 134 indicate requirement levels for compliant implementations. 136 The terminology defined in RTP [RFC3550], RTP Profile for Audio and 137 Video Conferences with Minimal Control [RFC3551], RTCP Extended 138 Report (XR) [RFC3611], Extended RTP Profile for RTCP-based Feedback 139 (RTP/AVPF) [RFC4585], RTP Retransmission Payload Format [RFC4588], 140 Forward Error Correction (FEC) Framework [RFC6363], and Support for 141 Reduced-Size RTCP [RFC5506] apply. 143 3. Concept: FEC for Congestion Control 145 FEC is one method for providing error-resilience, it improves 146 reliability by adding redundant data to the primary media flow, which 147 is used by received to recover packets that have been lost due to 148 congestion or bit-errors. The congestion control algorithm on the 149 other hand aims at maximizing the network path utilization, but risks 150 over-estimating the avaiable end-to-end network capacity leading to 151 congestion (and therefore losses). 153 Figure 2 shows the timeline of enabling and disabling FEC. The main 154 idea behind using FEC for congestion control is as follows: the 155 sending endpoint chooses a high FEC rate to aggressively probe for 156 available capacity and conversely chooses a low FEC rate to 157 conservatively probe for available capacity. During the ramp up, if 158 a packet is lost and the FEC packet arrives in time for decoding, the 159 receiver is be able to recover the lost packet; if no packet is lost, 160 the sender is able to increase the media encoding rate by swapping 161 out a part of the FEC rate. This method can be especially useful 162 when the transmission rate is close to the bottleneck link rate: by 163 choosing an appropriate FEC rate, the endpoint is able to probe for 164 available capacity without changing the target media rate and 165 therefore not affecting the user-experience. 167 Hence, the congestion control algorithm is always able to probe for 168 available capacity, as improved reliability compensates for possible 169 errors resulting from probing for additional capacity (i.e., increase 170 in observed latency and/or losses). 172 ^ ......... 173 | / \ / 174 t |/ \ / 175 h | +===+===+=\=+ / 176 r | | F | | \| +===+ +==/+ 177 o +===+---+ | \...........|.F.|...|./F|===+ 178 u | | | | | +===+===+---+===+---+---+ 179 g | | | | | | F | | | | | | 180 h | | | | |===+---+ | | | | | 181 p | | | | | | | | | | | | 182 u | | | | | | | | | | | | 183 t | | | | | | | | | | | | 184 | s | p | i | s | d | p | i | p | s | p | i | 185 +---+---+---+---+---+---+---+---+---+---+---+--> 186 time 188 Key: 189 +===+ Media with minimal FEC for error protection 191 +===+ 192 | F | Media with FEC for probing and error protection 193 +---+ 195 .... 196 / \ Available capacity 198 d,s,p,i: are the states: Decrease, Stay, Probe, Increase 200 Figure 2: Congestion Control enabling FEC. 202 +------------+ (B) Good conditions +-----------+ 203 | |------------------------------------>| | 204 | STEADY | | PROBE | 205 | |<------------------------------------| | 206 +------------+ Probed, but Loss recovered +-----------+ 207 /\ | | /\ | 208 | |(A) | | | 209 | |_______________________________________________| | |(C) 210 (B) | | (A) | | 211 | \/ (B) | \/ 212 +------------+ +------------+ 213 | | (A) Unstable conditions | | 214 | REDUCE |<------------------------------------| INCREASE | 215 | | | | 216 +------------+ +------------+ 218 Figure 3: State machine of a Congestion Control enabling FEC. 220 3.1. States 222 The Figure 3 illustrates the the state machine of a congestion 223 control algorithm incorporating FEC for probing. The state 224 transitions occur based on the information reported in the feedback 225 packet. In Figure 3 (A) indicates congestion, i.e., the congestion 226 control observes increasing delay and/or packet loss, or any other 227 congestion metric, and in response the congestion control reduces the 228 transmission rate. In Figure 3 (B) occurs when the congestion cues 229 report improvement in congestion metrics, and in response the 230 congestion cue increases the transmission rate. For probing using 231 FEC, the congestion control needs to map to the following 4 states: 232 STEADY, PROBE, INCREASE, and REDUCE. 234 o STEADY state: The congestion control keeps the same target media 235 rate and no additional FEC packets are generated for probing. 236 This is a transient state, after which the congestion control 237 either attempts to increase the transmission rate, or observes 238 congestion and reduces the transmission rate. 240 o REDUCE state: The congestion control reduces the transmission rate 241 based on the observed congestion cues, and generated no additional 242 FEC packets than the minimum required for error-resilience. If in 243 subsequent reports the conditions improve, the congestion control 244 can directly transition to the PROBE state. 246 o PROBE state: The congestion control observes no congestion for two 247 reporting intervals (i.e., the transmission rate should be 248 increased). The endpoint maintains the same target media bit 249 rate, and instead increases the amount of FEC packets, therby 250 increasing the transmission rate. 252 o INCREASE state: The endpoint is sending FEC packets and the 253 congestion control observes no congestion (as reported in RTCP 254 feedback), the media transmission rate is increased while 255 maintaining minimal amount of FEC for error protection. In this 256 case, the combined transmission rate (FEC+media) may remain the 257 same as in the PROBE state. If the feedback reports packet loss, 258 but some of these lost packets are recovered by the FEC packets, 259 the congestion control can keep the same media bit rate and adjust 260 the amount of FEC (compared to the previous PROBE state). If 261 congestion is observed (the target rate calculated by the 262 congestion control is much lower than the current media rate), the 263 congestion control can transition to the REDUCE state and decrease 264 the transmission rate. 266 3.2. Framework 268 The Figure 4 shows the interaction between the rate control module, 269 the RTP and the FEC module. 271 At the sender, the rate control module calculates the new bit rate. 272 If the new bit rate is higher than the previous than the previous bit 273 rate indicates to the FEC module that the congestion control intends 274 to probe. The FEC module depending on its internal state machine 275 decides to add FEC for probing or not. Thereafter it indicates to 276 the rate control module the bit rate remaining for the RTP media 277 stream, which may be less than equal to the calculated bit rate. 279 At the reciver, the FEC module reconstructs lost packets in the 280 primary stream from the packets received in the repair stream. If 281 packets are repair it generates the post-repair loss report 282 (discussed in Section 3.3) for the corresponding RTP packets. 284 At the sender, The FEC module also receives the RTCP Feedback related 285 to the primary stream and any post-repair loss report. It uses the 286 information from these RTCP reports to calculate the effectiveness of 287 FEC for congestion control and is also the basis for changing its 288 internal state. 290 + - - - - - - - - - - - - - - - - - - - - - - - -+ 291 | +--------------------------------------------+ | 292 | Media Encoder/Decoder | 293 | +--------------------------------------------+ | 294 | | 295 | +- -- -- -- -- -- -- -+ +- -- -- -- -+ | 296 | Rate Control | | RTP | 297 | | Module | | | | 298 +- -- -- -- -- -- -- -+ +- -- -- -- -+ 299 | ^ | | | 300 | | | Source 301 | | R +--------------------+ | RTP | 302 | T | | 303 | | C | | | 304 | P | | 305 | | +----------+ +----------------+ | 306 | F | FEC Code |<--->| FEC Module | 307 | | B +----------+ +----------------+ | 308 | | | | 309 | |------------------------+ | | | 310 | RTCP FB Repair | | Source 311 | | RTP | | RTP | 312 | | | 313 | +--------------------------------------------+ | 314 | RTP Processing Layer | 315 | | (Queue) | | 316 +--------------------------------------------+ 317 | | | 318 +--------------------------------------------+ 319 | | Transport Layer (UDP) | | 320 +--------------------------------------------+ 321 | | | 322 +--------------------------------------------+ 323 | | IP | | 324 +--------------------------------------------+ 325 | | 326 | Endpoint | 327 + - - - - - - - - - - - - - - - - - - - - - - - + 329 Figure 4: Interaction of Congestion Control and FEC Module. 331 3.3. FEC Scheme 333 [RFC6363] describes a framework for using Forward Error Correction 334 (FEC) codes with RTP and allows any FEC code to be used with the 335 framework. For this proposal, the FEC packets are created by XORing 336 RTP media packets, the resulting redundant RTP packets are encoded 337 using the scheme defined in [I-D.ietf-payload-flexible-fec-scheme]. 339 The endpoint MAY use a single-frame FEC (1-dimensional) or a multi- 340 frame FEC (2-dimensional) for protecting the primary RTP stream. A 341 single-frame FEC protects against a single packet loss and fails when 342 burst loss occurs. Using multi-frame FEC helps mitigate these issues 343 at the cost of higher overhead and latency in recovering lost 344 packets. [Holmer13] shows examples of using a single- and multi- 345 frame FEC. 347 The receiving endpoint may report the post-repair loss (or residual 348 loss) using either the report block defined in [RFC5725] (Run-length 349 encoding of packets repaired) or in [RFC7509] (packet count of 350 repaired packets). 352 Additionally, the receiving may report the occurance of losses and 353 discards via a run-length encoding (RLE) of lost [RFC3611] 354 (Section 4.1), which enables the sender to detect the burst loss 355 length and apply appropriate FEC scheme. 357 Packet that arrive too late to be played out by the receiver are 358 discarded by the de-jitter buffer. Typically, the de-jitter buffer 359 adjust the playout delay based on the observed frame inter-arrival 360 delay, so that packets are played out smoothly. Reporting RLE of 361 discarded packets [RFC7097] may further enable a sender to detect 362 losses that occur after packet discards. 364 3.4. Applicability to other RMCAT Schemes 366 [Open issue: The current implementation is delay based and is 367 documented in [Nagy14]. However, we would like to generalize the 368 concept and apply it to different RMCAT algorithms for e.g., Google's 369 Congestion Control algorithm [I-D.ietf-rmcat-gcc], SCReaM 370 [I-D.ietf-rmcat-scream-cc], etc.] 372 4. Security Considerations 374 The security considerations of [RFC3550], RTP/AVPF profile for rapid 375 RTCP feedback [RFC4585], circuit breaker 376 [I-D.ietf-avtcore-rtp-circuit-breakers], and Generic Forward Error 377 Correction [RFC5109] apply. 379 If non-authenticated RTCP reports are used, an on-path attacker can 380 send forged RTCP feedback packets that can disrupt the operation of 381 the underlying congestion control. Additionally, the forged packets 382 can either indicate no packet loss causing the congestion control to 383 ramp-up quickly, or indicate high packet loss or RTT causing the 384 circuit breaker to trigger. 386 5. IANA Considerations 388 There are no IANA impacts in this memo. 390 6. Acknowledgements 392 This document is based on the results published in [Nagy14]. 394 The work of Varun Singh, and Joerg Ott has been partially supported 395 by the European Institute of Innovation and Technology (EIT) ICT Labs 396 activity RCLD 11882 (2012-2014). The views expressed here are those 397 of the author(s) only. Neither the European Commission nor the 398 EITICT labs is liable for any use that may be made of the information 399 in this document. 401 Lars Eggert has received funding from the European Union's Horizon 402 2020 research and innovation programme 2014-2018 under grant 403 agreement No. 644866. This document reflects only the authors' views 404 and the European Commission is not responsible for any use that may 405 be made of the information it contains. 407 7. References 409 7.1. Normative References 411 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 412 Requirement Levels", BCP 14, RFC 2119, 413 DOI 10.17487/RFC2119, March 1997, 414 . 416 [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. 417 Jacobson, "RTP: A Transport Protocol for Real-Time 418 Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, 419 July 2003, . 421 [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and 422 Video Conferences with Minimal Control", STD 65, RFC 3551, 423 DOI 10.17487/RFC3551, July 2003, 424 . 426 [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., 427 "RTP Control Protocol Extended Reports (RTCP XR)", 428 RFC 3611, DOI 10.17487/RFC3611, November 2003, 429 . 431 [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, 432 "Extended RTP Profile for Real-time Transport Control 433 Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, 434 DOI 10.17487/RFC4585, July 2006, 435 . 437 [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size 438 Real-Time Transport Control Protocol (RTCP): Opportunities 439 and Consequences", RFC 5506, DOI 10.17487/RFC5506, April 440 2009, . 442 [I-D.ietf-avtcore-rtp-circuit-breakers] 443 Perkins, C. and V. Varun, "Multimedia Congestion Control: 444 Circuit Breakers for Unicast RTP Sessions", draft-ietf- 445 avtcore-rtp-circuit-breakers-14 (work in progress), March 446 2016. 448 [I-D.ietf-payload-flexible-fec-scheme] 449 Singh, V., Begen, A., Zanaty, M., and G. Mandyam, "RTP 450 Payload Format for Flexible Forward Error Correction 451 (FEC)", draft-ietf-payload-flexible-fec-scheme-01 (work in 452 progress), October 2015. 454 [RFC7509] Huang, R. and V. Singh, "RTP Control Protocol (RTCP) 455 Extended Report (XR) for Post-Repair Loss Count Metrics", 456 RFC 7509, DOI 10.17487/RFC7509, May 2015, 457 . 459 7.2. Informative References 461 [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. 462 Hakenberg, "RTP Retransmission Payload Format", RFC 4588, 463 DOI 10.17487/RFC4588, July 2006, 464 . 466 [RFC6363] Watson, M., Begen, A., and V. Roca, "Forward Error 467 Correction (FEC) Framework", RFC 6363, 468 DOI 10.17487/RFC6363, October 2011, 469 . 471 [I-D.ietf-rmcat-cc-requirements] 472 Jesup, R. and Z. Sarker, "Congestion Control Requirements 473 for Interactive Real-Time Media", draft-ietf-rmcat-cc- 474 requirements-09 (work in progress), December 2014. 476 [I-D.ietf-rmcat-gcc] 477 Holmer, S., Lundin, H., Carlucci, G., Cicco, L., and S. 478 Mascolo, "A Google Congestion Control Algorithm for Real- 479 Time Communication", draft-ietf-rmcat-gcc-01 (work in 480 progress), October 2015. 482 [I-D.ietf-rmcat-scream-cc] 483 Johansson, I. and Z. Sarker, "Self-Clocked Rate Adaptation 484 for Multimedia", draft-ietf-rmcat-scream-cc-03 (work in 485 progress), February 2016. 487 [I-D.ietf-rmcat-eval-test] 488 Sarker, Z., Varun, V., Zhu, X., and M. Ramalho, "Test 489 Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- 490 eval-test-03 (work in progress), March 2016. 492 [RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error 493 Correction", RFC 5109, DOI 10.17487/RFC5109, December 494 2007, . 496 [RFC5725] Begen, A., Hsu, D., and M. Lague, "Post-Repair Loss RLE 497 Report Block Type for RTP Control Protocol (RTCP) Extended 498 Reports (XRs)", RFC 5725, DOI 10.17487/RFC5725, February 499 2010, . 501 [RFC7097] Ott, J., Singh, V., Ed., and I. Curcio, "RTP Control 502 Protocol (RTCP) Extended Report (XR) for RLE of Discarded 503 Packets", RFC 7097, DOI 10.17487/RFC7097, January 2014, 504 . 506 [Nagy14] Nagy, M., Singh, V., Ott, J., and L. Eggert, "Congestion 507 Control using FEC for Conversational Multimedia 508 Communication", Proc. of 5th ACM Internation Conference on 509 Multimedia Systems (MMSys 2014) , 3 2014. 511 [Devadoss08] 512 Devadoss, J., Singh, V., Ott, J., Liu, C., Wang, Y-K., and 513 I. Curcio, "Evaluation of Error Resilience Mechanisms for 514 3G Conversational Video", Proc. of IEEE International 515 Symposium on Multimedia (ISM 2008) , 3 2014. 517 [Holmer13] 518 Holmer, S., Shemer, M., and M. Paniconi, "Handling Packet 519 Loss in WebRTC", Proc. of IEEE International Conference on 520 Image Processing (ICIP 2013) , 9 2013. 522 Appendix A. Simulations 524 This document is based on the results published in [Nagy14]. See the 525 paper for ns-2 and testbed results; more results based on the 526 scenarios listed in [I-D.ietf-rmcat-eval-test] will be published 527 shorty. 529 Authors' Addresses 531 Varun Singh 532 Nemu Dialogue Systems Oy 533 Runeberginkatu 4c A 4 534 Helsinki 00100 535 Finland 537 Email: varun.singh@iki.fi 538 URI: http://www.callstats.io/ 540 Marcin Nagy 541 Aalto University 542 School of Electrical Engineering 543 Otakaari 5 A 544 Espoo, FIN 02150 545 Finland 547 Email: marcin.nagy@aalto.fi 549 Joerg Ott 550 Aalto University 551 School of Electrical Engineering 552 Otakaari 5 A 553 Espoo, FIN 02150 554 Finland 556 Email: jo@comnet.tkk.fi 558 Lars Eggert 559 NetApp 560 Sonnenallee 1 561 Kirchheim 85551 562 Germany 564 Phone: +49 151 12055791 565 Email: lars@netapp.com 566 URI: http://eggert.org/