idnits 2.17.1 draft-shpiner-multi-path-synchronization-00.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There is 1 instance of too long lines in the document, the longest one being 1 character in excess of 72. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (October 15, 2012) is 4201 days in the past. Is this intentional? Checking references for intended status: Experimental ---------------------------------------------------------------------------- No issues found here. Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 1 Network Working Group A. Shpiner 2 Internet Draft Technion - Israel Institute of Technology 3 Intended status: Experimental R. Tse 4 Expires: April 2013 C. Schelp 5 PMC-Sierra 6 T. Mizrahi 7 Marvell 8 October 15, 2012 10 Multi-Path Time Synchronization 11 draft-shpiner-multi-path-synchronization-00.txt 13 Abstract 15 Clock synchronization protocols are very widely used in IP-based 16 networks. The Network Time Protocol (NTP) has been commonly deployed 17 for many years, and the last few years have seen an increasingly 18 rapid deployment of the Precision Time Protocol (PTP). As time- 19 sensitive applications evolve, clock accuracy requirements are 20 becoming increasingly stringent, requiring the time synchronization 21 protocols to provide high accuracy. Slave Diversity is a recently 22 introduced approach, where the master and slave clocks (also known as 23 server and client) are connected through multiple network paths, and 24 the slave combines the information received through all paths to 25 obtain a higher clock accuracy compared to the conventional one-path 26 approach. This document describes a multi-path approach to PTP and 27 NTP over IP netwokrs, allowing the protocols to run concurrently over 28 multiple communication paths between the master and slave clocks. The 29 multi-path approach can significantly contribute to clock accuracy, 30 security and fault protection. The Multi-Path Precision Time Protocol 31 (MPPTP) and Multi-Path Network Time Protocol (MPNTP) define an 32 additional layer that extends the existing PTP and NTP without the 33 need to modify these protocols. MPPTP and MPNTP also allow backward 34 compatibility with nodes that do not support the multi-path 35 extension. 37 Status of this Memo 39 This Internet-Draft is submitted to IETF in full conformance with the 40 provisions of BCP 78 and BCP 79. 42 Internet-Drafts are working documents of the Internet Engineering 43 Task Force (IETF), its areas, and its working groups. Note that 44 other groups may also distribute working documents as Internet- 45 Drafts. 47 Internet-Drafts are draft documents valid for a maximum of six months 48 and may be updated, replaced, or obsoleted by other documents at any 49 time. It is inappropriate to use Internet-Drafts as reference 50 material or to cite them other than as "work in progress." 52 The list of current Internet-Drafts can be accessed at 53 http://www.ietf.org/ietf/1id-abstracts.txt. 55 The list of Internet-Draft Shadow Directories can be accessed at 56 http://www.ietf.org/shadow.html. 58 This Internet-Draft will expire on April 15, 2013. 60 Copyright Notice 62 Copyright (c) 2012 IETF Trust and the persons identified as the 63 document authors. All rights reserved. 65 This document is subject to BCP 78 and the IETF Trust's Legal 66 Provisions Relating to IETF Documents 67 (http://trustee.ietf.org/license-info) in effect on the date of 68 publication of this document. Please review these documents 69 carefully, as they describe your rights and restrictions with respect 70 to this document. Code Components extracted from this document must 71 include Simplified BSD License text as described in Section 4.e of 72 the Trust Legal Provisions and are provided without warranty as 73 described in the Simplified BSD License. 75 Table of Contents 77 1. Introduction ................................................. 3 78 2. Conventions Used in this Document ............................ 5 79 2.1. Abbreviations ........................................... 5 80 2.2. Terminology ............................................. 5 81 3. Multiple Paths in IP Networks ................................ 5 82 3.1. Load Balancing .......................................... 5 83 3.2. Using Multiple Paths Concurrently ....................... 5 84 3.3. Two-Way Paths ........................................... 6 85 4. Solution Overview ............................................ 6 86 4.1. Path Configuration and Identification ................... 6 87 4.2. Combining ............................................... 7 88 5. Multi-Path Time Synchronization Protocols over IP Networks ... 7 89 5.1. One-Way Multi-Path Synchronization ...................... 8 90 5.1.1. One-Way MPPTP Synchronization Message Exchange ..... 8 91 5.1.2. One-Way MPNTP Synchronization Message Exchange ..... 9 92 5.2. Two-Way Multi-Path Synchronization ...................... 9 93 5.2.1. Two-Way MPPTP Synchronization Message Exchange .... 10 94 5.2.2. Two-Way MPNTP Synchronization Message Exchange .... 10 95 5.3. Using Traceroute for Path Discovery .................... 11 96 6. Combining Algorithm ......................................... 11 97 6.1. Averaging .............................................. 11 98 6.2. Switching / Dynamic Algorithm .......................... 12 99 6.3. NTP-like Filtering-Clustering-Combining Algorithm ...... 12 100 7. Security Considerations ..................................... 13 101 8. IANA Considerations ......................................... 13 102 9. Acknowledgments ............................................. 13 103 10. References ................................................. 13 104 10.1. Normative References .................................. 13 105 10.2. Informative References ................................ 13 107 1. Introduction 109 The two most common time synchronization protocols in IP networks are 110 the Network Time Protocol [NTP], and the Precision Time Protocol 111 (PTP), defined in the IEEE 1588 standard [IEEE1588]. 112 The accuracy of the time synchronization protocols directly depends 113 on the stability and the symmetry of propagation delays on both 114 directions between the master and slave clocks. Depending on the 115 nature of the underlying network, time synchronization protocol 116 packets can be subject to variable network latency or path asymmetry 117 (e.g. [ASSYMETRY], [ASSYMETRY2]). As time sensitive applications 118 evolve, accuracy requirements are becoming increasingly stringent. 120 Using a single network path in a clock synchronization protocol 121 closely ties the slave clock accuracy to the behavior of the specific 122 path, which may suffer from temporal congestion, faults or malicious 123 attacks. Relying on multiple clock servers as in NTP solves these 124 problems, but requires active maintenance of multiple accurate 125 sources in the network, which is not always possible. The usage of 126 Transparent Clocks (TC) in PTP solves the congestion problem by 127 eliminating the queueing time from the delay calculations, but 128 requires the intermediate routers and switches to support the TC 129 functionality, which is not always the case. 131 ____ 132 ______/ \_____ 133 ___/ \____ 134 ____/ \ 135 ____ / path 1 / ___ 136 / \ / ________________________ \ / \ 137 /Master\____\___/ \____\________/Slave\ 138 \Clock / / \________ _______________/ \ \Clock/ 139 \____/ \ / \__ / 140 \____ path 2 __/ 141 \_______ ______/ 142 \_________/ 144 Figure 1 Multi-Path Connection 146 Since master and slave clocks are often connected through more than 147 one path in the network, as shown in Figure 1, [SLAVEDIV] suggested 148 that a time synchronization protocol can be run over multiple paths, 149 providing several advantages. First, it can significantly increase 150 the clock accuracy as shown in [SLAVEDIV]. Second, this approach 151 provides additional security, allowing to mitigate man-in-the-middle 152 attacks against the time synchronization protocol [DELAY-ATT]. Third, 153 using multiple paths concurrently provides an inherent failure 154 protection mechanism with a negligible recovery time. 156 This document introduces Multi-Path PTP (MPPTP) and Multi-Path NTP 157 (MPNTP), respectively. These extensions are defined at the network 158 layer, and do not require any changes in the PTP or in the NTP 159 protocols. 161 MPPTP and MPNTP are defined over IP networks. As IP networks 162 typically combine ECMP routing, this property is leveraged for the 163 multiple paths used in MPPTP and MPNTP. The key property of the 164 multi-path extensions is that clocks in the network can use more than 165 one IP address. Each {master IP, slave IP} address pair defines a 166 path. Depending on the network topology and configuration, the IP 167 combination pairs can form multiple diverse paths used by the multi- 168 path synchronization protocols. 170 This document introduces two variants for each of the two multi-path 171 protocols; a variant that requires all nodes to support the multi- 172 path protocol, referred to as the two-way variant, and a backward 173 compatible variant that allows a multi-path clock to connect to a 174 conventional single-path clock, referred to as the one-way variant. 176 2. Conventions Used in this Document 178 2.1. Abbreviations 180 ECMP Equal Cost Multiple Path 182 LAN Local Area Network 184 MPNTP Multi-Path Network Time Protocol 186 MPPTP Multi-Path Precision Time Protocol 188 NTP Network Time Protocol 190 PTP Precision Time Protocol 192 2.2. Terminology 194 In the NTP terminology, a time synchronization protocol is run 195 between a client and a server, while PTP uses the terms master and 196 slave. Throughout this document, the sections that refer to both PTP 197 and NTP generically use the terms master and slave. 199 3. Multiple Paths in IP Networks 201 3.1. Load Balancing 203 Traffic sent across IP networks is often load balanced across 204 multiple paths. The load balancing decisions are typically based on 205 packet header fields: source and destination addresses, Layer 4 206 ports, the Flow Label field in IPv6, etc. 207 Three common load balancing criteria are per-destination, per-flow 208 and per-packet. The per-destination load balancers take a load 209 balancing decision based on the destination IP address. Per-flow load 210 balancers use various fields in the packet header, e.g., IP addresses 211 and Layer 4 ports, for the load balancing decision. Per-packet load 212 balancers use flow-blind techniques such as round-robin without 213 basing the choice on the packet content. 215 3.2. Using Multiple Paths Concurrently 217 To utilize the diverse paths that traverse per-destination load- 218 balancers or per-flow load-balancers, the packet transmitter can vary 219 the IP addresses in the packet header. The analysis in [PARIS2] shows 220 that a significant majority of the flows on the internet traverse 221 per-destination or per-flow load-balancing. It presents statistics 222 that 72% of the flows traverse per-destination load balancing and 39% 223 of the flows traverse per-flow load-balancing, while only a 224 negligible part of the flows traverse per-packet load balancing. 225 These statistics show that the vast majority of the traffic on the 226 internet is load balanced based on packet header fields. 228 The approaches in this draft are based on varying the source and 229 destination IP addresses in the packet header. Possible extensions 230 have been considered that also vary the UDP ports. However some of 231 the existing implementations of PTP and NTP use fixed UDP port values 232 in both the source and destination UDP port fields, and thus do not 233 allow this approach. 235 3.3. Two-Way Paths 237 A key property of IP networks is that packets forwarded from A to B 238 do not necessarily traverse the same path as packets from B to A. 239 Thus, we define a two-way path for a master-slave connection as a 240 pair of one-way paths: the first from master to slave and the second 241 from slave to master. 243 In a locally administered network, a traffic engineering approach can 244 be used to verify that time synchronization traffic is always 245 forwarded through bidirectional two-way paths, i.e., that each two 246 way path uses the same route on the forward and reverse directions. 247 However, in the general case two-way paths do not necessarily use the 248 same path for the forward and reverse directions. 250 4. Solution Overview 252 The multi-path time synchronization protocols we present are 253 comprised of two building blocks; one is the path configuration and 254 identification, and the other is the algorithm used by the slave to 255 combine the information received from the various paths. 257 4.1. Path Configuration and Identification 259 The master and slave clocks must be able to determine the path of 260 transmitted protocol packets, and to identify the path of incoming 261 protocol packets. A path is determined by a {master IP, slave IP} 262 address pair. The synchronization protocol message exchange is run 263 independently through each path. 265 Each IP address pair defines a two-way path, and thus allows the 266 clocks to bind a transmitted packet to a specific path, or to 267 identify the path of an incoming packet. 269 In locally administered IP networks, the routing tables across the 270 network can be configured with multiple traffic engineered paths 271 between the pair of clocks. By carefully configuring the routers in 272 such networks it is possible to create diverse paths for each of the 273 IP address pairs between two clocks in the network. However, in 274 public and provider networks the load balancing behavior is hidden 275 from the end users. In this case the actual number of paths may be 276 less than the number of IP address pairs, since some of the address 277 pairs may share common paths. 279 4.2. Combining 281 Various methods can be used for combining the time information 282 received from the different paths. This document surveys several 283 combining methods in Section 6 . The output of the combining algorithm 284 is the accurate time offset. 286 5. Multi-Path Time Synchronization Protocols over IP Networks 288 This section presents two variants of MPPTP and MPNTP; one-way multi- 289 path time synchronization and two-way multi-path time 290 synchronization. In the first variant the multi-path protocol is run 291 only by the slave, and the master is not aware of its usage. In the 292 second variant all clocks must support the multi-path protocol. 294 The two-way protocol provides higher path diversity by using multiple 295 IP addresses at both ends, the master and slave, while the one-way 296 protocol only uses multiple addresses at the slave. On the other 297 hand, the two-way protocol can only be deployed in networks where all 298 the clocks support this protocol, while the one-way protocol can be 299 used in hybrid networks. 301 Multi-path time synchronization, in both variants, requires clocks to 302 use multiple IP addresses. This approach introduces a tradeoff; using 303 a large number of IP addresses allows a large number of diverse 304 paths, providing the advantages of slave diversity discussed in 305 Section 1. On the other hand, a large number of IP addresses is more 306 costly, requires the network topology to be more redundant and yields 307 a management overhead. 309 The descriptions in this section refer to the end-to-end scheme of 310 PTP, but are similarly applicable to the peer-to-peer scheme. The 311 MPNTP protocol described in this document refers to the NTP client- 312 server mode, although the concepts described here can be extended to 313 include the symmetric variant as well. 315 Multi-path synchronization protocols by nature require protocol 316 messages to be sent as unicast. Specifically in PTP, the following 317 messages must be sent as unicast in MPPTP: Sync, Delay_Req, 318 Delay_Resp, PDelay_Req, PDelay_Resp, Follow_Up, and 319 PDelay_Resp_Follow_Up. Note that [IEEE1588] allows these messages to 320 be sent either as multicast or as unicast. 322 5.1. One-Way Multi-Path Synchronization 324 In the one-way approach, only the slave is aware of the fact that 325 multiple paths are used, while the master is agnostic to the usage of 326 multiple paths. This approach allows a hybrid network, where some of 327 the clocks are multi-path clocks, and others are conventional one- 328 path clocks. A one-way multi-path clock presents itself to the 329 network as N independent clocks, using N IP addresses, and N clock 330 identity values. Thus, the usage of multiple slave identities by a 331 slave clock is transparent from the master's point of view, such that 332 it treats each of the identities as a separate slave clock. 334 5.1.1. One-Way MPPTP Synchronization Message Exchange 336 The one-way MPPTP message exchange procedure is as follows. 338 o Each one-way MPPTP clock has a fixed set of N IP addresses and N 339 corresponding clockIdentities . One of the IP addresses and 340 clockIdentity values are defined as the clock primary identity. 342 o The BMC algorithm determines the master. 344 o Every one-way MPPTP port that is not in the 'slave' state (i.e., 345 either a master or a clock that has just joined the network) 346 periodically sends Announce messages using its primary identity. 348 o A one-way MPPTP clock that is in the 'slave' state periodically 349 transmits a set of N Announce messages using its N identities, 350 while a clock in the 'master' state only transmits Announce 351 messages using its primary identity. 353 o The master periodically sends unicast Sync messages from its 354 primary identity address to each of the slaves identified by the 355 sourcePortIdentity and IP address. 357 o The slave, upon receiving a Sync message, identifies its path 358 according to the destination IP address. In response to the Sync 359 message the slave sends a Delay_Req unicast message to the primary 360 identity of the master. This message is sent using the slave 361 identity corresponding to the path the Sync was received through. 363 o The master, in response to a Delay_Req message from the slave, 364 responds with a Delay_Resp message using the IP address and 365 sourcePortIdentity from the Delay_Req message. 367 o Upon receiving the Delay_Resp message, the slave identifies the 368 path using the destination IP address and the 369 requestingPortIdentity. The slave can then compute the 370 corresponding path delay and the offset from the master. 372 5.1.2. One-Way MPNTP Synchronization Message Exchange 374 The one-way MPNTP message exchange procedure is as follows. 376 o A one-way MPNTP client has N separate identities, i.e., N IP 377 addresses, and N corresponding Reference IDs. 379 o A one-way MPNTP client initiates the NTP protocol with an NTP 380 server N times, using each of its N identities. 382 o The NTP protocol is maintained between the server and each of the 383 N client identities. 385 o The client sends NTP messages to the master using each of its N 386 identities. 388 o The server responds to the client's NTP messages using the IP 389 address from the received NTP packet. 391 o The client, upon receiving an NTP packet, uses the IP destination 392 address to identify the path it came through, and uses the time 393 information accordingly. 395 5.2. Two-Way Multi-Path Synchronization 397 In two-way multi-path synchronization each clock has N IP addresses. 398 Time synchronization messages are exchanged between each combination 399 of {master IP, slave IP} addresses, allowing multiple paths between 400 the master and slave. Note that the actual number of paths between 401 the master and slave may be less than the number of {master, slave} 402 IP address pairs. 404 Once the multiple two-way connections are established, a separate 405 synchronization protocol exchange instance is run through each of 406 them. 408 5.2.1. Two-Way MPPTP Synchronization Message Exchange 410 The two-way MPPTP message exchange procedure is as follows. 412 o Every clock periodically sends a set of N Announce messages, using 413 its N addresses as the source IP address. The sourcePortIdentity 414 field in the PTP header remains the same for all PTP messages of a 415 given clock. 417 o The BMC algorithm determines the master. Clocks are identified by 418 the sourcePortIdentity and not by the IP address. 420 o Each clock learns the multiple IP addresses of other clocks from 421 the source IP addresses of the Announce packets it receives. 423 o The master periodically sends unicast Sync messages from each of 424 its N_m IP addresses to each of the slave's N_s IP addresses. 426 o The slave, upon receiving a Sync message, identifies its path 427 according to the {source, destination} IP addresses. In response 428 to the Sync message the slave sends a Delay_Req unicast message, 429 swapping the source and destination IP addresses from the Sync 430 message. 432 o The master, in response to a Delay_Req message from the slave, 433 responds with a Delay_Resp message using the sourcePortIdentity 434 from the Delay_Req message, and swapping the IP addresses from the 435 Delay_Req. 437 o Upon receiving the Delay_Resp message, the slave identifies the 438 path using the {source, destination} IP address pair. The slave 439 can then compute the corresponding path delay and the offset from 440 the master. 442 o The PTP protocol messages are sent through each of the N_m*N_s 443 paths, and the slave combines the information from all these 444 paths. 446 5.2.2. Two-Way MPNTP Synchronization Message Exchange 448 The MPNTP message exchange procedure is as follows. 450 o Each NTP clock has a set of N IP addresses. The assumption is that 451 the server information, including its multiple IP addresses is 452 known to the NTP clients. 454 o The MPNTP client initiates the N_s*N_c instances of the protocol, 455 one for each {server IP, client IP} pair, allowing the client to 456 combine the information from the N_s*N_c paths. 457 (N_s and N_c indicate the number of IP addresses used by the 458 server and client, respectively.) 460 o The client sends NTP messages to the master using each of the 461 source-destination address combinations. 463 o The server responds to the client's NTP messages using the IP 464 address combination from the received NTP packet. 466 o Using the {source, destination} IP address pair in the received 467 packets, the client identifies the path, and performs its 468 computations for each of the paths accordingly. 470 5.3. Using Traceroute for Path Discovery 472 The protocols presented above use multiple IP addresses in a single 473 clock to create multiple paths. However, although each two-way path 474 is defined by a different {master, slave} address pair, some of the 475 IP address pairs may traverse exactly the same network path, making 476 them redundant. Traceroute-based path discovery can be used for 477 filtering only the IP addresses that obtain diverse paths. 'Paris 478 Traceroute' [PARIS] and 'TraceFlow' [TRACEFLOW] are examples of tools 479 that discover the paths between two points in the network. 481 The Traceroute-based filtering can be implemented by both master and 482 slave nodes, or it can be restricted to run only on slave nodes to 483 reduce the overhead on the master. 485 6. Combining Algorithm 487 Previous sections discussed the methods of creating the multiple 488 paths and obtaining the time information required by the slave 489 algorithm. This section discusses the algorithm used to combine this 490 information into a single accurate time estimate. Note that the 491 choice of the combining algorithm is local to the slave, and does not 492 affect the interoperability of the protocol. 493 Several combining methods are examined next. 495 6.1. Averaging 497 In the first method the slave performs an autonomous time computation 498 for each of the master-slave paths, and obtains the combined time by 499 simply averaging the separate instances. This method can be further 500 enhanced by adding weights to each of the paths. For example, a 501 reasonable weighting choice is to use an inverse of the round-trip 502 delay between the peers. Another option is to use the inverse of the 503 path delay variance. , which is approximately the maximum likelihood 504 estimator under certain assumptions [WEIGHT-MEAN]. 506 6.2. Switching / Dynamic Algorithm 508 The switching and dynamic algorithms are presented in [SLAVEDIV]. The 509 switching algorithm periodically chooses a primary path, and performs 510 all time computations based on the protocol packets received through 511 the primary path. The primary path is defined as the path with the 512 minimal distance between the sampled delay and the average delay. The 513 dynamic algorithm dynamically chooses between the result of the 514 switching algorithm and the averaging. 515 6.3. NTP-like Filtering-Clustering-Combining Algorithm 517 NTP ([NTP], [NTP2]) provides an efficient algorithm of combining 518 offset samples from multiple peers. The same approach can be used in 519 MPPTP and MPNTP. 521 In the MPNTP, the selection and combining algorithms treat the offset 522 samples from multiple paths as NTP treats samples from distinct 523 peers. The rest of the selection and combining algorithms, as well as 524 clock control logic is the same as in conventional NTP. In MPPTP, a 525 similar approach to NTP can be adopted. 527 The combining algorithm [NTP3] contains three steps: filtering, 528 selection and clustering. 530 In the filtering step, the best of the last n (usually n=8) samples 531 of each peer is chosen. The choice criterion is the combination of a 532 round trip delay estimate of the sample and the distance from the 533 average offset of all n samples of a peer. 535 In the selection step the peers are divided into two groups: true- 536 chimers and false tickers. 538 The clustering step chooses a subset of the true-chimers, whose peer 539 jitter (the variance of peer offset samples) is smaller than the 540 total select jitter of all selected peer offsets (the variance of the 541 best offset of the selected peers). 543 The offset samples that passed through the three steps are combined 544 by a weighted average into a single offset estimate. Detailed 545 explanations are provided in [NTP2],[NTP3]. 547 7. Security Considerations 549 The security aspects of time synchronization protocols are discussed 550 in detail in [TICTOCSEC]. The methods describe in this document 551 propose to run a time synchronization protocol through redundant 552 paths, and thus allow to detect and mitigate man-in-the-middle 553 attacks, as described in [DELAY-ATT]. 555 8. IANA Considerations 557 There are no IANA actions required by this document. 559 RFC Editor: please delete this section before publication. 561 9. Acknowledgments 563 This document was prepared using 2-Word-v2.0.template.dot. 565 10. References 567 10.1. Normative References 569 [IEEE1588] IEEE Instrumentation and Measurement Society, "IEEE 570 Standard for a Precision Clock Synchronization 571 Protocol for Networked Measurement and Control 572 Systems", IEEE Std 1588, 2008. 574 [NTP] D. Mills, J. Martin, J. Burbank, W. Kasch, "Network 575 Time Protocol Version 4: Protocol and Algorithms 576 Specification", IETF, RFC 5905, 2010. 578 10.2. Informative References 580 [ASSYMETRY] Yihua He and Michalis Faloutsos and Srikanth 581 Krishnamurthy and Bradley Huffaker, "On routing 582 asymmetry in the internet", IEEE Globecom, 2005. 584 [ASSYMETRY2] Abhinav Pathak, Himabindu Pucha, Ying Zhang, Y. 585 Charlie Hu, and Z. Morley Mao, "A measurement study of 586 internet delay asymmetry", PAM'08, 2008. 588 [DELAY-ATT] T. Mizrahi, "A Game Theoretic Analysis of Delay 589 Attacks against Time Synchronization Protocols", 590 ISPCS, 2012. 592 [NTP2] Mills, D.L., "Internet time synchronization: the 593 Network Time Protocol", IEEE Trans. Communications 594 COM-39, 10 (October 1991), 1482-1493. 596 [NTP3] Mills, D.L., "Improved algorithms for synchronizing 597 computer network clocks", IEEE/ACM Trans. Networks 3, 598 3(June 1995), 245-254. 600 [PARIS] Brice Augustin, Timur Friedman and Renata Teixeira, 601 "Measuring Load-balanced Paths in the Internet", IMC, 602 2007. 604 [PARIS2] B. Augustin, T. Friedman, and R. Teixeira, "Measuring 605 Multipath Routing in the Internet", IEEE/ACM 606 Transactions on Networking, 19(3), p. 830 - 840, June 607 2011. 609 [SLAVEDIV] T. Mizrahi, "Slave Diversity: Using Multiple Paths to 610 Improve the Accuracy of Clock Synchronization 611 Protocols", ISPCS, 2012. 613 [TICTOCSEC] T. Mizrahi, K. O'Donoghue, "TICTOC Security 614 Requirements", IETF, draft-ietf-tictoc-security- 615 requirements, work in progress, 2012. 617 [TRACEFLOW] J. Narasimhan, B. V. Venkataswami, R. Groves and P. 618 Hoose, "Traceflow", IETF, draft-janapath-intarea- 619 traceflow, work in progress, 2012. 621 [WEIGHT-MEAN] http://en.wikipedia.org/wiki/Weighted_mean#Dealing_wi 622 th_variance 624 Authors' Addresses 626 Alex Shpiner 627 Department of Electrical Engineering 628 Technion - Israel Institute of Technology 629 Haifa, 32000 Israel 631 Email: shalex@tx.technion.ac.il 632 Richard Tse 633 PMC-Sierra 634 8555 Baxter Place 635 Burnaby, BC 636 Canada 637 V5A 4V7 639 Email: Richard.Tse@pmcs.com 641 Craig Schelp 642 PMC-Sierra 643 8555 Baxter Place 644 Burnaby, BC 645 Canada 646 V5A 4V7 648 Email: craig.schelp@pmcs.com 650 Tal Mizrahi 651 Marvell 652 6 Hamada St. 653 Yokneam, 20692 Israel 655 Email: talmi@marvell.com