idnits 2.17.1 draft-ietf-ntp-bcp-08.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (November 11, 2018) is 1990 days in the past. Is this intentional? -- Found something which looks like a code comment -- if you have code sections in the document, please surround them with '' and '' lines. Checking references for intended status: Best Current Practice ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Looks like a reference, but probably isn't: '1' on line 934 -- Looks like a reference, but probably isn't: '2' on line 937 -- Looks like a reference, but probably isn't: '3' on line 939 -- Looks like a reference, but probably isn't: '4' on line 941 -- Looks like a reference, but probably isn't: '5' on line 943 -- Looks like a reference, but probably isn't: '6' on line 946 -- Looks like a reference, but probably isn't: '7' on line 948 -- Looks like a reference, but probably isn't: '8' on line 950 -- Looks like a reference, but probably isn't: '9' on line 952 -- Looks like a reference, but probably isn't: '10' on line 966 -- Looks like a reference, but probably isn't: '11' on line 1061 -- Looks like a reference, but probably isn't: '12' on line 1065 ** Obsolete normative reference: RFC 1305 (Obsoleted by RFC 5905) ** Downref: Normative reference to an Informational RFC: RFC 7094 ** Downref: Normative reference to an Informational RFC: RFC 7384 == Outdated reference: A later version (-11) exists of draft-ietf-ntp-mode-6-cmds-05 == Outdated reference: A later version (-06) exists of draft-ietf-ntp-mac-04 == Outdated reference: A later version (-28) exists of draft-ietf-ntp-using-nts-for-ntp-12 Summary: 3 errors (**), 0 flaws (~~), 5 warnings (==), 14 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Internet Engineering Task Force D. Reilly, Ed. 3 Internet-Draft Orolia USA 4 Intended status: Best Current Practice H. Stenn 5 Expires: May 15, 2019 Network Time Foundation 6 D. Sibold 7 PTB 8 November 11, 2018 10 Network Time Protocol Best Current Practices 11 draft-ietf-ntp-bcp-08 13 Abstract 15 The Network Time Protocol (NTP), currently on its fourth version, has 16 been widely used since its initial publication. This documentation 17 is a collection of Best Practices for general operation of time 18 servers on the Internet from across the NTP community. 20 Status of This Memo 22 This Internet-Draft is submitted in full conformance with the 23 provisions of BCP 78 and BCP 79. 25 Internet-Drafts are working documents of the Internet Engineering 26 Task Force (IETF). Note that other groups may also distribute 27 working documents as Internet-Drafts. The list of current Internet- 28 Drafts is at https://datatracker.ietf.org/drafts/current/. 30 Internet-Drafts are draft documents valid for a maximum of six months 31 and may be updated, replaced, or obsoleted by other documents at any 32 time. It is inappropriate to use Internet-Drafts as reference 33 material or to cite them other than as "work in progress." 35 This Internet-Draft will expire on May 15, 2019. 37 Copyright Notice 39 Copyright (c) 2018 IETF Trust and the persons identified as the 40 document authors. All rights reserved. 42 This document is subject to BCP 78 and the IETF Trust's Legal 43 Provisions Relating to IETF Documents 44 (https://trustee.ietf.org/license-info) in effect on the date of 45 publication of this document. Please review these documents 46 carefully, as they describe your rights and restrictions with respect 47 to this document. Code Components extracted from this document must 48 include Simplified BSD License text as described in Section 4.e of 49 the Trust Legal Provisions and are provided without warranty as 50 described in the Simplified BSD License. 52 Table of Contents 54 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 55 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 56 2. Keeping NTP up to date . . . . . . . . . . . . . . . . . . . 3 57 3. General Network Security Best Practices . . . . . . . . . . . 4 58 3.1. BCP 38 . . . . . . . . . . . . . . . . . . . . . . . . . 4 59 4. NTP Configuration Best Practices . . . . . . . . . . . . . . 4 60 4.1. Use enough time sources . . . . . . . . . . . . . . . . . 5 61 4.2. Use a diversity of Reference Clocks . . . . . . . . . . . 6 62 4.3. Control Messages . . . . . . . . . . . . . . . . . . . . 6 63 4.4. Monitoring . . . . . . . . . . . . . . . . . . . . . . . 7 64 4.5. Using Pool Servers . . . . . . . . . . . . . . . . . . . 7 65 4.6. Leap Second Handling . . . . . . . . . . . . . . . . . . 8 66 4.6.1. Leap Smearing . . . . . . . . . . . . . . . . . . . . 9 67 5. NTP Security Mechanisms . . . . . . . . . . . . . . . . . . . 10 68 5.1. Pre-Shared Key Approach . . . . . . . . . . . . . . . . . 10 69 5.2. Autokey . . . . . . . . . . . . . . . . . . . . . . . . . 11 70 5.3. Network Time Security . . . . . . . . . . . . . . . . . . 11 71 6. NTP Security Best Practices . . . . . . . . . . . . . . . . . 11 72 6.1. Minimizing Information Leakage . . . . . . . . . . . . . 11 73 6.2. Avoiding Daemon Restart Attacks . . . . . . . . . . . . . 12 74 6.3. Detection of Attacks Through Monitoring . . . . . . . . . 13 75 6.4. Kiss-of-Death Packets . . . . . . . . . . . . . . . . . . 14 76 6.5. Broadcast Mode Should Only Be Used On Trusted Networks . 14 77 6.6. Symmetric Mode Should Only Be Used With Trusted Peers . . 15 78 7. NTP in Embedded Devices . . . . . . . . . . . . . . . . . . . 15 79 7.1. Updating Embedded Devices . . . . . . . . . . . . . . . . 15 80 7.2. Server configuration . . . . . . . . . . . . . . . . . . 16 81 7.2.1. Get a vendor subdomain for pool.ntp.org . . . . . . . 16 82 8. NTP over Anycast . . . . . . . . . . . . . . . . . . . . . . 16 83 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17 84 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 85 11. Security Considerations . . . . . . . . . . . . . . . . . . . 18 86 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 87 12.1. Normative References . . . . . . . . . . . . . . . . . . 18 88 12.2. Informative References . . . . . . . . . . . . . . . . . 19 89 12.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 20 90 Appendix A. NTP Implementation by the Network Time 91 Foundation . . . . . . . . . . . . . . . . . . . . . 21 92 A.1. Use enough time sources . . . . . . . . . . . . . . . . . 21 93 A.2. NTP Control and Facility Messages . . . . . . . . . . . . 21 94 A.3. Monitoring . . . . . . . . . . . . . . . . . . . . . . . 22 95 A.4. Leap Second File . . . . . . . . . . . . . . . . . . . . 22 96 A.5. Leap Smearing . . . . . . . . . . . . . . . . . . . . . . 23 97 A.6. Configuring ntpd . . . . . . . . . . . . . . . . . . . . 23 98 A.7. Pre-Shared Keys . . . . . . . . . . . . . . . . . . . . . 23 99 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 101 1. Introduction 103 NTP Version 4 (NTPv4) has been widely used since its publication as 104 RFC 5905 [RFC5905]. This documentation is a collection of best 105 practices from across the NTP community. 107 The recommendations in this document are intended to help operators 108 distribute time on their networks more accurately and more securely. 109 It is intended to apply generally to a broad range of networks. Some 110 specific networks may have higher accuracy requirements that require 111 additional techniques beyond what is documented here. 113 Among the best practices covered are recommendatons for general 114 network security, time protocol specific security, and NTP server and 115 client configuration. NTP operation in embedded devices is also 116 covered. 118 This document also contains information for protocol implementors who 119 want to develop their own RFC 5905 [RFC5905] compliant 120 implementations. 122 1.1. Requirements Language 124 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 125 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 126 "OPTIONAL" in this document are to be interpreted as described in 127 BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all 128 capitals, as shown here. 130 2. Keeping NTP up to date 132 Many network security mechanisms rely on time as part of their 133 operation. If attackers can spoof the time, they may be able to 134 bypass or neutralize other security elements. For example, incorrect 135 time can disrupt the ability to reconcile logfile entries on the 136 affected system with events on other systems. Important ways to 137 detect and protect computers and networks against undefined behavior 138 and security threats related to time are to keep their NTP 139 implementations current, use an appropriate number of trustworthy 140 time sources, and properly monitor their time infrastructure. 142 There are always new ideas about security on the Internet, and an 143 application which is secure today could be insecure tomorrow once an 144 unknown bug (or a known behavior) is exploited in the right way. 146 Even our definition of what is secure has evolved over the years, so 147 code which was considered secure when it was written may turn out to 148 be insecure after some time. By keeping NTP implementations current, 149 having "enough" trustworthy time sources (Section 4.1), and properly 150 monitoring their time infrastructure (Section 4.4), network operators 151 can make sure that their time infrastructure is operating correctly 152 and within specification, and is not being attacked or misused. 154 There are multiple versions of the NTP protocol in use, and multiple 155 implementations in use, on many different platforms. The practices 156 in this document are meant to apply generally to any implementation 157 of RFC 5905 [RFC5905]. It is recommended that that NTP users select 158 an implementation that is actively maintained. Users should keep up 159 to date on any known attacks on their selected implementation, and 160 deploy updates containing security fixes as soon as practical. 162 3. General Network Security Best Practices 164 3.1. BCP 38 166 Many network attacks rely on modifying the IP source address of a 167 packet to point to a different IP address than the computer which 168 originated it. UDP-based protocols such as NTP are generally more 169 susceptible to spoofing attacks then other connection-oriented 170 protocols. NTP control messages can generate a lot of data in 171 response to a small query, which makes it more attractive as a vector 172 for distributed denial-of-service attacks. (NTP Control messages are 173 discussed further in Section 4.3). One documented instance of such 174 an attack can be found here [1], and in [IMC14] and [NDSS14]. 175 Mitigating source address spoofing attacks should be a priority of 176 anyone administering NTP. 178 BCP 38 [RFC2827] was approved in 2000 to address this. BCP 38 179 [RFC2827] calls for filtering outgoing and incoming traffic to make 180 sure that the source and destination IP addresses are consistent with 181 the expected flow of traffic on each network interface. It is 182 recommended that large corporate networks (and ISP's of any size) 183 implement ingress and egress filtering. More information is 184 available at the BCP38 Info Web page [2] . 186 4. NTP Configuration Best Practices 188 This section provides general Best Practices. Best Practices that 189 are implementation specific are compiled in the Appendices. 191 4.1. Use enough time sources 193 An NTP implementation that is compliant with RFC 5905 [RFC5905] takes 194 the available sources of time and submits this timing data to 195 sophisticated intersection, clustering, and combining algorithms to 196 get the best estimate of the correct time. The description of these 197 algorithms is beyond the scope of this document. Interested readers 198 should read RFC 5905 [RFC5905] or the detailed description of NTP in 199 MILLS 2006 [MILLS2006]. These available sources must be truly 200 redundant and derive their time from independent sources. 202 o If there is only 1 source of time, the answer is obvious. It may 203 not be a good source of time, but it's the only source of time 204 that can be considered. Any issue with the time at the source 205 will be passed on to the client. 207 o If there are 2 sources of time and they agree well enough, then 208 the best "time" can be calculated easily. But if one source 209 fails, then the solution degrades to the single-source solution 210 outlined above. And if the two sources don't agree, then it's 211 impossible to know which one is correct by simply looking at the 212 time. 214 o If there are 3 sources of time, there is more data available to 215 converge on a "best" time, and this time is more likely to be 216 accurate. And the loss of one of the sources (by becoming 217 unreachable or unusable) can be tolerated. But at that point, the 218 solution degrades to the 2 source solution. 220 o 4 or more sources of time is better, as long as the sources are 221 diverse (Section 4.2). If one of these sources develops a problem 222 there are still at least 3 other time sources. 224 Operators who are concerned with maintaining accurate time SHOULD use 225 at least 4 independent, diverse sources of time. Four sources will 226 provide sufficient backup in case one source goes down. If four 227 sources are not available, operators MAY use fewer sources, subject 228 to the risks outlined above. 230 But even with 4 or more sources of time, systemic problems can 231 happen. For several hours before and after the June 2015 leap 232 second, several operators implemented leap smearing while others did 233 not, and many NTP end nodes could not determine an accurate time 234 source because 2 of their 4 sources of time gave them consistent UTC/ 235 POSIX time, while the other 2 gave them consistent leap-smeared time. 236 See Section 4.6.1 for more information. 238 Monitor your NTP instances. If your time sources do not generally 239 agree, find out why and either correct the problems or stop using 240 defective servers. See Section 4.4 for more information. 242 4.2. Use a diversity of Reference Clocks 244 When using servers with attached hardware reference clocks, it is 245 recommended that several different types of reference clocks be used. 246 Having a diversity of sources with independent implementations means 247 that any one issue is less likely to cause a service interruption. 249 Are all clocks on a network from the same vendor? They may have the 250 same bugs. Even devices from different vendors may not be truly 251 independent if they share common elements. Are they using the same 252 base chipset? Are they all running the same version of firmware? 253 Chipset and firmware bugs can happen, but they can be more difficult 254 to diagnose than application software bugs. When having the correct 255 time is of critical importance, it's ultimately up to operators to 256 ensure that their sources are sufficiently independent, even if they 257 are not under the operator's control. 259 A systemic problem with time from any satellite navigation service is 260 possible and has happened. Sunspot activity can render satellite or 261 radio-based time source unusable. If the time on your network must 262 be correct close to 100% of the time, then even if you are using a 263 satellite-based system, you must plan for those rare instances when 264 the system is unavailable (or wrong!). 266 4.3. Control Messages 268 Some implementations of NTPv4 provide the NTP Control Messages which 269 originally have been specified in Appendix B of [RFC1305] which 270 defined NTPv3, but never have been part of the NTPv4 specification. 271 (Work is being done to formally document the structure of these 272 control messages in draft-ietf-ntp-mode-6-cmds [CTRLMSG] .) 274 The NTP Control Messages are designed to permit monitoring and 275 optionally authenticated control of NTP and its configuration. Used 276 properly, these facilities provide vital debugging and performance 277 information and control. Used improperly, these facilities can be an 278 abuse vector. For this reason, it is recommended that publicly- 279 facing NTP servers should block mode 6 queries from outside their 280 organization. 282 The ability to use Mode 6 beyond its basic monitoring capabilities 283 can be limited to authenticated sessions that provide a 'controlkey'. 284 It can also be limited through mechanisms outside of the NTP 285 specification, such as Access Control Lists, that only allow access 286 from approved IP addresses. 288 The NTP Control Messages responses are much larger than the 289 corresponding queries. Thus, they can be abused in high-bandwidth 290 DDoS attacks. To provide protection for such abuse NTP server 291 operators should deploy ingress filtering BCP 38 [RFC2827]. 293 4.4. Monitoring 295 Use your NTP implementation's remote monitoring capabilities to 296 quickly identify servers which are out of sync, and ensure 297 correctness of the service. Monitor system logs for messages so 298 problems and abuse attempts can be quickly identified. 300 If a system starts getting unexpected time replies from its time 301 servers, that can be an indication that the IP address of the system 302 is being forged in requests to its time server, and these abusers are 303 trying to convince that time server to stop serving time to that 304 system. 306 If a system is a broadcast client and its system log shows that it is 307 receiving "early" time messages from its server, that is an 308 indication that somebody may be forging packets from a broadcast 309 server. 311 If a server's system log shows messages that indicates it is 312 receiving timestamps that are earlier than the current system time, 313 then either the system clock is unusually fast or somebody is trying 314 to launch a replay attack against that server. 316 4.5. Using Pool Servers 318 It only takes a small amount of bandwidth and system resources to 319 synchronize one NTP client, but NTP servers that can service tens of 320 thousands of clients take more resources to run. Users who want to 321 synchronize their computers should only synchronize to servers that 322 they have permission to use. 324 The NTP pool project is a group of volunteers who have donated their 325 computing and bandwidth resources to freely distribute time from 326 primary time sources to others on the Internet. The time is 327 generally of good quality, but comes with no guarantee whatsoever. 328 If you are interested in using the pool, please review their 329 instructions at http://www.pool.ntp.org/en/use.html [3]. 331 If you are a vendor who wishes to provide time service to your 332 customers or clients, consider joining the pool and providing a 333 "vendor zone" through the pool project. 335 If you want to synchronize many computers, consider running your own 336 NTP servers that are synchronized by the pool, and synchronizing your 337 clients to your in-house NTP servers. This reduces the load on the 338 pool. 340 4.6. Leap Second Handling 342 UTC is kept in agreement with the astronomical time UT1 [4] to within 343 +/- 0.9 seconds by the insertion (or possibly a deletion) of a leap 344 second. UTC is an atomic time scale whereas UT1 is based on the 345 rotational rate of the earth. Leap seconds are not introduced at a 346 fixed rate. They are announced by the International Earth Rotation 347 and Reference Systems Service (IERS) in its Bulletin C [5] when 348 necessary to keep UTC and UT1 aligned. 350 NTP time is based on the UTC timescale, and the protocol has the 351 capability to broadcast leap second information. Some Global 352 Navigation Satellite Systems (like GPS) or radio transmitters (like 353 DCF77) broadcast leap second information, so if you are synced to an 354 NTP server that is ultimately synced to a source that provides leap 355 second notification you will get advance notification of impending 356 leap seconds automatically. 358 Since the length of the UT1 day is generally slowly increasing [6], 359 all leap seconds that have been introduced since the practice started 360 in 1972 have been "positive" leap seconds, where a second is added to 361 UTC. NTP also supports a "negative" leap second, where a second is 362 removed from UTC, should that ever become necessary. 364 While earlier versions of NTP contained some ambiguity regarding when 365 a leap second that is broadcast by a server should be applied by a 366 client, RFC 5905 is clear that leap seconds are only applied on the 367 last day of a month. However, because some older clients may apply 368 it at the end of the current day, it is recommended that NTP servers 369 wait until the last day of the month before broadcasting leap 370 seconds. Doing this will prevent older clients from applying a leap 371 second at the wrong time. Note well that NTPv4's longest polling 372 interval exceeds one day and thus a leap second announcement may be 373 missed. 375 In circumstances where an NTP server is not receiving leap second 376 information from an automated source, certain organizations maintain 377 files which are updated every time a new leap second is announced: 379 NIST: ftp://time.nist.gov/pub/leap-seconds.list 381 US Navy (maintains GPS Time): ftp://tycho.usno.navy.mil/pub/ntp/leap- 382 seconds.list 384 IERS (announces leap seconds): 385 https://hpiers.obspm.fr/iers/bul/bulc/ntp/leap-seconds.list 387 4.6.1. Leap Smearing 389 Some NTP installations may instead make use of a technique called 390 "Leap Smearing". With this method, instead of introducing an extra 391 second (or eliminating a second), NTP time will be slewed in small 392 increments over a comparably large window of time (called the smear 393 interval) around the leap second event. The smear interval should be 394 large enough to make the rate that the time is slewed small, so that 395 clients will follow the smeared time without objecting. Periods 396 ranging from 2 to 24 hours have been used successfully. During the 397 adjustment window, all the NTP clients' times may be offset from UTC 398 by as much as a full second, depending on the implementation. But at 399 least all clients will generally agree on what time they think it is! 401 Operators should NOTE WELL that using a leap-smear can cause your 402 reported time to be "legally indefensible" and/or be a breach of 403 compliance regulations. 405 The purpose of Leap Smearing is to enable systems that don't deal 406 with the leap second event properly to function consistently, at the 407 expense of fidelity to UTC during the smear window. During a 408 standard leap second event, that minute will have 61 (or possibly 59) 409 seconds in it, and some applications (and even some OS's) are known 410 to have problems with that. 412 Clients that are connected to leap smearing servers MUST NOT apply 413 the "standard" NTP leap second handling. So these clients must never 414 have a leap second file loaded, and the smearing servers must never 415 advertise to clients that a leap second is pending. 417 Any use of leap smearing servers should be limited to within a 418 single, well-controlled environment. Leap Smearing MUST NOT be used 419 f or public-facing NTP servers, as they will disagree with non- 420 smearing servers (as well as UTC) during the leap smear interval, and 421 there is no standardized way for a client to detect that a server is 422 using leap smearing. However, be aware that some public-facing 423 servers may be configured this way anyway in spite of this guidance. 425 System Administrators are advised to be aware of impending leap 426 seconds and how the servers (inside and outside their organization) 427 they are using deal with them. Individual clients must never be 428 configured to use a mixture of smeared and non-smeared servers. If a 429 client uses smeared servers, the servers it uses must all have the 430 same leap smear configuration. 432 5. NTP Security Mechanisms 434 In the standard configuration NTP packets are exchanged unprotected 435 between client and server. An adversary that is able to become a 436 Man-In-The-Middle is therefore able to drop, replay or modify the 437 content of the NTP packet, which leads to degradation of the time 438 synchronization or the transmission of false time information. A 439 profound threat analysis for time synchronization protocols is given 440 in RFC 7384 [RFC7384]. NTP provides two internal security mechanisms 441 to protect authenticity and integrity of the NTP packets. Both 442 measures protect the NTP packet by means of a Message Authentication 443 Code (MAC). Neither of them encrypts the NTP's payload, because this 444 payload information is not considered to be confidential. 446 5.1. Pre-Shared Key Approach 448 This approach applies a symmetric key for the calculation of the MAC, 449 which protects authenticity and integrity of the exchanged packets 450 for an association. NTP does not provide a mechanism for the 451 exchange of the keys between the associated nodes. Therefore, for 452 each association, keys have to be exchanged securely by external 453 means, and they have to be protected from disclosure. It is 454 recommended that each association be protected by its own unique key. 455 It is recommended that participants agree to refresh keys 456 periodically. However, NTP does not provide a mechanism to assist in 457 doing so. 459 RFC 5905 [RFC5905] specifies a hash which must be supported for 460 calculation of the MAC, but other algorithms may be supported as 461 well. The MD5 hash is now considered to be too weak. 462 Implementations will soon be available based on AES-128-CMAC 463 [NTPMAC], and users are encouraged to use that when it is available. 465 To use this approach the communication partners have to exchange the 466 key, which consists of a keyid with a value between 1 and 65534, 467 inclusive, and a label which indicates the chosen digest algorithm. 468 Each communication partner adds this information to its own key file. 470 Some implementations store the key in clear text. Therefore it 471 should only be readable by the NTP process. Different keys are added 472 line by line to the key file. 474 An NTP client establishes a protected association by appending the 475 key to the server statement in its configuration file. Note that the 476 NTP process has to trust the applied key. 478 5.2. Autokey 480 Autokey was specified in 2010 to provide automated key management and 481 authentication of NTP servers. However, security researchers have 482 identified vulnerabilities in the Autokey protocol, which make the 483 protocol "useless". [7] 485 Autokey SHOULD NOT BE USED. 487 5.3. Network Time Security 489 Work is in progress on an enhanced replacement for Autokey, which is 490 called Network Time Security (NTS) [NTSFORNTP]. As of July 2018, 491 this effort was at draft #12, and in the 'Working Group Last Call' 492 process. Readers are encouraged to adopt its mechanisms. 494 6. NTP Security Best Practices 496 This section lists some general NTP security practices, but these 497 issues may (or may not) have been mitigated in particular versions of 498 particular implementations. Contact the maintainers of your 499 implementation for more information. 501 6.1. Minimizing Information Leakage 503 The base NTP packet leaks important information (including reference 504 ID and reference time) that may be used in attacks [NDSS16], 505 [CVE-2015-8138], [CVE-2016-1548]. A remote attacker can learn this 506 information by sending mode 3 queries to a target system and 507 inspecting the fields in the mode 4 response packet. NTP control 508 queries also leak important information (including reference ID, 509 expected origin timestamp, etc.) that may be used in attacks 510 [CVE-2015-8139]. A remote attacker can learn this information by 511 sending control queries to a target system and inspecting the 512 response. 514 As such, mechanisms outside of the NTP protocol, such as Access 515 Control Lists, should be used to limit the exposure of this 516 information to allowed IP addresses, and keep it from remote 517 attackers not on the list. 519 Hosts should only respond to NTP control queries from authorized 520 parties. One way to do this is to only allow control queries from 521 authenticated sources via authorized IP addresses. 523 A host that is not supposed to act as an NTP server that provides 524 timing information to other hosts may additionally log and drop 525 incoming mode 3 timing queries from unexpected sources. Note well 526 that the easiest way to monitor ntpd's status is to send it a mode 3 527 query. It is recommended that operators should filter mode 3 queries 528 at the edge, or make sure mode 3 queries are allowed only from 529 trusted systems or networks. 531 A "leaf-node host" is a host that is using NTP solely for the purpose 532 of adjusting its own system time. Such a host is not expected to 533 provide time to other hosts, and relies exclusively on NTP's basic 534 mode to take time from a set of servers. (That is, the host sends 535 mode 3 queries to its servers and receives mode 4 responses from 536 these servers containing timing information.) To minimize 537 information leakage, leaf-node hosts should drop all incoming NTP 538 packets except mode 4 response packets that come from known sources. 539 Note well that proper monitoring of an ntpd instance includes 540 checking the time of that ntpd instance. 542 6.2. Avoiding Daemon Restart Attacks 544 RFC 5905 [RFC5905] says NTP clients should not accept time shifts 545 greater than the panic threshold. Specifically, RFC 5905 says "PANIC 546 means the offset is greater than the panic threshold PANICT (1000 s) 547 and SHOULD cause the program to exit with a diagnostic message to the 548 system log." 550 However, this behavior can be exploited by attackers [NDSS16], when 551 the following two conditions hold: 553 1. The operating system automatically restarts the NTP daemon when 554 it quits. (Modern *NIX operating systems are replacing 555 traditional init systems with process supervisors, such as 556 systemd, which can be configured to automatically restart any 557 daemons that quit. This behavior is the default in CoreOS and 558 Arch Linux. It is likely to become the default behavior in other 559 systems as they migrate legacy init scripts to process 560 supervisors such as systemd.) 562 2. The NTP client is configured to ignore the panic threshold on all 563 restarts. 565 In such cases, if the attacker can send the target an offset that 566 exceeds the panic threshold, the client will quit. Then, when the 567 client restarts, it ignores the panic threshold and accepts the 568 attacker's large offset. 570 Hosts running with the above two conditions should be aware that the 571 panic threshold does not protect them from attacks. The recommended 572 and natural solution is not to run hosts with these conditions. 573 Specifically, only ignore the panic threshold in cold-start 574 situations if sufficient oversight and checking is in place to make 575 sure that this is appropriate. 577 As an alternative, the following steps could be taken to mitigate the 578 risk of attack. 580 o Monitor NTP system log to detect when the NTP daemon has quit due 581 to a panic event, as this could be a sign of an attack. 583 o Request manual intervention when a timestep larger than the panic 584 threshold is detected. 586 o Configure the ntp client to only ignore the panic threshold in a 587 cold start situation. 589 o Implementations should prevent the NTP daemon from taking time 590 steps that set the clock to a time earlier than the compile date 591 of the NTP daemon. 593 o Add "minsane" and "minclock" parameters to the ntp.conf file so 594 ntpd waits until "enough" trusted sources of time agree on the 595 correct time. 597 6.3. Detection of Attacks Through Monitoring 599 Users should monitor their NTP instances to detect attacks. Many 600 known attacks on NTP have particular signatures. Common attack 601 signatures include: 603 1. "Bogus packets" - A packet whose origin timestamp does not match 604 the value that expected by the client. 606 2. "Zero origin packet" - A packet with an origin timestamp set to 607 zero [CVE-2015-8138]. 609 3. A packet with an invalid cryptographic MAC [CCR16]. 611 The observation of many such packets could indicate that the client 612 is under attack. 614 Also, Kiss-o'-Death (KoD) packets can be used in denial of service 615 attacks. Thus, the observation of even just one KoD packet with a 616 high poll value could be sign that the client is under attack. See 617 Section 6.4 for more information. 619 6.4. Kiss-of-Death Packets 621 The "Kiss-o'-Death" (KoD) packet is a rate limiting mechanism where a 622 server can tell a misbehaving client to "back off" its query rate. 623 It is important for all NTP devices to respect these packets and back 624 off when asked to do so by a server. It is even more important for 625 an embedded device, which may not have exposed a control interface 626 for NTP. 628 That said, a client must only accept a KoD packet if it has a valid 629 origin timestamp. Once a RATE packet is accepted, the client should 630 increase its poll interval value (thus decreasing its polling rate) 631 up to a reasonable maximum. This maximum can vary by implementation 632 but should not exceed a poll interval value of 13 (2 hours). The 633 mechanism to determine how much to increase the poll interval value 634 is undefined in RFC 5905 [RFC5905]. If the client uses the poll 635 interval value sent by the server in the KoD packet, it must not 636 simply accept any value. Using large interval values may open a 637 vector for a denial-of-service attack that causes the client to stop 638 querying its server [NDSS16]. 640 The KoD mechanism relies on clients behaving properly in order to be 641 effective. Some clients ignore the KoD packet entirely, and other 642 poorly-implemented clients might unintentionally increase their poll 643 rate and simulate a denial of service attack. Server administrators 644 should be prepared for this and take measures outside of the NTP 645 protocol to drop packets from misbehaving clients when these clients 646 are detected. 648 6.5. Broadcast Mode Should Only Be Used On Trusted Networks 650 Per RFC 5905 [RFC5905], NTP's broadcast mode is authenticated using 651 symmetric key cryptography. The broadcast server and all of its 652 broadcast clients share a symmetric cryptographic key, and the 653 broadcast server uses this key to append a message authentication 654 code (MAC) to the broadcast packets it sends. 656 Importantly, all broadcast clients that listen to this server must 657 know the cryptographic key. This mean that any client can use this 658 key to send valid broadcast messages that look like they come from 659 the broadcast server. Thus, a rogue broadcast client can use its 660 knowledge of this key to attack the other broadcast clients. 662 For this reason, an NTP broadcast server and all its client must 663 trust each other. Broadcast mode should only be run from within a 664 trusted network. 666 6.6. Symmetric Mode Should Only Be Used With Trusted Peers 668 In symmetric mode, two peers Alice and Bob can both push and pull 669 synchronization to and from each other using either ephemeral 670 symmetric passive (mode 2) or persistent symmetric active (NTP mode 671 1) packets. The persistent association is preconfigured and 672 initiated at the active peer but not preconfigured at the passive 673 peer (Bob). Upon receipt of a mode 1 NTP packet from Alice, Bob 674 mobilizes a new ephemeral association if he does not have one 675 already. This is a security risk for Bob because an arbitrary 676 attacker can attempt to change Bob's time by asking Bob to become its 677 symmetric passive peer. 679 For this reason, a host (Bob) should only allow symmetric passive 680 associations to be established with trusted peers. Specifically, Bob 681 should require each of its symmetric passive association to be 682 cryptographically authenticated. Each symmetric passive association 683 should be authenticated under a different cryptographic key. 685 The use of a different cryptographic key per peer prevents a Sybil 686 attack, where a single malicious peer uses the same cryptographic key 687 to set up multiple symmetric associations a target, and thus bias the 688 results of the target's Byzantine fault tolerant peer selection 689 algorithms. 691 7. NTP in Embedded Devices 693 Readers of this BCP already understand how important accurate time is 694 for network computing. And as computing becomes more ubiquitous, 695 there will be many small "Internet of Things" devices that require 696 accurate time. These devices may not have a persistent battery- 697 backed clock, so using NTP to set the correct time on power-up may be 698 critical for proper operation. These devices may not have a 699 traditional user interface, but if they connect to the Internet they 700 will be subject to the same security threats as traditional 701 deployments. 703 7.1. Updating Embedded Devices 705 Vendors of embedded devices have a special responsibility to pay 706 attention to the current state of NTP bugs and security issues, 707 because their customers don't have the ability to update their NTP 708 implementation on their own. Those devices may have a single 709 firmware upgrade, provided by the manufacturer, that updates all 710 capabilities at once. This means that the vendor assumes the 711 responsibility of making sure their devices have the latest NTP 712 updates applied. 714 This should also include the ability to update information regarding 715 which NTP server to connect to on these devices. 717 There is a catalog of NTP server abuse incidents, some of which 718 involve embedded devices, on the Wikipedia page for NTP Server Misuse 719 and Abuse [8]. 721 7.2. Server configuration 723 Vendors of embedded devices that need time synchronization should 724 also carefully consider where they get their time from. There are 725 several public-facing NTP servers available, but they may not be 726 prepared to service requests from thousands of new devices on the 727 Internet. 729 Vendors are encouraged to invest resources into providing their own 730 time servers for their devices to connect to. 732 Vendors should read RFC 4085 [RFC4085], which advises against 733 embedding globally-routable IP addresses in products, and offers 734 several better alternatives. 736 7.2.1. Get a vendor subdomain for pool.ntp.org 738 The NTP Pool Project offers a program where vendors can obtain their 739 own subdomain that is part of the NTP Pool. This offers vendors the 740 ability to safely make use of the time distributed by the Pool for 741 their devices. Vendors are encouraged to support the pool if they 742 participate. For more information, visit http://www.pool.ntp.org/en/ 743 vendors.html [9] . 745 8. NTP over Anycast 747 Anycast is described in BCP 126 [RFC4786]. (Also see RFC 7094 748 [RFC7094]). With anycast, a single IP address is assigned to 749 multiple interfaces, and routers direct packets to the closest active 750 interface. 752 Anycast is often used for Internet services at known IP addresses, 753 such as DNS. Anycast can also be used in large organizations to 754 simplify configuration of a large number of NTP clients. Each client 755 can be configured with the same NTP server IP address, and a pool of 756 anycast servers can be deployed to service those requests. New 757 servers can be added to or taken from the pool, and other than a 758 temporary loss of service while a server is taken down, these 759 additions can be transparent to the clients. 761 NOTE WELL: Using a single anycast address for NTP should be done with 762 care. It means each client will likely use a single time server 763 source. A key element of a robust NTP deployment is each client 764 using multiple sources of time. With multiple time sources, a client 765 will analyze the various time sources, selecting good ones, and 766 disregarding poor ones. If a single Anycast address is used, this 767 analysis will not happen. 769 If clients are connected to an NTP server via anycast, the client 770 does not know which particular server they are connected to. As 771 anycast servers may arbitrarily enter and leave the network, the 772 server a particular client is connected to may change. This may 773 cause a small shift in time from the perspective of the client when 774 the server it is connected to changes. It is recommended that 775 anycast only be deployed in environments where these small shifts can 776 be tolerated. 778 Configuration of an anycast interface is independent of NTP. Clients 779 will always connect to the closest server, even if that server is 780 having NTP issues. It is recommended that anycast NTP 781 implementations have an independent method of monitoring the 782 performance of NTP on a server. If the server is not performing to 783 specification, it should remove itself from the Anycast network. It 784 is also recommended that each Anycast NTP server have at least one 785 Unicast interface, so its performance can be checked independently of 786 the anycast routing scheme. 788 One useful application in large networks is to use a hybrid unicast/ 789 anycast approach. Stratum 1 NTP servers can be deployed with unicast 790 interfaces at several sites. Each site may have several Stratum 2 791 servers with two ethernet interfaces, or a single interface which can 792 support multiple addresses. One interface has a unique unicast IP 793 address. The second has an anycast IP interface (with a shared IP 794 address per location). The unicast interfaces can be used to obtain 795 time from the Stratum 1 servers globally (and perhaps peer with the 796 other Stratum 2 servers at their site). Clients at each site can be 797 configured to use the shared anycast address for their site, 798 simplifying their configuration. Keeping the anycast routing 799 restricted on a per-site basis will minimize the disruption at the 800 client if its closest anycast server changes. Each Stratum 2 server 801 can be uniquely identified on their unicast interface, to make 802 monitoring easier. 804 9. Acknowledgments 806 The authors wish to acknowledge the contributions of Sue Graves, 807 Samuel Weiler, Lisa Perdue, Karen O'Donoghue, David Malone, Sharon 808 Goldberg, Martin Burnicki, Miroslav Lichvar, Daniel Fox Franke, and 809 Robert Nagy. 811 10. IANA Considerations 813 This memo includes no request to IANA. 815 11. Security Considerations 817 Time is a fundamental component of security on the internet. The 818 absence of a reliable source of current time subverts many common web 819 authentication schemes, e.g., by allowing the use of expired 820 credentials or by allowing for replay of messages only intended to be 821 processed once. 823 Much of this document directly addresses how to secure NTP servers. 824 In particular, see Section 3, Section 5, and Section 6. 826 There are several general threats to time synchronization protocols 827 which are discussed in RFC 7384 [RFC7384]. 829 [NTSFORNTP] is an Internet-Draft that specifies the Network Time 830 Security (NTS) mechanism and applies it specifically to NTP. Readers 831 are encouraged to check the status of the draft, and make use of the 832 methods it describes. 834 12. References 836 12.1. Normative References 838 [RFC1305] Mills, D., "Network Time Protocol (Version 3) 839 Specification, Implementation and Analysis", RFC 1305, 840 DOI 10.17487/RFC1305, March 1992, 841 . 843 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 844 Requirement Levels", BCP 14, RFC 2119, 845 DOI 10.17487/RFC2119, March 1997, 846 . 848 [RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering: 849 Defeating Denial of Service Attacks which employ IP Source 850 Address Spoofing", BCP 38, RFC 2827, DOI 10.17487/RFC2827, 851 May 2000, . 853 [RFC4085] Plonka, D., "Embedding Globally-Routable Internet 854 Addresses Considered Harmful", BCP 105, RFC 4085, 855 DOI 10.17487/RFC4085, June 2005, 856 . 858 [RFC4786] Abley, J. and K. Lindqvist, "Operation of Anycast 859 Services", BCP 126, RFC 4786, DOI 10.17487/RFC4786, 860 December 2006, . 862 [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, 863 "Network Time Protocol Version 4: Protocol and Algorithms 864 Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, 865 . 867 [RFC7094] McPherson, D., Oran, D., Thaler, D., and E. Osterweil, 868 "Architectural Considerations of IP Anycast", RFC 7094, 869 DOI 10.17487/RFC7094, January 2014, 870 . 872 [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in 873 Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, 874 October 2014, . 876 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 877 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 878 May 2017, . 880 12.2. Informative References 882 [CCR16] Malhotra, A. and S. Goldberg, "Attacking NTP's 883 Authenticated Broadcast Mode", SIGCOMM Computer 884 Communications Review (CCR) , 2016. 886 [CTRLMSG] Mills, D. and B. Haberman, "Control Messages Protocol for 887 Use with Network Time Protocol Version 4", draft-ietf-ntp- 888 mode-6-cmds-05 (work in progress), March 2018. 890 [CVE-2015-8138] 891 Van Gundy, M. and J. Gardner, "NETWORK TIME PROTOCOL 892 ORIGIN TIMESTAMP CHECK IMPERSONATION VULNERABILITY", 2016, 893 . 895 [CVE-2015-8139] 896 Van Gundy, M., "NETWORK TIME PROTOCOL NTPQ AND NTPDC 897 ORIGIN TIMESTAMP DISCLOSURE VULNERABILITY", 2016, 898 . 900 [CVE-2016-1548] 901 Gardner, J. and M. Lichvar, "Xleave Pivot: NTP Basic Mode 902 to Interleaved", 2016, 903 . 906 [IMC14] Czyz, J., Kallitsis, M., Gharaibeh, M., Papadopoulos, C., 907 Bailey, M., and M. Karir, "Taming the 800 Pound Gorilla: 908 The Rise and Decline of NTP DDoS Attacks", Internet 909 Measurement Conference , 2014. 911 [MILLS2006] 912 Mills, D., "Computer network time synchronization: the 913 Network Time Protocol", CRC Press , 2006. 915 [NDSS14] Rossow, C., "Amplification Hell: Revisiting Network 916 Protocols for DDoS Abuse", NDSS'14, San Diego, CA. , 2014. 918 [NDSS16] Malhotra, A., Cohen, I., Brakke, E., and S. Goldberg, 919 "Attacking the Network Time Protocol", NDSS'16, San Diego, 920 CA. , 2016, . 922 [NTPMAC] Malhotra, A. and S. Goldberg, "Message Authentication Code 923 for the Network Time Protocol", draft-ietf-ntp-mac-04 924 (work in progress), March 2018. 926 [NTSFORNTP] 927 Sibold, D., Roettger, S., and K. Teichel, "Using the 928 Network Time Security Specification to Secure the Network 929 Time Protocol", draft-ietf-ntp-using-nts-for-ntp-12 (work 930 in progress), July 2018. 932 12.3. URIs 934 [1] https://blog.cloudflare.com/technical-details-behind-a-400gbps- 935 ntp-amplification-ddos-attack/ 937 [2] http://www.bcp38.info 939 [3] http://www.pool.ntp.org/en/use.html 941 [4] https://en.wikipedia.org/wiki/Solar_time#Mean_solar_time 943 [5] https://www.iers.org/IERS/EN/Publications/Bulletins/ 944 bulletins.html 946 [6] https://en.wikipedia.org/wiki/Solar_time#Mean_solar_time 948 [7] https://lists.ntp.org/pipermail/ntpwg/2011-August/001714.html 950 [8] https://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse 952 [9] http://www.pool.ntp.org/en/vendors.html 954 [10] http://www.ntp.org/downloads.html 956 [11] http://bk1.ntp.org/ntp-stable/README.leapsmear?PAGE=anno 958 [12] https://support.ntp.org/bin/view/Support/ConfiguringNTP 960 Appendix A. NTP Implementation by the Network Time Foundation 962 The Network Time Foundation (NTF) provides the reference 963 implementation of NTP, well-known under the name "ntpd". It is 964 actively maintained and developed by NTF's NTP Project, with help 965 from volunteers and NTF's supporters. This NTP software can be 966 downloaded from ntp.org [10]. 968 A.1. Use enough time sources 970 In addition to the recommendation given in Section Section 4.1 the 971 ntpd implementation provides the 'pool' directive. Starting with 972 ntp-4.2.6, this directive will spin up "enough" associations to 973 provide robust time service, and will disconnect poor servers and add 974 in new servers as-needed. If you have good reason, you may use the 975 'minclock' and 'maxclock' options of the 'tos' command to override 976 the default values of how many servers are discovered through the 977 'pool' directive. 979 A.2. NTP Control and Facility Messages 981 In addition to NTP Control Messages the ntpd implementation also 982 offers the Mode 7 commands for monitoring and configuration. 984 If Mode 7 has been explicitly enabled to be used for more than basic 985 monitoring it should be limited to authenticated sessions that 986 provide a 'requestkey'. 988 As mentioned above, there are two general ways to use Mode 6 and Mode 989 7 requests. One way is to query ntpd for information, and this mode 990 can be disabled with: 992 restrict ... noquery 994 The second way to use Mode 6 and Mode 7 requests is to modify ntpd's 995 behavior. Modification of ntpd's configuration requires an 996 authenticated session by default. If no authentication keys have 997 been specified no modifications can be made. For additional 998 protection, the ability to perform these modifications can be 999 controlled with: 1001 restrict ... nomodify 1003 Users can prevent their NTP servers from considering query/ 1004 configuration traffic by default by adding the following to their 1005 ntp.conf file: 1007 restrict default -4 nomodify notrap nopeer noquery 1009 restrict default -6 nomodify notrap nopeer noquery 1011 restrict source nomodify notrap noquery 1012 # nopeer is OK if you don't use the 'pool' directive 1014 A.3. Monitoring 1016 The reference implementation of NTP allows remote monitoring. Access 1017 to this service is generally controlled by the "noquery" directive in 1018 NTP's configuration file (ntp.conf) via a "restrict" statement. The 1019 syntax reads: 1021 restrict address mask address_mask noquery 1023 If a system is using broadcast mode and is running ntp-4.2.8p6 or 1024 later, use the 4th field of the ntp.keys file to specify the IPs of 1025 machines that are allowed to serve time to the group. 1027 A.4. Leap Second File 1029 The use of leap second files requires ntpd 4.2.6 or later. After 1030 fetching the leap seconds file onto the server, add this line to 1031 ntpd.conf to apply and use the file: 1033 leapfile "/path/to your/leap-file" 1035 You may need to restart ntpd to apply this change. 1037 ntpd servers with a manually configured leap second file will ignore 1038 leap second information broadcast from upstream NTP servers until the 1039 leap second file expires. If no valid leap second file is available 1040 then a leap second notification from an attached reference clock is 1041 always accepted by ntpd. 1043 If no valid leap second file is available, a leap second notification 1044 may be accepted from upstream NTP servers. As of ntp-4.2.6, a 1045 majority of servers must provide the notification before it is 1046 accepted. Before 4.2.6, a leap second notification would be accepted 1047 if a single upstream server of a group of configured servers provided 1048 a leap second notification. This would lead to misbehavior if single 1049 NTP servers sent an invalid leap second warning, e.g. due to a faulty 1050 GPS receiver in one server, but this behavior was once chosen because 1051 in the "early days" there was a greater chance that leap second 1052 information would be available from a very limited number of sources. 1054 A.5. Leap Smearing 1056 Leap Smearing was introduced in ntpd versions 4.2.8.p3 and 4.3.47, in 1057 response to CLIENT requests. Support for leap smearing is not 1058 configured by default and must be added at compile time. In 1059 addition, no leap smearing will occur unless a leap smear interval is 1060 specified in ntpd.conf . For more information, refer to 1061 http://bk.ntp.org/ntp-stable/README.leapsmear?PAGE=anno [11]. 1063 A.6. Configuring ntpd 1065 See https://support.ntp.org/bin/view/Support/ConfiguringNTP [12] for 1066 additional information on configuring ntpd. 1068 A.7. Pre-Shared Keys 1070 Each communication partner must add the keyid information to their 1071 key file in the form: 1073 keyid label key 1075 An ntpd client establishes a protected association by appending the 1076 option "key keyid" to the server statement in ntp.conf: 1078 server address key keyid 1080 A key is deemed trusted when its keyid is added to the list of 1081 trusted keys by the "trustedkey" statement in ntp.conf. 1083 trustedkey keyid_1 keyid_2 ... keyid_n 1085 Starting with ntp-4.2.8p7 the ntp.keys file accepts an optional 4th 1086 column, a comma-separated list of IPs that are allowed to serve time. 1087 Use this feature. Note, however, that an adversarial client that 1088 knows the symmetric broadcast key could still easily spoof its source 1089 IP to an IP that is allowed to serve time. (This is easy to do 1090 because the origin timestamp on broadcast mode packets is not 1091 validated by the client. By contrast, client/server and symmetric 1092 modes do require origin timestamp validation, making it more 1093 difficult to spoof packets [CCR16]. 1095 Authors' Addresses 1097 Denis Reilly (editor) 1098 Orolia USA 1099 1565 Jefferson Road, Suite 460 1100 Rochester, NY 14623 1101 US 1103 Email: denis.reilly@orolia.com 1105 Harlan Stenn 1106 Network Time Foundation 1107 P.O. Box 918 1108 Talent, OR 97540 1109 US 1111 Email: stenn@nwtime.org 1113 Dieter Sibold 1114 Physikalisch-Technische Bundesanstalt 1115 Bundesallee 100 1116 Braunschweig D-38116 1117 Germany 1119 Phone: +49-(0)531-592-8420 1120 Fax: +49-531-592-698420 1121 Email: dieter.sibold@ptb.de