TICTOC Interim 16 to 18 June 2008, Paris Attendees (in person): Jean-Loup Ferrant - Alcatel-Lucent Michel Le Pallec - Alcatel-Lucent Alex Tal - Axerra Mike Gilson - British Telecom Kurt Erik Lindqvist - Consultant Peter Lothberg - Consultant Helmut Imlau - Deutsche Telekom Stevano Ruffini - Ericsson John Zhao - Huawei Kuiwen (Jacky) Ji - Huawei Zhang Zhan - Huawei Michael Mayer - Nortel Anti Pietilainen - NSN Sébastien JOBERT - Orange- Groupe FT Yaakov Stein - RAD Pat Diamond - Semtech Emmanuel Sicsik - Spectracom Tim Frost - Symmetricom Doug Arnold - Symmetricom Greg Dowd - Symmetricom Joseph Neil - Symmetricom Grégory LABORDE - Team Avionics Karen O'Donoghue - US Navy Silvana Rodrigues - Zarlink Stewart Bryant - Cisco Systems Mark Townsley - Cisco Systems Laurent Montini - Cisco Systems Leonid Goldin - Cisco Systems Attendees (by dialin) Stuart Venters - Adtran Marshal Eubanks - Iformata Communications TICTOC Interim Day 1 -- June 16, 2008, ~1300 Mark Townsley and Stewart Bryant called the meeting to order. Yaakov Stein was delayed by flights and arrived after the start of the session. There were approximately 30 attendees in the room and 2 dedicated remote attendees. Karen O’Donoghue agreed to take minutes, and Silvana Rodrigues and Kurtis Lindquist agreed to take notes to support the development of the requirements draft. Kurtis and Yaakov (after his arrival) supported the jabber room. The ietf jabber server was down and Yaakov established a temporary room at tictoc@conference.jabber.com Mark Townsley set the ground rules for the meeting. Monday would be spent discussing the application requirements gathered from the survey responses and generating a laundry list of requirements. Tuesday the group would divide into groups based on classes of applications and further refine the requirements. The desired output is a crisp set of requirements. Mark stated that the primary goal of the meeting is to address the first objective of the charter: “To develop a time and frequency distribution requirements document for the two cases listed above, including coexistence of the two as appropriate.” Consolidated Survey Responses, Stewart Bryant ============================================= Stewart Bryant went over the consolidated survey responses. Stewart and Yaakov generated a questionnaire that was sent to the tictoc mailing list by Stewart Bryant on Monday, 5/15/08. There were a total of 24 responses. Stewart consolidated the responses and distributed this to the mailing list on Sunday 6/15/08. This can be found at http://www.ietf.org/mail-archive/web/tictoc/current/msg00335.html The first discussion point was to group the applications into general classes and gauge the support and expertise of each in the room. There was immediate concern that we would address a specific application space based on popularity. The first step should be to gather all the requirements and generalize them for a general purpose protocol. This is deferred for further discussion later. The general application classes from the survey include the following: Cellular backhauling – This includes LTE, WiMAX, etc. 16 of 24 survey respondents were interested in this area as was a large majority of the room. Instrumentation / measurement – (augment not replace 1588) – 2 attendees interested Industrial – (also augment not replace 1588) – Are there new requirements beyond those addressed by 1588? What about sensor networks, mobile networks, power? - 2 attendees interested Networking – Question: Should performance monitoring be moved out to a separate category? – Several attendees interested Time of Day over the general purpose Internet – Defined as what NTP does today only better. 3 attendees interested Legal time – Defined as time traceable to a time source in a secure manner with an audit trail. Based on the currently defined classes, additional application classes were discussed. Already identified were power and sensors. Are there remote telco applications beyond cellular backhauling? Greg indicated that he thought there might be some less well defined applications beyond cellular backhauling. Karen asked about engineered campus networks from the charter. Examples may include the factory floor, ship, real time system, power distribution network. A discussion ensued on the definition of well-engineered. Concepts included networks where resources could be devoted to time, networks engineered to distribute time, constrained networks, networks within the same administrative domain. We agreed on the wording “Networks specifically engineered to deliver time and frequency.” The group reconvened and continued through the survey in order of the original survey questions. A number of questions were skipped when they were determined to be protocol design questions and not actual requirements. 3) What timescale do we want to distribute? There was no real consensus on this question in the survey responses or among the attendees. Peter indicated that the basic question is whether we have a timescale that jumps or not. It was mentioned that ITU-R may remove leap seconds from the UTC timescale, but this has been in discussion for some time already. Yaakov stated that we needed UTC and TAI at a minimum. We need to be careful in our discussions to distinguish the timescale distributed in the protocol from the timescale(s) presented to the user of time, one is orthogonal to the other. We need to provide enough information within the protocol that the consumer can derive the timescale required. We agreed that it was not an option to create our own timescale. 6) Should a separate “frequency only” service be defined? Are there optimizations you could do for the pure frequency deliver case? Yaakov points out that SyncE is an example of a unidirectional frequency service. Perhaps this is a special case of on-path support. You would want the pure frequency delivery to have as much commonality with TOD distribution as possible. 7) What about a special “time when frequency is locked” service? There was discussion about whether this could be done with NTP or not. Greg indicated that this isn’t a question for requirements. It might be in an application or implementation. 8) Should we divide the categories into stringent to undemanding? And 9) If so, how many categories and what are the dividing lines? How much do we want to subdivide the application space? Applications will have different demands, but we need to meet the ones with the highest requirements. We should not try to solve problems where we already have solutions. We should try and identify gaps where we can’t solve the current problems. Reuse is good. The definition of stringent to undemanding could be really difficult. What would define the categories: performance (accuracy, jitter, wander), scalability, cost, auto configuration, management. There are several differentiators that could be utilized. No consensus on this topic. 10) Are there other characteristics that would cause us to consider the creation of a unique timing class? None identified. 11) Separate protocols from algorithms This is designing the solution not identifying the requirements. 12) It is proposed that TICTOC exploit on-path support when available, but be able to function (with reduced performance) when it is not. Do you agree? Do we need to develop a requirement to the routing community to get additional information for tictoc protocols? Will we need routers to send timing packets along a specific path? Are there applications that require on-path support? Do we need a signaling protocol to the end system? There is a difference between discovering on-path support and using on-path support. The operator may contract and set it up (Network informs the user), or the user may ask the network if it is capable. Is on-path a hop by hop thing? The user has to know if the path taken had complete or partial on-path support. Same as the two routing models – traffic engineering model or hop by hop (RSVP). Need to include some concept of domain. Can’t modify every router to tell us it won’t meet our needs. How do you know that there is a failure in the on-path support? How do you manage on-path support for multiple operators? Do we have difference classes of timing within an operator domain? The only consensus achieved thus far is that we need a better understanding of on-path support. It includes too many things. We need to refine the definition of what on-path support is. 13) It is proposed that TICTOC exploit special hardware in server and clients, but be able to function (with reduced performance) when it is not. Do you agree? Deferred (based on solution) 14) Scalability 14a) Number of clients per server 14b) Number of servers on network – 14c) Number of hops / maximum propagation time – Scalability issue is the number of clients not the number of servers. There is a relation between the number of clients, the size of the network, etc. The protocol may need to help solve the issue. The protocol will need to scale, but the specific requirements for scalability are application dependent. 19) Multiple domains with multiple protocols Interworking requirements are an architectural constraint. What is meant by interworking? We don’t expect interoperability at the timing protocol level. NTP and TICTOC can’t break the network when they operate together in the same network. 20) How should client time be updated by TICTOC? The initial discussion was to defer this as an implementation detail, but Kurtis indicated that it is not completely so. Will we say anything about requirements on the API to make this time protocol more useful? Can we provide a definition of what the protocol supplies to other standards bodies defining APIs? We should table the API discussion until protocol is defined, but we should ultimately provide requirements to an external body to develop an API. 21) Do we need to handle multiple clocks? We do need to handle multiple clocks with no phase hit when switching amongst them for the constrained environment. What are the metrics to characterize this? An interesting but different requirement also surfaced. How do we handle multiple paths back to the same source clock. For packet based clocks, the clock plus the path defines the clock. In reality you have multiple clocks, multiple paths, and changing paths. 24) It is proposed that time resolution of TICTOC protocols be 1 picosecond. There was a long discussion and no consensus on the time resolution for tictoc. For the future, resolution should be extensible (not fixed). Hardware designers do not like to design around variable numbers of bits. 27) What are the maximum and minimum update rates needed to be supported ? We need to support a sufficient rate to meet requirements but not break the network. 28) Is a profiling mechanism required ? We need to look at the solutions we are pursuing. 30) How should potential clients determine where to find a time server ? We do need a mechanism to find a time server. There was some feeling that this was not an issue for us. We should assume there is a way to do it. 33) Should we distinguish between time delivered within an AS and time delivered outside it? Not enough time left to discuss this issue. Kurtis could talk at length on this 34) Which of the following authentication mechanisms is required ? 34a) of server by client - yes 34b) of client by server – yes (clarify available versus must be used) 34c) of middleboxes (nodes with on-path support and related) 34d) of the transaction (added during discussion) - maybe The server must be authenticated by client. The IETF has made a strong argument that IPSec should be used. There are no security protocols that understand the concept of zero knowledge proof of time. The IETF has an assumption of synchronized time underlying all its authentication protocols. We definitely need to reuse as much as possible in this space. Do we need to authenticate the transaction? The protocol needs to work in the presence of security. 35) What time source traceability is needed? We have source traceability as a requirement, the mechanism will be defined later. 36) Is encryption of timing packets needed? Most could not see a need for encryption of timing packets. Yaakov indicated that there might be a need for this in a fee based time service. The general consensus to not address encryption 37) Is authentication of timing packets needed? Yes – This was added to 34 as authentication of transactions. This completed the discussion of the survey responses received. Mark asked for the chairs, note takers, and requirements document editors to meet at 7:00 am on Tuesday to produce a requirements matrix from the Day 1 discussion to use Tuesday during break out sessions addressing individual application class requirements. There was a request to add a discussion of the relationship with other standards groups to Tuesday’s agenda as well. The meeting adjourned for the evening. TICTOC Interim Day 2 -- June 17, 2008, ~0900 ============================================ Mark Townsley started the meeting with a plan for the day. We will: - Address the ITU summary of work - Go over summary requirements from yesterday - Go over Kurtis' document - Go over Yaakov's matrix - Break into groups to fill out matrix ITU Summary of time related work efforts, Stefano Ruffini ========================================================= Slides were sent to TICTOC exploder. As they are not available from the email archive they have also been included with the IETF-72 TICTOC meeting materials as "Paris Interim - WD19-Applications-requirements" Stefano summarized the Q13 scope of work and products. He clarified that ITU won't develop a new protocol. Further cooperation/collaboration encouraged. Stefano will forward a link to the exploder on ITU documents. Summary requirements from yesterday, Stewart Bryant =================================================== We thought we could resolve a few specific topics from yesterday. The first item is the timescale. A lengthy discussion followed. Doug Arnold indicated that customer's inevitably will ask for the timescale not support by the protocol. This is driven by a desire for a really simple / lightweight client and the need for a protocol that can distribute multiple timescales. If the protocol can support any of the combinations, customers can do what they want. The protocol should support any kind of application. So if we deliver several timescales, do we need negotiation? The problem with negotiation is that there may be only one way communications. Ultimate protocol flexibility is going to require negotiation. It is useful to design a TLV structure that allows the flexibility. There will be scalability issues for the Internet if all machines could speak their own brand of timescale. Is this too much flexibility? Generally, alternative timescales have come from constrained environments. Perhaps we should have tacit support for allowing alternate timescales for distribution. We should have the mechanism available but don't allow negotiation. Every clock source should have its own identifier traceable to a national standard. It should be a requirement to have some type of flag that indicates you are doing something unusual. Timescales must be distinguishable. We need to have a default. Perhaps, the vast majority of clients donít care which UTC when differentiating between the various national variants of UTC that differ by nanoseconds. The ability to have all timescale operate in parallel is intractable. Are we designing to the simple client / intelligent server paradigm or the complex client / simple server. We concluded the discussion with two agreements: 1. We need to be able to identify the source (not by IP address) (open issue is whether it is in each timing packet and a separate mechanism) 2. We need to be able to distribute different timescales (one at a time from a given server) Furthermore, we had rough consensus that we need to choose a default or recommended timescale: Agree (20ish), Disagree (4) Second item for further discussion: On-path support. How do we address silent support? How do we detect on-path non support? Should we allow the protocol to modify the main time stamp or only tinker with the ancillary fields. If there are things on the wire that help you, how do you authenticate them (middlebox authentication from yesterday)? Do we do on the fly or follow-up corrections or is this a design decision? There was still some feeling in the room that there is not a clear definition of on-path support. There was a definition of on-path support sent to the list. On path support includes node support (transparent clock, boundary clock) and link support (synch E). On path support is correction information dealing with dwell time. On path support can be physical frequency pushed through the network. Yaakov indicated that on-path support is anything which increases the accuracy of the time distribution over pure IP. Alex clarified that it could be a device that only monitors the network and reports information to the endpoints to improve synchronization. On path / network helper functions - high quality clock references (frequency standards, synch e) - modify or correct timestamps in transit (e.g. transparent clocks) - routing decisions to support time/frequency distribution - monitor infrastructure and provide out of band information to end points - differential time transfer There was agreement on the definition list above. At this point, Stewart reviewed the requirements summary produced by the chairs, editors, and the note taker this morning. They took the consolidated survey response Stewart prepared and tried to capture and summarize agreements and remove items not related to requirements. A number of observations were made during the discussion. Clock jitter and PDV are not the same thing. Asymmetry is path asymmetry. There should be requirements categories beyond performance. We should distinguish time and frequency accuracy. Additional metrics including PDEV, TDEV should be added. There may be different scalability categories for constrained and unconstrained environments. Discussion on the requirements summary resumed. =============================================== See appendix for summary of issues from Day 1 On resolution, the consensus was that we need lots of bits. 232 picoseconds is the least common denominator for resolution. By specifying a resolution, we may eliminate one of the existing solutions (ntp, ptp) from consideration. The protocol requirement is that the polling interval is variable/configurable. On question 33, we should change AS to domain. This concluded the discussion of the requirements summary produced this morning. Internet Time Protocol requirements, Kurt Lindquist ====================================== draft-kurtis-tictoc-itp-req-00 Kurt Lindquist walked the attendees through the above posted internet draft. The table of contents is provided below and numbers in the subsequent discussion are referenced to this table. Table of Contents 1. Introduction 2. Terminology 3. Backward compatibility 4. New Packet format 5. Network topology dependencies 6. Resource management 7. Secure identification of time source 8. Size of time errors 9. Time scale 9.1. Concerns for picking a time scale 10. Event Flag (Leap seconds) 11. External data dependencies 12. ITP host API 12.1. ITP to OS 12.2. OS to application 13. Wire protocol and algorithms 14. Resolution requirements 15. IANA considerations 16. Security considerations 17. Acknowledgements 18. Normative References Kurtis clarified that the scope covers the entire protocol. 4) New Packet format The consensus is that we need extensibility. We haven't agreed that the extensibility mechanism is TLVs. We do agree that there needs to be a fixed frame portion for efficient hardware processing. 5) Network Topology Dependencies There was some concern about a 1 ms requirement in general. The protocol does not do clock recovery, we need to keep that distinction. Additionally, we are trying to address a new set of applications that do not degrade gracefully. We need to define limits. The protocol will degrade to a certain point and at some point stop providing service. Generally, a system should be able to constrain itself to operate within a band. ITU engineers network to fall-back in a defined manner. The server should be able to tell the client that it can meet its needs. In the ITU submission of a 1588 profile, if a server cannot provide all of the service requested it will provide none of the services. This document should reflect the WAN requirements. We may need to add requirements for end to end performance monitoring. 6) Resource Management Performance monitoring. Would you include KoD stuff here? Peter would implement something more flexible. Do we add flow control into the protocol to enhance scalability? The server could tell the client to back off. There is an assumption that there is a bi-directional link between client and server. Some would like to also have a server only based mechanism. Antti feels that like 1588 the service should provide what is asked or provide nothing. 7) Secure identification of time source Covered this morning 8) Size of time errors What are the error budgets? Greg indicated that you must measure at the egress of the service otherwise you have to measure three things - capture, transfer and generation of time information. How do you address upstream error? 9, 10 & 11 skipped because discussed earlier 12 ) APIs 12.1 Iinterface between info transfer protocol and local clock generation algorithms 12.2 Interface between local clock and applications utilizing the clock We need to specify 12.1 to properly separate protocol and algorithms. We should provide requirements on 12.2 to a body that develops APIs. Requirements Matrix Analysis ============================ Yaakov went over the matrix he generated during this morning's requirements scrub. This is a very preliminary effort to capture the specific requirements of classes of applications. The group divided into four groups to do a first cut at distilling the requirements of their application classes and using that information to populate the matrix. The groups were: 1) cellular backhaul / remote telcom; 2) instrumentation, power, industrial, engineered campus; 3) networking; 4) ToD/legal time 5) Sensors. At the completion of the breakout time, each subgroup reported on their effort. The results were consolidated into a matrix and distributed by Yaakov. 1) cellular backhaul / remote telecom - Antti Steffano agreed to provide input on LTE/ME column in the text few days. 2) instrumentation, power, industrial, engineered campus - Karen 3) Networking - Stewart 4) ToD/Internet, legal time ñ Peter Frequency stability / frequency accuracy requirement specified for a system over a year. Some feeling that the requirement represented a specialized application. 5) Sensors - Yaakov Mark wrapped up today's meeting with a summary of our progress to date at a reminder of what needs to be accomplished tomorrow. We need to generate a report back to the community on what has been accomplished. Additionally, we are going to address NTP and IEEE 1588 going through the same exercise we just went through for applications. Tomorrow we will be meeting from 0900-1300. <1800 - Adjourned for tictoc dinner> TICTOC Interim Day 3 -- June 18, 2008, ~0900 ============================================= Mark Townsley began the session by laying out the plan for today. First, we will perform yesterday's exercise looking at 1588 and NTP. Then we will summarize the meeting and wrap up. The discussion started with the whole group addressing the above exercise. There was a great deal of discussion on the meaning of terms and concern about how various numbers would be derived. After some thrashing, we split into two groups to address accomplish the task. When we reconvened, Karen presented the NTP output, and Stewart briefed the 1588 team output. Both outputs are captured in the spreadsheet referenced below. There was concern expressed about the difficulty in establishing baseline numbers for 1588. With the vast amount of NTP deployment experience, it was much easier to generate ballpark numbers. This comparison can be found in the IETF72 TICTOC meeting materials. Yaakov produced a consolidated spreadsheet that captured all the application results from yesterday along with and protocol results from today. This spreadsheet can be found in the IETF72 TICTOC meeting materials. Mark Townsley led a wrap up discussion for the meeting. He stepped through what had occurred over the past three days, relating the efforts back to charter. We began with requirements. We came to some convergence on understanding the two use case scenarios in the charter. We discussed constrained versus unconstrained networks. We made some progress on requirements, but work remains. We did not do much work on the second charter objective, modularization. We made an initial stab at the third objective of comparing existing solutions to the requirements, but much work remains here. We need to bring 1588 down to earth a bit more (running code, existence proof) to better understand what can be accomplished. Our next two steps are a 1588 profile and extensions to NTP. Mark indicated that everyone understands that we are going to do both. The final question was "Did we decide that we will address all the applications?" Mark indicated that we are going to look at a pplications to help focus discussion. We aren't designing for a specific application. We design protocols in the IETF for general functionality. We do however need to keep in mind that we have quite different classes of applications. We may have to split these applications into different classes. Stewart indicated that for example we may or may not choose to develop a 1588 profile for the general Internet. Our next step is to package this up and send to the email list. This includes the minutes and the spreadsheets with disclaimers. We need to revise the requirements document. Silvana and Kurtis will work together to produce a working group -00 requirements document for Dublin. We also need to revise the modularization draft. Karen and Tim will work together to produce an updated document by Dublin To clarify the 1588 situation, technical discussions will be based on 1588v2 draft 2.2. This has been made available to the tictoc working group. We can start immediately on a 1588 MIB. There is some prior art to be leveraged. Greg and Yaakov will find that material. Greg also agreed to put together a team for MIB editing. Greg will sign up to start it and draft some volunteers. Steffano offered to help fill in one of the columns from the matrix. We should schedule on the mailing list some editing sessions for Dublin. We should also point out other groups that may be relevant (routing, pwe3, ntp) for new ietf attendees. There was some discussion of IETF process and culture for newcomers in preparation for Dublin. Finally, Mark asked that folks send feedback on the meeting to him. <~1230 meeting adjourned> With MANY thanks to Karen Odonoghue for taking these comprehensive minutes. Stewart Bryant Yaakov Stein TICTIC co-chairs APPENDIX Key issues notes after first day ========================================= 3) What type of time do we need to distribute (multiple answers are possible) ? Is the timescale monotonic with events or does it contain timescales jumps. Is the timescale an applications requirement or something we can pick later Should the protocol handle a choice of timescales. 12) It is proposed that TICTOC exploit on-path support when available, but be able to function (with reduced performance) when it is not. Do you agree ? We know we need to do this Client vs Network discovery Need to way to idenify OPS Need to discover OPS How do we know that there is a failure in OPS Would net ops discose this info How do you manage for multiple ops Need to define OPS 1) Relevant applications that interest you - cellular backhauling remote telco networks instrumentation / measurement industrial networking time of day over the general purpose Internet legal time Power Sensors Engineered campus network metrology 6) Should a separate "frequency-only" service be defined ? There is no consensus on this Need cheap way to distribute frequency 7) What about a special "time when frequency is locked" service ? No consensus - impl specific Can be done with existing protocols. Revisit when we have time 9) If so, how may categories and where are the dividing lines ? Please consider accuracy resolution aquisition time Jitter Wander No consensus on the definitive list Assymetry 14) What scalability constraints do you envisage ? What are the scalability Constrained vs unconstrained environments Large vs Small Depends on the nw Depends on the app Mode of ops Load ballancing across co-located servers Load ballancing across non-co-located servers ????? Easy timestamping Scalability Network - hops 19) multiple domains with multiple protocols Make sure TT protocols do not break each other and do not break the network 20) How should client time be updated by TICTOC ? API needs to be defined 21) How are multiple sources of clock to be handled ? Yes How it works is an application issue Needs to be hitless (depends on application) 24) It is proposed that time resolution of TICTOC protocols be 1 picosecond. No agreement so far Need to discuss further 27) What are the maximum and minimum update rates needed to be supported ? Needs to be sufficient to meet applns needs NETWORKING ISSUES 30) How should potential clients determine where to find a time server ? Write requirements but do not design here 33) Should we distinguish between time delivered within an AS and time delivered outside it? Needs further discussion today SECURITY 34) Which of the following authentication mechanisms is required ? Required in protocol, not required to be turned on Need to identify components that need high security ] and components that would take lower security for scaling reasons. (light & heavy) 34a) of server by client Agreed - but concern over resources 34b) of client by server Agreed - but concern over resources 34c) of middleboxes We mean on path helpers 34d) Transactions - (inc authentication of timing pkts) Agreed 35) What time source traceability is needed? Need source tracability 36) Is encryption of timing packets needed ? * NO 37) Api definition 38) backwards compatibility 39) mappings (encapsulations) Walk through Peter and Kurtis' doc