======================================================================== Minutes for Internet Wideband Audio Codec BOF, IETF 75 Kongresshall C, City Conference Centre, Stockholm, Sweden Thursday, July 30, 2009, 13:00-15:00 local time, 11:00-13:00 UTC CHAIRS: Jason Fischl, Jean-Marc Valin NOTE TAKERS: Olle E. Johansson and Bruce Lowekamp JABBER SCRIBES: Alan Hawrylyshen and Peter Saint-Andre MINUTES EDITOR: Peter Saint-Andre AGENDA: 1. Administrivia (5 minutes) 2. Introduction and Scoping of the BoF (10 minutes) 3. Goals of the Working Group (15 minutes) 4. Engineering Work (25 minutes) 5. Codec Expertise in the IETF (15 minutes) 6. Gauging Likelihood of Success (15 minutes) 7. Call for Consensus on Whether to Form a WG (15 minutes) 8. Charter Discussion (15 minutes) 9. Conclusions and Next Steps (5 minutes) AUDIO: JABBER LOG: EDITOR'S NOTE: This was a very fast-pased BoF with a great many speakers at the mic and much back-and-forth conversation. Although the note-takers have attempted to capture all of the discussion, we cannot guarantee that the minutes are truly complete. Please refer to the audio stream and the Jabber logs for details. Approximate minute markings have been provided for reference, with time in UTC to synchronize with the Jabber logs. ======================================================================== 1. ADMINISTRIVIA (11:06) 1a. Note well slide presented. 1b. Remote logistics, Jabber room, meeting materials, audio, etc. 1c. Volunteers noted for note takers and Jabber scribes. 1d. Agenda bashing. Chairs note one modification: order of items 7 and 8 switched. Roni Even @ mic: The agenda topic "Engineering Work" does not explain what exactly will be discussed. Codecs or requirements? Jason Fischl (Chair): The topic will be the *kind* of work that we would be doing. We will look at some attributes that we would use as a way of evaluating codecs, then show how the 5 codecs that have been submitted would address those attributes. We will have short presentations (5 minutes) about individual codecs that have been submitted to indicate the kinds of work that might be in scope, but these codecs are merely representative. ======================================================================== 2. INTRODUCTION AND SCOPING OF THE BOF (11:11) 2a. Cullen Jennings (AD) presenting * SLIDE 1: Things for the IESG/IAB to Learn - Is there a well-defined goal? - Do standardized solutions exist? - Is it technically feasible to do this work in a reasonable time? - Are there people that can do this work in the IETF? - If the work is completed, would it be used? * SLIDE 2: Patent Related IPR at the IETF - If formed, working group would function within IETF IPR policy. - Not looking to change IETF IPR policy. * SLIDE 3: BCP 79 - [quotation from BCP 79] * SLIDE 4: Goal is Royalty Free - from BCP 79: "In general, IETF working groups prefer technologies with no known IPR claims..." * SLIDE 5: No Mandate of Royalty Free - from BCP 79: "... working groups have the discretion to adopt technology ... with no licensing commitment, if they feel that this technology is superior..." Xavier Marjou @ mic: Question about overlap of standardization bodies, referring to RFC 2418 (BCP 25). Cullen Jennings @ mic: Yes, it overlaps and we have a procedure for handling this with other liaison SDOs. Generally the IAB focuses on this. We need to make sure that our processes are consistent with that BCP, and we're doing that. We have already received two liason requests from two relevant organizations. New work is discussed on a mailing list read by other SDOs and issues are addressed between SDOs and IAB before deciding to form a working group. Patrik Fältström @ mic (as liaison to the ITU-T): I am the liason to the ITU-T and I am working together with the ITU-T on this. ======================================================================== 3. GOALS OF THE WORKING GROUP (11:18) 3a. Chairs on the goals Jean-Marc Valin presenting... * SLIDE 1: Today's Codec Landscape Jean-Marc (presenting): A representative list of some relevant codecs received from the ITU-T liaison. Henry Sinnreich @ mic: Why is the SILK codec not in the list? Jason Fischl (Chair): The list is not comprehensive. [unidentified] @ mic: There might be others that are royalty-free. Jean-Marc Valin presenting... * SLIDE 2: Applications Ted Hardie @ mic: How fixed is this list? Is this application list comprehensive or might there be others? AppArea might have a different set of requirements. Slava Borilin @ mic: Just an initial list to given an idea of the scope. Defining a list of target applications would be part of the working group's task, if it is approved. Roni Even @ mic: The charter says only doing "interactive" application, but not "streaming" applications (which might be similar). Ted Hardie @ mic: Is this list constraining or not? Jason Fischl (Chair): This list is intended to be a non-constraining list of motivating examples. Roni Even @ mic: I understood from the charter that there are some limitations. Jason Fischl (Chair): There is a slide later that defines what is out of scope. Joe Hildebrand @ mic: There are a number of AppArea people here as well and folks should take requirements from applications into account. Jason Fischl presenting... * SLIDE 3: Goals - Widely and easily distributable and accessible standard codecs - Small number of codecs (1 or 2) that cover the operating space Jason Fischl (presenting): Not proposing to start a "codec factory". Stefan Bruhn @ mic: What is so unique with these goals for the IETF? There are other SDOs who do work in this area. Why are we reformulating these goals here in the IETF? Harald Alvestrand @ mic: Is the operating space meant to be interactive voice as opposed to stored voice? Jean-Marc Valin (Chair): The scope is meant to be interactive audio, not just voice. Monty Montgomery @ mic: Within the IETF we would be solving problems that IETFers have. Also gives us an opportunity for cross area review. Feedback from application developers, transport protocol designers, etc. Stefan Bruhn @ mic: I don't understand the motivation. We have the MPEG (system agnostic), ITU-T (fixed line), 3GPP (wireless transports), so we have the whole range, I don't see why we need to address this in the IETF. What is so specific here to the Internet that it can't be addressed in MPEG, ITU-T, or 3GPP? Peter Saint-Andre @ mic: The IETF has an open participation model in which the entire Internet community can participate, many other SDOs and industry consortia do not have that model. Monty Montgomery @ mic: The development model in the IETF culturally matches the way we work in the developer community. BCP 79 avoids an accidental, systemic bias against producing royalty-free codecs. Here we have a fighting chance of doing something truly open. Joe Hildebrand @ mic: It's a valid point that applications considered at the IETF have requirements that applications considered by other SDOs don't have. Colin Perkins @ mic: I was chair of AVT WG. You need to pay attention to the way the Internet works. I don't know if that is an argument about whether to form a codec working group, but it is an argument that those who design codecs need to interact with those who understand Internet protocols. One could potentially make the argument that those who work on codecs should interact heavily with the IETF. Maybe that doesn't happen in other SDOs. Christian Hoehne @ mic: Need to consider other requirements than have traditionally been considered by other SDOs. Colin Perkins @ mic: I'm not saying that other SDOs have not been involved in that kind of interaction with the IETF, but that it doesn't always happen. Roni Even @ mic: IETF should produce requirements for submission to other SDOs, which have the expertise to do the codec work. As to open participation, anyone is welcome at ITU-T rapporteur meetings who is approved by rapporteur (but they cannot vote). Xavier Marjou @ mic: I confirm what Roni just said. I also add that there is a liaison relationship between ITU-T and IETF, and the ITU-T is open to taking on new work. Anisse Taleb @ mic: The ITU-T or even MPEG are not less open than the Internet community. Individuals are invited to submit requirements, and MPEG calls for proposals are open to companies that are not members of MPEG. I agree as well with Stefan Bruhn -- I don't see what is different here in terms of requirements. Joe Hildebrand @ mic: Definition of open used at IETF does not overlap with needing to be invited to participate. Roni Even @ mic: You need to pay to attend an IETF meeting. Joe Hildebrand @ mic: These meetings are not canonical, it is the open mailing lists that are canonical. Roni Even @ mic: It is the same in the ITU-T. Patrik Fältström @ mic (as ITU-T liaison): I'm a bit concerned about the language being used at the mic. We have agreement between ITU-T and IETF on how to communicate and how to cooperate regarding the possibility of doing overlapping work. We are following that process in the IETF and the ITU-T is following its process. It is not part of the process to have the kind of discussion we are having in the room, so please stop and let us move on to more productive topics. Jean-Marc Valin presenting... * SLIDE 4: Technical considerations - suitability for most fixed-point digital signal processors - medium- to high-quality speech at moderate bit-rates - high-quality music encoding at higher bit-rates - sampling rate profiles from narrow-band to full audio bandwidth - real-time adaptive bit-rate - source-controlled variable bit-rate (VBR) - low algorithmic delay Christian Hoehne @ mic: What's the background behind the choice of the chosen technical considerations? This looks like a description of CELT and SILK. Jean-Marc Valin (presenting): These considerations were originally posted in the first proposed charter and adapted based on feedback. They are by no means final and are open to modification. Roni Even @ mic: This is a typical list for codec requirements, nothing specific here to the Internet. Jason Fischl (Chair): These are technical considerations, there are other Internet-related considerations. Ted Hardie @ mic: Is the goal to "pick a winner" (just one or two). Is there a desire to avoid transcoding? We might want that for technical reasons, such as avoiding delay or preventing fragility. But let's be careful that the way to avoid transcoding is not to pick a winner because then we have a collusion issue. Must be very open about what our needs are to avoid the appearance of collusion. If this is not the issue, they do we really need to avoid the codec factory approach? Colin Perkins @ mic: I agree with Roni Even -- there is nothing here that is particular for the Internet. Surely there must be technical factors from the Internet transport that are relevant here. Francois Audet @ mic: I have a similar point. How does this relate to SIP or SDP? Issues like end-to-end negotiation of SDP parameters are important. In-band or out-of-band? Not good to take the most general approach. What are the IETF specifics? Identify what things IETF is best at and include those in the scope. Monty Montgomery @ mic: Transcoding points are excellent. It would make the system better to avoid transcoding. Second point, those technical considerations are items that are interesting to those of us who want to do the work, and this work is not being done elsewhere. Christian Hoehne @ mic: We haven't discussed transcoding because of the end-to-end principle for IP packets. If you have e2e instead of some other network principle (e.g., circuit-switched), then there's no need for transcoding. Mikael Abrahamsson @ mic: I'm a network engineer. I spend a lot of time debugging VoIP traffic. Jitter, latency, packet reordering, packet drops are Internet issues that are not commonly handled by codecs and that cause codecs to lack robustness. I'm baffled that this is missing from an IETF codec initiative. Jean-Marc (presenting): This is not a complete list. Julien Meuric @ mic: There are existing implementations that meet all these requirements. These considerations are not IETF-specific. Why not use or extend existing standards-based solutions? Jason Fischl (Chair): Could you be more specific? Julien Meuric @ mic: Transcoding not mentioned. There are standardized tools that make this possible. What's the new issue that needs to be solved? Slava Borilin @ mic: Existing codecs don't meet these requirements, either not easily distributable or not high quality. I agree that the Internet-specific requirements need to be included. Those of us who worked on the slides probably assumed that the Internet-specific issues were so obvious that unfortunately we omitted from the slides. New work will potentially be an extension of something existing. Anisse Taleb @ mic: These are kind of default considerations for codec work anywhere. What particular features would an Internet codec have? Be more specific about what requirements existing codecs do not meet. Need evidence. Robustness to frame losses is an important technical consideration, for example. Jean-Marc Valin (presenting): The requirement for robustness to frame losses is in the charter but we cut it off the list to fit on the slides. Anisse Taleb @ mic: Best way to avoid transcoding is to stop standardizing codecs. Keith Drage @ mic: You don't have a slide about non-goals. Jason Fischl (Chair): We do, it's next. Slava Borilin @ mic: Two requirements, easily distributable and good performance. Those are not met now. Jason Fischl (Chair): Let's move on to the next couple of slides, then have more comments. Jean-Marc Valin presenting... * SLIDE 5: What is out of scope? - Very low bitrate - Non-interactive music (e.g., MP3, AAC, Vorbis) - Unlimited number of codecs Jason Fischl presenting... * SLIDE 6: Why do this here? - IETF is where you find the people who understand the impact of the IP stack on codec design - Cross-area review - Whole Internet Community can participate - IETF will have change control and the resulting codecs will be different and better than the initial candidates Stefan Bruhn @ mic: Are you saying that in other SDOs there are no people who understanding the IP stack? Jason Fischl (presenting): Not saying anything about other SDOs, the only statement is that we have such people at the IETF. Stefan Bruhn @ mic: I don't see what is so unique in the IETF. We have all these things (cross-area review, whole Internet community can participate) in other SDOs. Keith Drage @ mic: I want to make sure there is a non-goal that we're not trying to define the default codec for the Internet, and that the resulting codec is not going to be mandated. Roni Even @ mic: I don't see codec expertise on this slide. Also, IETF doesn't mandate codec, this is done by other forums that create profiles. Free codecs are available. Why do you need a standard codec? Alan Duric @ mic: As an operator, I've been struggling for 6 years to deploy VoIP and none of the vendors want to deploy Speex because it's not a standard. If we don't have this work done as a standard, operators will not be able to convince vendors to deploy it. G.722 is falling apart on IP. Xavier Marjou @ mic: We have already deployed a wideband audio codec over IP. I think IP considerations are important, but other areas come into account in codec design. Alan Duric @ mic: Give me the evidence regarding G.722. Please post to mailing list what these codecs are that meet these requirements. Adam Roach @ mic (quoting from XMPP BoF minutes held at IETF 54): "We should be chartering work based on the number of folks who want to do things, not the number opposed." I think we've heard from a number of people here who want to do the work and have competence in this area. [unidentified] @ mic: [Comment about G.719?] Patrik Fältström @ mic (as liaison to the ITU-T): Regarding the comment that we should start work if there are enough people willing to do the work: there are two different discussions here. One is the IETF/IESG decision about whether to create a working group. There is a separate discussion between the IETF and the ITU-T according to RFC 3356, the agreed-upon procedure between IETF and ITU-T to determine if there is overlapping work. That discussion is happening and will continue after this BoF. Ingemar Johansson @ mic: There is no problem with interaction between SDOs. Let's define payload formats here and leave codec work to the ITU-T. Xavier Marjou @ mic: Why are people in the IETF saying that standards coming from other SDOs a joke? Does putting "IP stack" in the slide make the IETF qualified? Monty Montgomery @ mic: Regarding why to do this within the IETF: there are people who want to do this, are doing this, and who want to do it through the IETF. That's not true elsewhere. Christan Hoene @ mic: Things have to work together. This is a reason to do the work here. Are there enough Internet, codec, and VoIP experts sitting together in the same room? That is the question, and we need to decide whether that can be done in the IETF, between the IETF and ITU-T, or in some other way. I think it can be done in the IETF, but no matter what we need to start this work. Slava Borilin @ mic: Regarding why IETF and why standardized codec: Because IETF management of change control is important for vendors. Once change control moves from corporation / vendor to the IETF it becomes safer for others to implement and deploy. The IETF has a group of people that are willing to contribute, skilled and able to contribute. We have expertise here. The codec has to be developed with all aspects of the transport and IP stack considered for high quality on the Internet. Existing codecs do not address that factor, and the few codecs that are close to meeting those goals can't be freely distributed. We have goals of freely distributable and high quality in the real Internet. The IETF is the best organization that fits meeting both of those goals. Jason Fischl @ mic (as individual): How to we get wideband codecs broadly adopted? G.722 is brought out as an example and is starting to become actively deployed. The trigger is expiration of the patents. We haven't seen widely deployed codecs before. Widely distributed and freely available codecs lead to broad deployment. The IETF can try to achieve that, although there is no guarantee. Wilhelm Wimmtreuter @ mic: We have people willing and able to contribute. It matters who can fund it and support it. Why not let them do it? Stefan Bruhn @ mic: I doubt that you have the expertise in this group. Not traditionally at the IETF. Why are you not proposing this to the ITU-T, even royalty-free, and face competition? Slava Borilin @ mic: Because most of our stakeholders are here in the IETF. Jean-Marc Valin presenting some slides about codecs that have been submitted as Internet-Drafts so far... * SLIDE 7: CELT Codec Characteristics * SLIDE 8: SILK Codec Characteristics * SLIDE 9: IPMR Codec Characteristics * SLIDE 10: BV32 Codec Characteristics * SLIDE 11: A2DP SBC+PLC Dave Oran @ mic: Loss robustness is not a boolean. And it is not a scalar. Loss is a curve, not a scalar value. Colin Perkins @ mic: How is it relevant to the Internet that we have a list of codecs? How do these codecs solve Internet-specific problems? Roni Even @ mic: What is the point of a list of codecs? Jason Fischl (Chair): Trying to point out that there were a number of submissions of real codecs. Roni Even @ mic: That's interesting, but there are no requirements so everyone is qualified. Peter Saint-Andre @ mic (as individual, not Jabber scribe): People say there is no competence in the IETF. The point of the list of codecs is saying that we do have competence among the people who have developed these codecs and contributed them to the Internet Standards Process. Xavier Marjou @ mic: What is really new in terms of new requirements? Jason Fischl (Chair): These have been submitted in the open and intended to be open source. Xavier Marjou @ mic: For a new codec there are likely patents coming. Colin Perkins @ mic: I question the claim that they are royalty free. Jason Fischl (Chair): Claim is not that they are royalty-free, but that they are being provided without royalties on any of the patents. [?] Slava Borilin @ mic: People are contributing codecs that are believed to the best of their knowledge to be royalty free, and these people are contributing / have expertise to contribute royalty free codecs that are superior to existing alternatives. Based on what we know, we stand a good chance of doing better than what is out there, and keeping it freely available. We believe that this can be achieved such that quality is as good or better than existing codecs, and to do so in a way that is freely and easily distributable. Alan Duric @ mic: As to competence, there are people in this room who have developed codecs that run into the hundreds of millions of minutes each day or each week. These codecs are superior to the existing codecs out there and none of those are suitable for Internet or wireless LAN if there is any kind of packet loss. Peter Saint-Andre @ mic (as individual, not Jabber scribe): Could we please try to get through the agenda? ======================================================================== 4. ENGINEERING WORK (12:22) Jason Fischl (Chair): Quick presentations on SILK and CELP... 4a. Koen Vos presenting about the SILK codec (12:22) (draft-vos-silk-00) 4a. Monty Montgomery presenting about the CELT codec (12:24) (draft-valin-celt-codec-00) "If CELT comes out of a proposed working group intact, I will be sorely disappointed." Emphasizes the point that all of these codec developers want to merge their work to produce a best-of-breed codec. ======================================================================== 5. CODEC EXPERTISE IN THE IETF (12:27) Jason Fischl (Chair): These brief presentations were trying to establish that we do have codec expertise among IETF participants, but there are also other kinds of contributions -- requirements, testing, etc. Joel Halpern @ mic: I'm not going to question whether there are enough people here to do the codec work, because that's clearly true. The question is whether the *IETF* has the expertise. It's only sensible for the IETF to do it if it works with the rest of the process. Can the reviewers provide appropriate technical insights? Does the IESG have expertise to provide oversight? If we don't have expertise in those places, we can't do this as an organization. Henny Sinnreich @ mic: I'm willing to contribute with usage scenarios for Rich Internet Applications and the resulting requirements. Slava Borilin @ mic: We definitely have enough people to work on the codecs themselves, but do have enough qualified customers? I think so. Colin Perkins @ mic: I do not believe the wider IETF can meaningfully review the work (IESG and other areas). Bernard Aboba: We have chartered IETF working groups in this kind of situation before. For example, TRILL. We only looked at whether we had people who could do the work. What we put into the charter is review by appropriate outside experts and liaisons (in that case IEEE) to get comments. We have been successful in that approach. The only thing that was relevant is whether people are here with expertise and willing to do work. Roni Even @ mic: It's more complicated. You need to provide information correctly to liaisons and outside experts. For example, need to work on characterization. Christan Hoene @ mic: There need to be experts on judging the quality of the codecs. For example, there are standards for conducting listening tests. Jason Fischl (Chair): I want to make sure we are very focused on the question of whether we have the expertise to do this work in the IETF. Jean-Francois Mule @ mic: I do believe there is expertise. Anyone who has done codec development and testing knows how to define test vectors so that implementers can validate implementations. In this room we have 4-5 companies with codec test bench labs that could validate that the codec has been implemented correctly according to the spec and also that you have met your terms of reference and requirements. Also independent labs. There are plenty of resources, and this is not rocket science. Slava Borilin @ mic: At Spirit DSP have expertise in design and we are engaged on the Internet application side. A working group can serve most of all as an analyst to specify the right tasks for developing the kind of codec that is needed for Internet applications. The major value is identifying the technical and business requirements and developing technical solutions to meet those requirements. The working group might even end up with a decision that G.722 is the best choice. Monty Montgomery @ mic: Two hours ago the argument was that we didn't have the competence. That has shifted to "you are competent to design a codec but you don't have the competence to test". In my experience, testers outnumber implementers 10 to 1. This is irrelevant. Adam Roach @ mic for Eric Rescorla in the Jabber chatroom: I'm not concerned about Joel's point. We have similar problems in security, where the ability to evaluate cryptographic techniques outside the working group is extremely limited. But at least the performance of codecs is measurable, whereas with security even that isn't true. Adam Roach @ mic: At least codecs can be evaluated directly by everyone in the IETF, since we all have working ears. Stefan Bruhn @ mic: Do we have expertise? Depends on the companies sending their experts to this group. Experts are sent to ITU-T and MPEG and 3GPP, are you sure they will be sent here? For experts here, why aren't you at MPEP and ITU-T and 3GPP? Are you afraid of facing competition? Derek McDonald @ mic: It seems that we do have the expertise. But the IETF retains the right to fail. It isn't like we've lost a war if the working group fails. Don't let FUD block us. Mikael Abrahamsson @ mic: The Internet should demand things from the codec rather than the codec demanding things from Internet. We know IP, we know how the Internet works. Alan Duric @ mic: I fully agree, and if we brought this to ITU-T or ETSI, yes these proposals were brought forward 8 years ago and shot down without consideration. Julien Meuric @ mic: People are interested, but do you have any proof that the technical skill is present here? Roni Even @ mic: We are trying to define the whole process, need expertise in defining the whole process. But really need a beauty contest. Either approve all the specs that are submitted or you need to select a few and define how to do that. Xavier Marjou @ mic: IETF documents are good because they are so readable, but a codec RFC would be unreadable. Can't write formulas in an Internet-Draft. Jean-Francois Mule @ mic: Responding to request for evidence, Broadcom is willing to bring expertise and competence. Dr. Chen and his group have worked for many years in ITU-T. Slava Borilin @ mic: Spirit DSP has developed more than 10 proprietary codecs standardized under various bodies like radio standards. We are not going to ITU-T because there are no stakeholders interested in the result (freely distributable, high quality). In IETF there are. Monty Montgomery @ mic: First, we will mix inputs from different contributors. Second, Roni is right that a lot of characterization, testing, and documentation needs to be done. We're aware of that and we have a lot of work to do. Ye-Kui Wang @ mic: Who in IESG is capable of reviewing audio codecs? Ted Hardie @ mic: Please stop talking about stakeholders -- not a useful construct in the IETF. The decision is whether to charter work that would benefit the Internet. Serious question is whether this is the best place and whether this work benefits the Internet. Stakeholders are users, operators, vendors, etc. Please try to take a whole Internet view rather than a particular stakeholder view or we will ask ourselves the wrong question. Cullen Jennings @ mic, as AD who will be asked to evaluate codecs: Do we have competence on this topic? The IESG relies a lot on expert advisers across areas. We try to assess whether things cause harm. We have to listen to other people and try to evaluate whether claims of brokenness are legitimate or not. Often our expertise lies in finding the right people to provide input and in listening to them in order to judge which input is reliable. For example, will have to review IDNAbis (internationalized domain names) but there are no linguistics experts on the IESG, so we will have to rely on other people and work with them and try to integrate their feedback. Another factor is that IESG members are selected based on skills needed to do the work and review it. This codec work is really not different from other work and the important thing is that there's expertise to get the right comments so the IESG can make good decisions. [12:50, T-10 minutes] QUESTION from Chairs: Who here would call themselves codec experts and would be willing to develop codecs? Outcome: 10-12 real hands and 5+ virtual hands from the Jabber room. QUESTION from Chairs: Who is willing to participate in any way listed on slide 1 (guidelines, process, requirements, codec proposals, commenting, evaluation)? Outcome: 40-60 real hands and virtual hands. Lars Eggert @ mic: Of the codec experts, would you still participate if a proposal you contributed is ruled out by the working group? Outcome: Same as for the first question. HUM REQUEST from Chairs: Is or is not the technical scope of the work well defined? [one hum for yes, one hum for no] Outcome: Inconclusive. HUM REQUEST from Chairs: Do we or do we not have enough expertise to form a working group that can meet the goals? [one hum for yes, one hum for no] Outcome: Inconclusive. Ted Hardie @ mic: Large grey area here, what if you have no idea if there's enough expertise to complete the work because you don't know if it's well-scoped in the first place? Perhaps a separate hum is in order. Cullen Jennings @ mic: Could you type the question in a slide? My experience from other BoFs is that the question can get very confusing. [12:58 conclave at front of room among Chairs, AD, BoF Sponsor, etc.] HUM REQUEST from Chairs: Do we or do we not have enough expertise to properly scope the work? [one hum for yes, one hum for no] Outcome: Rough consensus in favor. [conclave continues] Cullen Jennings @ mic as Area Director: - I think we've identified that it's prematuve to ask whether to form a working group - We have the information we need from the BoF and will go forward with discussions on the mailing list [meeting adjourned at 13:01] ======================================================================== 6. Gauging Likelihood of Success - not discussed - ======================================================================== 7. Call for Consensus on Whether to Form a WG - not discussed - ======================================================================== 8. Charter Discussion - not discussed - ======================================================================== 9. Conclusions and Next Steps - not discussed - END