Skip to main content
  • IETF 111 Hackathon: Coding across time zones

    The IETF 111 Hackathon was held July 19-23, 2021. This was the 19th IETF Hackathon, and the 4th held as an online only event. For most people involved in the IETF the past several years, the IETF Hackathon marks the start of each IETF meeting.

    • Charles EckelIETF Hackathon Co-chair
    8 Sep 2021
  • IETF 111 post-meeting survey

    The results from our IETF 111 post-meeting survey are now available.

    • Jay DaleyIETF Executive Director
    23 Aug 2021
  • IETF Community Survey 2021

    In May 2021, the IETF Administration LLC (IETF LLC) on behalf of the IESG and in collaboration with the IAB distributed the first annual IETF community survey to all 56,000 addresses subscribed to IETF mailing lists. Its purpose was "To help better understand our community and its makeup, gather views on the IETF and how well it works for participants, and gain insight into how we compare to similar organisations".

    • Jay DaleyIETF Executive Director
    11 Aug 2021
  • Experiences from the first fully-online IAB workshop on Network Impacts of COVID-19

    The Internet Architecture Board (IAB) held its first fully-online workshop in November 2019, just before the IETF 109 meeting, to discuss the network impacts of the COVID-19 crisis.

    • Mirja KühlewindIAB Chair
    23 Jul 2021
  • Applied Networking Research Prize presentations at IETF 111

    Presentations on research into network specification and verification and on low-latency video streaming will be featured during the Internet Research Task Force Open session of the IETF 111 Online meeting scheduled for 26-30 July.

      22 Jul 2021

    Filter by topic and date

    Filter by topic and date

    IETF 111 post-meeting survey

    • Jay DaleyIETF Executive Director

    23 Aug 2021

    The results from our IETF 111 post-meeting survey are now available.

    The survey results for the IETF 111 post-meeting survey are now available on a web-based interactive dashboard.  We are always very grateful for the detailed feedback that we have received and will continue to process over the next few months.  The commentary below highlights a couple of areas where changes we have made based on feedback have been a success, and areas we still need to work on.  

    Analysis

    In total 166 responses were received.  Of those 164 participated in IETF 111 from a population of 1329 giving a margin of error of +/- 7.17%. The total numbers of meeting participants has grown (1196 for IETF 110, 1282 for IETF 109) while the survey response rate is significantly down on previous meetings (299 for IETF 110, 258 for IETF 109).

    The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1.  While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages.

    In this commentary a comparison is made with the IETF 110 Meeting Survey results using a comparison of means that assumes the two samples are independent even though they’re not but neither are they fully dependent.  A dependent means calculation may give a different result. A few comparisons are also made using a comparison of proportions.   

    Overall satisfaction

    The mean satisfaction score for IETF 111 (Q10) was 4.13 with 87% either ‘Satisfied’ or ‘Very satisfied’.  This is a statistically insignificant decline from the 4.20 mean satisfaction score for IETF 110.

    Preparedness

    Overall, preparation remains good with 83% reporting that they are either sufficiently prepared or well prepared. 4.35% were definitely under-prepared (Q8), well up on the 1.45% for IETF 110, which itself was down from 5% for IETF 109 but still a very small percentage of the total. The comments on this are mixed, with some pointing to the resources for practising with the tools, some bemoaning the lack of in-person meetings and some asking for WG chairs to provide the agenda earlier.

    Satisfaction with the agenda

    Overall satisfaction with the IETF 111 agenda was at 3.91 (Q12), a statistically insignificant decline from 4.04 for IETF 110 but now rated as 'acceptable' down from 'good'. Looking at the individual parts of the agenda (Q11), most of the satisfaction scores had no statistically significant change except for satisfaction with side meetings, which had a statistically significant improvement, and satisfaction with existing working groups, which had a statistically significant decline. Satisfaction with the opportunities for social interaction remains at 'very poor' and for side meetings, plenary and RGs it rates at 'acceptable'. For all other parts it rates 'good'.

    Satisfaction with the meeting structure

    Overall satisfaction with the structure of the IETF 111 meeting was 4.08 (Q14), a statistically insignificant drop from 4.20 for IETF 110. However, looking at the individual parts (Q13) there were statistically significant drops in satisfaction for the policy of scheduling online meetings in the timezone of the in-person meeting that they replace, for the overall length of the day and for starting at 12pm "local time", though this latter question was asked slightly differently this time. The first two are both rated as 'acceptable' and the latter is rated as 'poor' and only just above the boundary with 'very poor'. Satisfaction with 9 parallel tracks remains only 'acceptable' at 3.60 but for IETF 109 which has 8 parallel tracks, the score was 3.64 suggesting that a reduction to 8 would not improve satisfaction in any measurable way.

    Impact of being fully online

    In response to a hypothesis that moving fully online was allowing more people to participate in the IETF than would otherwise have been able to, a new question was added "If this meeting had been a normal (no COVID) in-person meeting in San Francisco with 8+ hour days and free remote participation then which of the follow best applies to you?" (Q15). Only 2 of the 148 who answered this question said that they would not have participated and 5 would have participated in fewer sessions, offset by 4 who would have participated in more sessions, suggesting that the hypothesis does not hold.

    Sessions

    41% experience no session conflicts (Q18) in IETF 111, up from both IETF 110 and IETF 109 at 35%, though a statistically insignificant increase. Satisfaction at conflict avoidance was 3.76 (Q20), a statistically insignificant increase from IETF 110 at 3.73. This again rates at 'acceptable' and suggests that more work is needed to get it up to 'good'.

    For this survey we omitted any questions on sessions running out of time to keep the survey short.

    Participation mechanisms

    Satisfaction with Meetecho remains 'good' at 4.29 (Q21) and only 'acceptable' for Gather at 3.77 and jabber at 3.68. After experimenting with Gather for four meetings we may need to experiment with alternative mechanisms for social interaction. Satisfaction with the audio streams and youtube videos both experienced statistically significant declines down to 3.84 and 4.09. Further investigation is required to understand this.

    Problem reporting

    Satisfaction with our response to problem reporting declined from 4.31 for IETF 110 to 4.00 (Q25) (the number of people responding to this question are so small that this is a statistically insignificant decline). From the comments it seems that much of this is down to confusion about what sessions require separate registration and the time it takes to fix that when someone try to join a session that is getting underway. A good improvement suggested in the text (Q26) is to put information on how to get support onto the agenda page.

    Final feedback

    Clearly, the majority just want to get back to in-person meetings with some using this survey (Q27) to complain about the decision to switch IETF 112 Madrid to an online meeting. Some are concerned that the lack of in-person meetings is making the IETF less relevant and less well supported by community volunteers.


    Share this page