Skip to main content
  • IETF mailing lists delivery issues resolved

    During a period from May 6 to May 9, a number of messages intended for IETF, IRTF, IAB, IESG, and RFC-Editor email lists were accepted by email services, but not forwarded to the list members or the list archives. All identified messages have now been processed by intended mailing lists for delivery to current list subscribers, and will appear in list archives.

    21 May 2024
  • Experimental survey of meeting non-returners

    We are experimenting with a new survey to help understand why people participate in one IETF meeting but not the meeting following.

    26 Apr 2024
  • IETF Community Survey 2023

    The final report on the IETF Community Survey 2023 is now available.

    25 Apr 2024
  • IETF Snapshot 2023

    Want to catch up on IETF activity in 2023? The IETF Snapshot provides a short summary of IETF activity for the previous year.

    17 Apr 2024
  • UN Report Calls for New Era for Digital Governance in which Tech Standards Respect Human Rights

    In a major milestone in the movement to consider human rights impacts of technology, the United Nations High Commissioner for Human Rights details the obstacles and opportunities posed by technical standards setting to the enjoyment of human rights in a new report.

    16 Apr 2024

Filter by topic and date

Filter by topic and date

IETF 119 post-meeting survey

12 Apr 2024

IETF 119 Brisbane was held 16-22 March 2024

The results of the IETF 119 Brisbane post-meeting survey are now available on a web-based interactive dashboard. Thank you to all of you who responded to this survey as we use your views to continually adjust the meeting experience.

IETF 119 Plenary 2

Analysis

We received 255 responses, 254 of whom participated in IETF 119, 190 onsite and 64 remote. With 1206 registered participants, this gives the survey a maximum margin of error of +/- 5.47%.

The results for satisfaction questions include a mean and standard deviation using a five point scale scoring system of Very satisfied = 5, Satisfied = 4, Neither satisfied nor dissatisfied = 3, Dissatisfied = 2, Very dissatisfied = 1. While there’s no hard and fast rule, a mean of above 4.50 is sometimes considered excellent, 4.00 to 4.49 is good, 3.50 to 3.99 is acceptable and below 3.50 is either poor or very poor if below 3.00. The satisfaction score tables also include a top box, the total of satisfied and very satisfied, and a bottom box, the total of dissatisfied and very dissatisfied, both in percentages. Please note that a small number of questions are on a four point scale.

Question changes since the last survey

For this survey we added a new question about the block on submitting I-Ds two-weeks before a meeting, and some new questions about time spent in Australia at the request of the local tourism agencies that offered us funding for this meeting. In order to fit in those questions we dropped the matrix questions about each individual part of the agenda and structure of the meeting and side meetings.

Actions taken following the last survey

For this meeting, we made the following changes, prompted by survey feedback:

  • The registration desk now opened on the Saturday and we no longer had a separate registration for Hackathon participants.
  • The onsite badge was changed to card stock for a lower environmental impact and redesigned for better visibility.
  • We implemented a new shared workspace, replacing the separate lounge and terminal room.

Satisfaction

Overall satisfaction is 4.24, down on recent meetings but still a good result. With some key exceptions, the satisfaction scores remain high, reflecting the various improvements made since we returned to onsite meetings.

The table below shows the satisfaction scores for the last six meetings, along with colour coded indicators for the five point scale above: excellent (πŸ”΅), good (🟒), acceptable (🟑), poor (πŸ”΄), very poor (⚫️)

Satisfaction scores for the last six meetings
IETF 119 Brisbane IETF 118 Prague IETF 117 San Francisco IETF 116 Yokohama IETF 115 London IETF 114 Phila.
Overall satisfaction 4.24 🟒 4.42 🟒 4.30 🟒 4.30 🟒 4.28 🟒 4.19 🟒
AGENDA
Overall agenda 4.05 🟒 4.20 🟒 4.16 🟒 4.18 🟒 4.22 🟒 4.06 🟒
Sessions for new WGs - - 4.19 🟒 4.17 🟒 4.12 🟒 4.15 🟒
Sessions for existing WGs - - 4.22 🟒 4.22 🟒 4.22 🟒 4.10 🟒
Sessions for WGs - 4.32 🟒 - - - -
BOFs - 4.12 🟒 3.95 🟑 4.11 🟒 4.10 🟒 4.09 🟒
Sessions for existing RGs - - 4.12 🟒 4.14 🟒 4.10 🟒 3.95 🟑
Sessions for RGs - 4.09 🟒 - - - -
Plenary - 3.94 🟑 3.99 🟑 3.98 🟑 3.98 🟑 3.98 🟑
Side meetings - 3.79 🟑 3.75 🟑 3.73 🟑 3.81 🟑 3.73 🟑
Hackathon - 4.27 🟒 4.25 🟒 4.34 🟒 4.35 🟒 4.30 🟒
HotRFC - 3.90 🟑 3.89 🟑 3.84 🟑 4.21 🟒 3.94 🟑
Pecha Kucha - 4.00 🟒 4.15 🟒 - - -
Office hours - 4.07 🟒 3.98 🟑 4.23 🟒 4.00 🟒 4.09 🟒
Opportunities for social interaction - 4.06 🟒 4.11 🟒 3.72 🟑 3.98 🟑 3.89 🟑
STRUCTURE
Overall meeting structure 4.15 🟒 4.34 🟒 4.28 🟒 4.28 🟒 4.28 🟒 4.19 🟒
Start time - 4.49 🟒 (9:30am) 4.28 🟒 (9:30am) 4.16 🟒 (9:30am) 4.28 🟒 (9:30am) 4.20 🟒 (10:00am)
Finish at 5pm Friday - 3.65 🟑 - - - -
Length of day - 4.20 🟒 4.30 🟒 4.30 🟒 4.32 🟒 4.10 🟒
Number of days - 4.18 🟒 (5+2) 4.27 🟒 (5+2) 4.30 🟒 (5+2) 4.32 🟒 (5+2) 4.30 🟒 (5+2)
Session lengths - 4.38 🟒 (60 / 90 / 120) 4.41 🟒 (60 / 90 / 120) 4.36 🟒 (60 / 90 / 120) 4.32 🟒 (60 / 90 / 120) 4.25 🟒 (60/120)
Break lengths - 4.38 🟒 (30/90) 4.32 🟒 (30/90) 4.38 🟒 (30/90) 4.36 🟒 (30/90) 4.25 🟒 (30/90)
Number of parallel tracks - 3.94 🟑 (8) 4.08 🟒 (8) 4.01 🟒 (8) 3.90 🟑 (8) 3.86 🟑 (8)
PARTICIPATION MECHANISMS
Meetecho 4.50 πŸ”΅ 4.47 🟒 4.35 🟒 4.45 🟒 4.45 🟒 4.23 🟒
Gather - - 3.52 🟑 3.46 πŸ”΄ 3.37 πŸ”΄ 3.06 πŸ”΄
Zulip 3.86 🟑 3.94 🟑 3.66 🟑 3.77 🟑 3.73 🟑 3.56 🟑
Jabber - - - - - -
Audio streams 4.27 🟒 4.27 🟒 4.02 🟒 4.21 🟒 4.04 🟒 4.05 🟒
YouTube streams 4.24 🟒 4.36 🟒 4.32 🟒 4.36 🟒 4.25 🟒 4.22 🟒
CONFLICTS
Conflict avoidance 3.88 🟑 4.00 🟒 3.90 🟑 3.94 🟑 3.91 🟑 3.78 🟑
VENUE & ACCOMM
Overall accommodation - 4.34 🟒 4.07 🟒 4.09 🟒 - -
Overall venue - - 3.90 🟑 - - -
Location - 4.52 πŸ”΅ 3.60 🟑 - - -
Venue facilities - 4.43 🟒 4.07 🟒 - - -
Cost of rooms - 3.54 🟑 2.87 ⚫️ - - -
Availability of rooms - 4.32 🟒 4.07 🟒 - - -
ONSITE
Overall 4.37 🟒 4.50 πŸ”΅ 4.29 🟒 - - -
Badge collection 4.71 πŸ”΅ 4.70 πŸ”΅ 4.69 πŸ”΅ - - -
WiFi 4.09 🟒 4.17 🟒 3.98 🟑 4.06 🟒 4.10 🟒 3.82 🟑
QR Codes 4.31 🟒 4.18 🟒 4.11 🟒 - - -
Break F&B 3.75 🟑 4.05 🟒 4.44 🟒 - - -
Breakout seating 3.93 🟑 3.80 🟑 4.08 🟒 - - -
Signage 3.96 🟑 4.15 🟒 4.22 🟒 - - -
Coffee carts 4.08 🟒 4.64 πŸ”΅ 4.56 πŸ”΅ - - -
Childcare 3.76 🟑 4.12 🟒 4.06 🟒 - - -
Welcome reception 3.97 🟑 4.08 🟒 - - - -
Farewell reception 4.15 🟒 4.34 🟒 - - - -

Special questions

Pre-meeting block on I-D submission

In this survey, a special question was asked "Internet-Drafts cannot be submitted or updated in the two weeks before the meeting (unless an AD approves an exception). Please rate how much you agree with the following:". With the caveat that the following results only reflect the views of 250 people while the total number of active IETF participants is over 7,000, the results of this questions indicate strong support for the current practice and either leaving it as is, or allowing WG chairs to also approve exceptions.

The full results are detailed in the table below:

Ratings of statements related to two-week prohibition
Coding Mean Strongly disagree Disagree Neutral Agree Strongly agree
This rule is helpful for my preparation 3.89 2.67% 7.11% 20.00% 39.11% 31.11%
This rule should stay as it is 3.81 4.95% 5.41% 22.07% 39.19% 28.38%
This rule should stay but WG chairs should also be allowed to approve exceptions 3.33 9.05% 12.67% 27.15% 38.46% 12.67%
This rule should be extended to other methods for updating I-Ds, such as GitHub merges 2.89 15.84% 21.29% 32.18% 19.31% 11.39%
This rule should stay but for a shorter period than two weeks 2.50 18.18% 33.64% 30.91% 14.55% 2.73%
This rule should be removed entirely 2.01 42.72% 27.70% 20.66% 3.76% 5.16%

Why people participated remotely

For the third time, we asked people who participated remotely (Q5), why they did and if they would have preferred to participate onsite (Q5a). Once again, the major factor, cited by 75% of people, was the lack funding to travel.

Funding

We asked the question "What was the primary source of funding for your onsite participation?", the results of which were:

  • "My employer" 75.14% (133 respondents)
  • "A grant or bursary" 9.04% (16 respondents)
  • "My own business" 5.65% (10 respondents)
  • "My personal funds" 5.08% (9 respondents)
  • "One or more clients" 5.08% (9 respondents)
  • "Other (please specify) - no respondents selected this option

Good results

Meetecho

The satisfaction score for Meetecho continues to slowly move up, though not in a statistically significant way, and for the first time Meetecho was rated as Excellent at 4.50.

Registration desk / badge collection process

This again achieved an exceptionally high score at 4.71.

Areas for improvement

Signage

This scored lower than previous meetings and analysis of the comments seems to indicate that the reason for this is because this was a very large space and therefore much harder to navigate than normal, which our signage did not adequately cover. We also made some changes to the signage, which we need to review.

Side meetings

While an overall satisfaction score for side meetings was unintentionally omitted, the satisfactions scores for individual elements (Q71) remain low. There are also multiple comments (Q16) about problems with side meetings.

Conflict avoidance

Again, satisfaction scores with this remain low and further work is needed to devise a new strategy to improve this.

Breakout seating

While there was plenty of breakout seating at this venue, it was generally a bit of a walk to get to it, indicating that our previous analysis is correct that participants want this immediately outside of meeting rooms.

Break food and beverage

We intentionally provided less food in breaks in an effort to tackle the extensive food waste we have had a previous meetings, and that combined with a significant shortfall in meeting sponsorship funds, meant that only limited breakfast catering was provided. Together these resulted in a lower satisfaction score. Feedback from the comments seems to indicate that we need to provide less sweet snack food and more healthy options.

Barista coffee

Yes, sorry.

Individual comments

The individual comments (Q9, Q16, Q27) cover a number of key themes, many of which have already been covered above:

  • Brisbane. Comments here are very mixed with some commenting that this was an excellent venue and it was about time that we went back to Australia, with others complaining that it was both too far and a difficult timezone for remote participation.
  • I-Ds and Agenda. There are multiple comments about how the timing of I-Ds and agenda affect effective preparation. Most of these ask for better adherence to the rules, while a few ask for more flexibility.
  • WiFi. There are multiple comments identifying issues people had with the WiFi during the meeting.
  • ALLDISPATCH. There are multiple comments about this, some in favour and some against, all of which will be passed to the IESG.

And finally

Thank you everyone who responded to this survey, your feedback is much appreciated.


Share this page