Handling Ballot Positions
statement-iesg-handling-ballot-positions-20220121-00

Document Type IESG Statement
Title Handling Ballot Positions
Published 2022-01-21
Metadata last updated 2024-02-23
State Active
Send notices to (None)
statement-iesg-handling-ballot-positions-20220121-00

Handling Ballot Positions

21 Jan 2022

This document is written to help authors and chairs (especially newer authors and chairs) understand the purpose and meaning of IESG ballot positions and how best to respond to them. The most important piece of advice is “Don’t Panic” but authors may have to do something!

The IETF exists because of people like you, people who are passionate about an idea, a protocol, a process…a way to make things better. You’ve come up with a protocol or an idea that you care deeply about and think the IETF should work on. You wrote an initial Internet-Draft, you made a number of presentations, and your document was finally adopted as a Working Group (WG) document. You then spent many months in discussions in the WG, discussing many topics at length; you laughed, you cried, you fought over complex technical details. Hundreds of emails have been exchanged, many revisions have been made, compromises have been agreed to. A few of the “discussions” might have been contentious, and many seemed like bikeshedding. It was an arduous journey, and you cared enough to persevere and now -- now, the document is done.

Except, “done” is really just the beginning.

After the document has been completed, it goes through a Working Group Last Call (WGLC). A few issues may be discovered, along with some comments and a pile of nits. These get addressed, and then consensus declared. Next, the document goes through much of the same process but with a broader audience in the IETF Last Call (LC).

After going through IETF LC, and addressing those issues, comments, and nits, you may feel that the document is REALLY, REALLY ready to be published. It’s done, checked, double checked… Done. And, while close, there is one more point of review to make absolutely sure that the document, when published, is as clear, actionable, and useful as it can be. The goal of the Internet Engineering Steering Group (IESG) Evaluation is not to reopen old issues or nit pick. The goal of the IESG Evaluation is to ensure operability, readability and clarity. In this review, the IESG -- some of whom you may know and some of whom you probably don’t know -- review your document with an eye towards interoperability and then issue comments, including DISCUSS ballots, for your document. And you may think, at this point, “Now what?”

Some of these are DISCUSS ballots, which can range from meaning “I cannot in good conscience send this document forward, but if it were fixed in these ways, I would change my ballot position to either Yes or No Objection", to just literally meaning "I think we need to talk about this." Many of the comments might be regarding things you’ve already discussed and settled, many of them are nits, and some of them seem to be from people who have no real knowledge about the topic… Considering your broader ultimate audience, who may also not have any real knowledge about the topic and/or who are coming to your document without knowing how the sausage was made, the IESG provides you with your first “fresh eyes” review, affording you a way to make sure that your document is understood in the way in which you intended.

How to address IESG comments

To better understand how to handle these comments, consider the basis of the Area Director’s (AD) viewpoint.

The IESG is composed of Area Directors, representing different areas: Applications and Real-Time (ART), General (GEN), Internet (INT), Ops & Management (OPS), Routing (RTG), Security (SEC), and Transport (TSV). Just like any other IETF participants, ADs care more deeply about, and have more interest in and knowledge of, some protocols than others; there are many many WGs in the IETF, and, as you’ve already noticed, many many emails.

The IESG operates on a 2-week cycle, reviewing around 7-14 documents (roughly 200-400 pages) each cycle. ADs ballot on the documents and then discuss the document and their ballot positions on biweekly telechats. More information on balloting is in the IESG Ballot Procedures.

ADs ballot by choosing a position and (often) providing comments to explain their position and point out places where they think the document could be improved. Discuss ballots (usually capitalized as DISCUSS, either to draw attention to them, or to strike terror into the recipient’s hearts) are special, in that they a: require an explanation of the position and b: are blocking.

In almost all cases, reviewing a document for the telechat is the first time an AD sees your document. This means that they are not familiar with the contentious discussions that led to the contrived wording in section 4.3.5.3, nor the history behind why the document uses the term ‘widget’ instead of ‘device’ or ‘router’.

This is a feature, not a bug - you are writing an RFC, an archival document, that will likely be read by many people who were not involved in its creation. If it isn’t clear to a random AD, it’s not likely to be clear to a random implementer.

While it is possible that the AD is an expert in your particular work, there are currently more than 120 WGs, and so the AD being an expert in your particular protocol is not probable.

ADs tend to be generalists. They have usually been participating in many WGs in the IETF, and they have a good idea of how the various protocols fit together.

In addition to having their pet protocols, ADs are mainly looking for a few things:

  1. Has the process been followed correctly?
  2. Is it clear enough that someone on a desert island could implement it using just the document and the normative references?
  3. Does it conflict with other RFCs or break existing implementations?
  4. How does it impact my area?

Unsurprisingly, SEC ADs pay lots of attention to the security implications, OPS ADs pay attention to the operations and manageability, TSV ADs focus on the transport implications, etc.

Understanding the background of how these ballots come about is helpful; however, it can still be disheartening to get a new set of comments, questions, and concerns -- requiring more work -- when you thought the process was close to competition. The DISCUSS ballots, in particular, will require your attention. Other comments received may just be comments, suggestions for you to implement or not. Some of them may be nits, pointing out typos and such. To you, the document author, what does this mean and how do they need to be handled?

DISCUSS

Authors often view DISCUSS as a nuclear issue. While a DISCUSS does need to be addressed, the name is important - it is a DISCUSSion, not a death knell, nor a personal affront. ADs are careful and considered when balloting DISCUSS. As generalists, focusing on their area, they are raising a (blocking) concern that they strongly believe must be addressed to make the document publishable.

From IESG Ballot Procedures: “Discuss” may mean, “I cannot in good conscience send this document forward -- but if it were fixed in these ways, I would change my ballot position to either Yes or No Objection”, or it may literally mean “I think we need to talk about this.” There is also the “Discuss Criteria” document, which explains which classes of concerns are DISCUSS worthy.

Sometimes the DISCUSS is pointing out technical errors (“You say the NEIGHBOURS field is 8 bits, but you are trying to stuff the value 257 into it in section 3. 257 > 255.”) or internal inconsistencies (ranging from "table 1 says this thing is MAY but table 7 says it is SHOULD" to "your example seems to not include this field that the protocol says is mandatory").

Often, however, the DISCUSS is raising issues with how the protocol interacts with other protocols or systems. Most often, though, the DISCUSS comes from the fact that the AD is coming to the document “fresh”. Remember, the AD probably has no idea on the background of why it is a MUST, not a SHOULD in step 42 of section 17… and neither will an implementer who wasn’t involved in the WG discussion. The DISCUSS is an opportunity to ensure clarity of intent and clarity for implementation.

There are also informally named “discuss discusses” - don’t worry, this isn’t “DISCUSS*2” or “DISCUSS^2” - this is more along the lines of “I think that there is something really important here, but I’m not sure…I suspect there is a good reason for why you did what you did. Let’s have a discussion.” An example of this is https://datatracker.ietf.org/doc/rfc6272/ballot/

The bottom line is that a DISCUSS ballot is a request to have a discussion. The authors are responsible for initiating and driving this discussion, and the DISCUSSing AD has a responsibility to review and respond in a timely manner. The Responsible AD (one who brought the document to the IESG in the first place) is also expected to help with this discussion.

The changes made are usually ones that serve to make the document clearer and more correctly actionable, which best serves the author’s purpose as well as the IETF and larger public as a whole. And sometimes the outcome is education of the AD, and “Nothing to see here; let’s move on.”

The important things to remember when handling DISCUSSes are:

  1. Take a deep breath; we are all here to try to write the best (where “best” = most technically sound, implementable, useful, and usable) RFCs possible. While criticism can sometimes be hard to bear, the goal is always for that criticism to be constructive and result in furthering the author’s goals.
  2. These are blocking, and you do need to address them to move the document forward. It’s your responsibility to follow up, etc. Consider this a MUST.
  3. ADs do not ballot DISCUSS lightly (apart from just not wanting to be jerks, a DISCUSS adds to the ADs’ workload as they need to discuss and clear it).
  4. The title is important - they are DISCUSS positions; DISCUSS with the balloting AD.
  5. Sometimes ADs are just wrong; we too have bad days, we sometimes rush through balloting, sometimes we accidentally put on a pot of decaf instead of regular coffee and misread something -- or may just not have enough context. If a DISCUSS makes no sense, see bullet 4.
  6. Even if you immediately agree with the DISCUSS position and know exactly what you need to do to fix it, it’s important to explicitly reply to the email message and let the AD know what the status is and what you’re doing about it. ADs are dealing with many documents, and without a direct response it’s very difficult for us to keep track of everything.
  7. If there are needed changes, the AD will generally change the DISCUSS to “No Objection” after those changes are in a new version of the draft, or a public working copy on GitHub. It helps to email the AD when this has happened.

There is no “magic bullet” to dealing with a DISCUSS. Some of them will be trivial to address (often the AD will include text which will address their concerns), but some will need a number of rounds of discussion and back and forth with the AD (and possibly the WG). While doing this, it's (still!) important to remember that this is a discussion between people, and that we are all here to try and make the best standards possible.

Work with the DISCUSSing AD, and feel free to include the responsible AD/WG chairs, and whomever else you need to talk to. Once the AD (or ADs!) holding the DISCUSS position(s) are satisfied, they will clear their DISCUSS position. The responsible AD will probably notice that this has occurred, but it’s also possible that this will fall through the cracks, so sending an email to let them know that all DISCUSSes are now cleared is always appreciated.

Comments

Comments do not have to be addressed; they are not blocking. However, ADs really do want to make your document better, and they took some time to consider and add these comments. You are writing an RFC to be part of a venerable, archival, and important document series, not a throwaway shopping list, and that is understood and appreciated.

Please consider the comments and take some time to think about what drove them.

Comments should be reviewed and digested carefully. The ADs might spot something they think is minor but actually turns out to be a problem in the protocol. If they had the specialist knowledge, they might have raised it as a DISCUSS but that didn't happen. ADs are fairly representative of the eventual audience of the document; if something wasn’t clear to them or seems confusing, it’s going to be unclear or confusing to others; make your best effort to address these to make the document better. In any case, it’s just courtesy to consider reviews and comments from people who took the time to make them.

Nits

Nits are included in the comments section of the ballot, and are almost always editorial in nature. Once again, ADs are reading your document with fresh eyes, and so are likely to stumble across things like typos and similar items that you missed. You already know what the sentence meant to say, and so your eyes are performing autocorrect, but when reviewing a document if an AD come across “the router send hte packet out the interfce”, the AD will likely point out the typos. Some authors believe that the IESG isn’t responsible for editorial matters - while that’s technically true, we are all here to write the best documents possible. While the RFC Editor is ultimately responsible for catching and fixing these things, they, too, are people, and it’s inefficient, not to mention discourteous, to have the view that the RFC Editor’s job is to clean up the mess. The RFC Editor’s job is to be a final editorial check before publishing the RFC.

In summary, we all share the goal of creating high quality, understandable, and implementable documents. IESG review provides the benefit of review by people with different knowledge bases and skill sets, and a focus on particular areas. In some ways, it’s a family and friends dress rehearsal, before the show to the public at large. A DISCUSS is a blocking position, designed to open a discussion to fix or refine a document so that it meets the goal. Comments are not blocking but generally make the document better (and authors are strongly encouraged to follow up to ask for clarification, etc.).

Once all of the DISCUSS ballots have been cleared, be sure to communicate that to the Responsible AD; this saves the author/WG time to publication and the ADs the need to follow up. After they’ve checked that everything is good, they can approve and send it to the RFC Editor to be published as an RFC. As part of their process, the RFC Editor may have further questions, suggestions or edits to ensure the document is as clear as possible and conforms to the RFC format and style, and will contact the authors for final approval (AUTH48).

A document goes through many sets of eyes on the way to becoming an RFC. From authors to WGs to ADs, we all share the same goal: Make the document as clear, correct, and actionable as possible. Sometimes, it just takes a village.