2.2.4 Requirements for IETF Technical Specificaiton Publication (techspec)

NOTE: This charter is a snapshot of the 64th IETF Meeting in Vancouver, British Columbia Canada. It may now be out-of-date.

Last Modified: 2005-10-03

Chair(s):

Leslie Daigle <leslie@thinkingcat.com>

General Area Director(s):

Brian Carpenter <brc@zurich.ibm.com>

General Area Advisor:

Brian Carpenter <brc@zurich.ibm.com>

Mailing Lists:

General Discussion:
To Subscribe:
Archive:

Description of Working Group:

The work of the IETF is to discuss, develop and disseminate
technical specifications to support the Internet's operation.
An important output of the IETF, then, is published technical
specifications. As the IETF progresses, documentation and review
of its requirements for the process and structure of technical
specification publication is important in order to ensure continued
support for the IETF's work.

The focus of this discussion is on the constructive enumeration and
expression of IETF publication requirements, without prejudice as to
whether the requirements are currently met or not.

As input to this discussion, draft-mankin-techspec-req has been
prepared as a first draft of the requirements, based on the IESG's
experience and perspective in shepherding documents through the
standards process. The intention going forward is to produce a
community-reviewed document (BCP or otherwise, as appropriate).

Goals and Milestones:

No Current Internet-Drafts

No Request For Comments

Current Meeting Report

Techspec BOF Notes BoF: TechSpec -- Requirements for IETF Technical Specification Publication

Chair: Leslie Daigle <leslie@thinkingcat.com>
Notes: Spencer Dawkins <spencer@mcsr-labs.org>
Mailing list: techspec@ietf.org

Agenda
------
  • <we actually need jabber monitors more than we need jabber scribes, these days?.
Overview & Introduction [BoF Chair]
  • BOF description didn't pass through an editor, of course...
  • Want to enumerate IETF publication requirements (BCP or otherwise)
  • See list of "what we are not doing" in the BOF description
  • Bob Braden - document formats should be discussed (somewhere, and why not here?) Document lifecycle should also include the part where people actually read documents?
?. Update from experiment in early copy editing [Bert Wijnen, RFC Editor]
  • ops.ietf.org/ece for documents that have gone through the experiment, description, etc.
  • Initiated by IAB, IESG, RFC-Editor
  • Alice Hagens did the actual work...
  • WGs participating are from OPS, TSV, SEC, and documents include a MIB
  • This was a very, very, very limited initial experiment with six documents - need to draw conclusions carefully
  • Hoping for positive impact on following steps, less copy-editing, shortened AUTH-48
  • Experiment does not mean documents have priority in queue
  • We have documents in AUTH-48 for six months now, this is not good, and changes cause churn late in the process
  • Bert serialized the experiment inputs/outputs to track timings, number of changes, etc.
  • RFC Editor committed to one-week turnaround
  • Using XML as format for the experiment
  • Average data points - 3 days at RFC Editor, 1 hour per 8/9 pages by RFC-Editor, 1149 changed lines, 1360 changes by authors after the authors got documents back (311 were for verb tense, 860 caused by author error submitting wrong version of document)
  • Still need to check number of changes after WGLC (only one doc has completed WGLC)
  • We skipped the source control steps, may need an IETF service for this
  • Positive experience, great turnaround, cannot serialize in production, need to be careful what gets shipped in each direction, need to follow what happens in the rest of the process
  • Next steps - follow up, do more experiments, figure out how to un-serialize
  • Aaron - "commodity copy-editors" - RFC Editor is using these now. Not IETF people, maybe not even computer scientists. They do lots of markups, but these get filtered before going to the authors. Alice has done both tasks in this experiment - think about what happens when authors interact with commidity copy-editors, who may be handing you changes on a sheet of paper and not new XML, for instance.
  • Pete - commodity copy editors may make a lot more changes and slow down the process?
  • Aaron - "yes", but even technical editors aren't familiar with document structure at the IETF
  • Bob - copy editors make stylistic changes, RFC Editor has been discouraged from doing this - this is "early editing", not "early copy-editing" - "agreement" substituted for "consensus"
Alice - take on experiment
  • Using paper, then entering edits into XML, add questions for authors and major revisions
  • Differences between documents - we only edit description clauses in MIBs, but in other documents we're figuring out how to do VSPACE and even looking at the wrong version of the document. 8-9 pages/hour seems typical
  • Patrik - VSPACE in lists is context-dependent - do you summarize problems you encounter with the tools? If Alice doesn't think the problem is learning experience..
  • Need to do some additional measures from RFC Editor perspective - time/changes, issues not resolved, AUTH48 progress
Miguel Garcia - editor for Diameter application for SIP draft
  • Document is 80 pages
  • Did two formal revisions over about 30 days
  • Some suggestions result in a large number of changed lines (inconsistent terminology, for example)
  • Still doing some changes at the end of the process (some technical, some formatting, etc.)
  • All suggestions were editorial, not technical
  • Very positive experience, especially not making global changes in AUTH48 with OLD/NEW
  • WG always had opportunity to see changes
  • No slowdown detected
  • Some documents have lots of dependencies - this may be a drawback for something like GRUU
Elwyn - editor for NAT-PT-to-Experimental draft
  • Had actually been through WGLC, perhaps this was a misunderstanding
  • Did about a three-day cycle, with lots of small changes ("which" instead of "that", commas). Almost all changes were editorial, did identify terminology overload and one other non-editorial issue
  • One "false positive" editorially, but we fixed the sentence in a better way anyhow
  • Resulting document is a lot better when it hits IETF LC (with Gen-ART reviewing hat on)
  • Not sure how much parallelization is possible
  • Bert - can't wait six months for a doc, can't pay $5M, it's a tradeoff
  • Allison - working group text is stable makes it better - what happens if the WG is still making lots of diffs? WGLC is when people read documents, need to take this into account
  • Bert - this document went through WGLC and was still difficult to read - process helped
Jari - co-chair of MOBIKE
  • Also went quite smoothly, had to wait in queue a little
  • Focus resources on documents that DO need the changes
  • Elwyn - would be easy for copy editors to do this, Gen-ART isn't copy editors
  • Bob - "some documents need few changes, therefore there should be no changes?"
Discussion
  • Spencer - Really like opportunity for global suggestions "early". Matches my Gen-ART experience - maybe experiment with one or two documents BEFORE WGLC? and sets of documents need to be consistent internally, too
  • David Black - Also Gen-ART - copy editors can clean up language, not technical lack of clarity
  • RFC Editor - global changes in AUTH48 - prefer NEW/OLD because many douments have sets of similar text, want to know exactly where the text is - and we keep sets of documents with one copy editor
. 3. Review and discussion of draft-mankin-pub-req-01 [Allison Mankin]
  • Have never really thought about document lifetime in the IETF before - we pick up readers early because of "running code", so adding readers to the lifecycle isn't easy to think about
  • Bob - at least add readers to the diagram?
  • (Req-3) - Detailed visibility into Tech Pub tracking
  • Aaron - what does Req-3 actually mean? for example? Correspondence with editors visible to WG, like correspondence with ADs - copying the WG mailing list would be sufficient?
  • Brian - to clarify the clarification - we are working toward interaction visibility with IESG, don't do as we do, do as we say
  • Aaron - want to know what the RFC Editor does now?
  • Leslie - yes, but not in this meeting, we're focused on what our requirements are first
  • Allison - are we ready to talk about adopting these requirements yet?
  • Bob - RFC Editor is part of the community and agrees with Req-2 and Req-3 strongly - need to think about resources required
  • Bert - sort of agree, but first requirement is that we hit a button and the document appears (applause in the room). How much time are we prepared to accept? We don't need detailed visiblity if the document appears in two weeks.
  • Leslie - we can move work, but it doesn't go away, and if we need visibility, we need it no matter where the work is
  • (Req-2) - do we want seamless tracking of document movement into Tech Pub?
  • Bert - why not one tracking system?
  • Eric - do we need detailed visibility of line-by-line editing? May not be appropriate
  • (Pre-Req-7) One coordinator for the editor process?
  • Bob - what's the issue here?
  • Allison - dealing with five authors, or one of them, or ...
  • Bob - we contact all authors to make sure they all exist and are still alive - want us to change our policy?
  • Leslie - have been on IAB telechats for years - there is lack of clarity about who is doing what
  • Pekka - sometimes one author isn't responsive, and it's not clear who steps up when that happens
  • Thomas - yes, we want to have one person. The current system works pretty well most of the time. We don't really have authors anymore with WG text, we have WG editors and the other names shouldn't be slowing down the process
  • Scott - no-brainer that either authors or ADs should pick one contact. Final signoff still needs to be by each author
  • Leslie - IETF decision for this is a requirement, need to have discussion about what we decide
  • Margaret - mix together concepts of credit and control. If an author dies, they deserve credit but don't have control (except maybe Bert, who will come back to haunt us). This is the extreme case, not the only case. Steve Deering deserves IPv6 credit but not IPv6 control, at this point. Don't know whose names go on the front page, and maybe we shouldn't even put names on the first page.
  • David - moving a lot of docs through this process, what Scott said was correct (masthead signs off). Proto shepherd is the right victim, and should be able to exempt and remove authors
  • Bob - more comfortable with ADs making this decision - is "DOES" in requirement MUST, MAY, or SHOULD? Margaret is right - first page is tending toward "control", with acknowledgements in a separate section.
  • (Pre-Req-8) Non-author editing affecting consensus wording
  • Bob - what happens if consensus wording is grammatically incorrect?
  • Allison - discuss bigger changes on WG mailing list
  • Leslie - when non-author editing affects wording, we need to resolve this
  • (Req-9) No style changes?
  • (Req-10) Small technical changes submitted in a narrow window following approval, not at time of publication
  • Document shepard and AD can filter these
  • Jari - queuing takes time, things happen, people implement after approval. These changes show up at AUTH48.
  • Margaret - after approval, something comes up - what should happen? We don't know what to do when something comes up. WG should decide how to handle large flaws. We have pulled documents out of editor queue - need to know what to do, in a process sense.
  • Allison - WGs are terrible at saying "no". Trust the shepherd to determine what's a show-stopper? We have finished the process and are collecting issues towards a -bis version. Ask the working group "broken beyond repair?" - don't go through process multiple times for small things.
  • Leslie - need to socialize our process from multiple perspectives, but can't do that in next 43 minutes. Need to discuss on the list, need to move on now. We will revise the document - we know that. Discuss creating a working group?
  • Bob - document is scattershot
  • Allison - someone else will be holding the pen
  • Bob - needs to address more issues and provide more context - not the basis for a working group yet
  • John - this document is part of our game of "whack-a-mole" - we're off to the land of requirements, not solutions. This topic is worth working on, this document is not the right way to go
  • Scott - there is a pony underneath the poop someplace. Some of this document is OK, but it's the problem set, not the document.
  • Dan Burnett - process question - also need to know who is going to do the work
  • Bert - yes, we combined both questions (should work on it, will work on it). A lot of this is darned complex. What happens before IESG approval is a lot cheaper than what happens after IESG approval, and part of the cost is delay.
  • Margaret - I don't have enough information (not in evidence). Lots of interaction with other short-term efforts. Do short-term efforts, at least in parallel.
  • Leslie - if we are doing competitive bids, we need to know what we're asking for
  • Margaret - we paid $800K based on an SOW, do we need more?
  • Leslie - SOW is what tasks are carried out, not what we WANT carried out
  • Brian - What Leslie said, plus - need general description text that we don't have in the SOW today
  • Bob, as member of community - please focus on readership! What an implementor has to focus on to implement anything. That was supposed to happen in Newtrk, but it's not. It's a big elephant.
  • Allison - what the document is supposed to do, is figure out what we really want, not just what we do, document by document, area by area, with no consistency. We are trying to guide IASA, and they don't know what we want them to do with their dollars
  • Leslie - need to reel all this back in ... for further work, we need more information about purpose, which is known as a charter in these parts
  • Bert - not trying to pick on our problems, we are putting forth requirements for fixing problems, not for producing documents
  • David - consistently publishing bug fixes, before and after approval, is a requirement
  • Margaret - we have substantial requirements for publication, not just for editing (archival quality, etc). In this document, or elsewhere?
  • Allison - "Cataloging" in the slide deck (which we didn't get to).
  • Thomas - must attach errata to the document in an obvious way
  • Sam - we need a low-overhead publication path - as part of what the IETF needs. We are trying to raise the bar for what goes into a working group - excluding work that needs to be published.
  • Harald - I'm worried (perhaps not for the first time, but) - technical publication means getting documents to the public so they can be referenced quickly. Need to know who to talk to in order to do something - open process - and how violently to talk to them. We have major issues with getting documents out, and making sure what comes out is what we approved. Two years to make up our minds isn't part of the process - don't use tech publishing as a holding queue
  • Pasi - Errata - if our documents took two weeks to publication, we wouldn't need errata
  • Allison - find a problem a year later, don't want to publish a new RFC to correct an RFC
  • Pasi - if it was easy and fast to publish a new RFC...
  • Leslie - interesting... counterpoint is that discussion also shows things down
  • Bob - the less editing you do, the more errata you publish. One guy in Germany reads all the RFCs and sends in "by the way, have you noticed?" - often up to 10 or so per RFC
  • Eric - why focus on errors that appear in RFCs? Knuth has an error on every third page and he's one of the most careful authors ever. We have to live with this
  • Leslie - please continue discussion on the mailing list.
5. Determination of where from here for requirements documentation.
  • (did we actually do this?)

Slides

ChairSlides
EarlyEditReport-BertWijnen
EarlyEditReport-AliceHagens
EarlyEditReport-MiguelGarcia
PubReq-AllisonMankin