Techspec BOF Notes
BoF: TechSpec -- Requirements for IETF Technical Specification
Publication
Chair: Leslie Daigle <leslie@thinkingcat.com>
Notes: Spencer Dawkins <spencer@mcsr-labs.org>
Mailing list: techspec@ietf.org
Agenda
------
- <we actually need jabber monitors more than we need jabber
scribes, these days?.
Overview & Introduction [BoF Chair]
- BOF description didn't pass through an editor, of course...
- Want to enumerate IETF publication requirements (BCP or otherwise)
- See list of "what we are not doing" in the BOF description
- Bob Braden - document formats should be discussed (somewhere, and
why not here?) Document lifecycle should also include the part where
people actually read documents?
?. Update from experiment in early copy editing
[Bert Wijnen, RFC Editor]
- ops.ietf.org/ece for documents that have gone through the
experiment, description, etc.
- Initiated by IAB, IESG, RFC-Editor
- Alice Hagens did the actual work...
- WGs participating are from OPS, TSV, SEC, and documents include a
MIB
- This was a very, very, very limited initial experiment with six
documents - need to draw conclusions carefully
- Hoping for positive impact on following steps, less copy-editing,
shortened AUTH-48
- Experiment does not mean documents have priority in queue
- We have documents in AUTH-48 for six months now, this is not
good, and changes cause churn late in the process
- Bert serialized the experiment inputs/outputs to track timings,
number of changes, etc.
- RFC Editor committed to one-week turnaround
- Using XML as format for the experiment
- Average data points - 3 days at RFC Editor, 1 hour per 8/9 pages
by RFC-Editor, 1149 changed lines, 1360 changes by authors after the
authors got documents back (311 were for verb tense, 860 caused by
author error submitting wrong version of document)
- Still need to check number of changes after WGLC (only one doc
has completed WGLC)
- We skipped the source control steps, may need an IETF service for
this
- Positive experience, great turnaround, cannot serialize in
production, need to be careful what gets shipped in each direction,
need to follow what happens in the rest of the process
- Next steps - follow up, do more experiments, figure out how to
un-serialize
- Aaron - "commodity copy-editors" - RFC Editor is using these now.
Not IETF people, maybe not even computer scientists. They do lots of
markups, but these get filtered before going to the authors. Alice has
done both tasks in this experiment - think about what happens when
authors interact with commidity copy-editors, who may be handing you
changes on a sheet of paper and not new XML, for instance.
- Pete - commodity copy editors may make a lot more changes and
slow down the process?
- Aaron - "yes", but even technical editors aren't familiar with
document structure at the IETF
- Bob - copy editors make stylistic changes, RFC Editor has been
discouraged from doing this - this is "early editing", not "early
copy-editing" - "agreement" substituted for "consensus"
Alice - take on experiment
- Using paper, then entering edits into XML, add questions for
authors and major revisions
- Differences between documents - we only edit description clauses
in MIBs, but in other documents we're figuring out how to do VSPACE and
even looking at the wrong version of the document. 8-9 pages/hour seems
typical
- Patrik - VSPACE in lists is context-dependent - do you summarize
problems you encounter with the tools? If Alice doesn't think the
problem is learning experience..
- Need to do some additional measures from RFC Editor perspective -
time/changes, issues not resolved, AUTH48 progress
Miguel Garcia - editor for Diameter application for SIP draft
- Document is 80 pages
- Did two formal revisions over about 30 days
- Some suggestions result in a large number of changed lines
(inconsistent terminology, for example)
- Still doing some changes at the end of the process (some
technical, some formatting, etc.)
- All suggestions were editorial, not technical
- Very positive experience, especially not making global changes in
AUTH48 with OLD/NEW
- WG always had opportunity to see changes
- No slowdown detected
- Some documents have lots of dependencies - this may be a drawback
for something like GRUU
Elwyn - editor for NAT-PT-to-Experimental draft
- Had actually been through WGLC, perhaps this was a
misunderstanding
- Did about a three-day cycle, with lots of small changes ("which"
instead of "that", commas). Almost all changes were editorial, did
identify terminology overload and one other non-editorial issue
- One "false positive" editorially, but we fixed the sentence in a
better way anyhow
- Resulting document is a lot better when it hits IETF LC (with
Gen-ART reviewing hat on)
- Not sure how much parallelization is possible
- Bert - can't wait six months for a doc, can't pay $5M, it's a
tradeoff
- Allison - working group text is stable makes it better - what
happens if the WG is still making lots of diffs? WGLC is when people
read documents, need to take this into account
- Bert - this document went through WGLC and was still difficult to
read - process helped
Jari - co-chair of MOBIKE
- Also went quite smoothly, had to wait in queue a little
- Focus resources on documents that DO need the changes
- Elwyn - would be easy for copy editors to do this, Gen-ART isn't
copy editors
- Bob - "some documents need few changes, therefore there should be
no changes?"
Discussion
- Spencer - Really like opportunity for global suggestions "early".
Matches my Gen-ART experience - maybe experiment with one or two
documents BEFORE WGLC? and sets of documents need to be consistent
internally, too
- David Black - Also Gen-ART - copy editors can clean up language,
not technical lack of clarity
- RFC Editor - global changes in AUTH48 - prefer NEW/OLD because
many douments have sets of similar text, want to know exactly where the
text is - and we keep sets of documents with one copy editor
. 3. Review and discussion of draft-mankin-pub-req-01
[Allison Mankin]
- Have never really thought about document lifetime in the IETF
before - we pick up readers early because of "running code", so adding
readers to the lifecycle isn't easy to think about
- Bob - at least add readers to the diagram?
- (Req-3) - Detailed visibility into Tech Pub tracking
- Aaron - what does Req-3 actually mean? for example?
Correspondence with editors visible to WG, like correspondence with ADs
- copying the WG mailing list would be sufficient?
- Brian - to clarify the clarification - we are working toward
interaction visibility with IESG, don't do as we do, do as we say
- Aaron - want to know what the RFC Editor does now?
- Leslie - yes, but not in this meeting, we're focused on what our
requirements are first
- Allison - are we ready to talk about adopting these requirements
yet?
- Bob - RFC Editor is part of the community and agrees with Req-2
and Req-3 strongly - need to think about resources required
- Bert - sort of agree, but first requirement is that we hit a
button and the document appears (applause in the room). How much time
are we prepared to accept? We don't need detailed visiblity if the
document appears in two weeks.
- Leslie - we can move work, but it doesn't go away, and if we need
visibility, we need it no matter where the work is
- (Req-2) - do we want seamless tracking of document movement into
Tech Pub?
- Bert - why not one tracking system?
- Eric - do we need detailed visibility of line-by-line editing?
May not be appropriate
- (Pre-Req-7) One coordinator for the editor process?
- Bob - what's the issue here?
- Allison - dealing with five authors, or one of them, or ...
- Bob - we contact all authors to make sure they all exist and are
still alive - want us to change our policy?
- Leslie - have been on IAB telechats for years - there is lack of
clarity about who is doing what
- Pekka - sometimes one author isn't responsive, and it's not clear
who steps up when that happens
- Thomas - yes, we want to have one person. The current system
works pretty well most of the time. We don't really have authors
anymore with WG text, we have WG editors and the other names shouldn't
be slowing down the process
- Scott - no-brainer that either authors or ADs should pick one
contact. Final signoff still needs to be by each author
- Leslie - IETF decision for this is a requirement, need to have
discussion about what we decide
- Margaret - mix together concepts of credit and control. If an
author dies, they deserve credit but don't have control (except maybe
Bert, who will come back to haunt us). This is the extreme case, not
the only case. Steve Deering deserves IPv6 credit but not IPv6 control,
at this point. Don't know whose names go on the front page, and maybe
we shouldn't even put names on the first page.
- David - moving a lot of docs through this process, what Scott
said was correct (masthead signs off). Proto shepherd is the right
victim, and should be able to exempt and remove authors
- Bob - more comfortable with ADs making this decision - is "DOES"
in requirement MUST, MAY, or SHOULD? Margaret is right - first page is
tending toward "control", with acknowledgements in a separate section.
- (Pre-Req-8) Non-author editing affecting consensus wording
- Bob - what happens if consensus wording is grammatically
incorrect?
- Allison - discuss bigger changes on WG mailing list
- Leslie - when non-author editing affects wording, we need to
resolve this
- (Req-9) No style changes?
- (Req-10) Small technical changes submitted in a narrow window
following approval, not at time of publication
- Document shepard and AD can filter these
- Jari - queuing takes time, things happen, people implement after
approval. These changes show up at AUTH48.
- Margaret - after approval, something comes up - what should
happen? We don't know what to do when something comes up. WG should
decide how to handle large flaws. We have pulled documents out of
editor queue - need to know what to do, in a process sense.
- Allison - WGs are terrible at saying "no". Trust the shepherd to
determine what's a show-stopper? We have finished the process and are
collecting issues towards a -bis version. Ask the working group "broken
beyond repair?" - don't go through process multiple times for small
things.
- Leslie - need to socialize our process from multiple
perspectives, but can't do that in next 43 minutes. Need to discuss on
the list, need to move on now. We will revise the document - we know
that. Discuss creating a working group?
- Bob - document is scattershot
- Allison - someone else will be holding the pen
- Bob - needs to address more issues and provide more context - not
the basis for a working group yet
- John - this document is part of our game of "whack-a-mole" -
we're off to the land of requirements, not solutions. This topic is
worth working on, this document is not the right way to go
- Scott - there is a pony underneath the poop someplace. Some of
this document is OK, but it's the problem set, not the document.
- Dan Burnett - process question - also need to know who is going
to do the work
- Bert - yes, we combined both questions (should work on it, will
work on it). A lot of this is darned complex. What happens before IESG
approval is a lot cheaper than what happens after IESG approval, and
part of the cost is delay.
- Margaret - I don't have enough information (not in evidence).
Lots of interaction with other short-term efforts. Do short-term
efforts, at least in parallel.
- Leslie - if we are doing competitive bids, we need to know what
we're asking for
- Margaret - we paid $800K based on an SOW, do we need more?
- Leslie - SOW is what tasks are carried out, not what we WANT
carried out
- Brian - What Leslie said, plus - need general description text
that we don't have in the SOW today
- Bob, as member of community - please focus on readership! What an
implementor has to focus on to implement anything. That was supposed to
happen in Newtrk, but it's not. It's a big elephant.
- Allison - what the document is supposed to do, is figure out what
we really want, not just what we do, document by document, area by
area, with no consistency. We are trying to guide IASA, and they don't
know what we want them to do with their dollars
- Leslie - need to reel all this back in ... for further work, we
need more information about purpose, which is known as a charter in
these parts
- Bert - not trying to pick on our problems, we are putting forth
requirements for fixing problems, not for producing documents
- David - consistently publishing bug fixes, before and after
approval, is a requirement
- Margaret - we have substantial requirements for publication, not
just for editing (archival quality, etc). In this document, or
elsewhere?
- Allison - "Cataloging" in the slide deck (which we didn't get to).
- Thomas - must attach errata to the document in an obvious way
- Sam - we need a low-overhead publication path - as part of what
the IETF needs. We are trying to raise the bar for what goes into a
working group - excluding work that needs to be published.
- Harald - I'm worried (perhaps not for the first time, but) -
technical publication means getting documents to the public so they can
be referenced quickly. Need to know who to talk to in order to do
something - open process - and how violently to talk to them. We have
major issues with getting documents out, and making sure what comes out
is what we approved. Two years to make up our minds isn't part of the
process - don't use tech publishing as a holding queue
- Pasi - Errata - if our documents took two weeks to publication,
we wouldn't need errata
- Allison - find a problem a year later, don't want to publish a
new RFC to correct an RFC
- Pasi - if it was easy and fast to publish a new RFC...
- Leslie - interesting... counterpoint is that discussion also
shows things down
- Bob - the less editing you do, the more errata you publish. One
guy in Germany reads all the RFCs and sends in "by the way, have you
noticed?" - often up to 10 or so per RFC
- Eric - why focus on errors that appear in RFCs? Knuth has an
error on every third page and he's one of the most careful authors
ever. We have to live with this
- Leslie - please continue discussion on the mailing list.
5. Determination of where from here for
requirements documentation.
- (did we actually do this?)
|